Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Introduction

Plenty is a general-purpose programming language. Plenty is statically-typed and compiles ahead of time to native code. Plenty is most heavily influenced by Python, Rust, and Zig.

Why the name “Plenty”?

I wanted a language without trade-offs: simple syntax that is easy to learn, easy to read and write, but also produces fast code, and is safe and reliable, and with a huge standard library to reduce the need for third-party dependencies. And more! I wanted everything that is good about each programming language. The programmer’s version of “a land of plenty” where you can have it all.

Goals

There are multiple goals for Plenty:

  • Convenience and safety: Could we merge the best parts of Python and Rust? This is Plenty.
  • Explicit ownership: Ownership and borrowing are explicit, with compile-time checks.
  • Sum types: Enums are tagged unions and are used to represent sum types. Very similar to Rust.
  • Fast compilation: The compiler is designed to compile quickly, even for large codebases.
  • Efficient: An optimizing compiler generating machine code (Cranelift). SIMD support. Parallel tools like Rust’s rayon. Exploit hardware features.
  • No garbage collector: Deterministic memory usage patterns.
  • No dynamic behaviour: Figure out as much as possible at compile time.
  • Fallible memory allocation: All operations that allocate memory can report allocation failure without exceptions.
  • Simple FFI interop with C: In both directions. There is an interface file format to specify lifetime information for resources from a C library so that Plenty can manage them safely.
  • No inheritance: Classes and enums have fixed declarations rather than dynamic attributes or inheritance.
  • Generics and Protocols: A native version of Python’s typing.Protocol. Something like Go interfaces, or a simpler version of Rust’s traits.
  • Batteries included: A standard library so stuffed to the gills with awesome that many programs can be written using the standard library alone.
  • Simple package management: No central package repo. Use golang-inspired git links for dependencies.

The primary influences on Plenty are:

  • Python, for syntax
  • Rust, for ownership and borrowing, and getting the details right
  • Zig, for emphasizing fallible memory allocation

Getting Started

This book provides:

  • Learn Plenty teaches the language with runnable examples. Start here.
  • Reference defines language rules, built-in facilities, and compiler/runtime architecture in separate areas.

Tested examples

cargo test --test integration test_tutorial:: compiles and runs every example in Learn Plenty. It compares the output with the expected output on the page. An example that must fail must give the expected diagnostic.

Build this book

mdbook serve book --open

Learning Plenty

Plenty combines Python-shaped syntax with explicit types, ownership, and native compilation. The examples are self-contained and teach implemented features.

Start with Setting up to install a release and put plenty on your PATH. Then follow the ten core parts to build a small command-line tool:

  1. Your first program: compile, run, and check.
  2. Values, functions, and control flow: name values, choose types, define functions, and repeat work.
  3. Absence and failure: handle missing values and errors with Option, Result, match, and ?.
  4. Collections, ownership, and borrowing: keep groups of values, transfer ownership, copy, and borrow.
  5. Iteration and transformation: visit values, transform them, and produce them incrementally.
  6. Defining your own data: write records, methods, constructors, and enums.
  7. Heterogeneous collections: combine different kinds of values with enums and anonymous sums.
  8. Special-purpose data structures: choose columnar storage for calculations over selected fields.
  9. Organizing a program: split a program into modules and choose its public interfaces.
  10. Useful programs and resources: use command-line arguments, text, standard streams, and files in a complete application.

Then choose an optional path when your program needs it:

The Reference describes detailed contracts and limits. Use the supplementary examples to look up collection methods, text operations, and advanced file usage.

Setting up

Install Plenty once, then use the plenty command throughout this tutorial. You do not need Rust, Cargo, or a copy of the compiler’s source repository.

Building from a source checkout instead? The local build recipes provide just build for the release compiler, both optional engines, and their licenses/metadata. Put that checkout’s target/release/ on PATH. These builds require Rust and native build tools and target your local system, unlike the portable binary download described here.

Check your system

Binary releases support x86-64 and ARM64 Linux with glibc 2.28 or later. This page uses the x86-64 archive names; on an ARM64 machine (uname -m prints aarch64), replace x86_64-unknown-linux-gnu with aarch64-unknown-linux-gnu in every command. Musl-based distributions such as Alpine are unsupported. Releases also include experimental macOS and Windows downloads, whose READMEs replace the Linux steps below. See the installation requirements for supported systems and troubleshooting.

Plenty also needs a C linker and development libraries to run or compile a program. Install the packages for your distribution before continuing:

# Debian or Ubuntu
sudo apt-get update && sudo apt-get install gcc libc6-dev

# Fedora, RHEL, AlmaLinux, or Rocky Linux
sudo dnf install gcc tar gzip

# Arch Linux
sudo pacman -Syu --needed gcc

Download and extract Plenty

Open Plenty’s GitHub Releases and choose a published release. Under Assets, download both:

  • plenty-VERSION-x86_64-unknown-linux-gnu.tar.gz
  • plenty-VERSION-x86_64-unknown-linux-gnu.tar.gz.sha256

Choose the binary archive, rather than GitHub’s Source code (zip) download. Plenty currently ships as a compressed tar archive, so extract it with tar. In every command below, replace VERSION with the version you downloaded; keep the same version in the archive name and directory name.

Assuming both files are in ~/Downloads, verify and extract them:

cd "$HOME/Downloads"
sha256sum -c plenty-VERSION-x86_64-unknown-linux-gnu.tar.gz.sha256
mkdir -p "$HOME/.local/opt"
tar -xzf plenty-VERSION-x86_64-unknown-linux-gnu.tar.gz -C "$HOME/.local/opt"

Check that the checksum command reports OK before extracting. The resulting directory contains bin/plenty and bin/plentyc. Keep the extracted directory; the shell setup below points to its bin directory.

Add Plenty to your PATH

PATH is the list of directories your shell searches for commands. Choose the instructions for your shell. These paths are quoted so they also work when your home directory contains spaces.

Bash

Add this line to ~/.bashrc, replacing VERSION:

export PATH="$HOME/.local/opt/plenty-VERSION-x86_64-unknown-linux-gnu/bin:$PATH"

Run that same line in your current terminal to use Plenty immediately. New interactive terminals read .bashrc. If you use a Bash login shell, ensure your login startup file also sources .bashrc; see Bash startup files.

Zsh

Add this line to ~/.zshrc, replacing VERSION:

export PATH="$HOME/.local/opt/plenty-VERSION-x86_64-unknown-linux-gnu/bin:$PATH"

Run that same line in your current terminal. New interactive Zsh terminals read .zshrc. If you configured ZDOTDIR, use $ZDOTDIR/.zshrc instead; see Zsh startup files.

Fish

Run this command once in Fish, replacing VERSION:

fish_add_path --universal --prepend --move "$HOME/.local/opt/plenty-VERSION-x86_64-unknown-linux-gnu/bin"

fish_add_path updates the current shell and remembers the directory for future Fish sessions. The directory must already exist, so extract the archive first. There is no need to edit config.fish for this universal setting.

Check the installation

In any of these shells, run:

command -v plenty
plenty --version
cc --version

The first command should name plenty inside your extracted archive; the second prints the installed version and target. If plenty is not found, check that the PATH entry matches the actual extracted directory. If another installation appears, put the new directory first on PATH. If cc is missing, install the linking packages above.

You are ready for Your first program. Save your programs in any working directory you choose. When installing a newer release, update the PATH entry to its new directory; see upgrading and switching versions.

Your first program

After setting up Plenty, run a complete program, then check and compile changes.

  1. Run a program
  2. Change and check it

Run a program

Save this as hello.plenty:

def main() -> Result[(), IoError | OverflowError[i64]]:
    print("Hello, Plenty!")?
    print(6 * 7)?
    Ok(())
Hello, Plenty!
42

In the directory where you saved the file, run:

plenty hello.plenty

A run compiles your file to a native executable and then runs it. If your shell cannot find plenty, follow Setting up first.

Plenty caches compiled code automatically. On supported Linux toolchains, unchanged runs also reuse the linked executable, so they do not link again. See the compile cache for invalidation and toolchain limits. The program still runs each time.

On Unix, you can put #!/usr/bin/env plenty before the first line, then run:

chmod +x hello.plenty
./hello.plenty

This uses the same automatic caches as plenty hello.plenty.

Execution begins at main. Indentation groups its statements; use four spaces. print prints one value followed by a newline. Put executable statements inside functions.

Printing can fail. The ? after each call continues when printing succeeds and returns an output error when it fails. Integer multiplication can overflow too, so the signature includes OverflowError[i64]. Its operands fit here, but the function still declares the arithmetic error contract. Ok(()) reports that the program finished successfully. Keep this pattern for printing; Absence and failure explains how to handle errors.

Each example is a complete program. Run it in a new file, along with any labelled companion modules shown on its page.

Change and check a program

Change the greeting or calculation, then run the file again. Each run starts a fresh process.

Use plenty --check hello.plenty to check syntax and types without running the program. A check needs no linker. A run or native compilation needs a cc-compatible linker driver installed.

To keep an executable:

plenty --compile hello.plenty -o /tmp/hello-plenty
/tmp/hello-plenty

The executable can run without Plenty installed, on a compatible host system. To ship an application directory instead:

plenty --compile hello.plenty --dist dist/hello
./dist/hello/hello

Choose a destination that does not already exist. This greeting needs only the executable. Applications reaching optional native engines also get the exact libraries and packaged notices they need, without development-machine lookup paths. This is not a static or cross-platform build; see application distributions for engine setup and platform requirements.

Parse and type errors prevent execution. A runtime error can happen after earlier effects, such as printing. Use --check when you want feedback without those effects, or print(value)? inside main to inspect a value.

Values, functions, and control flow

Name values, write typed functions, and choose or repeat work:

  1. Name values
  2. Choose numeric types
  3. Calculate with numbers Floating-point calculations extend these rules.
  4. Write text
  5. Build text with f-strings
  6. Define functions
  7. Complete work without returning data
  8. Choose between branches
  9. Return early
  10. Keep names in scope
  11. Repeat with a range
  12. Repeat while a condition holds
  13. Give types descriptive names

Next: Absence and failure.

Name values

Use = to create a binding. Plenty infers the type of a local value, so you usually do not need an annotation:

def main() -> Result[(), IoError | OverflowError[i64]]:
    # Comments begin with a hash.
    price = 20
    quantity = 3
    total = price * quantity
    print(total)?
    Ok(())
60

To see inferred types without writing annotations, enable your editor’s inlay hints. For example, the Plenty language server can show : i64 after price, quantity, and total. These labels are only displayed in the editor; they are not added to your source file. Hover also shows a binding’s type.

Bindings are immutable by default:

def main() -> ():
    score = 10
    score = 11
`score` is immutable; declare it with `mut`

Use mut when a value needs to change. Write it once, at the declaration:

def main() -> Result[(), IoError | OverflowError[i64]]:
    mut score: i64 = 10
    score += 5
    print(score)?
    Ok(())
15

A mutable binding keeps its original type. score: i64 states the type explicitly; without the annotation, this integer literal also infers i64.

score += 5 adds to the current score. Numeric values also support -=, *=, /=, //=, and %=. Use //= for integer division and /= for float division; %= requires integers. These updates require an existing writable destination and keep its type. Strings and collections use their explicit operations.

Choose the width of an integer

The built-in integer names specify their sizes:

TypesMeaning
i8, i16, i32, i64Signed integers of 8, 16, 32, or 64 bits
u8, u16, u32, u64Unsigned integers of those widths

For example, i8 can hold -128 through 127, and u8 can hold 0 through 255.

A whole-number literal without a suffix, such as 42, defaults to i64 when its type is unconstrained. An annotation, parameter, return type, or arithmetic operand can supply the type instead. Append a built-in integer type to choose another width:

def main() -> Result[(), IoError]:
    small: u8 = 200u8
    offset: i32 = -12i32
    large: u64 = 1_000_000u64
    print(small)?
    print(offset)?
    print(large)?
    Ok(())
200
-12
1000000

An annotation guides an unsuffixed literal, so small: u8 = 200 works. It does not convert an already typed initializer: convert the value explicitly for that.

Underscores separate digits for readability, as in 1_000_000u64 above. Integers may also be written in hexadecimal, octal, or binary, with the same suffixes and range checks:

def main() -> Result[(), IoError]:
    mask: u8 = 0b1111_0000
    print(0xFF, 0o755, mask, 0x1F600u32)?
    Ok(())
255 493 240 128512

Calculate with numbers

Arithmetic requires matching types. Integer failures are recoverable by default: list their error types in the function’s Result return, and use Ok(...) for success. Operators themselves still produce ordinary numbers.

def main() -> Result[(), IoError]:
    small = 7u8
    print(small + 1i64)?
    Ok(())
cannot combine u8 with i64; convert with `i64(small)`

Choose a matching literal or explicitly convert a value:

def main() -> Result[(), IoError | OverflowError[u8] | OverflowError[i64]]:
    small = 7u8
    print(small + 1u8)?
    print(i64(small) + 1)?
    Ok(())
8
8

i64(small) is safe because every u8 value fits in an i64. When the target cannot hold every value of the source, use try_from. It checks the range and returns a Result: Ok with the converted value, or Err(RangeError.OutOfRange) when the value does not fit. RangeError has only this one variant; parsing text uses the separate ParseError type.

def main() -> Result[(), RangeError | IoError | OverflowError[u8]]:
    wide = 200
    small = u8.try_from(wide)?
    print(small + 1u8)?
    Ok(())
201

? takes the converted value or returns its error to the caller. The RangeError | IoError error type preserves either a conversion error or a printing error. Absence and failure explains propagation and how to recover from an error.

A cast to an integer type requires the value to fit. A literal that does not fit is rejected when the program is compiled:

def main() -> ():
    small = u8(300)
300 is out of range for u8; u8.wrapping_from keeps the low bits

A value known only when the program runs is checked then. With wide = 300, u8(wide) stops the program with error: integer cast out of range. Casts retain this trapping behavior; arithmetic errors use typed returns. The same applies to a negative value cast to an unsigned type, such as u64(-1), and to an unsigned value above the signed maximum, such as i8(200u8). Cast when the value is known to fit, and use try_from when it might not.

When discarding the high bits is the intent, say so with wrapping_from, which never fails:

def main() -> Result[(), IoError]:
    wide = 300
    negative = -1
    print(u8.wrapping_from(wide))?
    print(u64.wrapping_from(negative))?
    print(i8.wrapping_from(200u8))?
    Ok(())
44
18446744073709551615
-56

Integer overflow returns OverflowError[T], where T is the operand type. An unsigned type has no negative values, so unary minus requires a signed integer or a float:

def main() -> Result[(), IoError]:
    small = 7u8
    print(-small)?
    Ok(())
unary minus is not defined for unsigned type u8

Convert first: -i16(small) is -7.

The same rule applies to literals, including zero:

def main() -> ():
    zero: u8 = -0
unary minus is not defined for unsigned type u8

+, -, and * have their usual arithmetic precedence. // divides integers and rounds down, including for negative values:

def main() -> Result[(), IoError | OverflowError[i64] | ZeroDivisionError]:
    print(1 + 2 * 3)?
    print((1 + 2) * 3)?
    print(-7 // 3)?
    Ok(())
7
9
-3

Integer division by zero returns ZeroDivisionError. Signed division also requires OverflowError[T] for T.MIN // -1. The contract follows the operators and operand types, even with a known nonzero divisor such as 3. Use / for floating-point division.

Integers also support bitwise operations and shifts. A shift count must be nonnegative and smaller than the integer width. Signed right shifts preserve the sign; unsigned right shifts fill with zero. Left shifts discard high bits.

def main() -> Result[(), IoError]:
    bits = 5u8
    print(bits & 3)?
    print(bits | 2)?
    print(bits ^ 3)?
    print(~bits)?
    print(bits << 1)?
    print(bits >> 1)?
    print(255u8.wrapping_add(1))?
    print(0u8.wrapping_sub(1))?
    print(128u8.wrapping_mul(2))?
    Ok(())
1
7
6
250
10
2
0
255
0

Use wrapping_add, wrapping_sub, or wrapping_mul when a calculation deliberately wraps at the integer width, as in a hash or a random-number generator. Ordinary arithmetic returns a typed overflow error. Parenthesize mixed bitwise expressions when that makes the intended grouping easier to read.

rotate_left and rotate_right move bits around the ends of an integer instead of discarding them. The count has the integer’s own type. No count loses bits, so rotation never traps: a count of the full width leaves the value unchanged.

def main() -> Result[(), IoError]:
    bits = 129u8
    print(bits.rotate_left(1))?
    print(bits.rotate_right(1))?
    print(bits.rotate_left(8))?
    print(bits << 1)?
    Ok(())
3
192
129
2

pow raises a number to a power, and each integer type names its limits as MIN and MAX. As with the operators, the number and its exponent must have one type, and the result has that type:

def main() -> Result[(), IoError | OverflowError[i64] | OverflowError[u8]]:
    change = -7
    small = 200u8
    print(change.pow(2))?
    print((2).pow(10))?
    print(u8.MAX - small)?
    print(i64.MIN)?
    Ok(())
49
1024
55
-9223372036854775808

There is no ** operator. An integer literal receiver needs parentheses, as in (2).pow(10).

A result that does not fit returns the same typed overflow error as the operators. An integer power with a negative exponent still stops with negative integer exponent; convert to a float for that.

Recover from overflow

Declare the errors a calculation can produce and propagate them to its caller. An arithmetic failure returns Err immediately; otherwise explicitly return Ok with the successful value.

def average(total: u8, count: u8) -> Result[u8, OverflowError[u8] | ZeroDivisionError]:
    rounded = total + count // 2
    Ok(rounded // count)

def main() -> Result[(), OverflowError[u8] | ZeroDivisionError | IoError]:
    print(average(200, 4)?)?
    print(average(250, 20).is_err())?
    print(average(250, 0).is_err())?
    Ok(())
50
True
True

The error type lists what the function’s operators can produce: OverflowError[u8] for + on u8, and ZeroDivisionError for //. Unsigned // cannot overflow, so it adds only ZeroDivisionError. ? passes either error to the caller, as with try_from. An OverflowError holds the wrapped result in its wrapped field: 250 + 10 on a u8 wraps to 4. Keep several error types shows how to handle each error instead of passing it on.

For local recovery, use checked:. Its final expression becomes the successful value of the block’s Result; no explicit Ok is needed there:

def increment_or_zero(value: i64) -> i64:
    result = checked:
        value + 1
    match result:
        case Ok(incremented):
            incremented
        case Err(_):
            0

def main() -> Result[(), IoError]:
    print(increment_or_zero(4), increment_or_zero(i64.MAX))?
    Ok(())
5 0

Every named function and closure keeps its own declared error contract, including when defined inside checked:. Calls returning Result still need explicit handling.

Use unchecked: only where you can show the arithmetic cannot fail. It yields an ordinary value, not a Result. A failure inside it terminates the process by default, but --unchecked assume builds omit that check, so do not use the suite to reject bad input. In assume builds, list, slice-view and Bytes indexing inside the suite also requires nonnegative, in-range indices; normal negative indexing is unavailable there. Keep these suites small and justify every indexed access as well as the arithmetic. Write unchecked trap: when termination is the intent. See arithmetic modes. Integer std.math helpers return ordinary overflow results; their float variants remain infallible. Remaining std implementations and parallel reductions still need the coordinated adoption described in B14. Calls never inherit a caller’s mode.

Validate a divisor once

When a divisor comes from input, validate it with NonZero.try_from and pass the resulting value to repeated divisions. A plain integer accepts that divisor directly; the compiler omits its zero check.

def per_group(total: i64, groups: NonZero[i64]) -> Result[i64, OverflowError[i64]]:
    Ok(total // groups)

def main() -> Result[(), IoError | RangeError | OverflowError[i64]]:
    count = 3
    groups = NonZero.try_from(count)?
    print(per_group(14, groups)?, 14 % groups)?
    Ok(())
4 2

The quotient and remainder are ordinary integers and can be zero. Signed i64.MIN // -1 can still overflow, so the function’s error type includes OverflowError[i64]. A locally handled checked block can instead keep this error out of the function signature. See nonzero integers for construction and arithmetic rules.

Work with floating-point numbers

A decimal or exponent literal defaults to f64. Use an f32 suffix when you want 32-bit arithmetic. Both operands must have the same type; casts are explicit:

def main() -> Result[(), IoError]:
    distance: f64 = 7.5
    time: f64 = 2.0
    print(distance / time)?
    print(1.25f32 + 0.5f32)?
    print(2.5e-2)?
    print(f64(3) / 2.0)?
    print(i32(-2.75))?
    Ok(())
3.75
1.75
0.025
1.5
-2

Float-to-integer casts discard the fractional part toward zero. The result must fit the integer type: u8(255.9) is 255, while a 300.7, an infinity, or NaN held in a variable stops the program with error: integer cast out of range. Integer-to-float casts and f64 to f32 casts may round. There is no implicit widening or narrowing:

def main() -> ():
    small: f32 = 1.5f64
expected f32, got f64

Write 1.5f32 or f32(1.5) instead. Floats follow IEEE arithmetic: division by zero can produce infinity or NaN, and arithmetic overflow can produce infinity. NaN is unequal to everything, including itself; ordered comparisons with it are false.

def main() -> Result[(), IoError]:
    zero = 0.0
    print(1.0 / zero)?
    unknown = zero / zero
    print(unknown == unknown)?
    print(unknown != unknown)?
    Ok(())
inf
False
True

When a zero divisor is an error rather than a source of infinity, x.div_checked(y) returns an Err holding a ZeroDivisionError for a zero y and Ok(x / y) otherwise; see recover from overflow.

Floats support +, -, *, /, unary signs, and comparisons.

// and % currently require integers. Floats cannot be dictionary keys or set elements. Float literals that overflow their declared width are compile errors.

Math functions are methods on a float, and each float type provides the constants PI and E:

def main() -> Result[(), IoError]:
    radius = 2.0
    print(f64.PI * radius.pow(2.0))?
    print((9.0).sqrt())?
    print(f64.E.log())?
    print((0.0).cos())?
    print((2.5).floor())?
    print((2.5).ceil())?
    print((7.5).fmod(2.0))?
    Ok(())
12.566370614359172
3.0
1.0
1.0
2.0
3.0
1.5

log is the natural logarithm, sin and cos take radians, and exp raises e to a power. floor and ceil return floats; cast the result for an integer, as in i64(x.floor()). fmod is the float remainder. These methods never stop the program: (-1.0).sqrt() is NaN. An integer needs a cast first, as in f64(n).sqrt().

Write text

The type str represents text. Single and double quotes both make strings; triple quotes allow physical newlines. Escapes include \n, \r, \t, escaped quotes, \\, \0 for an embedded NUL, and \u{H...} for any character by its hexadecimal code point, such as "e\u{301}" for an e followed by a combining accent. See escapes.

def main() -> Result[(), IoError]:
    name = 'Plenty'
    print(name)?
    print(contains(name, "lent"))?
    print(len("hé"))?
    print("hé".byte_len())?
    Ok(())
Plenty
True
2
3

print displays text without quotes. contains(text, part) tests for a substring. len counts Unicode scalar values, so the accented character above counts once; this is not a count of visible grapheme clusters. byte_len() gives the size of the UTF-8 encoding instead, which is what a file, a buffer, or a size limit measures.

Strings are immutable. A mut binding allows replacing a string, rather than editing its bytes.

Compare strings in Unicode scalar order, inspect ASCII characters, or convert a single character to its code point:

def main() -> Result[(), AllocError | IoError | ParseError]:
    print("apple" < "pear")?
    print("abc".isalpha())?
    print("123".isdigit())?
    print(ord("é")?)?
    print(chr(233)?)?
    print("Hello".lower()?)?
    print("Straße".upper()?)?
    print("  one\t two\n".split()?)?
    Ok(())
True
True
True
233
é
hello
STRASSE
["one", "two"]

ord requires exactly one Unicode scalar; chr takes a u32 code point and rejects surrogates and values outside Unicode. Both return Result and allocate nothing. Case conversion and splitting return fallible, independently owned results. The ASCII predicates return false for empty or non-ASCII text.

Build text with f-strings

Put an f before a string to fill in values. Each {field} is replaced by its value. An f-string builds a new str, which can fail to allocate, so it returns Result[str, AllocError] like a list literal does: write ? after it.

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    name = "Ada"
    year = 1843
    scores = [90, 85]?
    print(f"{name} published in {year}."?)?
    print(f"first score: {scores[0]}, count: {len(scores)}"?)?
    print(f"all scores: {scores}, {{braces}}"?)?
    print(f"total: {scores[0] + scores[1]}, passed: {scores[1] > 80}"?)?
    print(f"{name} was {'young' if 1843 - 1815 < 30 else 'older'}"?)?
    Ok(())
Ada published in 1843.
first score: 90, count: 2
all scores: [90, 85], {braces}
total: 175, passed: True
Ada was young

A field shows its value as print would. Double the braces for literal ones. A field can be any expression. A ? inside a field returns from your function, as ? does anywhere else; the ? after the f-string handles only failure to build the text.

After a colon, a format spec chooses width, alignment, fill, sign, digit grouping, precision, and a type, as in Python:

def main() -> Result[(), AllocError | IoError]:
    item = "tea"
    price = 3.14159
    count = 1234567
    print(f"[{item:<6}] [{item:>6}] [{item:*^7}]"?)?
    print(f"{price:.2f} {price:8.3f} {price:+.1e} {0.25:.0%}"?)?
    print(f"{count:,} {count:_} {255:#x} {255:08b} {-42:05}"?)?
    Ok(())
[tea   ] [   tea] [**tea**]
3.14    3.142 +3.1e+00 25%
1,234,567 1_234_567 0xff 11111111 -0042

A spec can take its width or precision from a nested field, and its fill or alignment from a nested str field. Such a value can be invalid, such as a negative width, so the f-string’s error becomes AllocError | FormatError; FormatError needs no import:

def main() -> Result[(), AllocError | IoError | FormatError]:
    width = 8
    digits = 2
    align = "^"
    print(f"[{3.14159:{width}.{digits}f}] [{'tea':*{align}{width}}]"?)?
    Ok(())
[    3.14] [**tea***]

!r shows the value’s representation, with quotes around text, and = shows the field’s source before its value, which helps when debugging:

def main() -> Result[(), AllocError | IoError]:
    name = "Ada"
    year = 1843
    print(f"{name!r} {name=} {year=}"?)?
    Ok(())
"Ada" name="Ada" year=1843

!a is like !r but spells every non-ASCII character as an escape, so f"{'café'!a}" gives "caf\u{e9}", which you can paste back into source as the string literal "caf\u{e9}".

The compiler checks every spec against the field’s type. A precision does not apply to an integer:

def main() -> Result[(), AllocError | IoError]:
    count = 3
    print(f"{count:.2f}"?)?
    Ok(())
format spec `.2f` does not apply to i64: integers take no precision; convert with f64(...) to format a float

The reference lists every option and conversion.

An f-string suits a fixed number of pieces of any type. A field whose type has its own formatting methods also adds FormatError, since that method can fail. Combine two strings with a.concat(b)? and any number of strings from a list, set, or generator with separator.join(pieces)?.

Define functions with clear interfaces

Every parameter and return type must be declared:

def double(value: i64) -> Result[i64, OverflowError[i64]]:
    """Return twice the supplied value."""
    Ok(value * 2)

def main() -> Result[(), IoError | OverflowError[i64]]:
    print(double(21)?)?
    Ok(())
42

value: i64 declares the parameter type. The return type Result[i64, OverflowError[i64]] describes either a doubled integer or overflow. Ok(value * 2) supplies success; the multiplication returns an error from double if it cannot fit. You can also write return Ok(value * 2). The caller uses double(21)? to obtain the successful integer or pass the error onward. Parameters are immutable. See recoverable errors for handling either outcome explicitly.

Indent statements in the same block equally, using spaces. Tabs are rejected.

A string at the beginning of a function is its documentation. Triple quotes allow a docstring to span lines. When a function should immediately return a string, write return "text" so it cannot be mistaken for documentation.

Functions are known throughout the file, so one function may call a function declared later. Name arguments when that makes their meaning clearer:

def rectangle(width: i64, height: i64 = 2) -> Result[i64, OverflowError[i64]]:
    Ok(width * height)

def main() -> Result[(), IoError | OverflowError[i64]]:
    print(rectangle(height=3, width=4)?)?
    print(rectangle(4)?)?
    Ok(())
12
8

Positional arguments come first. After the first named argument, all remaining arguments must be named; their order need not match the declaration. Supplied expressions run in the order you write them. Required parameters come before parameters with defaults.

Defaults run afresh for each omitted argument. Mutable defaults never share storage between calls:

def count(values: &mut list[i64] = []?) -> Result[i64, AllocError]:
    values.append(7)?
    Ok(len(values))

def main() -> Result[(), AllocError | IoError]:
    print(count()?)?
    print(count()?)?
    mut saved = [1, 2]?
    print(count(values=&mut saved)?)?
    print(len(saved))?
    Ok(())
1
1
3
3

An omitted borrowed argument gets temporary storage that lasts for the call. Pass your own buffer to keep its changes. Defaults run after supplied arguments, in declaration order; they cannot refer to another parameter. Allocation still needs ? and a compatible Result return type. If a default fails, that call returns the error without entering the body. Callable values and builtins keep positional arguments; splat unpacking is not supported.

Complete work without returning data

A function can complete without producing data. Its return type is the unit type, (). This is different from a missing value.

def show_if(enabled: bool, value: i64) -> Result[(), IoError]:
    if not enabled:
        return Ok(())
    print(value)?
    Ok(())

def main() -> Result[(), IoError]:
    show_if(False, 10)?
    show_if(True, 42)?
    Ok(())
42

Result[(), IoError] combines unit success with a possible output error. print(value)? therefore completes without supplying data.

In a function declared -> (), a bare return returns unit. Use pass for an intentionally empty block. Unit parameters and standalone unit bindings are not supported. Use Option for absence; there is no None value.

Make decisions that produce values

A condition must have type bool. Comparisons such as ==, !=, <, <=, >, and >= produce booleans. There is no implicit conversion of zero, an empty string, or another value to False.

An if can supply a function’s result:

def maximum(first: i64, second: i64) -> i64:
    if first > second:
        first
    else:
        second

def main() -> Result[(), IoError]:
    print(maximum(7, 12))?
    Ok(())
12

Both continuing branches need the same type. Use elif for additional cases. For a short choice, use a conditional expression:

def main() -> Result[(), IoError]:
    age = 20
    category = "adult" if age >= 18 else "child"
    print(category)?
    Ok(())
adult

and and or short-circuit: the right side is evaluated only when needed. not negates a boolean. All three require boolean operands:

def main() -> Result[(), IoError | ZeroDivisionError | OverflowError[i64]]:
    divisor = 0
    safe = divisor != 0 and 10 // divisor > 1
    print(safe)?
    print(not safe)?
    Ok(())
False
True

The division is skipped here. Chained comparisons such as 0 < x < 10 are not supported yet; write 0 < x and x < 10.

Return early and continue on the other path

A guard clause handles a special case before the main computation:

def clamp_low(value: i64, minimum: i64) -> i64:
    if value < minimum:
        return minimum
    value

def main() -> Result[(), IoError]:
    print(clamp_low(3, 10))?
    print(clamp_low(12, 10))?
    Ok(())
10
12

return exits the entire function, even when it appears in nested branches. Every return must match the function’s declared result. Code after a guaranteed exit is rejected. Every path that reaches the end must also supply the declared result; handling just one branch is not enough:

def incomplete(flag: bool) -> i64:
    if flag:
        return 42

def main() -> ():
    pass
expected i64, got ()

Keep branch-local names local

Assignments to an existing mutable binding survive a branch. New names declared inside a branch belong to that branch:

def main() -> Result[(), IoError | OverflowError[i64]]:
    mut price = 100
    discounted = True
    if discounted:
        discount = 20
        price = price - discount
    print(price)?
    Ok(())
80

Declare a mutable binding before the branch when subsequent code needs the updated value. For a new selected value, bind an inline conditional when each branch is one expression. A block conditional also supports branch-local work:

def main() -> Result[(), IoError | OverflowError[i64]]:
    discounted = True
    price = if discounted:
        discount = 20
        100 - discount
    else:
        100
    print(price)?
    Ok(())
80

Only price survives the block; discount still belongs to its branch. This keeps price immutable when it is selected once. The block must be the whole right-hand side; see conditional values.

Repeat with a range

A for loop repeats its body for each integer in a range:

def main() -> Result[(), IoError | OverflowError[i64]]:
    mut total = 0
    for n in range(1, 5):
        total = total + n
    print(total)?
    for n in range(5, 0, -2):
        print(n)?
    Ok(())
10
5
3
1

range(stop) starts at zero. range(start, stop) selects the start. The stop is always excluded: range(1, 5) yields 1, 2, 3, and 4. A third argument selects the step. A negative step counts down. A zero step would never reach the stop, so the step has type NonZero[i64]: an integer that cannot be zero. A literal step converts, and range(5, 0, 0) is a compile-time error. Convert a step computed while the program runs with NonZero.try_from, which works like u8.try_from and returns Err(RangeError.OutOfRange) for zero:

def main() -> Result[(), RangeError | IoError]:
    stride = 3
    step = NonZero.try_from(stride)?
    for n in range(0, 10, step):
        print(n)?
    Ok(())
0
3
6
9

Passing stride directly is a compile-time error whose message shows this conversion. step.get() reads the integer back.

The loop evaluates the range once. Each iteration binds n to the next integer. Names declared in the body stay there; updates to an enclosing mut binding persist. A loop completes with unit.

Creating and iterating a range do not allocate a list. Ranges default to i64. Write range[u8](8) to choose another integer type. Start and stop must fit that type, even though stop is excluded. The step stays a signed NonZero[i64], so an unsigned range can count down.

A configured count that must be greater than zero can use Positive[i64]. Construct positive_count = Positive.try_from(count)? once; NonZero.from(positive_count) then uses that positive domain value as a range step without another check. NonNegative[i64] similarly states that a value may be zero but not negative.

Some standard-library parameters also have a documented ceiling. A direct call with a literal above that ceiling is a compile error; computed values still need to satisfy it when the operation runs. A positive domain checks the lower bound, not every operation’s ceiling. See invalid std arguments.

import std.sketch

def main() -> ():
    drop(std.sketch.quantiles(1048577))
above its fixed ceiling of 1048576

Repeat until a condition changes

Use while when a condition determines how long to repeat. The condition must be bool, and it is checked before every iteration, including the first:

def main() -> Result[(), IoError | OverflowError[i64]]:
    mut remaining = 3
    while remaining > 0:
        print(remaining)?
        remaining = remaining - 1
    print("go")?
    Ok(())
3
2
1
go

continue skips the rest of the current iteration. break exits the loop:

def main() -> Result[(), IoError | OverflowError[i64] | ZeroDivisionError]:
    mut n = 0
    while True:
        n = n + 1
        if n % 2 == 0:
            continue
        if n > 5:
            break
        print(n)?
    Ok(())
1
3
5

Update the condition’s inputs before a continue when needed; otherwise a while loop can repeat forever. In a for loop, continue automatically advances to the next element:

def main() -> Result[(), IoError]:
    for n in range(6):
        if n == 1:
            continue
        if n == 4:
            break
        print(n)?
    Ok(())
0
2
3

break and continue affect only the innermost loop. Use return to leave the function. break cannot carry a value, and loop else clauses are not supported.

Even after while True, a function returning data still needs a result after the loop. Statements directly after an unconditional exit are rejected:

def main() -> Result[(), IoError]:
    while True:
        break
        print("unreachable")?
    Ok(())
unreachable statement after a control-flow exit

Give types names of your own

There is no built-in int. If you want that name, choose its meaning explicitly:

type int = i32

def increment(value: int) -> Result[int, OverflowError[int]]:
    Ok(value + int(1))

def main() -> Result[(), IoError | OverflowError[i32]]:
    answer: int = increment(int(41))?
    print(answer)?
    Ok(())
42

type int = i32 makes int another name for i32, including in casts. Aliases do not change unconstrained literal defaults, but an annotation such as answer: int = 41 selects the literal’s width. Aliases are not literal suffixes: write 41i32, not 41int.

Aliases can describe a value’s purpose and refer to other aliases:

type Count = u32
type ItemCount = Count

def add_one(count: ItemCount) -> Result[ItemCount, OverflowError[ItemCount]]:
    Ok(count + 1u32)

def main() -> Result[(), IoError | OverflowError[u32]]:
    print(add_one(41u32)?)?
    Ok(())
42

Count, ItemCount, and u32 are interchangeable. An alias does not create a distinct type or enforce units of measurement.

Declare aliases at module scope; they are visible throughout the file. Alias chains must reach a concrete type: cycles and unknown targets are errors. An alias cannot redefine a built-in name, another alias, or a function name.

Absence and failure

A calculation may have no answer, or it may fail with an explanation. Make those possibilities part of its type, and let the caller choose what to do.

  1. Represent absence
  2. Represent a recoverable error
  3. Supply a default or a typed error
  4. Give constructors enough type information
  5. Propagate with ?
  6. Choose which error details to keep
  7. Keep several error types
  8. Build text with recoverable allocation

Represent absence

Use Option[T] when a function may have no value to return. It has two variants: Some(value) holds a value of type T; Nothing has no payload.

def positive(n: i64) -> Option[i64]:
    Some(n) if n > 0 else Nothing

def show(n: i64) -> Result[(), IoError]:
    match positive(n):
        case Some(value):
            print(value)?
        case Nothing:
            print("not found")?
    Ok(())

def main() -> Result[(), IoError]:
    show(-1)?
    show(42)?
    Ok(())
not found
42

In case Some(value), the pattern binds the payload to value within that branch. A match must cover both Some and Nothing.

Some and Nothing are built in and need no type prefix. Nothing belongs to one concrete option type rather than acting as a universal null. A present zero, False, or empty string is still Some(value).

When only presence matters, ask with is_some() or is_none() instead of writing a match. Both borrow the option, so it is still usable afterwards.

def main() -> Result[(), IoError]:
    found: Option[str] = Some("key")
    missing: Option[str] = Nothing
    print(found.is_some(), found.is_none())?
    print(missing.is_some(), missing.is_none())?
    print(found.unwrap_or("default"))?
    Ok(())
True False
False True
key

Represent a recoverable error

Use Result[T, E] when failure has an explanation. Ok(value) carries success data of type T; Err(error) carries a cause of type E.

def divide(left: i64, right: i64) -> Result[i64, str]:
    outcome = checked:
        left // right
    match outcome:
        case Ok(value):
            Ok(value)
        case Err(error):
            match error:
                case ZeroDivisionError():
                    Err("division by zero")
                case OverflowError(_):
                    Err("division overflow")

def show(left: i64, right: i64) -> Result[(), IoError]:
    match divide(left, right):
        case Ok(value):
            print(value)?
        case Err(message):
            print(message)?
    Ok(())

def main() -> Result[(), IoError]:
    show(8, 2)?
    show(8, 0)?
    Ok(())
4
division by zero

The local checked: block collects division failures. Matching the error distinguishes division by zero from signed overflow (i64.MIN // -1), then gives each a message. This small example deliberately uses strings as its error API; applications can preserve those distinct typed errors instead.

The caller can recover and continue, as show does here. A result does not implicitly unwrap itself or raise an exception.

When success has no data, use unit as the success type:

def validate(name: str) -> Result[(), str]:
    if len(name) == 0:
        return Err("name is empty")
    Ok(())

def show(name: str) -> Result[(), IoError]:
    match validate(name):
        case Ok(_):
            print("valid")?
        case Err(message):
            print(message)?
    Ok(())

def main() -> Result[(), IoError]:
    show("Plenty")?
    show("")?
    Ok(())
valid
name is empty

The pattern _ ignores a payload you do not need. Ok(()) represents success without data; Nothing represents absence.

is_ok() and is_err() answer whether a result succeeded without looking at its payload. They borrow the result, so it can still be matched or returned.

def divide(left: i64, right: i64) -> Result[i64, str]:
    outcome = checked:
        left // right
    match outcome:
        case Ok(value):
            Ok(value)
        case Err(error):
            match error:
                case ZeroDivisionError():
                    Err("division by zero")
                case OverflowError(_):
                    Err("division overflow")

def main() -> Result[(), IoError]:
    outcome = divide(8, 0)
    print(outcome.is_ok(), outcome.is_err())?
    print(divide(8, 2).is_ok())?
    match outcome:
        case Ok(value):
            print(value)?
        case Err(message):
            print(message)?
    Ok(())
False True
True
division by zero

Supply a default or a typed error

option.unwrap_or(default) returns a present payload or the supplied default. dictionary.get(key, default) does the same for a dictionary lookup, while dictionary.get(key) still returns an Option. Zero, False, and an empty string count as present values.

def main() -> Result[(), AllocError | IoError]:
    numbers = {"answer": 42}?
    print(numbers.get("answer", 0))?
    print(numbers.get("missing", 0))?
    missing: Option[i64] = Nothing
    print(missing.unwrap_or(9))?
    Ok(())
42
0
9

option.ok_or(error) converts absence into a typed Result. Choose an error that explains the missing value, then use ? inside a function returning a compatible Result.

enum Missing:
    Answer

def answer(numbers: &dict[str, i64]) -> Result[i64, Missing]:
    value = numbers.get("answer").ok_or(Missing.Answer)?
    Ok(value)

def main() -> Result[(), AllocError | IoError]:
    numbers = {"answer": 42}?
    match answer(&numbers):
        case Ok(value):
            print(value)?
        case Err(_):
            print("missing answer")?
    Ok(())
42

These helpers add no allocation. They evaluate their arguments eagerly, even when a value is present. The Option methods consume owned payloads and drop an unused default or error. Dictionary lookup supports copyable values; use pop to remove and take an owned value. Use match when the fallback should run only on absence, or choose a lazy helper as described below. Bare Nothing may need an Option annotation or a typed result context to determine its payload type.

Compute a default only when needed

Both Option and Result provide unwrap_or_else: it calls a fallback only for Nothing or Err. A Result fallback receives the error. Option also has ok_or_else for constructing an absence error lazily. These take function values; functions as values and closures explain callbacks in more detail.

A fallback can itself fail. In that case, map_or_else lets both branches return a Result, and ? handles that Result after selection:

def default_port() -> Result[i64, ParseError]:
    i64.parse("8080")

def port(configured: Option[i64]) -> Result[i64, ParseError]:
    Ok(configured.map_or_else(default_port, Ok)?)

def main() -> Result[(), ParseError | IoError]:
    print(port(Some(9000))?, port(Nothing)?)?
    Ok(())
9000 8080

default_port runs only for Nothing. Writing configured.unwrap_or(default_port()?) would run it even for Some(9000). A match is equally suitable when either branch needs several statements. Result.unwrap_or(default) also exists for deliberate recovery with a cheap, already available value. See defaults and recovery for ownership, evaluation order, and the Result callback’s error argument.

Give constructors enough type information

A binding annotation, parameter, or return signature supplies missing type information. Some(42) infers Option[i64] from its payload. Nothing needs an option type from context; Ok and Err each need the other variant’s type:

def main() -> Result[(), IoError]:
    found = Some(42)
    missing: Option[i64] = Nothing
    success: Result[i64, str] = Ok(42)
    failure: Result[i64, str] = Err("not ready")
    print(found == Some(42))?
    print(missing == Option[i64].Nothing)?
    match failure:
        case Ok(value):
            print(value)?
        case Err(message):
            print(message)?
    Ok(())
True
True
not ready

Qualify a constructor, such as Option[i64].Nothing, to supply its type explicitly. Otherwise, insufficient context is an error:

def main() -> ():
    answer = Ok(42)
cannot infer `Ok`

Pass a failure back with ?

Put ? after a Result expression to extract its Ok payload or return its Err immediately. The example’s main preserves allocation and printing errors in AllocError | IoError; checked_name keeps its concrete str error.

def validate(name: str) -> Result[(), str]:
    if len(name) == 0:
        return Err("name is empty")
    Ok(())

def checked_name(name: str) -> Result[str, str]:
    validate(name)?
    Ok(name)

def main() -> Result[(), AllocError | IoError]:
    print(str.repr(checked_name("Plenty"))?)?
    print(str.repr(checked_name(""))?)?
    Ok(())
Result[str, str].Ok("Plenty")
Result[str, str].Err("name is empty")

validate(name)? stops checked_name on failure, skipping Ok(name). Error propagation cleans up local values and temporaries, just as return does.

str.repr formats a result wrapper so either outcome can be printed. This example displays both results for inspection; it does not propagate the validation error out of main. In an application where validation must succeed, use ? on checked_name and include str in the enclosing function’s error type. Ordinary print requires a handled result: print(checked_name("Plenty")?)? prints only the successful name. An expression statement cannot silently discard a result:

def main() -> Result[(), AllocError | IoError]:
    print("hello")
    Ok(())
cannot implicitly discard a Result

Propagate with ?, match to recover, or use drop(result) to intentionally ignore either outcome. The discard check does not cover unused bindings or Option values.

? works with Option too: it extracts Some or immediately returns Nothing.

def positive(n: i64) -> Option[i64]:
    Some(n) if n > 0 else Nothing

def same_positive(n: i64) -> Option[i64]:
    Some(positive(n)?)

def main() -> Result[(), IoError]:
    print(same_positive(42))?
    print(same_positive(-1))?
    Ok(())
Option[i64].Some(42)
Option[i64].Nothing

This example passes the positive value through unchanged to isolate optional propagation. Arithmetic that might overflow needs a Result contract or local checked recovery; Option does not silently convert arithmetic failure to absence.

The enclosing function must return the same family: Result for a Result operand or Option for an Option operand. Success payload types can differ, but a Result error must either match the declared error type or fit within its anonymous sum. For example, Result[(), AllocError | IoError] lets the same function propagate list allocation failures and printing failures with ?. The sum retains the original error and adds no allocation. Use match when you want to recover or translate an error rather than pass it back. ? is not supported inside generators.

def read_number() -> Result[i64, str]:
    Err("not a number")

def checked() -> Result[i64, i64]:
    Ok(read_number()?)

def main() -> ():
    print(checked()).unwrap()
`?` operand error str cannot propagate as i64; the function's error type must include it

Option and Result wrappers do not allocate. Their payloads can: Some([1, 2]?) allocates the list, with no additional allocation for Some.

Choose which error details to keep

Keep a concrete error type when the caller needs the cause. For example, IoError preserves an input/output cause, while str can carry a message.

Pass an error back unchanged when the caller can use it directly. When several operations have different errors, the next lesson shows how to keep them together without losing their details.

Sometimes an API needs to add meaning to a lower-level cause. Here, ConfigError.InvalidCount identifies what was being parsed and retains the original ParseError. A Result’s map_error method changes just the error type, forwarding success unchanged:

enum ConfigError:
    InvalidCount(ParseError)

def read_count(text: &str) -> Result[i64, ConfigError]:
    i64.parse(text).map_error(ConfigError.InvalidCount)

def main() -> Result[(), IoError]:
    match read_count("many"):
        case Ok(count):
            print(count)?
        case Err(ConfigError.InvalidCount(cause)):
            print(cause == ParseError.Invalid)?
    Ok(())
True

ConfigError.InvalidCount written without arguments is a function from ParseError to ConfigError. map_error calls it once with the original error, only when parsing fails. On success, the parsed count passes through unchanged. The conversion policy is explicit, and mapping adds no allocation. read_count returns that result directly because it has no more work to do. Use ? when further work depends on the successful value. See the mapping reference for callback evaluation and ownership details.

When the new error needs more than the cause, such as a second payload or captured context, pass an anonymous function that builds it instead.

The example’s main deliberately handles a failed parse so you can see that the cause survived. In an application that cannot continue, propagate the error instead. Returning an error from main reports it on standard error and exits with status one; printing an error and returning Ok(()) reports success to the program’s caller. The command-line application puts this policy into practice.

Avoid replacing useful causes with builtin Failure: propagation into Result[T, Failure] discards the original error and keeps only Failure.Unspecified. It is a last resort, not a way to combine errors. See the reference for its exact erasure rules.

.unwrap() explicitly extracts a successful payload or terminates on Err or Nothing, without normal scope cleanup:

def main() -> Result[(), IoError]:
    found = Some(42)
    print(found.unwrap())?
    Ok(())
42

Choose .unwrap() only when termination is your intended policy.

Keep several error types

A sum keeps one complete value from a finite set of types. Use | in the error type when a function combines operations with different errors:

type SmallNumberError = ParseError | RangeError

def parse_small(text: str) -> Result[u8, SmallNumberError]:
    number = i64.parse(text)?
    u8.try_from(number)

def main() -> Result[(), IoError]:
    match parse_small("300"):
        case Ok(number):
            print(number)?
        case Err(ParseError.Invalid):
            print("not an integer")?
        case Err(ParseError.OutOfRange):
            print("outside i64 range")?
        case Err(RangeError.OutOfRange):
            print("outside u8 range")?
    Ok(())
outside u8 range

? forwards the original parse error. The direct return widens the conversion result to the declared set. Neither step creates a heap wrapper or loses the payload. Each variant pattern selects its enum member and then its variant, so the errors can be handled directly inside Err. Every variant of every member needs coverage unless you use _.

Use case name: Type: when you need one whole member and its value. The matching reference describes how whole-member patterns and variant patterns combine.

Use a module-level type alias to name a recurring group: SmallNumberError means exactly ParseError | RangeError. It introduces no wrapper or allocation, so ? and exhaustive matching still work on the original members. An alias also composes with other errors: SmallNumberError | IoError is the same sum as ParseError | RangeError | IoError.

Name the errors the function actually propagates. An operation may already wrap a lower-level error in its own enum; that does not require adding the lower-level type separately. A new wrapper enum is useful when an API wants to hide or reorganize its causes, but naming a group needs only an alias.

A checked: block returns one of these sums. Its errors are classes, values with named fields, and a class pattern matches one and binds its fields in order:

def report(total: u8, count: u8) -> Result[(), IoError]:
    result = checked:
        rounded = total + count // 2
        rounded // count
    match result:
        case Ok(mean):
            print(mean)?
        case Err(OverflowError(wrapped)):
            print("overflow, wrapped to", wrapped)?
        case Err(ZeroDivisionError()):
            print("no values")?
    Ok(())

def main() -> Result[(), IoError]:
    report(200, 4)?
    report(250, 20)?
    report(250, 0)?
    Ok(())
50
overflow, wrapped to 4
no values

OverflowError(wrapped) selects the overflow member and binds its one field. ZeroDivisionError() has no fields, so its parentheses are empty. Write _ for a field you do not need. Classes explains fields, and class patterns gives the complete rules.

Sums also work outside errors:

def describe(value: i64 | str) -> Result[(), IoError]:
    match value:
        case number: i64:
            print(number)?
        case text: str:
            print(text)?
    Ok(())

def main() -> Result[(), AllocError | IoError]:
    describe(42)?
    describe("ready")?
    Ok(())
42
ready

The parameter supplies the destination type, so either member can be passed directly. str | i64 means the same type as i64 | str; duplicate members collapse. To inspect without moving an owned payload, match &value; use &mut value to change the active payload through a mutable borrow.

Heterogeneous collections uses sums and enum wrappers to store different kinds of values in lists and dictionaries.

Keep stream errors and progress

Use std.io.typed when a transfer needs the original cause and the number of bytes delivered before failure. Its copy error distinguishes reading from writing. Existing std.io streams can enter this API through bridges:

import std.http.typed

import std.bytes
import std.io
import std.io.typed
from std.http.typed import fixed_body
from std.io.typed import CopyFailure, copy_all

def main() -> Result[(), AllocError | IoError | std.http.typed.FramingError[std.io.StreamError] | std.bytes.BytesError]:
    mut bytes = std.bytes.with_capacity(3)?
    bytes.resize(3, 65u8)?
    mut scratch = std.bytes.with_capacity(2)?
    scratch.resize(2, 0u8)?
    mut body = fixed_body(std.io.typed.reader(std.io.cursor(bytes)), 3)?
    mut output = std.io.typed.writer(std.io.cursor(std.bytes.with_capacity(3)?))
    match copy_all(&mut body, &mut output, &mut scratch):
        case Ok(count):
            print(count)?
        case Err(CopyFailure.Read(count, cause)):
            print("read failed after")?
            print(count)?
            drop(cause)
        case Err(CopyFailure.Write(count, cause)):
            print("write failed after")?
            print(count)?
            drop(cause)
    cursor = std.io.typed.unwrap_writer(output)
    print(cursor.position())?
    Ok(())
3
3

The fixed body assumes an already validated Content-Length; it does not parse HTTP. close_body instead finishes at EOF and requires a connection that will never be reused. The copy helper retries explicit interruption and preserves other transport causes. A write failure can leave unwritten data from the last chunk in scratch; the delivered count alone is not a resume token.

For an HTTP message, use typed head parsing followed by request_body or response_body. They check framing fields and select the body decoder with explicit limits. This in-memory response uses chunked framing. Keep the entire buffered reader when handing it to the body, so prefetched bytes are preserved:

import std.bytes
import std.http
import std.io
import std.io.typed
from std.http.typed import FramingError, read_response, response_body
from std.io.typed import CopyFailure, Partial, ReadFailure, copy_all

type TransportCause = ReadFailure[std.io.StreamError]
type HttpCause = FramingError[TransportCause]

def main() -> Result[(), AllocError | IoError | std.bytes.BytesError | std.io.StreamError | std.http.HttpError | Partial[HttpCause] | HttpCause | CopyFailure[HttpCause, std.io.StreamError]]:
    wire = std.bytes.from_text("HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\n\r\n3\r\nABC\r\n0\r\nX-Tag: sample\r\n\r\n")?
    limits = std.http.Limits(64, 128, 4, 1024)
    mut conn = std.io.typed.buffered_reader(std.io.typed.reader(std.io.cursor(wire)), 128)?
    head = read_response(&mut conn, &limits)?.ok_or(std.http.HttpError.Truncated)?
    # Our GET request allows keep-alive; the response must also allow it.
    mut body = response_body(conn, &head, "GET", True, &limits)?
    mut output = std.io.typed.writer(std.io.cursor(std.bytes.with_capacity(3)?))
    mut scratch = std.bytes.with_capacity(2)?
    scratch.resize(2, 0u8)?
    print(copy_all(&mut body, &mut output, &mut scratch)?)?
    print(body.is_done(), body.trailers()[0].value)?
    # finish checks completion and persistence before handing back ownership.
    recovered = body.finish()?
    drop(recovered)
    Ok(())
3
True sample

Only decoded payload contributes to the copy count. The decoder keeps partial framing state across interruption and stops after the trailers; it does not parse an HTTP head. The parsed-head constructor validates framing; finish checks whether this completed message permits recovering its connection. Dropping an unfinished or failed body drops its connection. Close-delimited, closing, upgraded and successful CONNECT connections cannot be recovered by this API. The reference describes the remaining caller responsibilities and the lower-level chunked_body adapter.

For real requests, use std.http.client, which uses these typed paths and also handles deadlines, connection ownership and reuse. typed_response(response) adapts its response to the typed reader protocol while preserving ClientError; unwrap it before client.finish. response.read_all_progress is convenient when collecting a body and retaining both its original error and the number of bytes appended before failure.

See the typed streams reference for nested error payloads and the framing reference for terminal-state rules.

Build text with recoverable allocation

Concatenating strings can need new storage, so + returns Result[str, AllocError]. Handle that result before using the new text:

def main() -> Result[(), AllocError | IoError]:
    message = ('Hello, ' + "Plenty!")?
    print(message)?
    print(contains(message, "Plenty"))?
    Ok(())
Hello, Plenty!
True

Building the string can fail with AllocError; printing can fail with IoError. The AllocError | IoError sum lets this function propagate either cause while keeping its original type and details.

Use str.from(value) to convert a scalar to text, or str.repr(value) for a representation such as quoted text or a displayed result wrapper. Both return allocation results; see the text conversion lessons.

A helper can combine construction and printing behind one result:

def greet(name: str) -> Result[(), AllocError | IoError]:
    print(("Hello, " + name)?)?
    Ok(())

def greet_if(enabled: bool, name: str) -> Result[(), AllocError | IoError]:
    if not enabled:
        return Ok(())
    greet(name)

def main() -> Result[(), AllocError | IoError]:
    greet_if(False, "Ada")?
    greet_if(True, "Ada")?
    Ok(())
Hello, Ada

String-building operations return allocation results even when a particular short string fits inline. A slice such as name[1:]? is another: it copies part of the text.

Collections, ownership, and borrowing

Store values in collections and choose when to transfer, copy, or borrow them.

  1. Build and update a list
  2. Transfer or copy ownership
  3. Take part of a list or text
  4. Borrow a collection
  5. Borrow an element
  6. Handle allocation failures
  7. Associate keys with values Optional lookup handles missing entries explicitly.
  8. Keep distinct values in a set
  9. Remove values and take ownership
  10. Group different types in tuples
  11. Sort and reorder a list
  12. Clean up owned values
  13. Build singly and doubly linked lists

The collection examples provide additional operations.

Build and update a list

A list contains values of one type. Use list[T] in signatures and annotations. An empty list needs an annotation or a typed constructor such as list[i64]().

def main() -> Result[(), AllocError | IoError]:
    mut original: list[i64] = [10, 20]?
    mut changed = copy(original)?
    changed.append(30)?
    changed[0] = 99
    print(original)?
    print(changed)?
    print(changed[-1])?
    print(len(changed))?
    Ok(())
[10, 20]
[99, 20, 30]
30
3

copy(original)? makes an independent list. Updates happen in place and require a mut owner. append grows the list; indexed assignment replaces an existing element. Negative indices count from the end, and an invalid index is a runtime error.

List construction, copying, and growth return allocation results, handled here with ?.

Collections can nest. Update an inner list directly through its indices:

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    mut rows = [[1, 2]?, [3]?]?
    rows[0][-1] += 7
    print(rows)?
    Ok(())
[[1, 9], [3]]

Nested assignment does not copy or allocate the containing lists. Numeric elements support augmented assignment such as += too. Each target index runs once before the right side, which may read the list but cannot change or resize it during the update. A failed right-side result leaves the element unchanged. Indices can be expressions, including elements of another list:

def main() -> Result[(), AllocError | IoError]:
    mut values = [30, 10, 20]?
    order = [1, 2, 0]?
    print(values[order[0]])?
    values[order[2]] = 31
    print(values)?
    Ok(())
10
[31, 10, 20]

Transfer or copy ownership

A list has one owner responsible for its mutable contents. Assigning it to another binding transfers that responsibility:

def main() -> Result[(), AllocError | IoError]:
    original = [10, 20]?
    mut changed = original
    changed.append(30)?
    print(changed)?
    Ok(())
[10, 20, 30]

The assignment moves the list. The old binding cannot be used afterward:

def main() -> Result[(), AllocError | IoError]:
    original = [10, 20]?
    changed = original
    print(original)?
    Ok(())
use of moved binding `original`

Passing a collection to an owned parameter or returning it also transfers ownership. A move transfers the existing storage rather than duplicating it.

Use copy(value)? when you need independent contents. Copying observes the source and returns Result[T, AllocError], so the source remains usable whether copying succeeds or fails. Nested mutable contents are copied too; immutable strings may share their existing storage.

Numbers and booleans copy on assignment. Strings remain usable after assignment because their storage is immutable. Tuples containing only these values copy; tuples containing collections move.

A value that copies needs no copy(...)?: write name = original for a string. plenty --check warns about copy(x)? when the type of x already copies. Use copy for values that move, such as lists, when you need two independent owners.

A mutable owner can receive a new value after moving its previous one. The compiler checks all continuing paths: a move on one branch can cause a use of possibly moved binding diagnostic later.

Take part of a list or text

A slice copies part of a list or string into a new value. Write the start and stop positions inside the brackets, separated by :. The start is included and the stop is excluded:

def main() -> Result[(), AllocError | IoError]:
    scores = [70, 85, 90, 65, 88]?
    print(scores[1:3]?)?
    print(scores[:2]?)?
    print(scores[3:]?)?
    print(scores[-2:]?)?
    print(scores[::-1]?)?
    print(scores)?
    Ok(())
[85, 90]
[70, 85]
[65, 88]
[65, 88]
[88, 65, 90, 85, 70]
[70, 85, 90, 65, 88]

An omitted start or stop runs to that end of the list. A negative position counts from the end, so scores[-2:] takes the last two elements. A third number is the step: [::2] takes every second element and [::-1] walks backwards. The source list is unchanged.

A slice allocates its new list, so it returns a Result like a list literal and needs ?:

def main() -> Result[(), AllocError | IoError]:
    scores = [70, 85, 90]?
    top: list[i64] = scores[:2]
    Ok(())
slice returns Result[list[i64], AllocError]; use `?` to propagate allocation failure

Positions outside the list do not stop the program. They clamp to its ends, as in Python, so scores[3:100] and scores[100:] are fine; the second is empty.

Slices are copies

A slice is an explicit copy. Each selected element is copied, including nested lists, so changing the source afterwards does not change the slice:

def main() -> Result[(), AllocError | IoError]:
    mut rows = [[1, 2]?, [3]?]?
    first = rows[:1]?
    rows[0].append(99)?
    print(first)?
    print(rows)?
    Ok(())
[[1, 2]]
[[1, 2, 99], [3]]

Copying costs time and memory in proportion to the selected elements. To read elements without copying them, borrow them as a slice view, &scores[1:], which the element borrowing lesson introduces. A loop over their positions also avoids the copy:

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    scores = [70, 85, 90, 65, 88]?
    mut total = 0
    for i in range(1, len(scores)):
        total += scores[i]
    print(total)?
    Ok(())
328

Text

String slices count Unicode characters, the same positions as len and indexing, so a slice never splits a character:

def main() -> Result[(), AllocError | IoError]:
    word = "café!"
    print(word[:4]?)?
    print(word[-1:]?)?
    print(word[::-1]?)?
    Ok(())
café
!
!éfac

To read part of a string without copying it, borrow it as &word[:4], which the element borrowing lesson introduces.

The step must not be zero. A literal 0 is a compile error, and a computed step is converted with NonZero.try_from(step)?, which reports zero as an error. The slice reference covers steps, tuples, bytes, columnar lists, and views.

Borrow instead of transferring ownership

A shared reference, &T, permits observation. An exclusive reference, &mut T, permits mutation. The owner remains responsible for cleanup. Function signatures state which access is needed:

def first(values: &list[i64]) -> i64:
    values[0]

def add(values: &mut list[i64], value: i64) -> Result[(), AllocError]:
    values.append(value)?
    Ok(())

def main() -> Result[(), AllocError | IoError]:
    mut numbers = [1, 2]?
    print(first(numbers))?
    add(&mut numbers, 3)?
    print(numbers)?
    Ok(())
1
[1, 2, 3]

Collection operations such as append, indexing, and len work directly through references.

The parameter determines borrowing: first(numbers) implicitly lends numbers because first expects &list[i64]. Writing first(&numbers) also works. Use explicit &mut for mutable arguments, as in add(&mut numbers, 3). An owned parameter still takes ownership.

References can also be local bindings. &mut permits changing the target, not reassigning the reference binding. Use *reference to read or replace a scalar target:

def main() -> Result[(), IoError | OverflowError[i64]]:
    mut score = 10
    reference = &mut score
    *reference = *reference + 5
    print(*reference)?
    score = 20
    print(score)?
    Ok(())
15
20

The exclusive borrow ends after the last use of reference, so assigning to score afterward is allowed. A reference that will be used later keeps its loan live, including across branches and loop iterations:

def main() -> Result[(), AllocError | IoError]:
    mut numbers = [1, 2]?
    view = &numbers
    numbers.append(3)?
    print(view)?
    Ok(())
conflicting borrow: cannot modify or exclusively borrow `numbers` while it is borrowed

The diagnostic notes identify the borrow’s creation and the later use that keeps it live.

An exclusive reference can temporarily lend shared or exclusive access. Conflicting access through it is suspended until the child borrow’s last use. Passing an existing reference to a reference parameter reborrows it; the owner is not transferred.

When a conflict goes through a reference, the diagnostic names both the owner and the reference written at the failing use:

def main() -> Result[(), AllocError | IoError]:
    mut numbers = [1, 2]?
    reference = &mut numbers
    view = &reference
    reference.append(3)?
    print(view)?
    Ok(())
conflicting borrow: cannot modify or exclusively borrow `numbers` (through `reference`) while it is borrowed

Borrowing collection elements

Borrow an element when you want to inspect or change it without moving its owner:

def main() -> Result[(), AllocError | IoError]:
    mut counts = [1, 5]?
    value = &mut counts[0]
    *value = 2
    print(counts)?
    Ok(())
[2, 5]

The loan ends after its last use. While an element reference is live, the collection cannot grow, remove entries, or move. An out-of-range index traps. Dictionaries lend a value through get_mut, which returns an Option instead of trapping on a missing key.

Two list elements written as different number literals are different places, so both can be borrowed at once:

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    mut scores = [10, 20, 30]?
    first = &mut scores[0]
    last = &mut scores[2]
    *first = *first + *last
    *last = 0
    print(scores)?
    Ok(())
[40, 20, 0]

The compiler proves only this literal case. Any other index, such as i or -1, may name the same element, so its loan covers the whole list:

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    mut scores = [10, 20, 30]?
    i = 2
    first = &mut scores[0]
    last = &mut scores[i]
    *first = *first + *last
    Ok(())
while `scores[0]` is exclusively borrowed

Dictionary entries are never separated this way. To exchange two list elements at computed indices, swap them instead.

Borrowed loops always give references, including when the elements are numbers. Printing and comparison inspect those references directly. Use * when a function takes a value or a new collection needs the element itself:

def count_eq[T](values: &list[T], target: &T) -> Result[i64, OverflowError[i64]]:
    mut count = 0
    for value in values:
        if value == target:
            count = count + 1
    Ok(count)

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    values = [1, 2, 1]?
    target = 1
    print(count_eq(&values, &target)?)?
    for value in &values:
        print(*value)?
    Ok(())
2
1
2
1

The same body works for a list[str]. Shared dictionary items() loops likewise give &K keys and &V values; mutable views give &mut V values. A borrowed collection stays protected against changes throughout its loop.

Borrow part of a list

&values[a:b] borrows a run of elements in place. This slice view copies nothing, so unlike a copying slice it needs no ?. A function takes a view as &[T], or as &mut [T] to change the elements:

def total(values: &[i64]) -> Result[i64, OverflowError[i64]]:
    mut sum = 0
    for value in values:
        sum += *value
    Ok(sum)

def reset(values: &mut [i64]) -> ():
    for value in values:
        *value = 0

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    mut scores = [70, 85, 90, 65, 88]?
    print(total(&scores)?, total(&scores[1:])?)?
    reset(&mut scores[:2])
    print(scores)?
    Ok(())
398 328
[0, 0, 90, 65, 88]

The positions clamp as a copying slice’s do. A view supports len, indexing, narrower views, loops, and == with another view, and prints as a list of its elements. An exclusive view also takes indexed assignment and swap. A &[T] parameter also takes a whole list as &scores, which is a view that copies nothing. A view borrows its run of the list, and the list cannot grow while the view is live. Two exclusive views of one list can be live together when one stops where the other starts:

def main() -> Result[(), AllocError | IoError | ZeroDivisionError | OverflowError[i64]]:
    mut scores = [70, 85, 90, 65, 88]?
    half = len(scores) // 2
    low = &mut scores[:half]
    high = &mut scores[half:]
    low[0] = high[0]
    print(scores)?
    Ok(())
[90, 85, 90, 65, 88]

Views whose ranges may overlap, such as &mut scores[:3] and &scores[2:], conflict. Byte buffers have views too. A view cannot be returned or stored yet; the slice reference lists its operations and limits.

Borrow part of a string

&text[a:b] borrows part of a string the same way. Its type is &substr, and its positions count characters, as a copying string slice’s do. Loop over it, compare it with a string, or search it without copying:

def count(text: &substr, letter: &str) -> Result[i64, OverflowError[i64]]:
    mut n = 0
    for c in text:
        if c == letter:
            n += 1
    Ok(n)

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    line = "name: Ada Lovelace"
    value = &line[6:]
    print(value, len(value))?
    print(&line[:4] == "name", "Love" in value, count(&line, "a")?)?
    first = value[:3]?
    print(first)?
    Ok(())
Ada Lovelace 12
True True 3
Ada

A function takes a substring as &substr, and a whole string as &line or a literal. Copy the text with value[a:b]? when you need an owned str, for example to call a str method or keep it after the string changes. The substring view reference lists every operation.

Handle allocation failures

Collection construction returns Result[collection, AllocError]. Use ? to propagate a failure, or match to recover:

def answers() -> Result[list[i64], AllocError]:
    mut values = list[i64].with_capacity(2)?
    values.append(21)?
    values.append(42)?
    Ok(values)

def main() -> Result[(), IoError]:
    match answers():
        case Ok(values):
            print(values)?
        case Err(AllocError.OutOfMemory):
            print("not enough memory")?
        case Err(AllocError.CapacityOverflow):
            print("requested capacity is too large")?
    Ok(())
[21, 42]

list[i64].with_capacity(2)? constructs an empty list with room for at least two elements. Appending those elements needs no further list storage allocation. Use list[i64]() when you do not know the capacity ahead of time. dict[str, i64]() and set[str]() construct the other collection kinds. Displays such as [21, 42] return the same kind of allocation result; extract their collection with ? too. List displays reserve their full element count automatically. Both mutable and immutable list bindings benefit. Element expressions may still allocate.

For an existing mutable collection, values.reserve(n)? reserves room for n additional elements beyond its current length. Existing contents remain unchanged if reservation fails.

Growth also returns a result: append for lists, insert for dictionaries, and add for sets. A failed insertion keeps existing contents but consumes its arguments, cleaning up any owned argument rather than returning it.

AllocError has two payload-free variants. OutOfMemory means the allocator rejected a request. CapacityOverflow means its size cannot be represented. Neither variant needs allocation.

Use capacity and reservation to obtain storage before inserting values you want to keep if reservation fails.

Associate keys with values

A dict[K, V] maps keys of one type to values of another. Keys may be integers, booleans, strings, bytes, or tuples of supported keys; values can include collections. Named enums can also be keys when their payloads are supported key types; see heterogeneous dictionaries. The keys() and values() loops below borrow their elements: *name reads the value behind a reference. The borrowing lesson explains references in detail.

def main() -> Result[(), AllocError | IoError]:
    mut scores: dict[str, i64] = {"Ada": 10, "Grace": 20}?
    scores.insert("Ada", 12)?
    scores.insert("Lin", 30)?
    print(scores["Ada"])?
    print("Grace" in scores)?
    for name in scores.keys():
        print(*name)?
    for score in scores.values():
        print(*score)?

    Ok(())
12
True
Ada
Grace
Lin
12
20
30

insert adds a key or replaces the value of an existing one, which keeps its insertion position. Adding a key can allocate, so insert returns an allocation result. Dictionaries have no indexed assignment: scores["Ada"] = 12 is a compile error. Reading scores["Ada"] traps when the key is missing.

To change a value in place, get_mut returns Option[&mut V]: a mutable reference when the key is present, or Nothing:

def main() -> Result[(), AllocError | IoError]:
    mut groups = {"first": [10, 20]?}?
    match groups.get_mut("first"):
        case Some(group):
            group[0] = 11
            group.append(30)?
        case Nothing:
            pass
    print(groups)?
    Ok(())
{"first": [11, 20, 30]}

get_mut allocates nothing. The reference borrows the whole dictionary exclusively, so the dictionary cannot be read or changed while it is live. To change every value, loop over (&mut groups).values() instead of calling get_mut per key; dictionary key/value loops show the mutable form.

To count or group, a missing key needs a starting value. get_or_insert(key, default) inserts the default when the key is absent, then returns a mutable reference to the stored value. It finds the key with one lookup. Inserting can allocate, so it returns a Result; ? propagates the AllocError:

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    mut counts: dict[str, i64] = {}?
    for word in ["red", "blue", "red"]?:
        *counts.get_or_insert(word, 0)? += 1
    print(counts)?
    Ok(())
{"red": 2, "blue": 1}

The default is built before the lookup, even when the key is present. A default like 0 costs nothing; one that allocates, such as []?, allocates on every call, so use get_mut when the key is usually present.

keys() and values() borrow elements in insertion order without allocating. The references protect the dictionary from mutation while they are used. Keep a key view in a local when checking several properties:

def main() -> Result[(), AllocError | IoError]:
    scores = {"Ada": 12, "Grace": 20}?
    names = scores.keys()
    expected = {"Ada", "Grace", "Lin"}?
    print(len(names), "Ada" in names, names.issubset(&expected))?
    print(names.union(&expected)? == expected)?
    Ok(())
2 True True
True

Read-only checks return plain values. Set algebra and names.to_set() allocate an owned set and return an allocation result. The view stays local: use scores.keys_snapshot()? or scores.values_snapshot()? when an owned ordered list must outlive the borrow. For mutable payloads, copy(groups)?.values_snapshot()? requests independent copies explicitly. into_keys() and into_values() consume a dictionary into an ordered list. See the full view contract for restrictions and cost details. {} is an empty dictionary; select its types with an annotation or a constructor such as dict[str, i64]()?.

Reuse a borrowed view

Bind a shared view when several observations use the same dictionary. The view keeps the dictionary borrowed until its last use; it creates no snapshot.

def main() -> Result[(), AllocError | IoError]:
    scores = {"Ada": 10, "Grace": 20}?
    values = scores.values()
    entries = scores.items()
    print(len(values), len(entries))?
    for score in values:
        print(*score)?
    for name, score in entries:
        print(*name, *score)?
    print(iter(values).count())?
    Ok(())
2 2
10
20
Ada 10
Grace 20
2

Use iter(view) for transformations and terminals. Item-view callbacks take a key and a value parameter, just like direct items() pipelines.

Look up a value that may be absent

Missing keys in scores[key] are runtime errors. Use scores.get(key) when absence is expected. It returns an Option: Some(value) if found, or Nothing if absent. Match the two possibilities explicitly:

def main() -> Result[(), AllocError | IoError]:
    scores = {"Ada": 12, "Grace": 0}?
    match scores.get("Ada"):
        case Some(score):
            print(score)?
        case Nothing:
            print("unknown player")?
    print(scores.get("Grace"))?
    print(scores.get("Lin"))?
    print(scores["Ada"])?
    Ok(())
12
Option[i64].Some(0)
Option[i64].Nothing
12

A stored zero, False, empty string, or Nothing counts as present; looking up a stored Nothing returns Some(Nothing). Use the option match to choose a fallback; get has no default argument.

get leaves the dictionary and key available and does not allocate. It supports numbers, booleans, strings, and enums of such values. It cannot return mutable collections or classes: use pop to take ownership or &dictionary[key] to borrow an existing value. Dictionaries have no optional borrowed lookup yet; your own functions can return a borrow that may be absent.

Read a list element that might be missing

items.get(index) returns Some(value) for an existing element or Nothing for an out-of-range index. It takes one i64 index, including negative indices:

def main() -> Result[(), AllocError | IoError]:
    names = ["Ada", "Bea"]?
    print(names.get(-1))?
    match names.get(2):
        case Some(name):
            print(name)?
        case Nothing:
            print("no name at that position")?
    print(names)?
    Ok(())
Option[str].Some("Bea")
no name at that position
["Ada", "Bea"]

List get has the same element restrictions as dictionary get. For owned elements, use pop to transfer ownership, copy(items[index])? to duplicate one, or &items[index] to borrow it. Ordinary indexing still traps when the index is out of range. The report gives the source position, the index, and the length: error: main.plenty:3:11: index out of bounds: index 5, length 3.

Both forms of get leave their source unchanged. Numbers, booleans, and enums copy; strings share immutable storage. The returned value remains valid if the source is updated or dropped.

Keep distinct values in a set

A set[T] stores each distinct value once. Its elements can be integers, booleans, or strings:

def main() -> Result[(), AllocError | IoError]:
    mut names: set[str] = set()?
    names.add("Ada")?
    names.add("Ada")?
    print(len(names))?
    print("Ada" in names)?
    names.add("Grace")?
    print(names.discard("Ada"))?
    print(names.discard("Ada"))?
    print(len(names))?
    Ok(())
1
True
True
False
1

add needs mutable access and returns an allocation result. discard removes a member without allocating and returns whether it was present.

Use set[str]()? for a typed empty set, or provide an annotation as above. A display such as {"Ada", "Grace"}? builds a set directly. {} is an empty dictionary. Sets have no promised iteration order.

Dictionary and set equality compare contents independently of order.

Remove an entry and take its value

pop(key) removes a dictionary entry and returns Some(value), or Nothing when the key is absent. It requires a mutable dictionary and transfers ownership of the stored value, including nested collections:

def main() -> Result[(), AllocError | IoError]:
    mut groups = {"ready": [1, 2]?, "waiting": [3]?}?
    match groups.pop("ready"):
        case Some(items):
            mut work = items
            work.append(4)?
            print(work)?
        case Nothing:
            print("no work")?
    print(groups)?
    print(groups.pop("missing"))?
    Ok(())
[1, 2, 4]
{"waiting": [3]}
Option[list[i64]].Nothing

The returned owner remains valid if the dictionary is dropped. Discarding a successful pop result also cleans up its removed value.

Removal preserves the order of remaining entries without allocating. Reinserting a removed key puts it at the end. pop takes one key and has no default argument.

Remove list elements

Lists also have pop. With no argument it removes the last element; an i64 index selects another position. Negative indices count from the end:

def main() -> Result[(), AllocError | IoError]:
    mut tasks = [[1]?, [2]?, [3]?]?
    match tasks.pop(1):
        case Some(task):
            mut work = task
            work.append(4)?
            print(work)?
        case Nothing:
            print("no task")?
    print(tasks)?
    print(tasks.pop())?
    print(tasks.pop(-1))?
    print(tasks.pop())?
    Ok(())
[2, 4]
[[1], [3]]
Option[list[i64]].Some([3])
Option[list[i64]].Some([1])
Option[list[i64]].Nothing

List pop needs mutable access. Empty lists and out-of-range indices return Nothing. Removal retains capacity, preserves element order, and does not allocate. Removing the last element is constant time; removing an earlier one shifts subsequent elements.

Returning and unpacking tuples

A tuple groups values with different types. Write (value,) for one component; parentheses without a comma simply group an expression. Function signatures may spell a tuple as (i64, str) or tuple[i64, str]. Unpacking transfers owned components, and _ discards a component. Indices must be integer literals; as with lists, a negative index counts from the end, so pair[-1] is the last component.

def measurement() -> (i64, str):
    (42, "cm")

def main() -> Result[(), IoError]:
    value, unit = measurement()
    print(value)?
    print(unit)?
    print((3, "three"))?
    Ok(())
42
cm
(3, "three")

Creating (a, b) does not allocate a tuple, so it needs no ?. Operations that produce its components, such as the list construction below, can still fail.

Nested tuples can be unpacked in one statement. Match each tuple’s shape, using _ for a component you do not need:

def main() -> Result[(), IoError | OverflowError[i64]]:
    ((x, y), label) = ((3, 4), "point")
    print(label, x + y)?
    Ok(())
point 7

Tuples of numbers, strings, or nested tuples of those can be compared. The first differing component decides the result, just as in sort(). Comparisons leave both values available:

def main() -> Result[(), IoError]:
    first = (1, "apple")
    second = (1, "pear")
    print(first < second, first <= first)?
    print(first, second)?
    Ok(())
True True
(1, "apple") (1, "pear")

Tuple float components follow sorting’s total order: -0.0 precedes 0.0, and NaNs come last. This differs from scalar float comparison; tuple equality still uses the components’ ordinary equality.

Distinct tuple components can be borrowed and read independently. The borrow below protects component 1 while component 0 remains readable:

def main() -> Result[(), AllocError | IoError]:
    mut pair = (1, [2]?)
    items = &mut pair[1]
    first = pair[0]
    print(first)?
    print(pair[0])?
    items.append(3)?
    print(pair)?
    Ok(())
1
1
(1, [2, 3])

Copyable components can be changed through mutable references too. A saved copy has its own inline storage:

def main() -> Result[(), IoError]:
    mut pair = (1, 2)
    saved = pair
    first = &mut pair[0]
    *first = 5
    print(pair)?
    print(saved)?
    Ok(())
(5, 2)
(1, 2)

Use a reference for the write; direct pair[0] = 5 is not supported. Reading an exclusively borrowed component is rejected:

def main() -> Result[(), AllocError | IoError]:
    mut pair = (1, [2]?)
    items = &mut pair[1]
    print(pair[1])?
    items.append(3)?
    Ok(())
conflicting borrow: cannot read or borrow `pair[1]` while it is exclusively borrowed

Print, unpack, and use tuple keys

Print several values with spaces between them. Unpack an owned list when its length is known; the list must contain exactly as many elements as targets. The statement moves its elements, and _ discards an element.

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    first, second = "north east".split(" ")?
    print(first, second, 2)?
    mut visits = dict[(i64, i64), i64].new()?
    cell = (3, 4)
    *visits.get_or_insert(cell, 0)? += 1
    *visits.get_or_insert(cell, 0)? += 1
    print(visits.get(cell, 0))?
    visited = {(3, 4), (3, 4), (4, 3)}?
    print(len(visited), cell in visited)?
    Ok(())
north east 2
2
2 True

Tuple keys contain integers, booleans, strings, or further tuples of those types. Values compare by their contents; a grid coordinate needs no integer encoding or string allocation. get_or_insert starts a missing count at 0 and returns a reference to the stored count; see dictionaries.

Consume dictionary values directly when the dictionary is finished:

def main() -> Result[(), AllocError | IoError]:
    groups = {"first": [1, 2]?, "second": [3]?}?
    for group in groups.into_values()?:
        print(group)?
    Ok(())
[1, 2]
[3]

into_values() consumes the dictionary and transfers its owned values into an ordered list. Use copy(groups)? first if the original is still needed.

Keep a queue with two ends

Use std.deque.Deque for queues and breadth-first traversal. Removing the first element does not shift the remaining elements.

from std.deque import Deque

def main() -> Result[(), AllocError | IoError]:
    mut queue = Deque[(i64, i64)]()?
    queue.append((0, 0))?
    queue.append((0, 1))?
    queue.appendleft((-1, 0))?
    print(len(queue))?
    while not queue.is_empty():
        match queue.popleft():
            case Some(value):
                print(value)?
            case Nothing:
                break
    print(queue.pop())?
    Ok(())
3
(-1, 0)
(0, 0)
(0, 1)
Option[tuple[i64, i64]].Nothing

len(queue) borrows the queue and calls its shared len method, so the queue remains available for removal. The same spelling works for built-in collections and standard containers with a shared len(self) -> i64 method.

Construction and appending return Result with AllocError. End removals return Option and transfer ownership. Appending takes ownership even on failure, while the existing queue remains intact. Growth is occasional; appending within capacity and removing an end allocate nothing. clear() drops all remaining values and retains the buffer for reuse.

See the module reference for exact operations and costs. A deque is owned; it has no borrowed iterator or indexed API yet.

Sort and reorder a list

sort() puts a list in ascending order. sort(True) gives descending order. Both change the list in place, need a mut owner, and allocate nothing, so neither takes ?.

def main() -> Result[(), AllocError | IoError]:
    mut scores = [72, 95, 58, 81]?
    scores.sort()
    print(scores)?
    scores.sort(True)
    print(scores)?
    mut names = ["pear", "fig", "apple"]?
    names.sort()
    print(names)?
    Ok(())
[58, 72, 81, 95]
[95, 81, 72, 58]
["apple", "fig", "pear"]

sort orders integers, floats, strings, and tuples of those. Tuples compare their first components, then the next where the first are equal. To sort records by a key, put the key first:

def main() -> Result[(), AllocError | IoError]:
    mut ranked = [(72, "Ada"), (95, "Bea"), (72, "Abe")]?
    ranked.sort()
    print(ranked)?
    print(ranked[-1][1])?
    Ok(())
[(72, "Abe"), (72, "Ada"), (95, "Bea")]
Bea

Other element types need a key function; natural sorting rejects them:

def main() -> Result[(), AllocError | IoError]:
    mut flags = [True, False]?
    flags.sort()
    Ok(())
sort does not order bool elements; it orders integers, floats, str, and tuples of those

Exchange two elements

swap(i, j) exchanges two elements of any type. Negative indices count from the end, and a missing index is a runtime error.

def main() -> Result[(), AllocError | IoError]:
    mut rows = [[0, 1]?, [2, 3]?, [4, 5]?]?
    rows.swap(0, -1)
    print(rows)?
    Ok(())
[[4, 5], [2, 3], [0, 1]]

The rows are exchanged where they are. Nothing is copied.

An exclusive slice view has the same swap, so a function can reorder part of a list. Its indices count within the view:

def ends(values: &mut [i64]) -> ():
    values.swap(0, -1)

def main() -> Result[(), AllocError | IoError]:
    mut scores = [70, 85, 90, 65, 88]?
    ends(&mut scores[1:4])
    print(scores)?
    Ok(())
[70, 65, 90, 85, 88]

Assign several targets at once

An assignment can name several targets, separated by commas. The whole right side is evaluated first, and then each value is assigned to its target from left to right. That makes an exchange a single statement:

def main() -> Result[(), AllocError | IoError]:
    mut values = [10, 20, 30]?
    values[0], values[2] = values[2], values[0]
    print(values)?
    mut low = 9
    mut high = 4
    low, high = high, low
    print((low, high))?
    Ok(())
[30, 20, 10]
(4, 9)

The right side is a tuple written without its parentheses. Reading values[2] copies an integer. A nested list is owned by its row list and cannot be moved out by reading it, so this form does not exchange owned elements:

def main() -> Result[(), AllocError | IoError]:
    mut rows = [[0, 1]?, [2, 3]?]?
    rows[0], rows[1] = rows[1], rows[0]
    Ok(())
cannot move out of indexed storage; borrow it with &collection[index], take it with replace(collection[index], value) or collection.pop(index), or copy it with copy(collection[index])?

Use rows.swap(0, 1) instead.

Sort by a key or create a sorted copy

sort_by_key reads each element through a shared reference. Its key can be a number, string, or tuple of those, so class and collection elements can be ordered too. An optional positional True requests descending order.

class Point:
    x: i64
    y: i64

def horizontal(point: &Point) -> i64:
    point.x

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    mut points = [Point(3, 1), Point(1, 2), Point(2, 4)]?
    points.sort_by_key(horizontal)
    for point in &points:
        print(point.x)?
    points.sort_by_key(horizontal, True)
    print(points[0].x)?
    points.sort_by_key(def(point: &Point) -> Result[i64, OverflowError[i64]]: Ok(-point.y))?
    print(points[0].x)?
    Ok(())
1
2
3
3
2

The operation allocates no key buffer. It calls the key repeatedly, so use a consistent key without side effects. Equal keys may change relative order; include a tie breaker in a tuple key when the order matters. The key can also be an anonymous function written in place, as the last call shows, or a reusable shared-call closure.

sorted creates a copy in natural order, leaving its source available. Copying is fallible and therefore needs ?:

def main() -> Result[(), AllocError | IoError]:
    scores = [72, 95, 58]?
    print(sorted(&scores)?)?
    print(scores)?
    print(sorted(&scores, True)?)?
    Ok(())
[58, 72, 95]
[72, 95, 58]
[95, 72, 58]

For a copy ordered by a key, explicitly copy into a mut binding, then call sort_by_key. Keys may return Result; handle the sorting call with ? and include its error in the enclosing signature. On failure, elements may already be reordered. One-shot or mutable-call closures are not supported.

Clean up owned values

Owned values clean up when their scope exits, including through return, break, continue, and error propagation. A move transfers this responsibility. Borrowing lends access while leaving cleanup with the owner.

Use drop(value) to release an owner early:

def main() -> Result[(), AllocError | IoError]:
    mut numbers = [1, 2, 3]?
    drop(numbers)
    numbers = [4]?
    print(numbers)?
    Ok(())
[4]

The old contents are released and the consumed binding is unavailable until reinitialized. Borrow checking prevents dropping an owner while a reference still needs it. A borrow ending at its last use does not destroy the owner.

Dropping a string does not invalidate other strings sharing its immutable storage. Fatal runtime traps terminate without running scope cleanup.

Build singly and doubly linked lists

A linked list connects nodes rather than treating adjacent storage positions as adjacent elements. It is a useful way to see ownership and borrowing at work. For an ordinary stack, prefer list[T]; for a queue with two ends, prefer std.deque. Separate heap allocations and pointer chasing often make linked nodes more expensive than contiguous storage.

These examples preview classes: a class groups typed fields, and its generated constructor takes those fields in declaration order. node.value accesses a field. The pub declarations make the teaching modules and their fields accessible to the example programs; they are not encapsulated collection APIs.

A singly linked list owns its successors

Each node owns one value and optionally a boxed successor. Box(value) moves its argument into a heap allocation and returns a Result; Nothing ends the chain. The box gives this recursive type a finite size.

pub class Node:
    pub value: i64
    pub next: Option[Box[Node]]

pub class Linked:
    pub head: Option[Box[Node]]

pub def push(xs: &mut Linked, value: i64) -> Result[(), AllocError]:
    mut node = Box(Node(value, Nothing))?
    node.next = replace(xs.head, Nothing)
    xs.head = Some(node)
    Ok(())

pub def pop(xs: &mut Linked) -> Option[i64]:
    match replace(xs.head, Nothing):
        case Some(node):
            mut removed: Node = node
            xs.head = replace(removed.next, Nothing)
            Some(removed.value)
        case Nothing:
            Nothing

pub def bump(xs: &mut Linked) -> Result[(), OverflowError[i64]]:
    match &mut xs.head:
        case Some(head):
            mut cursor: &mut Node = head
            while True:
                cursor.value += 1
                match &mut cursor.next:
                    case Some(next):
                        cursor = next
                    case Nothing:
                        break
        case Nothing:
            pass
    Ok(())

replace(place, replacement) installs a new value and returns the old one. It lets push take the head without leaving the borrowed list uninitialized. Allocate the detached node before taking that head: if allocation fails, the caller still owns the original chain. Each successful push allocates one box; popping and walking allocate nothing.

In pop, the annotation removed: Node extracts the contents of the owned box and frees its allocation. Taking removed.next transfers the remaining chain back to the list before the removed node is destroyed.

In bump, matching &mut borrows rather than consumes each link. A borrowed box converts to a reference to its contents, so cursor can advance from one node to its successor without copying or recursion. Incrementing a value can overflow, so bump returns a Result. Earlier nodes remain changed if a later node fails; this traversal is not transactional. This is a preview of borrowed matching.

from singly import Linked, push, pop, bump

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    mut xs = Linked(Nothing)
    for value in range(1, 4):
        push(&mut xs, value)?
    bump(&mut xs)?
    print(pop(&mut xs))?
    print(pop(&mut xs))?
    print(pop(&mut xs))?
    print(pop(&mut xs))?
    Ok(())
Option[i64].Some(4)
Option[i64].Some(3)
Option[i64].Some(2)
Option[i64].Nothing

Push and pop operate at the head, so this example behaves as a stack. Each operation takes constant time; walking takes time proportional to the number of nodes. Dropping a remaining chain uses bounded native stack space and allocates nothing. Automatic printing, equality, and copying of recursive owners are unsupported; inspect values by traversing the chain instead.

A doubly linked list uses IDs for both directions

Adding an owning prev: Box[Node] would make neighbours own each other. Plenty does not support ownership cycles or references stored in fields. Instead, let a built-in list own every node, and use integer indices as node IDs:

pub class Node:
    pub value: i64
    pub prev: Option[i64]
    pub next: Option[i64]

pub class Linked:
    pub nodes: list[Node]
    pub head: Option[i64]
    pub tail: Option[i64]

pub def push_back(xs: &mut Linked, value: i64) -> Result[(), AllocError]:
    id = len(xs.nodes)
    xs.nodes.append(Node(value, xs.tail, Nothing))?
    match xs.tail:
        case Some(old_tail):
            xs.nodes[old_tail].next = Some(id)
        case Nothing:
            xs.head = Some(id)
    xs.tail = Some(id)
    Ok(())

The empty list has no head or tail. Appending creates a node whose prev is the old tail, connects that tail’s next, and then updates the tail ID. The first node also becomes the head. Append happens before existing links change, so allocation failure leaves them intact.

IDs borrow nothing. Updating neighbouring nodes in separate statements avoids holding simultaneous mutable references into the backing list. Growing that list may move nodes in memory, but their indices remain valid.

from doubly import Linked, push_back

def main() -> Result[(), AllocError | IoError]:
    mut xs = Linked([]?, Nothing, Nothing)
    push_back(&mut xs, 10)?
    push_back(&mut xs, 20)?
    push_back(&mut xs, 30)?

    mut cursor = xs.head
    while True:
        match cursor:
            case Some(id):
                print(xs.nodes[id].value)?
                cursor = xs.nodes[id].next
            case Nothing:
                break

    cursor = xs.tail
    while True:
        match cursor:
            case Some(id):
                print(xs.nodes[id].value)?
                cursor = xs.nodes[id].prev
            case Nothing:
                break
    Ok(())
10
20
30
30
20
10

Forward traversal follows next; backward traversal follows prev. This is a doubly linked list logically, even though links are indices rather than machine pointers. Appending is amortized constant time because the backing list can need to grow; each traversal is linear.

Keep the teaching limits explicit

This example supports append and traversal, not deletion or reusable public handles. It relies on valid IDs and consistent links; public fields let callers break those invariants. Invalid indices trap, and a cycle in the links could make traversal loop forever.

Do not remove a backing-list entry with pop(id): shifting later entries would invalidate their IDs. To support deletion, use vacant slots, for example a list[Option[Node]], and reconnect the removed node’s neighbours. If slots are reused and callers retain IDs, add a generation check to reject stale handles; handles crossing between collections also need an identity policy. This is more than adding a prev field to an owned chain.

See representation choices for trade-offs, recursive storage for ownership limits, and parent links with arena IDs for another use of index-based links. The balanced-tree examples apply the same representation to B-tree and red-black insertion, deletion, and sorted traversal. The trie example uses parent and sibling IDs for iterative prefix retrieval and pruning.

Iteration and transformation

Transform collections with loops and comprehensions. Use generators to produce values incrementally, and collect them when you need a stored result.

Count positions and pair inputs with enumerate and zip directly in loops and comprehensions.

Traverse backward with reversed.

Chain iteration steps with iter to count, test, skip, pair, and collect items without writing the loop.

Iterate over collections and text

for visits list elements, dictionary keys, set elements, range integers, or characters of a string. A character is a one-character str.

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    mut total = 0
    for n in range(1, 5):
        total = total + n
    print(total)?

    scores = {"Ada": 10, "Grace": 20}?
    for name in &scores:
        print(name)?
        print(scores[name])?

    print(list(range(5, 0, -2))?)?
    for character in "hé":
        print(character)?
    Ok(())
10
Ada
10
Grace
20
[5, 3, 1]
h
é

range(stop) starts at zero; the stop is excluded. A step may be negative but never zero: it is a NonZero[i64]. Ranges can be reused and do not allocate; list(range(5))? allocates the list. String iteration yields characters without allocating.

Ranges default to i64. Use range[u8](8) or a typed bound to choose another element type. Start and stop must fit it; the step remains a signed NonZero[i64].

The iterable is evaluated once. Iterating an owned collection transfers it into the loop; use for item in &values to preserve the owner. Borrowed iteration prevents conflicting mutation; owned list elements become shared references. Use &mut values for mutable element references. To iterate a snapshot while mutating the original, use for item in copy(values)?.

Dictionary key/value loops

items() borrows a dictionary in insertion order. It works directly in loops and comprehensions, without allocating item tuples or a snapshot. Keys and values are shared references. Borrow the dictionary exclusively, (&mut counts).items(), to update every value in place; keys stay shared.

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    mut counts = {"apple": 2, "pear": 3}?
    for fruit, count in (&mut counts).items():
        *count += 1
    for fruit, count in counts.items():
        print(fruit)?
        print(count)?
    for a, b in [(1, 2), (3, 4)]?:
        print(a + b)?
    Ok(())
apple
3
pear
4
3
7

(&mut counts).values() lends only the values. Neither loop looks a key up or copies a value, so they suit values that cannot be copied. The dictionary cannot grow, shrink, or be read through another path during a mutable loop. A shared items() or values() view can be bound to a name and reused; a mutable one is used directly as the loop source. See updating every value for the full contract.

Count positions and pair inputs

Use enumerate(iterable) to get an i64 index starting at zero beside each element. Supply a second argument to choose the first index. Use zip(left, right) to pair elements until the shorter input ends.

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    for index, letter in enumerate("ab", 10):
        print((index, letter))?
    for number, letter in zip(range(3), "xy"):
        print((number, letter))?
    print([index + value for index, value in enumerate(range(3))]?)?
    Ok(())
(10, "a")
(11, "b")
(0, "x")
(1, "y")
[0, 2, 4]

An owned list transfers its elements. Borrow a list to preserve it; its elements retain their reference types. Borrow mutably to edit them.

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    mut values = [10, 20]?
    for index, value in enumerate(&mut values):
        *value += index
    for value, offset in zip(&values, range(2)):
        print(*value + offset)?
    print(values)?
    Ok(())
10
22
[10, 21]

Both helpers work directly in loop and comprehension sources. They add no allocation and evaluate their input expressions once, from left to right. They do not yet produce a value you can store or pass to next. Borrowed inputs require two unpacked bindings; references cannot be placed into an item tuple. Nested helpers are supported when their inner items are owned values. To pair a dictionary’s keys, pass the dictionary itself; items() remains a separate loop form.

zip tests the left input before the right. A left generator may advance once more when the right input ends; that unmatched item is dropped. Early exit drops remaining owned inputs. An enumeration index overflows only when another element requires an index beyond i64.MAX.

Traverse backward

Use reversed directly in a loop or comprehension to visit a list, text, or range backward. Borrow a list to keep it afterward. Text yields Unicode scalars, so a multibyte character stays intact.

def main() -> Result[(), AllocError | IoError | ZeroDivisionError]:
    values = [10, 20, 30]?
    for index, value in enumerate(reversed(&values)):
        print((index, *value))?
    for letter in reversed("aé🙂"):
        print(letter)?
    print([x for x in reversed(range(5)) if x % 2 == 0]?)?
    print(values)?
    Ok(())
(0, 30)
(1, 20)
(2, 10)
🙂
é
a
[4, 2, 0]
[10, 20, 30]

reversed adds no allocation and does not change a list’s order. Use reversed(&mut values) to edit its elements in reverse order. It currently accepts only lists, text, and ranges as loop/comprehension sources and pipeline sources; it cannot be stored or passed to next.

Borrowing and unpacking while iterating

Borrow a list when a helper needs to inspect or mutate it without taking ownership:

def total(values: &list[i64]) -> Result[i64, OverflowError[i64]]:
    mut result = 0
    for value in values:
        result = result + value
    Ok(result)

def add(values: &mut list[i64], value: i64) -> Result[(), AllocError]:
    values.append(value)?
    Ok(())

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    mut numbers = [1, 2]?
    print(total(numbers)?)?
    add(&mut numbers, 3)?
    print(numbers)?
    Ok(())
3
[1, 2, 3]

Unpack a tuple in a loop

def measurement() -> (i64, str):
    (42, "cm")

def main() -> Result[(), AllocError | IoError]:
    value, unit = measurement()
    print(value)?
    print(unit)?
    for number, word in [(1, "one"), (2, "two")]?:
        print((number, word))?
    print((3, "three"))?
    Ok(())
42
cm
(1, "one")
(2, "two")
(3, "three")

Unpacking a borrowed list keeps the list and borrows each component of the current tuple in place: in for number, word in &pairs, number is a &i64 and word a &str. Over &mut pairs, the names are &mut references, so the loop can change the tuples:

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    mut pairs = [(1, "one"), (2, "two")]?
    for number, _ in &mut pairs:
        *number *= 10
    for number, word in &pairs:
        print(*number, word)?
    print(pairs)?
    Ok(())
10 one
20 two
[(10, "one"), (20, "two")]

The names borrow different components, so both can be passed as &mut arguments to one call:

def swap(a: &mut i64, b: &mut i64) -> ():
    t = *a
    *a = *b
    *b = t

def main() -> Result[(), AllocError | IoError]:
    mut pairs = [(1, 2), (3, 4)]?
    for low, high in &mut pairs:
        swap(low, high)
    print(pairs)?
    Ok(())
[(2, 1), (4, 3)]

Tuple targets can nest. Each name still borrows its own component when the source is borrowed:

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    mut rows = [((1, 2), "point")]?
    x = 99
    for (x, y), _ in &mut rows:
        *x += 10
        *y += 20
    print(rows)?
    print(x)?
    Ok(())
[((11, 22), "point")]
99

The loop’s x is a new binding to the borrowed component. The outer x remains available after the loop.

Borrow elements while iterating

An exclusive loop lends each element in turn for mutation:

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    mut numbers = [10, 20]?
    for number in &mut numbers:
        *number = *number + 2
    print(numbers)?
    mut groups = [[1]?, [2, 3]?]?
    for group in &mut groups:
        group.append(4)?
    print(groups)?
    Ok(())
[12, 22]
[[1, 4], [2, 3, 4]]

Use &items when inspection is enough. Copyable elements such as integers and strings are yielded as values; owned elements such as inner lists are shared references. Use &mut items for mutable references, including scalar elements. The outer list cannot grow, shrink, or move during either loop. Afterward, the owner is available again.

Build collections with comprehensions

A comprehension produces a new collection from an iterable and optional filters. The element expression runs only after the filters pass.

def main() -> Result[(), AllocError | IoError | ZeroDivisionError | OverflowError[i64]]:
    squares = [n * n for n in range(8) if n % 2 == 0]?
    print(squares)?
    print({n: n * n for n in [2, 3]?}?)?
    print(len({n // 2 for n in range(8)}?))?
    print([x * 10 + y for x in range(3) for y in range(x)]?)?
    Ok(())
[0, 4, 16, 36]
{2: 4, 3: 9}
4
[10, 20, 21]

Multiple clauses nest from left to right. Later iterables can use earlier variables, and you can use multiple if filters. Variables remain local to the comprehension. Parenthesize a conditional expression used as an iterable or filter. Dictionary comprehensions evaluate each key before its value.

% is modulo: the result follows the divisor’s sign, so -7 % 3 is 2. Both operands must have the same integer type.

Every element must have the same type; there is no implicit numeric widening:

def main() -> Result[(), AllocError | IoError]:
    values = [1, True]?
    Ok(())
expected list[i64]

A comprehension over a range or list with no if clause reserves its full length once; with a filter or another source it grows the output in place. The ? handles allocation failure. A comprehension with one for clause compiles to the same loop as the equivalent pipeline.

Collect a snapshot with unpacking

Unpacking works in comprehensions too. This one collects dictionary items into a list of tuples:

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    mut counts = {"apple": 2, "pear": 3}?
    for fruit, count in (&mut counts).items():
        *count = *count + 1
    print([(*fruit, *count) for fruit, count in counts.items()]?)?
    print([a + b for a, b in [(1, 2), (3, 4)]?]?)?
    Ok(())
[("apple", 3), ("pear", 4)]
[3, 7]

For dictionaries containing owned values, use copy(value)? when the snapshot needs independent owners. Shared item views yield references for every key and value, so the * above reads each copyable value before storing it in the snapshot tuple.

Text comprehensions collect Unicode scalars:

def main() -> Result[(), AllocError | IoError]:
    print([character for character in "hé"]?)?
    Ok(())
["h", "é"]

Produce values incrementally

A generator function declares Generator[T] and uses yield statements:

def countdown(start: i64) -> Generator[i64]:
    mut remaining = start
    while remaining > 0:
        yield remaining
        # The loop proves 1 <= remaining <= i64.MAX.
        unchecked:
            remaining -= 1

def main() -> Result[(), IoError]:
    numbers = countdown(3)
    print("created")?
    for number in numbers:
        print(number)?
    Ok(())
created
3
2
1

Calling a generator evaluates its arguments immediately but starts its body only when iteration requests a value. Each yield pauses the body, retaining its locals until the next request. A bare return or the end of the body finishes it; a generator cannot return a value.

Creating the generator needs no allocation or ?. Allocating arguments and operations in its body still need their usual error handling. A generator cannot propagate arithmetic errors to its caller. The countdown uses one small unchecked: suite: the loop proves that subtracting one stays in range. For unbounded calculations, yield Result items and let the consumer handle them, as in lazy expressions.

Comprehensions and collection constructors also consume generators:

def numbers(limit: i64) -> Generator[i64]:
    for n in range(limit):
        yield n

def main() -> Result[(), AllocError | IoError | ZeroDivisionError | OverflowError[i64]]:
    print([n * n for n in numbers(6) if n % 2 == 1]?)?
    print(list(numbers(3))?)?
    Ok(())
[1, 9, 25]
[0, 1, 2]

Call next(&mut messages) to advance a mutable generator without consuming it. The shorthand next(messages) also works. It returns Some(value) or Nothing; an exhausted generator keeps returning Nothing:

def once() -> Generator[str]:
    yield "hello"

def main() -> Result[(), IoError]:
    mut messages = once()
    print(next(&mut messages))?
    print(next(&mut messages))?
    print(next(&mut messages))?
    Ok(())
Option[str].Some("hello")
Option[str].Nothing
Option[str].Nothing

Breaking out of consuming iteration drops the generator without resuming its body. Yielding an owned value transfers ownership; to keep it, handle copy(value)’s result and yield the copy.

Generators cannot yield other generators, references, or closures. For a lazy expression producer, use a generator expression. yield from, send, and async operations are unsupported.

Pass a lazy expression to a consumer

A generator expression evaluates one element at a time, when its consumer asks for the next one:

def square(n: i64) -> Result[i64, OverflowError[i64]]:
    Ok(n * n)

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    squares = (square(n) for n in range(4) if n != 2)
    mut values: list[i64] = []?
    for outcome in squares:
        values.append(outcome?)?
    print(values)?
    words = ["tea", "", "café"]?
    print(", ".join(w for w in words if w != "")?)?
    Ok(())
[0, 1, 9]
tea, café

The square helper returns a Result because arbitrary i64 squares can overflow. The generator yields those results without unwrapping them; the loop uses ? in main before appending each value. A caller’s arithmetic mode never changes a producer’s declared contract.

separator.join accepts any iterable of str, so the selected words are joined without first collecting them into a list. A generator cannot be measured in advance, so join grows a scratch buffer as pieces arrive; a list, set, or dictionary is measured first and needs only the result’s allocation. A consumer that stops early, such as the pipeline method any, leaves the remaining elements unevaluated.

The outer source evaluates when the generator is created. Filters, elements, and inner for sources evaluate on demand. The generator owns its sources and copies or moves any enclosing local values it uses; copied captures are snapshots. Borrowed inputs are unsupported because a suspended frame cannot retain a reference into its caller. Use copy(values)? to preserve an owned list, or consume a borrowing pipeline locally instead.

Parentheses can be omitted for a sole argument. With additional arguments, write consume((n for n in source), other). Generator expressions follow the same ownership, cleanup, and yield restrictions as generator functions; they require no intermediate list or heap-allocated frame. A fallible operation in their body must be handled explicitly, since ? cannot propagate out of a suspended generator.

Transfer a generator’s progress

A generator owns a position in an advancing computation. Assignment transfers that owner instead of copying the position:

def once() -> Generator[i64]:
    yield 42

def main() -> Result[(), AllocError | IoError]:
    first = once()
    second = first
    print(list(second)?)?
    Ok(())
[42]

After a move, the old binding cannot be used:

def once() -> Generator[i64]:
    yield 42

def main() -> Result[(), AllocError | IoError]:
    first = once()
    second = first
    list(first)?
    pass
    Ok(())
use of moved binding `first`

Owned arguments, returns, and for iteration also move generators. next(&mut it) borrows a mutable generator without consuming its owner; next(it) remains a shorthand. A mutable binding can be reinitialized after moving its previous value.

The compiler checks all possible continuing paths. A move on just one branch makes later reuse unsafe, reported as use of possibly moved binding. Within a loop, an outer generator must be reinitialized before any path repeats the loop. Generators cannot be stored in collections or user-defined enum payloads yet.

See Borrow a collection for the borrowing rules.

Drop without resuming

Use drop(value) to release an owner earlier. The consumed binding becomes unavailable; a mutable binding may then receive another value:

def pending() -> Generator[i64]:
    yield 1

def main() -> Result[(), AllocError | IoError]:
    mut numbers = [1, 2, 3]?
    drop(numbers)
    numbers = [4]?
    print(numbers)?

    task = pending()
    drop(task)
    print("done")?
    Ok(())
[4]
done

Dropping a generator cleans its captured values without resuming its body. Borrow checking prevents dropping an owner while a reference still needs it. A borrow can end at its last use; that does not implicitly destroy its owner.

Passing and returning generators

Use Generator[T] to accept or return a generator:

def once(n: i64) -> Generator[i64]:
    yield n

def pair() -> Generator[i64]:
    yield 10
    yield 20

def source(n: i64) -> Generator[i64]:
    once(n)

def total(values: Generator[i64]) -> Result[i64, OverflowError[i64]]:
    mut result = 0
    for value in values:
        result = result + value
    Ok(result)

def main() -> Result[(), IoError | OverflowError[i64]]:
    print(total(source(7))?)?
    print(total(pair())?)?
    Ok(())
7
30

total accepts either producer. Passing a generator transfers ownership; use &mut Generator[T] when a helper should advance an existing owner with next. Once you know closures, Transform a stream of values shows how to filter, map and reduce a generator without writing the loop.

A factory must return the same producer on every path. Two generators yielding the same type are not interchangeable return values:

def one() -> Generator[i64]:
    yield 1

def two() -> Generator[i64]:
    yield 2
    yield 3

def choose(flag: bool) -> Generator[i64]:
    if flag:
        return one()
    two()

def main() -> ():
    drop(choose(True))
expected Generator[i64] from `one`, got Generator[i64] from `two`

Put such a branch inside one generator body instead. A generator may contain another generator, but recursive generator state is rejected: its inline storage would have no finite size.

Creating generators

An ordinary function can create and return a generator:

def numbers() -> Generator[i64]:
    yield 10
    yield 20

def source() -> Generator[i64]:
    numbers()

def total() -> Result[i64, OverflowError[i64]]:
    values = source()
    mut result = 0
    for n in values:
        result = result + n
    Ok(result)

def main() -> Result[(), IoError | OverflowError[i64]]:
    print(total()?)?
    Ok(())
30

A factory that can fail can return Result[Generator[T], E]. Write Ok(numbers()) for success and use ? at the factory’s call site. Wrapping a generator adds no allocation; dropping the wrapper releases its captures without resuming the body.

Collect results into owned collections

Return a comprehension’s Result directly when the caller should handle allocation failure:

def squares() -> Result[list[i64], AllocError | ZeroDivisionError | OverflowError[i64]]:
    [n * n for n in range(6) if n % 2 == 0]

def main() -> Result[(), AllocError | IoError | AllocError | OverflowError[i64] | ZeroDivisionError]:
    print(squares()?)?
    Ok(())
[0, 4, 16]

Collect an owned iterator with recoverable list growth:

def numbers() -> Generator[i64]:
    yield 3
    yield 6

def collect() -> Result[list[i64], AllocError]:
    source = numbers()
    list[i64].from(source)

def main() -> Result[(), AllocError | IoError]:
    print(collect()?)?
    Ok(())
[3, 6]

from consumes the source. If allocation fails, it drops the partial list and remaining iterator. Neither from nor comprehensions undo earlier effects or handle failures inside the producer’s body.

Sets provide the same constructor, removing duplicates in first-seen order:

def main() -> Result[(), AllocError | IoError]:
    print(set[i64].from([3, 1, 3, 2]?)?)?
    Ok(())
{3, 1, 2}

Dictionary sources iterate over keys. from does not support borrowed sources or string iteration.

Chain iteration steps with iter

iter(source) starts a pipeline: a chain of iteration steps that runs only when a loop or a final method such as count() consumes it. No intermediate collection is built.

def main() -> Result[(), AllocError | IoError]:
    rolls = [3, 6, 2, 6, 6, 1]?
    print(iter(&rolls).filter_value(6).count())?
    for turn, roll in iter(&rolls).enumerate(1).skip(2).take(3):
        print(turn, *roll)?
    print(len(rolls))?
    Ok(())
3
3 2
4 6
5 6
6

filter_value(6) keeps the items equal to 6, and count() counts them without building a list. enumerate(1) numbers the items from one, skip(2) discards the first two, and take(3) stops after three more. Steps apply in the order written: enumerate comes first, so the turn numbers count every roll, including skipped ones.

The source follows the ordinary iteration rules. Borrowing rolls keeps the list, and each item is a reference, so the loop reads *roll. filter_value still takes a plain i64: a borrowed item is compared by value. It also accepts a borrowed value, such as filter_value(&wanted), so a string can be matched without copying it; wanted then stays borrowed until the pipeline ends.

A pipeline is single-use. Consuming it again, or using an owned source after the pipeline has taken it, is an ownership error.

Add up and find extremes

sum(), min(), and max() are final methods that need no function either:

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    rolls = [3, 6, 2, 6, 6, 1]?
    print(iter(&rolls).sum()?, iter(&rolls).min(), iter(&rolls).skip(4).max())?
    none: list[i64] = []?
    print(iter(&none).sum()?, iter(&none).max().unwrap_or(0))?
    Ok(())
24 Option[i64].Some(1) Option[i64].Some(6)
0 0

A borrowed item is read by value, so the results are i64, not references. sum() gives zero for no items. Integer overflow returns OverflowError[i64], handled with ? here; floating-point sums retain IEEE behavior. min() and max() return an Option, Nothing for no items. They accept numbers and str and keep the first of equal items. Floats follow the IEEE 754 rules for NaN and signed zero, as std.math.min_all and max_all do; the pipeline reference gives them.

Test boolean items

any() reports whether at least one item is True, and all() whether every item is. Both stop as soon as they know the answer:

def positive(values: list[i64]) -> Generator[bool]:
    for value in values:
        yield value > 0

def main() -> Result[(), AllocError | IoError]:
    present = [True, False, True]?
    print(iter(&present).any(), iter(reversed(&present)).all())?
    print(positive([2, -1, 3]?).any(), positive([2, -1, 3]?).all())?
    none: list[bool] = []?
    print(iter(&none).any(), iter(&none).all())?
    Ok(())
True False
True False
False True

The generator computes each comparison only when advanced, so all() stops at -1 and never tests 3; its frame is cleaned up when the method returns. For no items, any() gives False and all() gives True.

Without an argument, these methods need bool items: numbers and text have no implicit truth value. To test other items, pass a predicate function; that form is taught in Reduce a borrowed collection with a pipeline.

Pair, join, and collect

zip walks two sources together, and chain continues with a second source of the same item type. collect() builds the collection named by the binding’s annotation and returns a Result for allocation failure:

def main() -> Result[(), AllocError | IoError]:
    names = ["Ada", "Bob", "Cal"]?
    present = [True, False, True]?
    for name, here in iter(&names).zip(&present):
        print(name, *here)?
    print(iter(&present).filter_value().count())?
    morning = [3, 1]?
    evening = [4, 3]?
    readings: list[i64] = iter(morning).chain(evening).collect()?
    print(readings)?
    Ok(())
Ada True
Bob False
Cal True
2
[3, 1, 4, 3]

Without an argument, filter_value() keeps True items. The two owned lists move into the chain, so their numbers move into readings without copying.

The value must have the item’s type. There is no implicit conversion, and the no-argument form works only for bool items:

def main() -> Result[(), AllocError | IoError]:
    rolls = [3, 6]?
    print(iter(&rolls).filter_value("6").count())?
    Ok(())
filter_value compares &i64 items with a str value; it requires i64 or &i64

Take one item, then continue

next takes one item from a pipeline, as it does from a generator. The pipeline keeps its place, so a loop or final method afterwards continues with the remaining items:

def main() -> Result[(), AllocError | IoError]:
    lines = ["name,score", "ada,3", "bob,5"]?
    mut rows = iter(lines)
    match next(rows):
        case Some(header):
            print("columns:", header)?
        case Nothing:
            print("no header")?
    for number, row in rows.enumerate(1):
        print(number, row)?
    Ok(())
columns: name,score
1 ada,3
2 bob,5

The binding needs mut because next advances it. next returns an owned Option, so it needs owned items: here iter(lines) moves the list into the pipeline. For a borrowed source, first map each item to an owned value. Once next returns Nothing, the pipeline stays empty.

Lend a pipeline to a function

A function can take a pipeline as &mut Iter[T] and consume part of it. The caller lends it with &mut and continues after the items the function took:

def print_header(rows: &mut Iter[str]) -> Result[(), AllocError | IoError]:
    match next(rows):
        case Some(header):
            print("columns:", header)?
        case Nothing:
            print("no header")?
    Ok(())

def main() -> Result[(), AllocError | IoError]:
    lines = ["name,score", "", "ada,3", "bob,5"]?
    mut rows = iter(lines).filter(def(line: &str) -> bool: line != "")
    print_header(&mut rows)?
    print(rows.count(), "rows")?
    Ok(())
columns: name,score
2 rows

The function can also loop over rows and break or return early; the rest stays with the caller. It cannot keep or return the pipeline.

A pipeline over a borrowed list yields references, written &mut Iter[&str] here. The function can use each item while it runs, but cannot return one, because the caller lent the list only for the call:

def greet_first(names: &mut Iter[&str]) -> Result[(), IoError]:
    for name in names:
        print("hello", name)?
        break
    Ok(())

def main() -> Result[(), AllocError | IoError]:
    names = ["ada", "bob", "ada", "cal"]?
    mut adas = iter(&names).filter_value("ada")
    greet_first(&mut adas)?
    print(adas.count(), "more")?
    print(len(names))?
    Ok(())
hello ada
1 more
4

A lent pipeline whose callbacks borrow local values needs those closures written inside the adapter calls; see Lend a pipeline whose callbacks borrow.

Generators have the same methods without iter:

def countdown(start: i64) -> Generator[i64]:
    mut remaining = start
    while remaining > 0:
        yield remaining
        # The loop proves 1 <= remaining <= i64.MAX.
        unchecked:
            remaining -= 1

def main() -> Result[(), AllocError | IoError]:
    last: list[i64] = countdown(10).skip(7).collect()?
    print(last)?
    Ok(())
[3, 2, 1]

skip still advances the generator past the first seven values; it only discards them.

Prefer a pipeline when a chain of these steps states the task directly, and a plain loop or comprehension when the work needs its own expression or several statements. Most other pipeline steps, such as map, filter, and fold, take a function. They are taught after functions as values, starting with Reduce a borrowed collection with a pipeline. The pipeline reference lists every step and its cost.

Defining your own data

Use classes for records with fields and methods, and enums for alternatives. Their fields follow the same ownership and cleanup rules as collections.

Heterogeneous collections combines classes and enums in lists and dictionary values, and explains the restrictions on sets and dictionary keys. Special-purpose data structures introduces columnar lists for scanning selected primitive fields.

Group fields and methods in a class

A class is a record with typed fields. Without __init__, its constructor takes the fields in declaration order:

class Point:
    x: i64
    y: i64

    def squared_length(self) -> Result[i64, OverflowError[i64]]:
        Ok(self.x * self.x + self.y * self.y)

    def shift(self: &mut Point, amount: i64) -> Result[(), OverflowError[i64]]:
        x = self.x + amount
        y = self.y + amount
        self.x = x
        self.y = y
        Ok(())

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    mut point = Point(y=4, x=3)
    print(point.squared_length()?)?
    point.shift(amount=1)?
    mut changed = copy(point)?
    changed.x = 20
    print(point)?
    print(changed)?
    Ok(())
25
Point(x=4, y=5)
Point(x=20, y=5)

Ordinary methods borrow self read-only by default: self is shorthand for self: &Point here. A method that changes fields declares self: &mut Point. Calling it requires a mut binding or an exclusive reference. shift calculates both coordinates before changing either, so overflow leaves the point unchanged. Arithmetic failure does not undo earlier assignments automatically. An explicit self: Point instead takes ownership; see consuming methods for state transitions and finalization.

Methods can share names with integer built-ins. A class receiver selects its own method, including its parameter and return types:

class Counter:
    total: i64

    def wrapping_add(self: &Counter, other: i64) -> Result[i64, OverflowError[i64]]:
        Ok(self.total + other)

def main() -> Result[(), IoError | OverflowError[i64]]:
    counter = Counter(1)
    print(counter.wrapping_add(2)?)?
    Ok(())
3

This method uses ordinary checked addition; its name does not change that.

If you forget the receiver annotation, the error names the required signature:

class Counter:
    value: i64
    def increment(self) -> Result[(), OverflowError[i64]]:
        self.value = self.value + 1
        Ok(())

def main() -> ():
    pass
declare `self: &mut Counter`

Change the parameter to self: &mut Counter, as in shift above. The lifecycle methods __init__ and __del__ give bare self exclusive access automatically.

Instances store their fields inline: Point(3, 4) needs no allocation or ?. Copying list fields can still fail to allocate.

By default, class instances move even when their fields are all integers. other = point transfers ownership; copy(point) requests an independent instance. Integer and string fields can be read directly. Owned fields such as lists or other classes must be borrowed, observed, or explicitly copied. Equality compares fields; printing shows the class name and fields.

Copy small value types

Write @copy on the line before class when instances are plain values, like integers. Assignment, argument passing, returning, and storing in a collection then copy the instance and leave the original usable:

@copy
class Money:
    cents: i64
    currency: str

def add(a: Money, b: Money) -> Result[Money, OverflowError[i64]]:
    Ok(Money(a.cents + b.cents, a.currency))

def main() -> Result[(), IoError | OverflowError[i64]]:
    price = Money(250, "EUR")
    tax = Money(50, "EUR")
    paid = add(price, tax)?
    mut discounted = price
    discounted.cents = 200
    print(price, paid, discounted)?
    Ok(())
Money(cents=250, currency="EUR") Money(cents=300, currency="EUR") Money(cents=200, currency="EUR")

Every field must itself copy: a scalar, str, another @copy class, or a tuple or enum of those. Copying duplicates the inline fields and never allocates, so it needs no ?. A field that owns storage is rejected:

@copy
class Basket:
    items: list[str]

def main() -> ():
    pass
every field of a `@copy` class must copy

A @copy class also cannot define __del__. A generic @copy class copies when its type arguments copy, so Pair[i64] copies and Pair[list[i64]] moves. Keep the default for instances that own resources or should be used only once; see copy classes.

Hide secrets when printing

Printing a class shows every field, so a password or key reaches any log that prints its holder. Write @redacted before class or enum to replace the contents with (@redacted) wherever the value is printed or formatted, however deeply it is nested:

@redacted
class Password:
    text: str

class Login:
    user: str
    password: Password

def main() -> Result[(), AllocError | IoError]:
    login = Login("ada", Password("hunter2"))
    print(login)?
    print(str.repr(login.password)?)?
    print(login.password.text == "hunter2")?
    Ok(())
Login(user="ada", password=Password(@redacted))
Password(@redacted)
True

The fields stay readable in code, so redaction prevents accidental printing rather than enforcing secrecy; see redacted types.

Limits

There is no inheritance, dynamic attribute creation, or class-variable syntax. Fields currently have no default values. Constructors and methods accept named arguments; an explicit __init__ can give its parameters defaults.

Give arithmetic meaning to your types

Use arithmetic operators when their meaning is natural for your type. A method named __add__ defines +, and __neg__ defines unary -. These are statically compiled calls, so using the operator does not introduce runtime method lookup.

The distance wrappers here demonstrate hook and forwarding mechanics. For physical calculations, use quantities: they also check the dimensions of products and quotients, which these wrappers do not.

class Distance:
    metres: i64

    def __add__(self, other: &Distance) -> Result[Distance, OverflowError[i64]]:
        Ok(Distance(self.metres + other.metres))

    def __neg__(self) -> Result[Distance, OverflowError[i64]]:
        Ok(Distance(-self.metres))

    def __mul__(self, scale: i64) -> Result[Distance, OverflowError[i64]]:
        Ok(Distance(self.metres * scale))

    def __rmul__(self, scale: i64) -> Result[Distance, OverflowError[i64]]:
        self * scale

    def add(self: &mut Distance, other: &Distance) -> Result[(), OverflowError[i64]]:
        self.metres += other.metres
        Ok(())

def main() -> Result[(), IoError | OverflowError[i64]]:
    mut walk = Distance(10)
    extra = Distance(5)
    print((walk + extra)?, (-extra)?, (3 * extra)?)?
    walk.add(&extra)?
    print(walk, extra)?
    Ok(())
Distance(metres=15) Distance(metres=-5) Distance(metres=15)
Distance(metres=15) Distance(metres=5)

The ordinary hooks borrow their operands, so extra stays usable. The reflected hook __rmul__ handles a distance on the right of 3 * extra. The named add method takes exclusive access and updates walk in place. Its integer addition can fail, so it returns a Result; the binding must be mutable.

Integer hooks declare overflow just like ordinary functions. A hook can also report a domain error; here the extra check prevents negative combined stock:

class Stock:
    amount: i64

    def __add__(self, other: &Stock) -> Result[Stock, str | OverflowError[i64]]:
        total = self.amount + other.amount
        if total < 0:
            return Err("negative stock")
        Ok(Stock(total))

def main() -> Result[(), IoError | str | OverflowError[i64]]:
    a = Stock(5)
    b = Stock(3)
    combined = (a + b)?
    print(combined.amount)?
    Ok(())
8

Parenthesize the operation before ?. Allocation errors follow the same pattern. For a fallible update, write value = (value + other)? or call a named method; += requires a unit-returning in-place hook. For large data, a named method that writes into reusable output storage can avoid intermediate allocations.

A class that only gives a distinct name to one value can forward its field’s operators instead of defining a hook for each. Annotate it @newtype:

@newtype
class Metres:
    value: i64

@newtype
class Seconds:
    value: i64

def main() -> Result[(), IoError | OverflowError[i64]]:
    run = Metres(400)
    mut total = run + run
    total += Metres(100)
    print(total, total > run)?
    lap = Seconds(70)
    print(lap * Seconds(2))?
    Ok(())
Metres(value=900) True
Seconds(value=140)

run + run adds the two value fields and wraps the sum back in Metres, and comparisons such as > return bool. Typing stays nominal: run + lap and run + 100 are compile errors, which keeps metres and seconds apart. An explicit hook such as __add__ takes over its own operator. @newtype does not make the class copy; add @copy for that.

Equality still compares fields automatically. Python special methods that Plenty never calls, such as __eq__, __hash__, __str__ and __len__, are compile errors rather than silently ignored; see equality, ordering and hashing and reserved method names. For supported hooks, exact signatures, and resolution rules, see user-defined arithmetic operators.

Return a reference to a field

A function that returns a reference borrows it from its first reference parameter; for a method, that is self. The returned value must borrow that parameter, never a later one or a local owner. The caller keeps the original value alive and borrowed until the result’s last use.

class Pair:
    left: i64
    right: i64
    def left_ref(self: &mut Pair) -> &mut i64:
        &mut self.left

def main() -> Result[(), IoError]:
    mut pair = Pair(1, 2)
    left = pair.left_ref()
    *left = 8
    print(pair.left)?
    Ok(())
8

A mutable returned reference requires the first reference parameter to be &mut. A temporary receiver cannot supply a reference that outlives it.

Later reference parameters only lend their values for the call. A lookup can take its key by reference and still return a borrow of the collection:

def score(scores: &dict[str, i64], name: &str) -> &i64:
    &scores[*name]

def main() -> Result[(), AllocError | IoError]:
    scores = {"ada": 3, "bob": 5}?
    mut name = "bob"
    points = score(&scores, &name)
    name = "eve"
    print(*points)?
    print(name)?
    Ok(())
5
eve

The key’s loan ends when score returns, so name can change while points is live. To return a borrow of name, it would have to be the first parameter.

Return a borrow that may be absent

A search may find nothing. Return Option[&T], or Option[&mut T] to let the caller change what was found. Some must borrow the first reference parameter; Nothing borrows nothing:

class Item:
    name: str
    count: i64

def find(items: &mut list[Item], name: &str) -> Option[&mut Item]:
    for item in items:
        if item.name == name:
            return Some(item)
    Nothing

def add(items: &mut list[Item], name: str) -> Result[(), AllocError | OverflowError[i64]]:
    match find(items, &name):
        case Some(item):
            item.count += 1
        case Nothing:
            items.append(Item(name, 1))?
    Ok(())

def main() -> Result[(), AllocError | IoError | AllocError | OverflowError[i64]]:
    mut items: list[Item] = []?
    add(&mut items, "ada")?
    add(&mut items, "bob")?
    add(&mut items, "ada")?
    for item in items:
        print(item.name)?
        print(item.count)?
    Ok(())
ada
2
bob
1

Match the result directly. In the Some arm, item borrows items, so the list cannot change there. The Nothing arm holds no borrow, so it can append. Result[&T, E] works the same way with Ok; its error cannot hold a reference. ? and unwrap() also produce the reference.

The result can be held in a local until it is matched, but it cannot be stored in a list, a field, or a mut binding: match it and store an owned copy or an index. The reference lists every rule.

Shared arguments can borrow temporaries, as in size("ada") for a size(s: &str) function. The temporary lives through the expression. A result borrowing it may be used immediately, but cannot be stored or returned. Mutable temporaries are unsupported.

An exclusive method can use a completed read of its receiver as an argument. This also works through an existing exclusive reference:

import std.bytes
from std.bytes import Bytes

def clear(buffer: &mut Bytes) -> ():
    buffer.fill(0, buffer.len(), 0u8)

def main() -> Result[(), AllocError | IoError]:
    mut buffer = std.bytes.from_list([1u8, 2u8]?)?
    clear(&mut buffer)
    print(buffer.to_list()?)?
    Ok(())
[0, 0]

The receiver is reserved first, arguments run left to right, and exclusive access begins when the method starts. Arguments may read the receiver, but cannot mutate or move it, or pass a reference into it that remains live during the call. For nested mutations, compute the inner result in a separate local before calling the outer method.

Control construction with init

Define __init__ to calculate or check fields during construction. It replaces the generated constructor; bare self is an exclusive reference:

class Span:
    start: i64
    end: i64

    def __init__(self, start: i64, length: i64) -> Result[(), OverflowError[i64]]:
        self.start = start
        self.end = start + length
        Ok(())

    def length(self) -> Result[i64, OverflowError[i64]]:
        Ok(self.end - self.start)

def main() -> Result[(), IoError | OverflowError[i64]]:
    span = Span(10, 5)?
    print(span)?
    print(span.length()?)?
    Ok(())
Span(start=10, end=15)
5

The initializer calculates the endpoint from a start and length. Overflow returns its declared OverflowError[i64], so Span(10, 5) returns Result[Span, OverflowError[i64]]; ? propagates failure through main. An initializer returning () instead gives an infallible constructor. Construction itself allocates nothing and adds no allocation error; see constructor contracts.

Every field must be initialized on every path out of the constructor. You may read a field already initialized, but cannot pass the whole unfinished self to another function or method. A loop might never execute, so initializing a field only inside a loop is insufficient.

class Pair:
    first: i64
    second: i64

    def __init__(self, first: i64) -> ():
        self.first = first

def main() -> ():
    pass
fields not initialized: second

Initialize fallible fields

An initializer can return Result[(), E] when setting up its fields may fail. The constructor then returns Result[Class, E]. Here, reserving list capacity can fail to allocate:

class Buffer:
    values: list[i64]

    def __init__(self: &mut Buffer, size: i64) -> Result[(), AllocError]:
        self.values = list[i64].with_capacity(size)?
        Ok(())

def main() -> Result[(), AllocError | IoError]:
    print(Buffer(8)?)?
    Ok(())
Buffer(values=[])

If initialization fails, the fields already initialized are dropped, but the incomplete instance’s __del__ is skipped. Successful initialization must fill every field and return Ok(()).

Borrow distinct fields

Borrow a field directly with & or &mut:

class Basket:
    count: i64
    items: list[str]

def main() -> Result[(), AllocError | IoError]:
    mut basket = Basket(0, []?)
    count = &mut basket.count
    *count = 2
    basket.items.append("apple")?
    basket.items.append("pear")?
    print(basket)?
    Ok(())
Basket(count=2, items=["apple", "pear"])

Distinct fields can be borrowed or changed independently. Borrowing a whole record overlaps all its fields. Collection element references such as &basket.items[0] protect the collection against changes that could invalidate the element’s address. You can replace a class-valued field by assignment, but cannot replace a whole class through *reference = new_instance yet.

class Position:
    x: i64
    y: i64

def main() -> Result[(), IoError]:
    mut position = Position(1, 2)
    x = &mut position.x
    y = &mut position.y
    *x = 10
    *y = 20
    print(*x)?
    print(*y)?
    Ok(())
10
20

Borrowing a field through a getter

A simple getter that directly returns a field preserves the field’s identity for the borrow checker. After the call, unrelated fields remain available:

class Pair:
    x: i64
    y: i64
    def x_ref(self: &mut Pair) -> &mut i64:
        &mut self.x

def main() -> Result[(), IoError]:
    mut pair = Pair(1, 2)
    x = pair.x_ref()
    pair.y = 7
    *x = 9
    print(pair)?
    Ok(())
Pair(x=9, y=7)

Getters that choose between fields borrow the whole argument. Indexed references also borrow the whole collection. The call still requires the access declared by the receiver type.

Describe alternatives with enums

An enum defines alternatives, each of which may carry typed data. Match its variants to extract that data:

enum Reading:
    Missing
    Value(i64)
    Invalid(str)

def describe(reading: Reading) -> str:
    match reading:
        case Reading.Missing:
            "no reading"
        case Reading.Value(number):
            "positive" if number > 0 else "nonpositive"
        case Reading.Invalid(reason):
            reason

def main() -> Result[(), IoError]:
    print(describe(Reading.Missing))?
    print(describe(Reading.Value(42)))?
    print(describe(Reading.Invalid("sensor offline")))?
    Ok(())
no reading
positive
sensor offline

Enum payloads are stored inline: Reading.Value(42) needs no allocation or ?. Variants without data omit parentheses. Payloads may have several positions, such as Pair(i64, str). Case bindings are immutable and local to that case. Use _ to ignore a payload position or a final case _: to cover remaining variants; case _ as name: also binds the remaining value. Duplicate or missing cases are errors:

enum Switch:
    On
    Off

def main() -> Result[(), IoError]:
    match Switch.On:
        case Switch.On:
            print("on")?
    Ok(())
non-exhaustive match; missing Switch.Off

A final match produces the function’s result; cases can also return early or break/continue a loop. A match can also be the value of a binding. Separately declared enums remain distinct types, even with identical variants.

A payload position can hold another pattern instead of a name, so one case reaches into an enum stored inside an enum. | joins patterns that share an arm; every alternative must bind the same names:

enum Shape:
    Dot
    Circle(i64)
    Rect(i64, i64)

enum Slot:
    Empty
    Full(Shape)

def width(slot: Slot) -> i64:
    match slot:
        case Slot.Full(Shape.Circle(size) | Shape.Rect(size, _)):
            size
        case Slot.Full(Shape.Dot) | Slot.Empty:
            0

def main() -> Result[(), IoError]:
    print(width(Slot.Full(Shape.Rect(3, 4))))?
    print(width(Slot.Full(Shape.Circle(5))))?
    print(width(Slot.Empty))?
    Ok(())
3
5
0

Coverage is checked through the nested patterns: leaving out Slot.Full(Shape.Dot) is an error that names it.

Payloads can contain strings, collections, and other enums. Matching an enum with mutable payloads consumes it and transfers those payloads; use match copy(value)? to preserve the owner.

An enum can keep behavior beside its variants. A method’s bare self is a shared borrow, so matching it borrows the payload. The same method can satisfy a structural protocol:

protocol Numbered:
    def number(self) -> i64:
        pass

enum Reading:
    Missing
    Value(i64)

    def number(self) -> i64:
        match self:
            case Reading.Missing:
                0
            case Reading.Value(value):
                *value

def number[T: Numbered](value: &T) -> i64:
    value.number()

def main() -> Result[(), IoError]:
    reading = Reading.Value(42)
    print(number(&reading))?
    print(reading.number())?
    Ok(())
42
42

Use pub def when callers in other modules need the method, and self: &mut Reading when it changes a payload. Variant and method names must be distinct. The formatting lesson uses an enum method to produce a message while preserving the error data.

Match combinations with tuple patterns

Match a tuple when the answer depends on several values together. Each component takes a pattern, and the cases cover combinations of components:

def combine(left: Option[i64], right: Option[i64]) -> Result[i64, OverflowError[i64]]:
    match (left, right):
        case (Some(a), Some(b)):
            Ok(a + b)
        case (Some(a), Nothing):
            Ok(a)
        case (Nothing, Some(b)):
            Ok(b)
        case (Nothing, Nothing):
            Ok(0)

def main() -> Result[(), IoError | OverflowError[i64]]:
    print(combine(Some(3), Some(4))?)?
    print(combine(Nothing, Some(5))?)?
    Ok(())
7
5

Leaving out (Nothing, Nothing) is an error: both inputs can be absent. Tuple patterns also nest inside enum payloads, such as Some((value, offset)). Write (pattern,) for a singleton tuple; (pattern) groups a pattern.

Matching an owned tuple transfers owned components into the case’s bindings and destroys ignored components. Match &value or &mut value to borrow its components, as in borrowed matching. The matching reference describes coverage and pattern limits.

Match literal values

A pattern can be an integer, string, or boolean literal. It matches a value equal to it, so match also selects on an integer, a str, or a bool directly, and on such a value inside an enum payload:

enum Command:
    Move(i64)
    Say(str)

def opcode(name: str) -> i64:
    match name:
        case "cpy":
            1
        case "inc" | "dec":
            2
        case _:
            0

def describe(command: Command) -> str:
    match command:
        case Command.Move(0):
            "stay"
        case Command.Move(_):
            "move"
        case Command.Say("" | "..."):
            "silence"
        case Command.Say(words):
            words

def main() -> Result[(), IoError]:
    print(opcode("cpy"))?
    print(opcode("dec"))?
    print(opcode("jmp"))?
    print(describe(Command.Move(0)))?
    print(describe(Command.Move(3)))?
    print(describe(Command.Say("...")))?
    print(describe(Command.Say("hello")))?
    Ok(())
1
2
0
stay
move
silence
hello

Cases are tried in order. Literals cannot name every integer or string, so such a match ends with case _:, and a payload position needs _ or a name after its literals:

def main() -> Result[(), IoError]:
    count = 3
    match count:
        case 1:
            print("one")?
    Ok(())
non-exhaustive match; cases cannot name every i64 value, so end with `case _:`

True and False together cover a bool, so no case _: follows them:

def label(enabled: bool) -> str:
    match enabled:
        case True:
            "on"
        case False:
            "off"

def main() -> Result[(), IoError]:
    print(label(True))?
    print(label(3 > 4))?
    Ok(())
on
off

An integer literal takes the type of the value it matches and must fit it: case 300: is an error for a u8. A float is not a pattern; compare floats with if. A bare name still binds only at a payload position, so the catch-all for the whole value is case _:, or case _ as name: to bind it.

Each literal case costs one comparison, tried from the top, exactly like an if/elif chain. Put the common cases first in a long match.

Bind the value a match leaves over

case _: covers the remaining values but names nothing. To handle a few variants and pass every other value on, end with case _ as name:, which binds the whole matched value:

enum Event:
    Tick
    Key(str)
    Resize(i64, i64)

def describe(event: Event) -> str:
    match event:
        case Event.Tick:
            "tick"
        case Event.Key(name):
            name
        case Event.Resize(_, _):
            "resize"

def without_ticks(event: Event) -> str:
    match event:
        case Event.Tick:
            "ignored"
        case _ as other:
            describe(other)

def main() -> Result[(), IoError]:
    print(without_ticks(Event.Tick))?
    print(without_ticks(Event.Key("enter")))?
    print(without_ticks(Event.Resize(80, 24)))?
    Ok(())
ignored
enter
resize

without_ticks keeps working when Event gains a variant; only describe must name it.

The binding follows the matched value. Matching an owned value moves it into the name; matching &value or &mut value binds a shared or exclusive reference to it:

def bump(count: &mut Option[i64]) -> Result[(), OverflowError[i64]]:
    match count:
        case Some(number):
            *number = *number + 1
        case _ as empty:
            *empty = Some(1)
    Ok(())

def main() -> Result[(), IoError | OverflowError[i64]]:
    mut count: Option[i64] = Nothing
    bump(&mut count)?
    bump(&mut count)?
    print(count)?
    Ok(())
Option[i64].Some(2)

_ as name matches every value, so it is the last case and stands alone: a case after it is unreachable, and it cannot be a | alternative. The _ as is required. A bare name is rejected, because it is usually a variant written without its enum:

enum Switch:
    On
    Off

def main() -> ():
    match Switch.On:
        case On:
            pass
        case _:
            pass
a case cannot bind the whole value to the bare name `On`; bind it with `_ as On`, match any value with `_`, or qualify a variant with its enum type

Inside a payload a name alone binds, as in Event.Key(name); _ as name is only for the whole value.

Bind the value a match selects

A match at the end of a function supplies the function’s result. To use the selected value in the middle of a function, write the match as the value of a binding. The final expression of the selected case becomes the value:

enum Shape:
    Dot
    Square(i64)
    Rect(i64, i64)

def paint_cost(shape: Shape) -> Result[i64, OverflowError[i64]]:
    area = match shape:
        case Shape.Dot:
            return Ok(1)
        case Shape.Square(side):
            side * side
        case Shape.Rect(width, height):
            width * height
    Ok(area * 10)

def main() -> Result[(), IoError | OverflowError[i64]]:
    print(paint_cost(Shape.Dot)?)?
    print(paint_cost(Shape.Square(3))?)?
    print(paint_cost(Shape.Rect(2, 5))?)?
    Ok(())
1
90
100

The cases that continue must produce one type. A case may instead leave with return, break, continue, or ?, as Shape.Dot does; it then supplies no value and nothing is bound.

The same block is the value of an assignment to a mut local, a field, or a collection element, and of return. An annotation gives every case its expected type, which Nothing needs:

def half(number: i64) -> Option[i64]:
    two: Positive[i64] = 2
    return match number % two:
        case 0:
            Some(number // two)
        case _:
            Nothing

def main() -> Result[(), AllocError | IoError]:
    mut halves = [0, 0]?
    for index in range(2):
        found: Option[i64] = match index:
            case 0:
                half(8)
            case _:
                Nothing
        halves[index] = match found:
            case Some(value):
                value
            case Nothing:
                -1
    print(halves)?
    print(half(3))?
    Ok(())
[4, -1]
Option[i64].Nothing

return match returns the final expression of the selected case, so nothing after the block runs.

The block must be the whole right-hand side. Inside a larger expression it is rejected; bind it first:

def main() -> Result[(), IoError]:
    number = 3
    doubled = 2 * match number:
        case 3:
            30
        case _:
            0
    print(doubled)?
    Ok(())
a `match` block is a value only as the whole right side of a binding, a single-target assignment, or `return`; bind its value to a name first

Cases that disagree are rejected at the case that differs:

def main() -> Result[(), IoError]:
    number = 3
    label = match number:
        case 3:
            "three"
        case _:
            0
    print(label)?
    Ok(())
expected str, got i64

The reference lists the remaining rules, including why the value cannot be a reference.

Creating enum values

Payload arguments move into the enum. Constructing it adds no allocation:

enum Message:
    Empty
    Text(str)

def main() -> Result[(), IoError]:
    print(Message.Text("hello"))?
    print(Message.Empty)?
    Ok(())
Message.Text("hello")
Message.Empty

Allocations inside argument expressions, such as building a string, still return their own Result.

Match borrowed values

Use match &value to inspect an enum while retaining its owner. Payload bindings are shared references. With match &mut value, they are exclusive references and can change the original payload.

An existing enum reference can be matched directly. A function can also return a payload reference when it comes from the function’s first reference parameter:

def number(value: &Result[i64, i64]) -> &i64:
    match value:
        case Ok(payload):
            payload
        case Err(payload):
            payload

def main() -> Result[(), IoError]:
    mut outcome: Result[i64, i64] = Err(7)
    selected = number(&outcome)
    print(*selected)?
    outcome = Ok(9)
    print(number(&outcome))?
    Ok(())
7
9

Replacing outcome is allowed after selected’s last use; doing it before that use is rejected. References cannot outlive their owner.

Assign through a mutable payload reference to change the original slot, keeping its enclosing variants:

def main() -> Result[(), IoError | OverflowError[i64]]:
    mut nested: Option[Result[i64, str]] = Some(Ok(4))
    match &mut nested:
        case Some(Ok(number)):
            *number = *number + 1
        case Some(Err(_)) | Nothing:
            pass
    print(nested)?
    Ok(())
Option[Result[i64, str]].Some(Result[i64, str].Ok(5))

match &mut works for every enum, whatever its payloads contain. A copy made before the match keeps its own payload:

enum Shape:
    Circle(i64)
    Square(i64)

def main() -> Result[(), IoError | OverflowError[i64]]:
    mut shape = Shape.Circle(3)
    before = shape
    match &mut shape:
        case Shape.Circle(radius):
            *radius = *radius + 1
        case Shape.Square(side):
            *side = *side * 2
    print(shape)?
    print(before)?
    Ok(())
Shape.Circle(4)
Shape.Circle(3)

_ ignores a payload without moving it. Borrowed matching uses the same exhaustiveness checks and qualified variant names as owned matching.

Clean up owned instances

Use __del__ for cleanup that belongs to an owned instance. It runs automatically when the instance leaves scope, is replaced, or is explicitly dropped. Its bare self is an exclusive reference, like in __init__:

class Resource:
    name: str

    def __del__(self) -> ():
        drop(print("release", self.name))

class Pair:
    first: Resource
    second: Resource

    def __del__(self) -> ():
        drop(print("release pair"))

def work() -> Result[(), IoError]:
    pair = Pair(Resource("first"), Resource("second"))
    spare = Resource("spare")
    print("working")?
    Ok(())

def main() -> Result[(), IoError]:
    work()?
    print("done")?
    Ok(())
working
release spare
release pair
release first
release second
done

Locals clean up in reverse declaration order. A class’s __del__ runs first; its fields then clean up automatically in declaration order. Moving an instance transfers cleanup responsibility without running the destructor.

Each field finishes cleaning up its children before the next field begins:

class Leaf:
    name: str
    def __del__(self) -> ():
        drop(print(self.name))
class Pair:
    first: list[Leaf]
    second: Leaf
def main() -> Result[(), AllocError]:
    pairs = [Pair([Leaf("a")]?, Leaf("b"))]?
    drop(pairs)
    Ok(())
a
b

Lifecycle methods are not called directly. A class with custom cleanup, or one containing a value with custom cleanup, cannot be copied yet. A destructor cannot yield or report a recoverable error; use an explicit closing method returning Result when failures matter. Fatal traps do not run cleanup.

The lifecycle trace uses drop(print(...)) deliberately: cleanup cannot return an error, and a failed diagnostic write must not interrupt cleanup. For output that must succeed, provide an explicit fallible operation and propagate its result before leaving the scope.

Consume a value to change its state

Use self: Class when an operation finishes an object or replaces it with a different state. The method receives ownership, so the caller cannot use the old class instance again. Bare self still borrows; self: &mut Class mutates an instance that remains available to the caller.

Here a draft must be validated before it can be finished. Separate classes make invalid operation ordering a type error. Keeping Ready’s field and constructor private prevents callers from skipping validation:

pub class Draft:
    pub text: str

    pub def validate(self: Draft) -> Result[Ready, Draft]:
        if self.text == "":
            return Err(self)
        Ok(Ready(self.text))

pub class Ready:
    text: str

    pub def finish(self: Ready) -> str:
        self.text
import workflow

def main() -> Result[(), IoError]:
    draft = workflow.Draft("report")
    ready = draft.validate().unwrap()
    print(ready.finish())?
    # An error can explicitly return the draft for correction and retry.
    match workflow.Draft("").validate():
        case Ok(ready):
            print(ready.finish())?
        case Err(retry):
            mut corrected = retry
            corrected.text = "corrected"
            print(corrected.validate().unwrap().finish())?
    Ok(())
report
corrected

The unwrap calls assert that the nonempty strings in this example pass validation. For input that might fail, recover the owner using match, as the empty-draft example does. A fallible method does not automatically restore the receiver after failure. The same rule applies to a simple finalization method:

class Message:
    text: str

    def finish(self: Message) -> str:
        self.text

def main() -> ():
    message = Message("report")
    text = message.finish()
    print(message.text).unwrap()
    drop(text)
use of moved binding `message`

Ownership moves into the method before its other arguments are evaluated. Read any needed fields into locals before making the consuming call. If a later argument returns early with ?, the pending receiver is cleaned up. Inside the method, owned self is an immutable binding; move it into a mut local if you need to change fields.

This example reads a copyable string field. A transition can also move an owned field, such as a list, out of the consumed receiver into the next state:

class Pending:
    id: i64
    items: list[str]

    def ship(self: Pending) -> Shipped:
        Shipped(self.id, self.items)

class Shipped:
    id: i64
    items: list[str]

def main() -> Result[(), AllocError | IoError]:
    order = Pending(7, ["book", "pen"]?)
    shipped = order.ship()
    print(shipped.id, len(shipped.items))?
    Ok(())
7 2

self.items transfers the list without copying it, and the rest of self is cleaned up when the method returns. After a field moves out, the object cannot be used as a whole until that field is assigned again:

class Pending:
    items: list[str]

    def ship(self: Pending) -> Pending:
        items = self.items
        drop(items)
        self

def main() -> ():
    pass
use of partly moved binding `self`: `self.items` is moved; assign `self.items` before using `self` as a whole

Fields cannot move out through a reference, or out of a class that defines __del__, because its hook receives the whole object. There, keep the field initialized with Option plus replace. See moving fields for the full rules.

Consuming a value does not require callers to finish it: they can still drop it. Normal cleanup runs exactly once unless the method transfers ownership onward. Enums also accept by-value receivers, but a copyable enum is copied according to ordinary argument rules. Use affine state types when the old state must cease to be usable. See method contracts.

Borrow records in a collection

For a list of records, &items yields shared references and &mut items yields mutable references. The list cannot grow, shrink, or move during either loop:

class Score:
    value: i64

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    mut scores = [Score(2), Score(5)]?
    for score in &mut scores:
        score.value = score.value + 1
    print([score.value for score in &scores]?)?
    mut numbers = [10, 20]?
    for number in &mut numbers:
        *number = *number + 2
    print(numbers)?
    Ok(())
[3, 6]
[12, 22]

For large primitive-field records scanned a few fields at a time, consider columnar storage. Ordinary lists retain the full record-borrowing interface used here.

Heterogeneous collections

A heterogeneous collection holds values of different kinds. Plenty still gives it one element type: an enum or an anonymous sum describes the alternatives. The possible types are known at compile time.

Use the classes and enums you have defined to choose a representation:

CollectionRepresenting different kinds of data
ListsUse an enum wrapper for a named interface, or `list[A
SetsUse an enum or anonymous sum whose alternatives are supported keys.
DictionariesUse enums or anonymous sums for mixed keys, values, or both.

An anonymous sum, written A | B, holds one value of either type. It is not a set collection: set[T] holds distinct values. An enum gives its alternatives names and can define methods; an anonymous sum needs a match to use each member’s own interface.

These techniques do not create an open-ended collection accepting any type. Adding a new kind means extending the enum or sum and handling it wherever matching must be exhaustive. Protocols constrain generic calls; they are not runtime element types such as list[MyProtocol].

Lists of different kinds of values

A list has one element type, but that type can describe alternatives. Use a wrapper enum when the alternatives form a concept in your program and share operations. Use an anonymous sum when a local choice of whole types is enough.

Give the alternatives a shared interface

A message and a status can both appear in a feed. FeedItem names that concept and keeps the dispatch in its label method:

class Message:
    text: str

    def label(self) -> str:
        self.text

enum Status:
    Ready
    Busy

    def label(self) -> str:
        match self:
            case Status.Ready:
                "ready"
            case Status.Busy:
                "busy"

enum FeedItem:
    Message(Message)
    Status(Status)

    def label(self) -> str:
        match self:
            case FeedItem.Message(message):
                message.label()
            case FeedItem.Status(status):
                status.label()

def main() -> Result[(), AllocError | IoError]:
    mut items: list[FeedItem] = []?
    items.append(FeedItem.Message(Message("hello")))?
    items.append(FeedItem.Status(Status.Ready))?
    for item in &items:
        print(item.label())?
    Ok(())
hello
ready

The wrapper can carry class instances, enum values, or other owned data. Constructing a variant moves an affine payload into it; appending moves that wrapper into the list. The loop borrows the list, and matching self borrows the active payload, so neither operation needs a copy.

Each kind has its own implementation of label, but the wrapper’s method is what lets the loop call one interface. No protocol declaration is necessary. Adding a variant makes the compiler check that the wrapper’s match handles it.

Use an anonymous sum for a local choice

If you do not need a named wrapper or its methods, declare the alternatives in the list’s element type. The annotation lets append accept either member directly:

class Message:
    text: str

enum Status:
    Ready
    Busy

def main() -> Result[(), AllocError | IoError]:
    mut items: list[Message | Status] = []?
    items.append(Message("hello"))?
    items.append(Status.Ready)?
    for item in &items:
        match item:
            case message: Message:
                print(message.text)?
            case status: Status:
                match status:
                    case Status.Ready:
                        print("ready")?
                    case Status.Busy:
                        print("busy")?
    Ok(())
hello
ready

case message: Message: selects a whole member type. Because item is a borrow, its matched payload is borrowed too. The Status member contains any of that enum’s variants, so its own match handles both.

The sum does not automatically forward methods, even if every member has a method with the same signature. Match to select a member, or choose the wrapper pattern when several callers need the same operation.

Both forms store a tag and payload inline; wrapping adds no heap allocation. List creation and growth still return allocation results, as does constructing any payload that itself needs storage. See anonymous sums and enums for the full contracts.

For implementations supplied independently of a fixed list of alternatives, see protocol objects. Each object uses an explicit heap allocation and exposes the methods of one protocol.

Sets and different kinds of values

Sets accept named enums when every payload is a supported key type: integers, bool, str, bytes, or tuples, classes, enums and anonymous sums of those types. The variant is part of the identity, so different variants with equal payloads remain distinct.

enum Selection:
    Account(i64)
    Group(str)
    All

def main() -> Result[(), AllocError | IoError]:
    mut selected: set[Selection] = set()?
    selected.add(Selection.Account(7))?
    selected.add(Selection.Group("7"))?
    selected.add(Selection.Account(7))?
    print(len(selected))?
    print(Selection.Account(7) in selected)?
    Ok(())
2
True

This is one element type with several alternatives. Enums carrying collections, floats or boxes cannot be set elements. Anonymous sums qualify when every member is a supported key type.

Class payloads

A wrapper can also carry classes whose fields are supported keys. Ordinary classes move into the set; use a borrowed probe to keep a lookup value:

class Account:
    region: str
    number: i64

enum Selection:
    Account(Account)
    Group(str)

def main() -> Result[(), AllocError | IoError]:
    selected = {Selection.Account(Account("eu", 7)), Selection.Group("editors") }?
    probe = Selection.Account(Account("eu", 7))
    print(&probe in selected)?
    for item in &selected:
        match item:
            case Selection.Account(account):
                print(account.region, account.number)?
            case Selection.Group(group):
                print(group)?
    Ok(())
True
eu 7
editors

The lookup compares fields rather than object addresses. Borrowed iteration keeps the stored keys immutable. For keys without custom cleanup, set algebra creates independent key copies; its result can outlive either input. See sets.

Keep each kind in its own set

Suppose a selection contains numeric account IDs and textual group names. Separate sets preserve both kinds without converting IDs into strings:

def main() -> Result[(), AllocError | IoError]:
    mut account_ids: set[i64] = set()?
    mut group_names: set[str] = set()?
    account_ids.add(7)?
    account_ids.add(7)?
    group_names.add("editors")?
    group_names.add("7")?
    print(len(account_ids), len(group_names))?
    print(7 in account_ids, "7" in group_names)?
    Ok(())
1 2
True True

The integer 7 and the name "7" remain distinct identities. Each set removes its own duplicates and supports the ordinary set operations. Group the two sets in a class if they need to travel together.

Use an anonymous sum

When the types themselves express the alternatives, no wrapper is needed:

def main() -> Result[(), AllocError | IoError]:
    mut selected: set[i64 | str] = {7, "editors"}?
    selected.add("7")?
    selected.add(7)?
    print(len(selected), 7 in selected, "7" in selected)?
    Ok(())
3 True True

The member type is part of each key’s identity. Every member must support structural hashing; set[i64 | f64] is rejected because floats are not keys. Use a named enum when alternatives need different meanings despite sharing a payload type. Anonymous sums use hash sets, including sums of finite enums.

Small enums without payloads

Use ordinary sets for flags or selections too. An enum with no payloads and at most 256 variants automatically gets compact bitset storage. The compiler selects a 64-, 128-, or 256-bit mask from the number of possible values:

enum Permission:
    Read
    Write
    Execute

def main() -> Result[(), AllocError | IoError]:
    mut granted = {Permission.Read}?
    granted.add(Permission.Write)?
    required = {Permission.Read, Permission.Write}?
    print(required.issubset(&granted))?
    granted.discard(Permission.Write)
    print(Permission.Write in granted)?
    Ok(())
True
False

Keep the usual ? handling: constructing a set still allocates its owner. Adding members to this representation needs no further allocation. The same programming style works for larger enums and other payload types, which use hash sets. See small enum bitsets.

Small finite payloads

Payloads can use bitsets too when the complete type has at most 256 possible values. Booleans and nested enums or tuples of these types qualify. Here State has three values: Idle, Enabled(False), and Enabled(True).

enum State:
    Idle
    Enabled(bool)

def main() -> Result[(), AllocError | IoError]:
    mut allowed = {State.Idle, State.Enabled(False)}?
    allowed.add(State.Enabled(True))?
    print(len(allowed), State.Enabled(True) in allowed)?
    Ok(())
3 True

The limit counts all possible payload combinations, regardless of how many members the set currently contains. For example, eight boolean fields have 256 combinations; adding a separate payload-free variant makes 257 and selects a hash set. Integer and string payloads use hash sets. No special container or manual integer encoding is needed.

Keys with custom cleanup

A key may define __del__, or contain a value that does. Its fields must still be supported key types. The collection owns the inserted key and runs cleanup when that owner is removed or destroyed. Borrow keys for lookup and inspection; transfer them when a second owner cannot be created.

This example logs cleanup to make ownership visible. It unwraps diagnostic printing inside the destructor because cleanup cannot return an error:

class Ticket:
    id: i64

    def __del__(self) -> ():
        print("released", self.id).unwrap()

def main() -> Result[(), AllocError | IoError]:
    mut selected = {Ticket(7)}?
    additional = {Ticket(8)}?
    selected.update(additional)?
    print(len(selected))?
    for ticket in &selected:
        print(ticket.id)?
    # Transfer the owners out of the set before destroying them.
    for ticket in selected:
        drop(ticket)
    Ok(())
2
7
8
released 7
released 8

Iteration order is unspecified. update consumes the source set; an equal incoming key is destroyed while the stored owner is retained. Use intersection_update and difference_update to filter in place, and relationship checks such as issubset to compare without copying.

union, intersection, difference, symmetric_difference, copy(selected) and dictionary keys().to_set() require independent key copies, so they reject custom cleanup, including cleanup nested inside an enum or anonymous sum. A borrowed keys() view still works. To keep dictionary keys after consuming the dictionary, use into_keys(); see dictionary snapshots and views.

Dictionaries with different kinds of values

A dictionary has one key type and one value type. The value type can be an enum wrapper or an anonymous sum, so a string key can select any of several kinds of stored data. Keys retain the narrower key-type contract: integers, bool, str, bytes, or tuples, classes, enums and anonymous sums recursively containing those types. Keys with custom cleanup can be borrowed through keys() or transferred into an owned list with into_keys(); they cannot be copied with keys().to_set(). See cleanup-bearing keys.

Give mixed values a named wrapper

An enum makes the kinds of settings explicit. Here each value carries either a retry policy class or a mode enum:

class RetryPolicy:
    attempts: i64

enum Mode:
    Fast
    Careful

enum Setting:
    Retries(RetryPolicy)
    Mode(Mode)

def main() -> Result[(), AllocError | IoError]:
    mut settings: dict[str, Setting] = {}?
    settings.insert("network", Setting.Retries(RetryPolicy(3)))?
    settings.insert("processing", Setting.Mode(Mode.Careful))?
    for name, setting in settings.items():
        match setting:
            case Setting.Retries(policy):
                print(*name, policy.attempts)?
            case Setting.Mode(mode):
                print(*name, *mode)?
    Ok(())
network 3
processing Mode.Careful

insert adds or replaces an entry and handles possible allocation with ?. The dictionary owns each wrapper and its payload. items() borrows them in insertion order, so inspecting a class payload does not copy or remove it. A wrapper can also provide shared methods, as in the list example.

For a lookup that may miss, get requires a copyable value: this Setting contains an affine class, so get cannot return it. Use get_mut to borrow an optional value for an update, or pop to remove and take ownership of it. See dictionary lookup and removal.

Use a sum for simple values

An anonymous sum avoids declaring variants when the whole types express the choice. The annotation supplies the expected type for both a literal and later insertions:

def main() -> Result[(), AllocError | IoError]:
    mut settings: dict[str, i64 | str] = {"retries": 3}?
    settings.insert("mode", "careful")?
    for name, value in settings.items():
        match value:
            case count: i64:
                print(*name, *count)?
            case text: str:
                print(*name, *text)?
    Ok(())
retries 3
mode careful

The sum admits only the listed types. If two alternatives have the same payload type but different meanings, such as seconds and attempts both stored as i64, use separate named enum variants instead: i64 | i64 is just i64.

Different kinds of keys

An anonymous sum can combine supported key types without merging their identities:

def main() -> Result[(), AllocError | IoError]:
    labels: dict[i64 | str, str] = {7: "numeric account", "7": "named account"}?
    print(labels[7], labels["7"])?
    Ok(())
numeric account named account

A named enum can represent these distinct key domains while preserving their identity:

enum Key:
    Id(i64)
    Name(str)

def main() -> Result[(), AllocError | IoError]:
    labels: dict[Key, str] = {
        Key.Id(7): "numeric account",
        Key.Name("7"): "named account",
    }?
    print(labels[Key.Id(7)], labels[Key.Name("7")])?
    Ok(())
numeric account named account

Every enum payload must itself be a supported key type. You can also keep separate dictionaries when numeric IDs and textual names are different lookup domains:

def main() -> Result[(), AllocError | IoError]:
    by_id: dict[i64, str] = {7: "numeric account"}?
    by_name: dict[str, str] = {"7": "named account"}?
    print(by_id.get(7).unwrap(), by_name.get("7").unwrap())?
    Ok(())
numeric account named account

These lookups unwrap because the example has just inserted both keys. For external input, match the Option or supply an intentional default instead. Do not merge distinct identities by converting every key to a string.

Different kinds on both sides

Keys and values can independently use sums. Only the key alternatives must satisfy the key-type contract:

def main() -> Result[(), AllocError | IoError]:
    mut settings: dict[i64 | str, i64 | str] = {7: "paused", "retries": 3}?
    settings.insert("mode", "careful")?
    print(settings[7], settings["retries"], settings["mode"])?
    Ok(())
"paused" 3 "careful"

The annotations provide the expected types for insertion and lookup. The integer key 7 remains distinct from the string key "7". Use a named enum when domain-specific variant names make the meaning clearer.

For implementations supplied independently of a fixed list of alternatives, see protocol objects. Each object uses an explicit heap allocation and exposes the methods of one protocol.

Special-purpose data structures

Choose a data structure to suit the operations your program performs most often. This section introduces collections with specialized storage and access patterns, along with the tradeoffs that make them useful.

Start with Columnar lists: keep an ordinary class-shaped interface while storing each primitive field in its own contiguous column.

For analytical queries and Parquet datasets, use Dataframes: explicit expression plans executed by an optional Polars engine. Columnar lists remain the lightweight mutable collection.

Query dataframes with Polars

Use std.dataframe when your task is an analytical query over columns or Parquet files. Plenty describes the query; a separately compiled Polars engine optimizes and executes it. This exposes native columnar processing, parallelism and SIMD without asking you to manage vector instructions. Linux binary releases include the engine beside the compiler; no local Polars build is needed. When distributing a compiled application, follow shipping dataframe applications to include its shared library and notices. These examples require it when compiled and run.

clist[T] remains useful for lightweight mutable records, a compile-time class schema and ordinary Plenty computations. A dataframe has a runtime schema and native query semantics. It is a different tool for larger analytical operations, not the default replacement for a collection.

Build a query

Copy numeric lists into named columns, create a frame, and build expressions. An expression is a native query description, not a Plenty function called once per row. ? handles query errors, separately from null values. Borrowed parameters accept values directly, so explicit & is optional here.

import std.dataframe as df

def main() -> Result[(), df.DataFrameError | AllocError | IoError]:
    readings = [18.0, 23.0, 29.0]?
    columns = [df.column_f64("temperature", readings)?]?
    data = df.frame(columns)?
    temperature = df.col("temperature")?
    warm = temperature.gt(df.lit(20.0)?)?
    aggregates = [temperature.mean()?.alias("average")?]?
    result = data.lazy()?.filter(warm)?.select(aggregates)?.collect()?
    print(result.get_f64("average", 0)?)?
    Ok(())
Option[f64].Some(26.0)

Column construction copies values in bulk. Plans borrow and share native buffers; chaining operations does not copy the whole dataset at every step. collect() executes the plan. A getter returns Result[Option[T], ...]: Nothing means a null cell, while Err means the operation failed. Bulk values_f64 and values_i64 return ordinary Plenty lists, but reject columns containing nulls. Their _nullable variants return Option elements. Strings and booleans have matching typed APIs; see Types, nulls and schemas.

Derive columns and group

Select explicit expressions and names. Group order is unspecified, so this example prints the number of groups rather than assuming which comes first.

import std.dataframe as df

def main() -> Result[(), df.DataFrameError | AllocError | IoError]:
    stations = [1, 2, 1]?
    readings = [10.0, 20.0, 30.0]?
    columns = [df.column_i64("station", stations)?, df.column_f64("reading", readings)?]?
    data = df.frame(columns)?
    derived = [df.col("reading")?.mul(df.lit(2.0)?)?.alias("adjusted")?]?
    keys = [df.col("station")?]?
    totals = [df.col("adjusted")?.sum()?.alias("total")?]?
    result = data.lazy()?.with_columns(derived)?.group_by(keys, totals)?.collect()?
    print(result.height()?)?
    data.write_parquet("readings.parquet")?
    restored = df.scan_parquet("readings.parquet")?.collect()?
    print(restored.values_f64("reading")?)?
    Ok(())
2
[10.0, 20.0, 30.0]

For an existing Parquet file, start with scan_parquet, add filters and projections, then collect. Polars can push those operations into the scan. The write above creates or overwrites its file.

Native query semantics differ from Plenty scalar arithmetic: nulls and NaN are distinct, NaN ordering differs, integer overflow is not checked like Plenty arithmetic, and parallel floating-point reductions can change rounding. Native allocation failure can abort the process. Read the reference contracts before using these operations where those distinctions matter.

Continue with Join and rank results to combine datasets, order results inside Polars and select a row window. For cleaning text, conditional values and per-group comparisons, see Clean and compare column values. Query CSV and write Parquet adds file-backed pipelines and explains when to collect or write directly.

Types, nulls and schemas

A missing reading differs from a reading of zero. Use Option elements to preserve that distinction, and keep names and flags in their native string and boolean columns.

This example keeps enabled sensors and reads their nullable values back. The expressions run in Polars; ordinary Plenty code handles the results. It needs the dataframe engine, which Linux releases include.

import std.dataframe as df

def main() -> Result[(), df.DataFrameError | AllocError | IoError]:
    names = ["north", "café", "南"]?
    enabled = [True, False, True]?
    readings: list[Option[f64]] = [Some(18.5), Some(22.0), Nothing]?
    data = df.frame([
        df.column_str("sensor", names)?,
        df.column_bool("enabled", enabled)?,
        df.column_f64_nullable("reading", readings)?
    ]?)?
    result = data.lazy()?.filter(df.col("enabled")?)?.collect()?
    print(result.values_str("sensor")?)?
    print(result.values_f64_nullable("reading")?)?
    for field in result.schema()?:
        print(field.name, field.dtype)?
    Ok(())
["north", "南"]
[Option[f64].Some(18.5), Option[f64].Nothing]
sensor std.dataframe.DType.String
enabled std.dataframe.DType.Bool
reading std.dataframe.DType.F64

Nothing is missing data; Some(""), Some(False), zero and NaN are present values. Use values_TYPE_nullable for nullable results. The plain values_TYPE methods reject nulls rather than dropping rows. Scalar get_TYPE(name, row) returns an Option inside a Result: absence and query failure are different outcomes.

For an unfamiliar file, inspect df.scan_parquet(path)?.schema()? before choosing columns or collecting. Its ordered fields describe the native runtime schema. DType.Other(description) reports a native type that Plenty cannot yet extract, such as an unsupported nested value. Inspection reads metadata as needed; collection can still encounter errors.

Keep numeric widths and unsigned identifiers

Every Plenty numeric type has matching column constructors and getters. Choose the type your data needs: a u64 identifier can exceed i64.MAX, while f32 readings use less native column memory than f64 readings. Neither needs a conversion through the default numeric types.

import std.dataframe as df

def main() -> Result[(), df.DataFrameError | AllocError | IoError]:
    data = df.frame([
        df.column_u64("device", [u64.MAX, 42u64]?)?,
        df.column_f32("reading", [18.5f32, 22.0f32]?)?
    ]?)?
    selected = data.lazy()?.filter(df.col("device")?.eq(df.lit_u64(u64.MAX)?)?)?.collect()?
    print(selected.values_u64("device")?)?
    print(selected.values_f32("reading")?)?
    Ok(())
[18446744073709551615]
[18.5]

Use column_TYPE_nullable and values_TYPE_nullable when values may be missing. Extraction checks the exact dtype; it does not widen integers or floats implicitly. An expression or aggregation can change the result’s dtype, so inspect its schema or make a deliberate cast before extracting.

Lists are copied across the native boundary. Keep filtering, grouping and sorting inside the query, then extract only the columns and rows your Plenty code needs. See the type contracts for Unicode, ownership and conversion details, then continue with Join and rank results or Clean and compare column values. For logical calendar data, see Dates and daily reports.

Store binary payloads

Use binary columns for file contents, encoded messages, or other bytes that are not necessarily text. Values use owned Bytes buffers; there is no UTF-8 conversion at the dataframe boundary. This lesson requires the optional engine.

Keep selection inside Polars, then extract only the payloads your application needs. Here, null means missing data and an empty buffer is a present payload:

import std.dataframe as df
import std.bytes as buffers
from std.bytes import Bytes, BytesError

def main() -> Result[(), df.DataFrameError | BytesError | AllocError | IoError]:
    payloads: list[Option[Bytes]] = [
        Some(buffers.from_bytes(b"\xff\0")?),
        Nothing,
        Some(buffers.empty()?)
    ]?
    data = df.frame([df.column_binary_nullable("payload", payloads)?]?)?
    matches = data.lazy()?.filter(
        df.col("payload")?.eq(df.lit_binary(b"\xff\0")?)?
    )?.collect()?
    print(matches.height()?)?
    for payload in matches.values_binary("payload")?:
        print(payload.len(), payload[0])?
    print(data.get_binary("payload", 1)?.is_none())?
    print(data.get_binary("payload", 2)?.unwrap().len())?
    Ok(())
1
2 255
True
0

Use values_binary_nullable when the selected result may contain nulls. values_binary rejects nulls; it never silently drops rows. Both return new buffers that can outlive the dataframe and be changed independently.

write_parquet and scan_parquet preserve binary values and nulls. CSV does not support binary columns. Converting a binary expression to DType.String is explicit and checks UTF-8; it is unnecessary for writing Parquet.

Binary literals accept a byte literal or borrowed view directly, including &buffer[a:b]. Column construction packs values into a temporary Bytes buffer and copies them into Polars; extraction returns independent buffers. Prefer native filtering/projection before extraction, especially for large payloads. See binary transfer details.

Lists and records inside columns

A row can contain several values or a record of related fields. Keep these values in native columns when queries will process them. Nested columns compose with the same expressions and Parquet I/O as scalar columns.

Store a list per row

Build one flat child column and supply the boundaries between rows. In this example the rows contain [8, 5], [], and [3].

import std.dataframe as df

def main() -> Result[(), df.DataFrameError | AllocError | IoError]:
    values = df.column_i64("item", [8, 5, 3]?)?
    scores = df.column_list("scores", values, [0, 2, 2, 3]?)?
    report = scores.to_frame()?.lazy()?.select([
        df.col("scores")?.list_len()?.alias("count")?,
        df.col("scores")?.list_get(-1)?.alias("last")?
    ]?)?.collect()?
    print(report.values_i64("count")?)?
    print(report.values_i64_nullable("last")?)?
    Ok(())
[2, 0, 1]
[Option[i64].Some(5), Option[i64].Nothing, Option[i64].Some(3)]

list_get returns null for an out-of-bounds index. Use nullable extraction when an empty or null list can reach the query. A null child element also remains null; it is not replaced with zero.

column_list_nullable additionally accepts a boolean per row. False marks a null list; its offsets must describe an empty range. True with an empty range is an empty list. These states remain distinct through Parquet.

Group fields into records

A child dataframe supplies the fields of a struct column. Select a field with struct_field; it stays a native expression.

import std.dataframe as df

def main() -> Result[(), df.DataFrameError | AllocError | IoError]:
    fields = df.frame([
        df.column_str("name", ["Ada", "Lin"]?)?,
        df.column_i64("score", [8, 5]?)?
    ]?)?
    people = df.column_struct("person", fields)?.to_frame()?
    result = people.lazy()?.select([
        df.col("person")?.struct_field("name")?.alias("name")?
    ]?)?.collect()?
    print(result.values_str("name")?)?
    for node in people.schema_tree()?:
        print(node.parent, node.name, node.dtype)?
    Ok(())
["Ada", "Lin"]
-1 person std.dataframe.DType.Struct
0 name std.dataframe.DType.String
0 score std.dataframe.DType.I64

The schema tree uses parent indexes: -1 is a top-level column, and both fields above belong to node 0. Lists introduce an item child, which may itself be a struct or another list.

When Plenty needs the values, frame.column(name)?.list_parts() returns native child values plus owned offsets and parent validity. Its child column is named item: use parts.values.to_frame()?.values_i64_nullable("item") for nullable integer elements. struct_parts() similarly returns a child dataframe and the parent mask. Prefer native expressions for filtering or aggregation; extracting parts is for application code that actually needs them.

Read the nested-column contracts for ownership, null semantics, reconstruction and Polars’ Parquet map representation.

Join and rank dataframe results

An order table and a price table usually have different shapes. Join them by product, derive revenue, then group and rank the results. These operations stay inside the optional Polars engine until collect().

Methods borrow their inputs automatically, including temporary expressions. The example uses named intermediate plans to keep the steps readable; each step still belongs to one lazy query.

import std.dataframe as df

def main() -> Result[(), df.DataFrameError | AllocError | IoError]:
    orders = df.frame([df.column_i64("product", [1, 2, 1, 3]?)?, df.column_i64("units", [2, 1, 3, 5]?)?]?)?
    prices = df.frame([df.column_i64("product", [1, 2]?)?, df.column_f64("price", [10.0, 25.0]?)?]?)?
    keys = [df.col("product")?]?
    joined = orders.lazy()?.join(prices.lazy()?, keys, keys, how=df.JoinKind.Left)?
    revenue = df.col("price")?.fill_null(df.lit(0.0)?)?.mul(df.col("units")?)?.alias("revenue")?
    enriched = joined.with_columns([revenue]?)?
    totals = enriched.group_by(keys, [df.col("revenue")?.sum()?.alias("total")?]?)?
    ranking = [df.SortKey(df.col("total")?, descending=True), df.SortKey(df.col("product")?)]?
    best = totals.sort(ranking)?.limit(2)?.collect()?
    print(best.values_i64("product")?)?
    print(best.values_f64("total")?)?
    Ok(())
[1, 2]
[50.0, 25.0]

The left join keeps product 3 even though it has no price. This example deliberately treats a missing price as zero with fill_null; a real application may instead reject or separately report unpriced orders. The default inner join would drop unmatched orders.

Duplicate keys produce every matching pair. The price table here has one row per product; duplicate price rows would multiply revenue contributions. The join does not validate uniqueness. Null keys do not match unless you pass nulls_equal=True.

Both key columns survive the join. Since both are named product, the right one becomes product_right; use suffix="_price" to choose another suffix. A suffix collision is an error, never silent replacement.

The ranking sorts totals descending, then product ascending to break ties. Nulls go last by default; each SortKey can specify nulls_last=False. Sorting is stable for equal keys, but a join or group has no promised input order, so use a meaningful tie-breaking key when you need reproducible output.

For pagination, replace limit(2) with slice(2, 2) to skip two rows and take up to two more. Dataframe slicing takes an offset and count, not start/stop endpoints. Negative offsets count from the end; invalid negative lengths return an error. See the query contracts.

Continue with Membership and full joins to filter by another dataset or reconcile unmatched rows from both sides.

Membership and full joins

Use a semi join to keep rows whose keys appear in another dataset, or an anti join to find keys that are absent. Both return only the left columns. They are useful for checking allowed IDs, finding missing records and filtering against a lookup table without multiplying rows.

These examples require the optional engine.

Keep matching rows or find missing ones

import std.dataframe as df

def main() -> Result[(), df.DataFrameError | AllocError | IoError]:
    orders = df.frame([
        df.column_i64("id", [1, 2, 2, 3]?)?,
        df.column_str("order", ["one", "two-a", "two-b", "three"]?)?
    ]?)?
    allowed = df.frame([df.column_i64("id", [2, 2]?)?]?)?
    keys = [df.col("id")?]?
    order = [df.SortKey(df.col("order")?)]?
    matching = orders.lazy()?.join(
        allowed.lazy()?, keys, keys, how=df.JoinKind.Semi
    )?.sort(order)?.collect()?
    missing = orders.lazy()?.join(
        allowed.lazy()?, keys, keys, how=df.JoinKind.Anti
    )?.sort(order)?.collect()?
    print(matching.values_str("order")?)?
    print(missing.values_str("order")?)?
    Ok(())
["two-a", "two-b"]
["one", "three"]

The duplicate allowed ID does not duplicate matching orders. Each left row is kept at most once; duplicate left rows would still be separate rows. No right-side columns or suffixes appear in the result.

Null keys do not match by default, so a null left key goes into the anti result. Set nulls_equal=True only when null should be an ordinary matching key. Sorting above makes presentation deterministic.

Reconcile both sides with a full join

A full join also keeps rows found only on the right. Both key columns remain, so explicitly derive a common display key if that is what your report needs.

import std.dataframe as df

def main() -> Result[(), df.DataFrameError | AllocError | IoError]:
    before = df.frame([
        df.column_i64("id", [1, 2]?)?,
        df.column_i64("price", [10, 20]?)?
    ]?)?
    after = df.frame([
        df.column_i64("id", [2, 3]?)?,
        df.column_i64("price", [25, 30]?)?
    ]?)?
    keys = [df.col("id")?]?
    joined = before.lazy()?.join(after.lazy()?, keys, keys, how=df.JoinKind.Full)?
    display_id = df.col("id")?.fill_null(df.col("id_right")?)?.alias("item")?
    result = joined.with_columns([display_id]?)?.sort([
        df.SortKey(df.col("item")?)
    ]?)?.collect()?
    print(result.values_i64("item")?)?
    print(result.values_i64_nullable("price")?)?
    print(result.values_i64_nullable("price_right")?)?
    Ok(())
[1, 2, 3]
[Option[i64].Some(10), Option[i64].Some(20), Option[i64].Nothing]
[Option[i64].Nothing, Option[i64].Some(25), Option[i64].Some(30)]

Full joins preserve duplicate matching pairs, just like inner and left joins. They do not verify key uniqueness. The nulls above describe missing sides; they are not query failures. See the join contracts for composite keys, null matching, name collisions and ordering.

Continue with Match nearby observations when a lookup should use the most recent or nearest key rather than an identical key.

Match nearby observations

An equality join requires identical keys. An as-of join instead finds the most recent, next or nearest observation. This is useful for attaching a calibration value to sensor readings or a price to an event.

Use integer keys with a unit your application defines. Here, keys represent elapsed seconds. They are ordinary i64 values, not a date/time type. These examples require the optional dataframe engine. As-of validation materializes both inputs when the query executes, including with collect_streaming; filter and select the needed columns beforehand when working with larger data.

Use the most recent calibration

The source below is already ordered. For unsorted data, sort both query inputs by their key before joining. The engine checks ascending order rather than silently producing a misleading match.

import std.dataframe as df

def main() -> Result[(), df.DataFrameError | AllocError | IoError]:
    readings = df.frame([
        df.column_i64("second", [2, 6, 12]?)?,
        df.column_f64("reading", [10.0, 20.0, 30.0]?)?
    ]?)?
    calibration = df.frame([
        df.column_i64("second", [1, 5]?)?,
        df.column_f64("offset", [0.5, 1.5]?)?
    ]?)?
    joined = readings.lazy()?.join_asof(
        calibration.lazy()?, "second", "second", tolerance=Some(3)
    )?
    corrected = df.col("reading")?.add(df.col("offset")?)?.alias("corrected")?
    result = joined.with_columns([corrected]?)?.collect()?
    print(result.values_f64_nullable("corrected")?)?
    Ok(())
[Option[f64].Some(10.5), Option[f64].Some(21.5), Option[f64].Nothing]

Backward matching is the default: calibration at second 5 can apply to a reading at second 6, but cannot apply to a reading at second 2. The final reading has no calibration within three seconds; its null offset propagates through addition. This is missing data, not a query failure. Both original key columns remain available as second and second_right for inspection.

Choose the nearest observation

Nearest matching can look into the future. Choose it deliberately when that is meaningful for your data; do not use it for a causal lookup by accident.

import std.dataframe as df

def main() -> Result[(), df.DataFrameError | AllocError | IoError]:
    events = df.frame([df.column_i64("second", [2, 4]?)?]?)?
    samples = df.frame([
        df.column_i64("sample_second", [1, 3, 5]?)?,
        df.column_str("label", ["early", "middle", "late"]?)?
    ]?)?
    result = events.lazy()?.join_asof(
        samples.lazy()?, "second", "sample_second",
        strategy=df.AsOfStrategy.Nearest, tolerance=Some(1)
    )?.collect()?
    print(result.values_str("label")?)?
    Ok(())
["middle", "late"]

Each event is equally far from two samples; nearest matching selects the greater key. AsOfStrategy.Forward always chooses the next qualifying key. For backward/forward matching, set allow_exact_matches=False when a strictly earlier/later sample is required. See as-of join contracts for duplicate keys, nulls, limits and failure behavior.

Dates and daily reports

Parse a timestamp column once, then keep calendar calculations inside the query. The example below groups events by their UTC calendar date. This is different from splitting timestamp strings or assuming every input has an identical textual layout.

These examples need the dataframe engine, which Linux releases include. utc=True declares that the input text already represents UTC; it does not convert another timezone.

import std.dataframe as df

def main() -> Result[(), df.DataFrameError | AllocError | IoError]:
    events = df.frame([
        df.column_str("text", ["2024-02-28 09:00:00", "2024-02-29 12:30:00", "2024-02-29 14:00:00"]?)?,
        df.column_i64("amount", [10, 20, 5]?)?
    ]?)?
    stamp = df.col("text")?.str_to_datetime(
        "%Y-%m-%d %H:%M:%S", df.TimeUnit.Microseconds, utc=True
    )?.alias("stamp")?
    daily = events.lazy()?.with_columns([stamp]?)?.group_by(
        [df.col("stamp")?.dt_date()?.alias("day")?]?,
        [df.col("amount")?.sum()?.alias("total")?]?
    )?.sort([df.SortKey(df.col("day")?)]?)?.select([
        df.col("day")?.dt_format("%Y-%m-%d")?.alias("day")?,
        df.col("total")?
    ]?)?.collect()?
    print(daily.values_str("day")?)?
    print(daily.values_i64("total")?)?
    Ok(())
["2024-02-28", "2024-02-29"]
[10, 25]

dt_date() retains a logical Date while grouping and sorting; formatting happens only for presentation. The chosen unit controls stored precision. Microseconds keep six fractional decimal places; milliseconds keep three and nanoseconds keep nine. Extra fractional precision is discarded toward the earlier instant, so select the unit intentionally.

Keep invalid input visible as missing data

Parsing is strict by default. Use strict=False when invalid input should become null and your application has a policy for missing data.

import std.dataframe as df

def main() -> Result[(), df.DataFrameError | AllocError | IoError]:
    source = df.frame([
        df.column_str("text", ["1970-01-01", "1969-12-31", "2001-02-29"]?)?
    ]?)?
    dates = source.lazy()?.select([
        df.col("text")?.str_to_date(strict=False)?.alias("day")?
    ]?)?.collect()?
    print(dates.values_date_days_nullable("day")?)?
    Ok(())
[Option[i32].Some(0), Option[i32].Some(-1), Option[i32].Nothing]

The raw boundary representation is days since 1970-01-01; datetimes use integer ticks with an explicit unit. In most reports, prefer calendar expressions and formatting to manipulating these representations yourself. Getters check the logical dtype and, for datetimes, the requested unit and UTC metadata. They do not silently rescale timestamps.

Use Parquet when temporal metadata should survive file round-trips. For CSV, choose an explicit text format and parse it again on input. Named timezones, offset parsing and leap seconds are outside this foundation. See the temporal contracts for nulls, calendar bounds, overflow and conversion costs.

Clean and compare column values

Keep cleaning and derived calculations inside a query so Polars can optimize them with the rest of the plan. These examples require the optional engine.

Parse values and label missing data

The explicit non-strict cast below treats invalid numeric text as missing. That is a deliberate data-cleaning policy; omit strict=False when bad input should fail the query instead.

import std.dataframe as df

def main() -> Result[(), df.DataFrameError | AllocError | IoError]:
    data = df.frame([
        df.column_str("team", ["North", "South", "NORTH"]?)?,
        df.column_str("amount", ["12", "oops", "8"]?)?
    ]?)?
    cleaned = data.lazy()?.with_columns([
        df.col("team")?.str_lower()?,
        df.col("amount")?.cast(df.DType.I64, strict=False)?
    ]?)?
    state = df.when(
        df.col("amount")?.is_null()?,
        df.lit_str("missing")?,
        df.lit_str("ready")?
    )?.alias("state")?
    result = cleaned.with_columns([state]?)?.collect()?
    print(result.values_str("team")?)?
    print(result.values_i64_nullable("amount")?)?
    print(result.values_str("state")?)?
    Ok(())
["north", "south", "north"]
[Option[i64].Some(12), Option[i64].Nothing, Option[i64].Some(8)]
["ready", "missing", "ready"]

The second with_columns sees the parsed amount. Expressions within a single with_columns all see its input schema, so separate dependent steps. when is a native conditional expression, not a short-circuiting Plenty if: both branches must be valid. It cannot guard a failing strict cast.

For literal matching, write df.col("team")?.str_contains(df.lit_str("north")?)?. The pattern is text, not a regular expression. A column expression can also supply a different pattern per row.

Compare each row with its group

A grouped aggregate produces one row per group. A window calculation instead places the group’s result alongside each original row.

import std.dataframe as df

def main() -> Result[(), df.DataFrameError | AllocError | IoError]:
    data = df.frame([
        df.column_str("team", ["north", "south", "north", "north"]?)?,
        df.column_i64("units", [2, 4, 3, 2]?)?
    ]?)?
    teams = [df.col("team")?]?
    units = df.col("units")?
    result = data.lazy()?.with_columns([
        units.sum()?.over(teams)?.alias("team_total")?,
        units.n_unique()?.over(teams)?.alias("distinct_units")?
    ]?)?.collect()?
    print(result.values_i64("team_total")?)?
    print(result.values_i64("distinct_units")?)?
    Ok(())
[7, 4, 7, 7]
[2, 1, 2, 2]

over uses the entire partition, not a running window. It maps results back to the original rows and does not sort them. Distinct counts include null as one distinct value, whereas count() counts only present values. Read the expression contracts for casting, Unicode and null semantics.

Query CSV and write Parquet

Use a lazy CSV scan when Polars should filter, derive or aggregate file-backed data. A direct Parquet sink writes the query’s output without first returning a complete dataframe to Plenty.

This example creates a small input file so it can run by itself. With real data, begin at scan_csv. The examples need the dataframe engine; run them in an empty directory: the new output methods refuse to overwrite existing files.

import std.dataframe as df

def main() -> Result[(), df.DataFrameError | AllocError | IoError]:
    readings = df.frame([
        df.column_str("station", ["001", "002", "001"]?)?,
        df.column_f64("reading", [10.0, 20.0, 30.0]?)?
    ]?)?
    readings.write_csv("readings.csv")?
    query = df.scan_csv("readings.csv", schema_overrides=Some([
        df.Field("station", df.DType.String),
        df.Field("reading", df.DType.F64)
    ]?))?
    warm = query.filter(df.col("reading")?.gt(df.lit(15.0)?)?)?
    warm.sink_parquet("warm.parquet")?
    result = df.scan_parquet("warm.parquet")?.collect_streaming()?
    print(result.values_str("station")?)?
    print(result.values_f64("reading")?)?
    Ok(())
["002", "001"]
[20.0, 30.0]

The String override keeps leading zeros in station identifiers. CSV contains text rather than a typed schema. For unfamiliar files, inspect query.schema()?. With infer_rows=0, unspecified columns stay strings, so you can validate them and choose explicit casts. The default samples 100 rows; infer_rows=-1 scans the full file for inference.

Missing values and text

A quoted empty string and an unquoted empty field have different meanings. The default writer/reader preserves that distinction. When copying a frame through CSV, pass its schema to preserve column types too.

import std.dataframe as df

def main() -> Result[(), df.DataFrameError | AllocError | IoError]:
    notes: list[Option[str]] = [Some(""), Nothing, Some("雪, café")]?
    data = df.frame([df.column_str_nullable("note", notes)?]?)?
    data.write_csv("notes.csv", separator=";")?
    restored = df.scan_csv(
        "notes.csv",
        separator=";",
        schema_overrides=Some(data.schema()?)
    )?.collect_streaming()?
    print(restored.values_str_nullable("note")?)?
    Ok(())
[Option[str].Some(""), Option[str].Nothing, Option[str].Some("雪, café")]

A custom null token is available through write_csv(null_value="NA") and scan_csv(null_value=Some("NA")). Use it only when that token cannot be a real value: even a quoted "NA" is treated as null on reading.

Streaming execution does not imply a memory bound. collect_streaming retains all final rows, and sorts, groups, joins or windows may buffer data. Prefer sink_parquet when the desired result is a file rather than a Plenty dataframe. A failed write can leave a partial new file; existing files are never overwritten by these two new output methods. See the I/O contracts for options, error timing and execution limits.

Continue with Application: a daily sales report to combine file scans, cleaning, dates, joins and exports in a complete program.

Application: a daily sales report

The repository’s examples/sales_report/ combines dataframe operations into a command-line application. It reads order lines, joins a product catalogue, separates bad records, and exports daily sales totals. The directory includes small input files, reusable query functions, a CLI and independent output checks.

Start with the engine installation. From a source checkout with the compiler and matching engine installed:

target/release/plenty --compile examples/sales_report/main.plenty -o /tmp/sales-report
target/release/plenty --compile examples/sales_report/convert.plenty -o /tmp/sales-convert
RUN=$(mktemp -d)
mkdir "$RUN/input" "$RUN/output"
/tmp/sales-convert ingest examples/sales_report/data/orders-01.csv "$RUN/input/part-01.parquet"
/tmp/sales-convert ingest examples/sales_report/data/orders-02.csv "$RUN/input/part-02.parquet"
/tmp/sales-report parquet "$RUN/input/part-*.parquet" examples/sales_report/data/products.csv "$RUN/output"
cat "$RUN/output/daily.csv"

The application prints 12 input lines, 5 accepted lines, 7 rejected lines and 4 daily groups. The resulting CSV is:

day,category,lines,total_cents
2024-02-28,books,1,1200
2024-02-29,books,2,1000
2024-02-29,games,1,2500
2024-03-01,games,1,0

It also writes daily.parquet, retaining logical Date values, and rejected.csv, retaining the original fields with rejection reasons. For a single CSV input, use sales-report csv ORDERS.csv PRODUCTS.csv OUTPUT_DIR. Only Parquet supports the quoted glob. The converter keeps the raw String columns; it does not pre-clean or calculate the report.

Follow the query through the application

Read report.plenty in this order:

  1. scan_orders selects four required String columns. CSV inference is disabled so product IDs such as 001 keep their leading zeros. Parquet input uses that same raw ingestion schema.
  2. catalogue checks nonempty IDs/categories and unique product IDs before joining. Duplicate reference keys would otherwise multiply sales totals.
  3. classify parses UTC timestamps and integer cents non-strictly, then performs a left join. A native conditional expression assigns each row an acceptance or rejection reason. Missing prices or dates are not silently replaced with useful-looking values.
  4. daily filters accepted rows, groups by calendar date and lowercased category, counts lines and sums cents. Sorting gives stable report order.
  5. run exports the report and rejected records, then prints reconciliation counts. It propagates structural/query/I/O failures through its fallible entrypoint, giving a nonzero exit status.

These functions build native expressions rather than invoking Plenty once per row. Named intermediate plans make the cleaning policy readable without copying the whole dataset at each step. The classified rows are collected once so both outputs describe the same input; that is an explicit memory cost, not a promise that streaming execution bounds memory.

Amounts are already line totals, expressed as whole cents. Zero is valid; negative amounts, missing amounts and amounts above one billion cents are rejected. Together with a 100-million-line cap before aggregation, this keeps native i64 sums in range. The row cap is checked after collection and cannot protect against running out of memory while reading. Repeated order IDs are separate lines, not automatically deduplicated transactions. This is a batch reporting example, not an accounting system.

Check and ship it

Run the independent checks with the same engine installation:

python3 -B examples/sales_report/check.py --plenty target/release/plenty -v

The checks compile the actual application, compare CSV and Parquet results with Python stdlib calculations, and exercise invalid rows, duplicate keys, empty inputs and failures. Add --benchmark-rows 100000 to measure a larger generated workload, including all output writes. Python is a development dependency of these checks, not a runtime dependency of the report.

Use a new output directory for each run. Files are not overwritten, but the three writes are not one transaction: a failure may leave partial output. Publish the directory only after the process succeeds. Rejected rows are an intentional successful outcome; inspect the reconciliation counts and file.

The application README describes the exact input schema, rejection precedence, resource costs and measurement procedure. Follow shipping dataframe applications to distribute the executable with its matching engine and license notices.

Columnar lists

clist[T] makes data-oriented performance accessible through ordinary Plenty code. Declare a class normally, then choose clist[YourClass] when a large collection of its records is usually processed one or two fields at a time. You still append ordinary objects and refer to their named fields. The collection stores each field in its own packed, contiguous column.

For example, averaging temperatures does not need the station identifier from every reading. Keeping temperatures together lets a scan bring more useful values into the CPU cache with each memory access. Column pipelines make this kind of bulk calculation natural: select a field, then use the familiar fluent operations. The storage choice belongs to this collection; it does not change how the class is stored everywhere else.

This is useful as a representation as well as a container: it makes efficient execution easier to express for the programmer and easier to recognize for the compiler. Start with the access pattern you need, rather than expecting every operation on a columnar list to become faster. Gathering a complete row requires reading separate columns. The current implementation accepts only primitive fields; use ordinary lists for strings, nested objects, or borrowed references to whole rows.

For analytical expression plans, grouping and Parquet datasets, consider dataframes. Their optional Polars engine has runtime schemas and native query semantics; clist keeps a lightweight mutable collection and Plenty’s own arithmetic and type checking.

Build and scan columnar records

Declare a class normally, then create a clist to store its fields in separate columns. This example reserves space for three readings, averages their temperatures, and updates and copies a row:

class Reading:
    station: i64
    temperature: f64

def add(total: f64, temperature: f64) -> f64:
    total + temperature

def main() -> Result[(), AllocError | AppendError[Reading] | IoError]:
    mut readings = clist[Reading].with_capacity(3)?
    readings.append(Reading(10, 18.0))?
    readings.append(Reading(20, 21.0))?
    readings.append(Reading(30, 24.0))?
    total = readings.column(Reading.temperature).fold(0.0, add)
    print(total / f64(len(readings)))?
    readings[0].temperature = 19.0
    match readings.copy_row(0):
        case Some(reading):
            print(reading.station, reading.temperature)?
        case Nothing:
            ()
    Ok(())
21.0
10 19.0

Reading.temperature selects that field’s column. The pipeline borrows the collection and yields scalar values, so add takes f64 directly. Reserving capacity makes the three appends allocation-free after construction. Errors from append retain the submitted reading; this example propagates that error so callers can decide whether to retry.

A column pipeline is consumed once. While it remains live, the collection cannot be changed or moved. Create a new column pipeline for another pass. copy_row gathers a row’s fields from the separate columns into an independent ordinary object. It allocates nothing, but gathering all the fields can cost more than accessing a record in an ordinary list. Changing the copy does not change the stored row. Use a column for a selected-field calculation and a row copy when you actually need the complete object.

The primitive-only restriction keeps this first version’s ownership and packed storage model simple and sound. Prefer an ordinary list for strings, nested objects, row references, or workloads that usually access whole objects. Indexed field access and row copying compile to direct scalar memory operations, with bounds checks. This avoids dispatching through a generic runtime helper for each field, but does not make every access pattern faster. Use column pipelines for repeated scans when the calculation needs selected fields.

Next, count matching integer values with an automatically optimized column pipeline. See the columnar-list reference for the allocation, borrowing and representation contracts.

Count matching integer values

Write the ordinary calculation you need. For a simple comparison over an i32 or u32 column, the default optimized profile can count matches with SIMD: the CPU compares four values together. You do not choose a vector width or handle leftover rows.

class Sample:
    score: i32

def main() -> Result[(), AppendError[Sample] | IoError]:
    mut samples = clist[Sample]()
    for score in range(9):
        samples.append(Sample(i32(score)))?
    matches = samples.column(Sample.score).filter(def(x: &i32) -> bool: *x > 5i32).count()
    print(matches)?
    Ok(())
3

Use runtime bounds

Bounds do not need to be constants. Capture their scalar values to count a range:

class Sample:
    score: i32

def count_between(samples: &clist[Sample], low: i32, high: i32) -> i64:
    samples.column(Sample.score).filter(def [low, high](x: &i32) -> bool: *x >= low and *x < high).count()

def main() -> Result[(), AppendError[Sample] | IoError]:
    mut samples = clist[Sample]()
    for score in range(9):
        samples.append(Sample(i32(score)))?
    print(count_between(&samples, 2i32, 7i32))?
    Ok(())
5

[low, high] captures copies of the bounds. They can come from function arguments or parsed input. The closure keeps those values even if an original binding later changes. Write two comparisons with and; Plenty does not use chained comparison syntax.

This is the same program in normal and optimized builds. The SIMD path recognizes one comparison, or two comparisons joined by and, against constants or up to two immutable scalar captures of the column’s type. Borrowed captures, arithmetic inside the predicate, or, and extra adapters keep scalar execution. Other calculations, including the floating-point average in the previous lesson, keep ordinary scalar execution. All scans still run on one thread. See the columnar-list reference for the precise optimization limits, allocation, borrowing and representation.

Remove rows and reuse storage

Use remove(index) when you need to take one complete record out of a columnar list. It returns Some(record) or Nothing for an out-of-bounds index. Remaining records keep their order, and negative indices count from the end.

class Reading:
    station: i64
    temperature: f64

def main() -> Result[(), AllocError | AppendError[Reading] | IoError]:
    mut readings = clist[Reading].with_capacity(3)?
    readings.append(Reading(10, 18.0))?
    readings.append(Reading(20, 21.0))?
    readings.append(Reading(30, 24.0))?
    match readings.remove(1):
        case Some(reading):
            print("removed", reading.station, reading.temperature)?
        case Nothing:
            ()
    for station in readings.column(Reading.station):
        print(station)?

    capacity = readings.capacity()
    readings.clear()
    print(len(readings), readings.capacity() == capacity)?
    readings.append(Reading(40, 19.0))?
    print(readings[0].station)?
    Ok(())
removed 20 21.0
10
30
0 True
40

Removal assembles an independent ordinary object, just like copy_row, then shifts the later values left in every column. It allocates nothing, but removing near the front of a large collection moves much more data than removing the last row. Repeated front removals are a poor way to filter a whole dataset.

clear() empties all columns in constant time and retains their allocation. This is useful when processing successive batches: clear the old batch and append the next one into the same storage. Removal also retains capacity. Appends within that capacity do not allocate, although their return type still requires error handling. Use drop(readings) when you want to release the storage instead of reusing it.

Both methods require mutable access. A live column pipeline borrows the entire collection, so finish its traversal before removing rows or clearing the owner. See the reference for the complete contract and costs.

Organizing a program

Imports divide a program into source modules. pub controls which declarations other modules may use.

Split a program into modules

An import names a source file without executing it. Modules contain declarations and imports; execution starts at the application’s main. Names are private to their module unless marked pub.

Save this companion file as geometry.plenty:

pub class Point:
    pub x: i64
    pub y: i64

    pub def squared_length(self) -> Result[i64, OverflowError[i64]]:
        Ok(self.x * self.x + self.y * self.y)

pub def make_point() -> Point:
    Point(3, 4)

Save the application beside it, for example as main.plenty:

import geometry
from geometry import Point as Position

def main() -> Result[(), IoError | OverflowError[i64]]:
    point: Position = geometry.make_point()
    print(point.squared_length()?)?
    print(point.x)?
    Ok(())
25
3

import geometry makes declarations reachable through geometry.Name. from geometry import Point as Position binds just that type under a local name. You can also write import geometry as geo, then use geo.Point. Aliases refer to the same declaration. Two modules may each define Point; those are distinct types.

Imports are absolute. import tools.geometry reads tools/geometry.plenty under the source root. Directories provide namespaces and need no __init__ file. The root defaults to the entry file’s directory. If the entry file is deeper in the tree, select the root explicitly:

plenty --module-root src src/app/main.plenty
plenty --module-root src --compile src/app/main.plenty -o app
plenty --module-root src --check-module src/tools/geometry.plenty

--check checks a complete application and requires main. --check-module checks a library and its imports without that requirement.

Choose a public API

Mark each public field and method explicitly. Without __init__, the generated field constructor is public only when the class and every field are public. To keep fields private while allowing construction, declare pub def __init__:

Save this module as counter.plenty:

pub class Counter:
    value: i64

    pub def __init__(self, start: i64) -> ():
        self.value = start

    pub def increment(self: &mut Counter) -> Result[(), OverflowError[i64]]:
        self.value = self.value + 1
        Ok(())

    pub def read(self) -> i64:
        self.value

    def __del__(self) -> ():
        drop(print("counter closed"))

Application:

from counter import Counter

def main() -> Result[(), IoError | OverflowError[i64]]:
    mut count = Counter(41)
    count.increment()?
    print(count.read())?
    Ok(())
42
counter closed

The private destructor still runs automatically. Private fields and methods are accessible throughout their module, but not from another module. Reads, writes, and borrows all enforce this. Save the next module as secret.plenty:

pub class Secret:
    value: i64

    pub def __init__(self, value: i64) -> ():
        self.value = value
from secret import Secret

def main() -> Result[(), IoError]:
    item = Secret(42)
    print(item.value)?
    Ok(())
secret.Secret.value` is private

pub also applies to functions, enums, and type aliases. A public enum exposes all its variants. Public signatures cannot mention private classes or enums, even through aliases or containers. Ordinary imports are private bindings; importing a name does not re-export it. Relative imports, wildcards, circular imports, and pub import are not supported yet.

Private fields are still included in automatic printing and equality.

The lifecycle trace uses drop(print(...)) deliberately: cleanup cannot return an error, and a failed diagnostic write must not interrupt cleanup. For output that must succeed, provide an explicit fallible operation and propagate its result before leaving the scope.

Use the standard math library

The compiler includes std.math. Import it like any other module; no companion source file or download is needed. Its floating-point API accepts f64.

import std.math as math
from std.math import pi

const half_turn: f64 = f64.PI

def main() -> Result[(), AllocError | IoError | math.MathError | OverflowError[i64]]:
    print(math.sqrt(81.0)?)?
    print(math.factorial(5)?)?
    print(math.comb(5, 2)?)?
    print(math.isclose(math.radians(180.0), pi, 1e-9, 0.0)?)?
    print(pi == half_turn)?
    print(math.sqrt(-1.0) == Result[f64, math.MathError].Err(math.MathError.Domain))?
    values = [2, 3, 4]?
    print(math.sum(&values, 0)?)?
    print(math.prod(&values, 1)?)?
    match math.min_all(&values):
        case Some(value):
            print(value)?
        case Nothing:
            print("empty input")?
    match math.max_all(&values):
        case Some(value):
            print(value)?
        case Nothing:
            print("empty input")?
    print(math.round(2.5)?)?
    print(math.round(3.5)?)?
    quotient, remainder = math.divmod(-7, 3)?
    print(quotient)?
    print(remainder)?
    interval = range(1, 5)
    print(math.sum_iter(&interval, 0)?)?
    Ok(())
9.0
120
10
True
True
True
9
24
2
4
2
4
-3
2
10

? propagates mathematical errors just like I/O errors. Match on a result when you want to recover from math.MathError.Domain or math.MathError.Overflow. Integer combinatorics return u64, reporting overflow instead of wrapping. Integer sum, sum_iter, and prod return OverflowError[T], where T is the integer type; their calls need ? too. Floating-point variants such as sum_float keep IEEE behavior. floor, ceil, trunc, and round return checked i64 results. round chooses the even integer at halfway values. divmod returns an integer floor quotient and remainder with the divisor’s sign; use divmod_float for floats. Division by zero is a recoverable Domain error.

min_all and max_all borrow a list and return Nothing for an empty list. They retain the first value on ties. Floats follow the IEEE 754 standard’s minimum and maximum: -0.0 counts as less than 0.0, and any NaN makes the result Some(NaN), regardless of its position. The two-value min and max follow the same rule. sum_iter and prod_iter also borrow ranges, sets, and dictionary keys; their explicit start supplies the numeric type and the result for an empty input. The original collections remain usable.

isclose takes both tolerances explicitly. Module constants are math.pi, math.e, math.tau, math.inf, and math.nan, all f64. Import one as a value, as the example does with pi. The type constants f64.PI, f64.E, f64.TAU, f64.INF, and f64.NAN also remain available. Low-level methods such as x.sqrt() follow IEEE arithmetic and may return NaN; the imported functions provide recoverable domain errors.

Declare your own immutable constants with const name: T = value, as with half_turn. The compiler evaluates the initializer, which can combine literals, type constants, and other constants with operators, as in const full_turn: f64 = 2.0 * half_turn. Use pub const to export one from a module. See constants for the accepted expressions, static assertions, and file contents included as constants.

See the math reference for available functions and their fixed-width and numerical limits. The standard library reference lists every module.

Work with explicit SIMD lanes

Prefer ordinary pipelines for repeated column scans: the columnar-list lessons show how Plenty can accelerate them automatically. Use std.simd when the algorithm itself is clearer as a fixed set of lanes.

This example compares four measurements with one threshold, then selects the threshold for lanes below it. The mask is a vector of logical booleans; count counts its true lanes, and select chooses between already-computed values.

import std.simd as simd

def main() -> Result[(), IoError]:
    readings = simd.f32x4.Lanes(18.0f32, 21.0f32, 16.0f32, 24.0f32)
    floor = simd.splat_f32x4(20.0f32)
    below = readings.simd_lt(floor)
    adjusted = below.select(floor, readings)
    print(below.count(), adjusted.to_tuple())?
    print(readings.to_tuple())?
    Ok(())
2 (20.0, 21.0, 20.0, 24.0)
(18.0, 21.0, 16.0, 24.0)

Vectors are copyable and allocate nothing. Operations borrow their inputs and produce new values, so the original readings remain unchanged. i32x4, u32x4 and f64x2 provide the same pattern with other lane types. Mask shapes must match the vectors they select.

Use add, sub and mul for ordinary arithmetic. Integer versions return a Result with a lane-typed overflow error; propagate it with ? or handle it with match. Integer wrapping_add, wrapping_sub and wrapping_mul explicitly choose modular arithmetic. Floating sum_ordered(initial) preserves scalar addition order. Selection is eager: compute and handle both inputs before selecting between them.

The module also provides checked list loads and stores; they require a complete vector to fit and never allocate. Handle any leftover elements with a scalar loop. Ordinary lists and these initial vector values use tagged scalar storage, so explicit lanes do not promise packed storage or faster code. The default optimized profile enables supported native SIMD operations on x86-64. AArch64 currently uses the scalar implementation, which preserves the same results. See the reference for the exact operations, bounds contracts and current limits.

Measure time and wait

Import std.time for spans, clock readings, and sleep. A Duration is a nonnegative span; an Instant is a reading of a clock that never goes backwards.

from std.time import monotonic_now, sleep, milliseconds, seconds

def main() -> Result[(), AllocError | IoError | RangeError]:
    pause = milliseconds(5)
    start = monotonic_now()
    sleep(&pause)
    elapsed = monotonic_now().duration_since(&start)?
    print(elapsed.as_milliseconds() >= pause.as_milliseconds())?
    minutes = seconds(60).checked_mul(90)?
    print(minutes.as_seconds())?
    print(seconds(1) == milliseconds(1000))?
    Ok(())
True
5400
True

sleep blocks for at least the span; a busy machine can make it longer, so the example prints a comparison, not a measurement. Unit constructors take u32 counts and never fail. Arithmetic that could leave the representable range returns Result with RangeError, including a subtraction that would be negative:

from std.time import milliseconds, seconds

def main() -> Result[(), AllocError | IoError | RangeError]:
    short = milliseconds(250)
    long = seconds(1)
    match short.checked_sub(&long):
        case Ok(span):
            print(span.as_milliseconds())?
        case Err(RangeError.OutOfRange):
            print("a span cannot be negative")?
    print(short.saturating_sub(&long).is_zero())?
    print(long.checked_sub(&short)?.as_milliseconds())?
    Ok(())
a span cannot be negative
True
750

Time values are class instances, so they move. Pass them by reference, as these calls do, and they stay usable.

Wall-clock time is a different type

wall_now() returns a Timestamp: seconds and nanoseconds since the Unix epoch. The system clock can be set backwards, so the difference between two timestamps is fallible. This example builds two readings by hand to show a clock that was stepped back by two seconds:

from std.time import Timestamp, wall_now

def main() -> Result[(), AllocError | IoError | RangeError]:
    first = Timestamp(1700000010, 0)
    second = Timestamp(1700000008, 500000000)
    match second.duration_since(&first):
        case Ok(span):
            print(span.as_milliseconds())?
        case Err(RangeError.OutOfRange):
            print("the clock moved backwards")?
    print(first.duration_since(&second)?.as_milliseconds())?
    print(wall_now().seconds > first.seconds)?
    Ok(())
the clock moved backwards
1500
True

Use instants to measure elapsed time and timestamps to record when something happened. The compiler rejects mixing them:

from std.time import monotonic_now, wall_now

def main() -> Result[(), AllocError | IoError]:
    started = monotonic_now()
    print(wall_now().duration_since(&started)?.as_seconds())?
    Ok(())
expected &std.time.Timestamp, got &std.time.Instant

The module has no calendar, time-zone, or formatting support. See the time reference for every operation, and Share one deadline across waits for timeouts.

Take the most urgent item first

Import Heap from std.collections for a priority queue. pop always removes the least element, however the elements were pushed.

from std.collections import Heap

def main() -> Result[(), AllocError | IoError]:
    mut times = Heap[i64]()?
    for time in [40, 10, 30, 20]?:
        times.push(time)?
    print(times.peek(), times.len())?
    while not times.is_empty():
        match times.pop():
            case Some(value):
                print(value)?
            case Nothing:
                break
    print(times.pop())?
    Ok(())
Option[i64].Some(10) 4
10
20
30
40
Option[i64].Nothing

Construction and push return Result with AllocError. pop returns Option and transfers the element. peek returns a copy of the first element without removing it, or Nothing when the heap is empty.

Heap orders integers, floats, and strings with <. For any other element, HeapBy takes a function that says which of two elements leaves first:

from std.collections import HeapBy

class Job:
    priority: i64
    name: str

def urgent(a: &Job, b: &Job) -> bool:
    a.priority > b.priority

def main() -> Result[(), AllocError | IoError]:
    mut jobs = HeapBy(urgent)?
    jobs.push(Job(1, "sweep"))?
    jobs.push(Job(9, "deploy"))?
    jobs.push(Job(5, "review"))?
    match jobs.peek_ref():
        case Some(job):
            print("next:", job.name)?
        case Nothing:
            pass
    while not jobs.is_empty():
        match jobs.pop():
            case Some(job):
                print(job.priority, job.name)?
            case Nothing:
                break
    Ok(())
next: deploy
9 deploy
5 review
1 sweep

The heap owns its jobs; pop moves one out without copying it. A Job cannot be copied, so peek is rejected for this heap; peek_ref borrows the first job instead, as in returning a borrow that may be absent. The heap cannot change while that borrow is in use. Elements that compare equal leave in no promised order. See the module reference for costs and limits.

Find a route through a graph

std.graph stores nodes and the edges between them. UnGraph edges join two nodes both ways; DiGraph edges run from a source to a target. Each node and edge owns a value, and adding one returns a handle that names it.

from std.graph import UnGraph, GraphError

def minutes(walk: &i64) -> f64:
    f64(*walk)

def main() -> Result[(), AllocError | IoError | GraphError]:
    mut town = UnGraph[str, i64]()?
    home = town.add_node("home")?
    shop = town.add_node("shop")?
    park = town.add_node("park")?
    school = town.add_node("school")?
    town.add_edge(home, shop, 10)?
    town.add_edge(home, park, 4)?
    town.add_edge(park, shop, 3)?
    town.add_edge(shop, school, 6)?
    paths = town.dijkstra(home, &minutes)?
    for stop in paths.path(school)?:
        print(*town.node(stop))?
    print(paths.distance(school))?
    Ok(())
home
park
shop
school
Option[f64].Some(13.0)

The graph does not know what edge values mean, so dijkstra takes a function that turns one into a non-negative weight. The result answers distance and path queries for every node reachable from the start.

Handles stay valid after removals: removing park leaves shop and school naming the same places, and the path now takes the direct road.

from std.graph import UnGraph, GraphError

def minutes(walk: &i64) -> f64:
    f64(*walk)

def main() -> Result[(), AllocError | IoError | GraphError]:
    mut town = UnGraph[str, i64]()?
    home = town.add_node("home")?
    shop = town.add_node("shop")?
    park = town.add_node("park")?
    town.add_edge(home, shop, 10)?
    town.add_edge(home, park, 4)?
    town.add_edge(park, shop, 3)?
    print(town.remove_node(park))?
    print(town.contains_node(park), town.edge_count())?
    print(town.get_node_ref(park), town.get_node_ref(shop))?
    print(town.dijkstra(home, &minutes)?.distance(shop))?
    Ok(())
Option[str].Some("park")
False 1
Option[&str].Nothing Option[&str].Some("shop")
Option[f64].Some(10.0)

Using a removed handle with node or add_edge traps. When a handle may be stale, get_node_ref and get_edge_ref return Nothing instead of trapping, and get_node_mut and get_edge_mut do the same for changes. See the module reference for traversal, topological order, components, spanning trees and costs.

Encode bytes and pack numbers

Binary formats store integers at fixed widths in a stated byte order, and text channels carry binary data as hex or base64. std.encoding does both over std.bytes buffers.

import std.bytes
import std.encoding
from std.encoding import Endian

def main() -> Result[(), AllocError | IoError | std.encoding.EncodingError | std.bytes.BytesError]:
    mut frame = std.bytes.empty()?
    std.encoding.append_u16(&mut frame, 513u16, Endian.Big)?
    std.encoding.append_u32(&mut frame, 7u32, Endian.Little)?
    frame.extend(&std.bytes.from_text("hi")?)?
    print(std.encoding.hex_text(&frame)?)?
    print(std.encoding.base64_text(&frame)?)?
    print(std.encoding.base64url_text(&frame[:7])?)?
    print(std.encoding.unpack_u16(&frame, 0, Endian.Big), std.encoding.unpack_u16(&frame, 0, Endian.Little))?
    std.encoding.pack_u32(&mut frame, 2, 8u32, Endian.Little)
    print(std.encoding.unpack_u32(&frame, 2, Endian.Little), frame.len())?
    Ok(())
0201070000006869
AgEHAAAAaGk=
AgEHAAAAaA
513 258
8 8

Every integer function names its width and takes the byte order as an argument; there is no default order. Endian.Big stores the most significant byte first. The same two bytes read as 513 or 258 depending on the order you state. append_ grows the buffer by the integer’s width. pack_ overwrites bytes that already exist and never grows it. unpack_ reads at an offset.

Mixed integer and float fields

Use f32 or f64 when the format specifies IEEE 754 binary32 or binary64. The same append, overwrite, and offset rules apply. A numeric cast to an integer changes the value; these codecs preserve its float representation:

import std.bytes
import std.encoding
from std.encoding import Endian

def main() -> Result[(), AllocError | IoError | std.encoding.EncodingError]:
    mut record = std.bytes.with_capacity(14)?
    std.encoding.append_u16(&mut record, 1u16, Endian.Big)?
    std.encoding.append_f32(&mut record, 1.5f32, Endian.Big)?
    std.encoding.append_f64(&mut record, -2.5, Endian.Little)?
    print(std.encoding.hex_text(&record)?)?
    print(std.encoding.unpack_u16(&record, 0, Endian.Big))?
    print(std.encoding.unpack_f32(&record, 2, Endian.Big))?
    print(std.encoding.unpack_f64(&record, 6, Endian.Little))?
    Ok(())
00013fc0000000000000000004c0
1
1.5
-2.5

Widths count bytes, so the fields begin at offsets 0, 2, and 6. Floats need no alignment padding. The float packing reference explains signed zero, infinities, NaNs and exact bit preservation.

The codecs take a byte view: &buffer for a whole buffer, &buffer[start:stop] for part of one, or a b"..." literal. The _text forms return a str; hex_encode, base64_encode, and base64url_encode return the encoded bytes as a new buffer. base64 uses + and / and always pads with =. base64url uses - and _ and never pads.

Ordinary decoding accepts exactly one form and reports what is wrong with anything else. Nothing is skipped or repaired, including spaces and line breaks:

import std.bytes
import std.encoding
from std.encoding import Endian

def main() -> Result[(), AllocError | IoError | std.bytes.BytesError | std.encoding.EncodingError]:
    print(std.encoding.base64_from_text("aGk=")?.to_text()?)?
    print(str.repr(std.encoding.hex_from_text("abc"))?)?
    print(str.repr(std.encoding.hex_from_text("0g"))?)?
    print(str.repr(std.encoding.base64_from_text("aGk"))?)?
    print(str.repr(std.encoding.base64_from_text("aG k"))?)?
    print(str.repr(std.encoding.base64_from_text("aGl="))?)?
    print(str.repr(std.encoding.base64url_from_text("aGk="))?)?
    Ok(())
hi
Result[std.bytes.Bytes, std.encoding.EncodingError].Err(std.encoding.EncodingError.InvalidLength)
Result[std.bytes.Bytes, std.encoding.EncodingError].Err(std.encoding.EncodingError.InvalidByte(1))
Result[std.bytes.Bytes, std.encoding.EncodingError].Err(std.encoding.EncodingError.InvalidLength)
Result[std.bytes.Bytes, std.encoding.EncodingError].Err(std.encoding.EncodingError.InvalidByte(2))
Result[std.bytes.Bytes, std.encoding.EncodingError].Err(std.encoding.EncodingError.NonCanonical(2))
Result[std.bytes.Bytes, std.encoding.EncodingError].Err(std.encoding.EncodingError.InvalidPadding(3))

The number in an error is the index of the offending byte. aGl= is NonCanonical because its last symbol carries bits that belong to no byte: aGk= is the only accepted text for hi. Reading four bytes from a three-byte buffer traps. Validate untrusted record lengths before unpacking; Cursor field reads return Truncated when input can legitimately end before a field is complete.

Decoded bytes are not text. Call to_text() on the result when you expect UTF-8; it validates and can fail on its own.

Base32 and protocol-specific forms

Choose the encoding the receiving protocol expects. Base32 uses uppercase letters and digits 2 through 7, with padding. For a protocol requiring a different alphabet or no padding, build a validated encoding once and reuse it:

import std.bytes
import std.encoding

def main() -> Result[(), IoError | AllocError | std.bytes.BytesError | std.encoding.EncodingError]:
    data = std.bytes.from_text("hello")?
    print(std.encoding.base32_text(&data)?)?
    lower = std.encoding.custom("abcdefghijklmnopqrstuvwxyz234567", Nothing)?
    text = lower.text(&data)?
    print(text, lower.from_text(&text)?.to_text()?)?
    print(std.encoding.base64mime_from_text("aG Vs\r\nbG8=")?.to_text()?)?
    Ok(())
NBSWY3DP
nbswy3dp hello
hello

base32hex offers the alternate alphabet 0–9, A–V. For email body transfer, base64mime_text wraps padded base64 at 76 columns using CRLF between lines. Only the MIME decoder skips ASCII space, tab, CR and LF; other invalid characters, incorrect padding and nonzero trailing bits still fail. It does not parse MIME headers or append a final CRLF. Ordinary base64_from_text continues to reject all whitespace.

See the encoding reference for every operation, its errors, and its costs.

Take a URL apart and build one

A URL is several components joined by delimiters, and each component has its own rules about which characters may appear. std.url splits a URL into those components and puts data into one of them safely.

import std.url

def main() -> Result[(), AllocError | IoError | std.url.UrlError]:
    url = std.url.parse("https://example.com:8443/docs/a%20b?tag=x%26y&page=2#top", 2048)?
    print(url.scheme, url.path)?
    print(str.repr(&url.host)?, str.repr(&url.port)?)?
    print(str.repr(&url.userinfo)?, str.repr(&url.fragment)?)?
    match &url.query:
        case Some(query):
            for pair in &std.url.query_pairs(query)?:
                print(pair.key, "=", pair.value)?
        case Nothing:
            pass
    print(std.url.decode(&url.path)?)?
    print(url.to_string()?)?
    Ok(())
https /docs/a%20b
Option[str].Some("example.com") Option[i64].Some(8443)
Option[str].Nothing Option[str].Some("top")
tag = x&y
page = 2
/docs/a b
https://example.com:8443/docs/a%20b?tag=x%26y&page=2#top

parse takes the text and the longest length in bytes you are willing to accept. Components come back exactly as written, still percent-encoded: nothing is lowercased, decoded, or tidied, so format returns the original text. An optional component is Nothing when its delimiter is absent.

Decode a component only after the URL has been split. %26 in the query is data, and query_pairs splits at the real & before it decodes, which is why tag holds x&y.

Put data into a URL

To build a URL, encode each piece for the component it is going into, then fill in a Url:

import std.url
from std.url import Component, Pair, Url

def main() -> Result[(), AllocError | IoError | std.url.UrlError]:
    path = "/files/".concat(&std.url.encode("a/b report.txt", Component.Segment)?)?
    mut pairs = list[Pair]()?
    pairs.append(Pair("q", "fish & chips"))?
    pairs.append(Pair("lang", "en"))?
    query = std.url.query_text(&pairs)?
    url = Url("https", Nothing, Some("example.com"), Nothing, path, Some(query), Nothing)
    print(url.to_string()?)?
    print(std.url.encode("a/b", Component.Path)?, std.url.encode("a/b", Component.Segment)?)?
    Ok(())
https://example.com/files/a%2Fb%20report.txt?q=fish%20%26%20chips&lang=en
a/b a%2Fb

The Component says where the data is going. As a Segment, the / in a file name is data and is escaped; as a Path, it is a separator and is kept. query_text encodes every key and value as a QueryPart, which escapes & and =.

What is rejected

std.url follows RFC 3986 strictly and never repairs input:

import std.url
from std.url import Url

def main() -> Result[(), AllocError | IoError]:
    print(str.repr(std.url.parse("/docs/page", 2048))?)?
    print(str.repr(std.url.parse("http://example.com/a b", 2048))?)?
    print(str.repr(std.url.parse("http://example.com/100%", 2048))?)?
    print(str.repr(std.url.parse("http://good.example@evil.example@host/", 2048))?)?
    print(str.repr(std.url.parse("http://example.com:99999/", 2048))?)?
    print(str.repr(std.url.parse("http://example.com/", 10))?)?
    forged = Url("https", Nothing, Some("example.com/x"), Nothing, "", Nothing, Nothing)
    print(str.repr(forged.to_string())?)?
    Ok(())
Result[std.url.Url, std.url.UrlError].Err(std.url.UrlError.MissingScheme)
Result[std.url.Url, std.url.UrlError].Err(std.url.UrlError.InvalidByte(20))
Result[std.url.Url, std.url.UrlError].Err(std.url.UrlError.InvalidEscape(22))
Result[std.url.Url, std.url.UrlError].Err(std.url.UrlError.InvalidByte(32))
Result[std.url.Url, std.url.UrlError].Err(std.url.UrlError.InvalidPort(19))
Result[std.url.Url, std.url.UrlError].Err(std.url.UrlError.LimitExceeded)
Result[str, std.url.UrlError].Err(std.url.UrlError.InvalidByte(19))

The number is the byte offset of the problem. A URL must start with a scheme; relative references are not parsed. A second @ is an error rather than a guess about which host was meant. format checks a Url you built yourself with the same rules, so a host containing / cannot turn into a different URL.

Non-ASCII text must be percent-encoded, which encode does for you. A Unicode host name must already be in its xn-- form.

See the URL reference for the exact syntax, every operation, its errors, and its costs.

Draw random values and make secrets

Plenty has two sources of random values, and they are not interchangeable. std.random gives repeatable sequences for simulations, games, and tests. std.secrets gives unpredictable bytes from the operating system for tokens and keys.

For record and request identifiers, see std.uuid, which builds standard UUIDs from OS entropy.

Seeded generators

A generator is a value you create from a seed and own. Nothing is seeded for you and there is no shared generator behind the scenes, so the same seed always produces the same run:

import std.random

def main() -> Result[(), AllocError | IoError | std.random.RandomError]:
    mut rng = std.random.seeded(1234567u64)
    print(rng.next_u64())?
    print(rng.between(1, 7)?, rng.between(1, 7)?, rng.between(1, 7)?)?
    print(rng.next_bool(), rng.next_f64() < 1.0)?
    mut cards = ["ace", "king", "queen", "jack", "ten"]?
    rng.shuffle(&mut cards)
    print(cards)?
    print(rng.sample_indices(10, 3)?)?
    print(cards[rng.index(len(cards))?])?
    Ok(())
3504822795582309479
5 2 1
True True
["ace", "king", "ten", "queen", "jack"]
[3, 1, 9]
king

Drawing changes the generator, so it must be a mut binding. Ranges are half-open: between(1, 7) is a die roll from 1 to 6. Every bounded draw is unbiased. shuffle reorders a list in place without copying its elements. sample_indices(10, 3) picks three different positions out of ten; index your collection with them. index(count) picks one.

Because the sequence depends only on the seed, a failing simulation can be replayed exactly. state() and from_state checkpoint a generator part-way through:

import std.random

def main() -> Result[(), AllocError | IoError | std.random.RandomError]:
    mut first = std.random.seeded(42u64)
    mut second = std.random.seeded(42u64)
    print(first.next_u64() == second.next_u64())?
    (a, b, c, d) = first.state()
    mut resumed = std.random.from_state(a, b, c, d)?
    print(first.between(0, 100)? == resumed.between(0, 100)?)?
    print(str.repr(first.between(5, 5))?)?
    print(str.repr(first.below(0u64))?)?
    Ok(())
True
True
Result[i64, std.random.RandomError].Err(std.random.RandomError.EmptyRange)
Result[u64, std.random.RandomError].Err(std.random.RandomError.EmptyRange)

An empty range is an error, never a clamped or default value, because it can come from data such as an empty list. A rejected request does not advance the generator. An argument no call could accept, such as sample_indices(3, 4) asking for four of three positions, is a bug in the caller: it stops the program with a trap report.

The sequence for a seed is part of the language’s contract and does not change between Plenty versions. It is also predictable by anyone who sees a few outputs, so never use a seeded generator for a password, token, or key.

Secrets

std.secrets asks the operating system for every byte. Its results differ on every run, so this program prints sizes rather than values:

import std.bytes
import std.secrets

def main() -> Result[(), AllocError | IoError | std.bytes.BytesError | std.secrets.SecretsError]:
    token = std.secrets.token_urlsafe(32)?
    print(len(token))?
    print(len(std.secrets.token_hex(16)?))?
    key = std.secrets.token_bytes(32)?
    print(key.len())?
    print(token != std.secrets.token_urlsafe(32)?)?
    print(std.secrets.below(6u64)? < 6u64)?
    mut nonce = std.bytes.empty()?
    nonce.resize(12, 0u8)?
    std.secrets.fill(&mut nonce, 0, 12)?
    print(nonce.len())?
    print(str.repr(std.secrets.token_hex(-1))?)?
    Ok(())
43
32
32
True
True
12
Result[str, std.secrets.SecretsError].Err(std.secrets.SecretsError.Allocation(AllocError.CapacityOverflow))

The argument is a number of secret bytes, not characters: 32 bytes become 43 base64url characters or 64 hex digits. token_urlsafe is safe to put in a URL or a file name. below is an unbiased integer, for example to pick a winner.

A negative count is a size that cannot be allocated, so it returns Allocation(AllocError.CapacityOverflow).

If the system cannot supply entropy, these functions return SecretsError.Entropy. They never fall back to a weaker source, and a failed fill leaves zeros, not a partly random buffer.

To start a simulation from an unpredictable seed, pass std.secrets.random_u64()? to std.random.seeded and print the seed so the run can be repeated.

For normal, exponential or floating uniform samples, pass that generator to a std.stat distribution. Each distribution documents its draw count, and failed sampling leaves the generator unchanged.

See the random reference for the algorithm and its stability contract, and the secrets reference for the failure policy and costs.

Summarize observations and model uncertainty

Use std.stat for small numerical datasets and probability calculations. Choose a population statistic when the observations are the complete group you want to describe; choose a sample statistic when estimating the spread of a larger population from a sample.

import std.stat as stat

def main() -> Result[(), AllocError | IoError | stat.StatError]:
    measurements = [4.0, 1.0, 3.0, 2.0]?
    print(stat.mean(&measurements)?, stat.median(&measurements)?)?
    print(stat.quantiles(&measurements)?)?
    print(measurements)?
    Ok(())
2.5 2.5
[1.75, 2.5, 3.25]
[4.0, 1.0, 3.0, 2.0]

Median and quantiles copy and sort the input; mean and standard deviation do not allocate. Quantiles interpolate between observed values with the smallest and largest at probabilities zero and one. Inputs must be finite, and sample variance needs at least two observations.

Distributions model probabilities. A uniform distribution over a two-to-six second delay has accumulated one quarter of its probability by three seconds:

import std.stat as stat
import std.random as random

def main() -> Result[(), IoError | stat.StatError]:
    delays = stat.uniform(2.0, 6.0)?
    print(delays.cdf(3.0)?, delays.inverse_cdf(0.25)?)?
    mut first = random.seeded(42u64)
    mut repeated = random.seeded(42u64)
    print(delays.sample(&mut first) == delays.sample(&mut repeated))?
    bell = stat.normal()?
    print(bell.cdf(0.0)?)?
    Ok(())
0.25 3.0
True
0.5

normal(mean, sigma) and exponential(rate) also provide density, CDF, inverse CDF and sampling. Their samples can overflow and therefore return Result; failure leaves the supplied generator unchanged. Seed explicitly to reproduce a simulation, and use std.secrets for security-sensitive randomness. Normal/exponential calculations use platform math, so their last bits may differ between math-library versions.

The reference describes numeric limits, errors, costs and sampling’s fixed draw counts.

Identify records with UUIDs

Use std.uuid when records need identifiers that can be created independently, without asking a central counter. Choose v4() for a random identifier, or v7() when placing recently created records close together in a sorted index is useful. Both use operating-system entropy and report failure through Result.

import std.uuid as uuid
from std.secrets import SecretsError

def main() -> Result[(), IoError | SecretsError | RangeError]:
    request_id = uuid.v4()?
    record_id = uuid.v7()?
    print(request_id.version(), record_id.version())?
    Ok(())
Option[u8].Some(4) Option[u8].Some(7)

The identifiers change each run, so this example prints only their versions. v7 embeds a wall-clock timestamp in milliseconds. It does not promise creation order within a millisecond, or when the clock moves backwards. Use std.secrets tokens rather than UUIDs for passwords and access credentials.

Read, store and compare

Keep identifiers as Uuid values while working with them. Convert to canonical text for display, or to 16 bytes for binary storage:

import std.uuid as uuid
from std.bytes import BytesError

def main() -> Result[(), IoError | ParseError | BytesError]:
    id = uuid.parse("017F22E2-79B0-7CC3-98C4-DC0C0C07398F")?
    print(id.to_string()?)?
    raw = id.to_bytes()?
    restored = uuid.from_bytes(&raw)?
    print(raw.len(), restored == id)?
    print(id.unix_milliseconds())?
    Ok(())
017f22e2-79b0-7cc3-98c4-dc0c0c07398f
16 True
Option[u64].Some(1645557742000)

Parsing accepts either case but requires the hyphens exactly as shown. It rejects whitespace, braces and malformed digits. Formatting always produces lowercase text. The byte representation is big-endian, matching the text’s left-to-right byte order. Use to_string() explicitly: ordinary class printing is intended for inspecting fields, not interchange.

For a sorted list, borrow each UUID and use its pair of unsigned integer halves as the key. This avoids allocating a string per comparison:

import std.uuid as uuid
from std.bytes import BytesError

def uuid_key(id: &uuid.Uuid) -> (u64, u64):
    id.as_u64_pair()

def main() -> Result[(), AllocError | IoError | ParseError | BytesError]:
    mut records = [uuid.parse("017f22e2-79b1-7000-8000-000000000000")?,
                   uuid.parse("017f22e2-79b0-7cc3-98c4-dc0c0c07398f")?]?
    records.sort_by_key(uuid_key)
    for id in records:
        print(id.to_string()?)?
    Ok(())
017f22e2-79b0-7cc3-98c4-dc0c0c07398f
017f22e2-79b1-7000-8000-000000000000

UUIDs follow ordinary class ownership rules. Their comparison, parsing and numeric sorting keys need no allocation; creating output text or byte buffers does. See the reference for allocation-free writing into existing buffers and deterministic generator testing.

Summarizing a large stream

Use std.sketch when keeping every value costs too much memory and an approximate answer is useful. Keep exact data when correctness requires exact membership or counts. A sketch’s memory budget and accuracy are part of the choice.

This example counts distinct numeric customer identities, samples latency quantiles, and tracks frequent customer identities without retaining the stream:

import std.sketch as sketch
from std.sketch import SketchError
from std.bytes import BytesError

def main() -> Result[(), AllocError | IoError | SketchError | BytesError | ZeroDivisionError]:
    mut customers = sketch.hyperloglog(10)?
    mut latency = sketch.quantiles(128, seed=42u64)?
    mut frequent = sketch.frequent_items(16)?
    for index in range(1000):
        customer = u64(index % 100)
        customers.add(customer)
        latency.add(f64(index % 50))?
        frequent.add(customer)?
    print(latency.count(), latency.retained())?
    print(latency.quantile(0.0)?, latency.quantile(1.0)?)?
    print(customers.estimate() > 80.0 and customers.estimate() < 120.0)?
    Ok(())
1000 128
0.0 49.0
True

The quantile sketch retains a uniform sample and exact extrema. Its interior quantiles are sample estimates, not guaranteed rank bounds. Frequent-item queries return lower/upper count bounds, and candidates() returns possible heavy hitters. Increase capacity for better accuracy. See the reference for the different guarantees.

Skipping unnecessary lookups

A static xor filter can cheaply rule out keys before an expensive exact lookup. A positive answer still needs that lookup:

import std.sketch.xor as xor
from std.sketch import SketchError
from std.bytes import BytesError

def main() -> Result[(), AllocError | IoError | SketchError | BytesError]:
    keys = [10u64, 20u64, 30u64]?
    filter = xor.build(&keys, max_keys=100)?
    print(filter.len(), filter.contains(20u64))?
    Ok(())
3 True

For a changing set, std.sketch.cuckoo supports bounded growth and deletion. Insertion returns a layer receipt. Keep it with the real record; deletion must use that receipt and a known live occurrence, never just a positive membership test:

import std.sketch.cuckoo as cuckoo
from std.sketch import SketchError
from std.bytes import BytesError

def main() -> Result[(), AllocError | IoError | SketchError | BytesError]:
    mut filter = cuckoo.cuckoo(initial_buckets=16, max_buckets=112)?
    layer = filter.insert(42u64)?
    print(filter.contains(42u64))?
    print(filter.remove(42u64, layer)?, filter.len())?
    Ok(())
True
True 0

All three sketches and both filters provide versioned to_bytes() checkpoints and bounded readers. Save the application’s key-hashing convention and cuckoo receipts too. See xor filters and cuckoo filters before choosing one.

Draw a chart

std.plot turns data into SVG without an external plotting program. Its names follow ggplot2: a plot combines a table, mappings from column names to visual properties, and a list of layers and styling choices.

This example renders into a memory stream so the lesson creates no files:

import std.plot as gg
import std.io
import std.bytes
from std.plot import Column, PlotError
from std.bytes import BytesError

def main() -> Result[(), AllocError | IoError | PlotError | BytesError]:
    data = gg.table([
        ("hour", Column.Num([9.0, 12.0, 15.0]?)),
        ("temperature", Column.Num([16.0, 22.0, 20.0]?)),
    ]?)?
    chart = gg.ggplot(data, gg.aes("hour", "temperature"), [
        gg.geom_line(colour="navy"),
        gg.geom_point(colour="navy", size=2),
        gg.labs(title="Afternoon temperatures", x="Hour", y="Degrees C"),
        gg.theme_minimal(),
    ]?)?
    mut output = std.io.cursor(std.bytes.empty()?)
    notes = chart.write_svg(&mut output)?
    print(notes)?
    print(output.data.to_text()?.startswith("<svg"))?
    Ok(())
[]
True

For a file, replace the cursor and write_svg with notes = chart.save("temperatures.svg")?. Open the file in a browser or include it in a document. Saving borrows the chart, so you can save it again at another size. The default is 6 by 4 inches; named width, height, and units control dimensions. Only SVG is supported currently.

The distinction between mapping and setting matters. aes(colour="kind") looks up a column and creates a colour scale and legend. geom_point(colour="navy") uses one constant colour. Unknown column names fail when the plot is constructed.

Plot your own records

Typed accessors turn lists of ordinary classes into columns. Compute derived values in these accessors, where Plenty can type-check them; aes does not evaluate strings as expressions.

import std.plot as gg
import std.io
import std.bytes
from std.plot import PlotError
from std.bytes import BytesError

class Sale:
    region: str
    amount: f64

def main() -> Result[(), AllocError | IoError | PlotError | BytesError]:
    sales = [Sale("North", 12.0), Sale("South", 18.0), Sale("North", 5.0)]?
    data = gg.rows(&sales, [
        gg.text("region", def(s: &Sale) -> str: s.region)?,
        gg.num("amount", def(s: &Sale) -> f64: s.amount)?,
    ]?)?
    base = gg.ggplot(data, gg.aes("region", "amount", fill="region"), [
        gg.geom_col(),
        gg.scale_fill_manual({"North": "navy", "South": "#D55E00"}?),
        gg.theme_minimal(),
    ]?)?
    labelled = base.plus([gg.labs(title="Sales by region", y="Sales")]?)?
    mut output = std.io.cursor(std.bytes.empty()?)
    drop(labelled.write_svg(&mut output)?)
    print(len(sales), output.data.to_text()?.startswith("<svg"))?
    Ok(())
3 True

geom_col uses supplied heights and stacks rows at the same x by default. Use geom_bar with no y mapping to count rows instead, or geom_histogram(bins=20) with a continuous x mapping to count numeric intervals. For grouped bars, position=gg.position_dodge() places groups side by side.

The named palette keeps each region’s colour stable even if the input order changes. Include every observed region; extra names are allowed. For a numeric colour mapping, use gg.scale_colour_gradient(low="#132B43", high="#56B1F7") to choose endpoints instead. The legend follows the chosen colours. See manual palettes and gradients for interpolation and validation rules.

plus leaves the original plot unchanged and shares its immutable data. rows copies selected values, so the original records also remain usable. For columns you already own, table moves the lists without copying them.

Missing and non-finite mapped values are omitted, with a note returned by the renderer. Check these notes when interpreting a chart. The final example discards them deliberately because its small, fixed dataset contains no missing values. File/stream errors and allocation failures still propagate through ?. See the reference for supported layers, layout limits, and differences from the larger plotting proposal.

Combine plot layers and control axes

A summary often has different columns and fewer rows than its observations. Give that layer its own table and mapping. Use inherit_aes=False when it should not inherit grouping or other aesthetics from the observations.

import std.plot as gg
import std.io
import std.bytes
from std.plot import Column, PlotError
from std.bytes import BytesError

def main() -> Result[(), AllocError | IoError | PlotError | BytesError]:
    observations = gg.table([
        ("dose", Column.Num([1.0, 10.0, 100.0]?)),
        ("response", Column.Num([2.0, 4.0, 8.0]?)),
        ("site", Column.Text(["North", "South", "North"]?)),
    ]?)?
    reference = gg.table([
        ("dose", Column.Num([1.0, 100.0]?)),
        ("expected", Column.Num([2.0, 7.0]?)),
    ]?)?
    chart = gg.ggplot(observations, gg.aes("dose", "response", colour="site"), [
        gg.geom_point(size=2),
        gg.geom_line(data=reference, mapping=gg.aes("dose", "expected"),
                     inherit_aes=False, colour="navy"),
        gg.scale_x_log10(breaks=Some([1.0, 10.0, 100.0]?),
                         labels=Some(["1 mg", "10 mg", "100 mg"]?)),
        gg.labs(title="Observed and expected response"),
        gg.theme_minimal(),
    ]?)?
    mut output = std.io.cursor(std.bytes.empty()?)
    notes = chart.write_svg(&mut output)?
    print(notes, output.data.to_text()?.startswith("<svg"))?
    Ok(())
[] True

The line inherits no site grouping, so it connects the two reference points. Both layers train the same axes. The logarithmic x scale gives equal horizontal spacing to each tenfold increase, while breaks and labels use original doses. The explicit labels include units without changing the data.

To override only one mapping while keeping the others, use a layer argument such as mapping=gg.aes(y="expected") with the default inherit_aes=True. Every inherited name must exist in that layer’s table.

For a categorical axis, gg.scale_x_discrete(limits=Some(["South", "North"]?)) sets the display order and excludes any unlisted categories. Its labels argument can supply display names in the same order. It requires a discrete x mapping; the dose example uses a continuous one.

Use coord_cartesian(xlim=(1.0, 10.0)) to zoom while retaining all rows for statistics. A scale’s limits instead removes values before computation, which can change histogram counts. Values outside a transformation’s domain are also omitted and reported in the renderer’s notes. Check these notes when working with real input.

Long titles and labels wrap automatically. The renderer reserves space for their text boxes and checks for overlapping ticks before writing SVG. If it reports that the chart cannot fit, increase width or height, shorten the labels, or provide fewer breaks. For example, a presentation chart can use chart.write_svg(&mut output, width=10.0, height=6.0)?. Labels are never silently dropped to make the chart fit.

See position scales for the full contracts, including square-root and reversed axes, and draw a chart for file output and record accessors.

Compare panels and show ranges

Use facets to compare the same measurements across groups. Add error bars or a ribbon when your data includes lower and upper bounds. The plot draws these supplied values; calculating a confidence interval or another statistical summary is a separate step.

This example uses illustrative measurement bounds, not fitted confidence intervals:

import std.plot as gg
import std.io
import std.bytes
from std.plot import Column, PlotError
from std.bytes import BytesError

def main() -> Result[(), AllocError | IoError | PlotError | BytesError]:
    data = gg.table([
        ("day", Column.Num([1.0, 2.0, 3.0, 1.0, 2.0, 3.0]?)),
        ("value", Column.Num([2.0, 4.0, 3.0, 3.0, 5.0, 4.0]?)),
        ("lower", Column.Num([1.0, 2.0, 1.5, 2.0, 3.0, 2.5]?)),
        ("upper", Column.Num([3.0, 6.0, 4.5, 4.0, 7.0, 5.5]?)),
        ("site", Column.Text(["North", "North", "North",
                             "South", "South", "South"]?)),
    ]?)?
    chart = gg.ggplot(data, gg.aes("day", "value", ymin="lower", ymax="upper"), [
        gg.geom_ribbon(fill="blue", alpha=0.2),
        gg.geom_errorbar(colour="navy", width=0.15),
        gg.geom_line(colour="navy"),
        gg.geom_point(colour="navy"),
        gg.facet_wrap("site", ncol=2),
        gg.labs(title="Measurement ranges by site", x="Day", y="Value"),
        gg.theme_minimal(),
    ]?)?
    mut output = std.io.cursor(std.bytes.empty()?)
    notes = chart.write_svg(&mut output, width=10.0, height=4.0)?
    print(notes, output.data.to_text()?.startswith("<svg"))?
    Ok(())
[] True

Fixed axes let you compare heights directly between sites. Choose scales=gg.Scales.FreeY when each panel needs its own vertical range, but then compare the axis values rather than the visual heights. With two grouping variables, use gg.facet_grid(rows="region", cols="period").

A reference layer whose table omits site appears in every panel. Give it its own mapping with inherit_aes=False when it should not inherit aesthetics whose columns are absent; see combine plot layers. Histogram counts and bar stacks remain independent within each panel.

Both interval endpoints contribute to the axes. Missing bounds omit an error bar and interrupt a ribbon at that x; inspect the returned notes with real data. geom_linerange draws the same interval without caps. To display horizontal intervals, add coord_flip.

The larger output size gives two panels room for their labels. All panels are checked before writing SVG, so a chart that cannot fit reports an error instead of hiding labels. See facets and uncertainty and intervals for the full contracts and current limits.

Read and write JSON

std.json decodes and encodes JSON under explicit limits. It offers two levels: a value tree for documents that fit in memory, and events for documents that should not be held whole.

Parse a document into a tree

import std.json
from std.json import Json

def main() -> Result[(), AllocError | IoError | std.json.JsonError]:
    bounds = std.json.limits()
    value = std.json.parse_text("{\"name\": \"Ada\", \"tags\": [\"x\", \"y\"], \"age\": 36, \"ratio\": 0.5}", &bounds)?
    match &value:
        case Json.Object(members):
            for member in members:
                match &member.value:
                    case Json.Text(text):
                        print(member.key, "is the text", text)?
                    case Json.Int(number):
                        print(member.key, "is the integer", *number)?
                    case Json.Float(number):
                        print(member.key, "is the float", *number)?
                    case Json.Array(items):
                        print(member.key, "has", len(items), "items")?
                    case _:
                        print(member.key, "is something else")?
            print(std.json.find(members, "age"), std.json.find(members, "email"))?
        case _:
            print("not an object")?
    print(std.json.encode_text(&value, &bounds)?)?
    Ok(())
name is the text Ada
tags has 2 items
age is the integer 36
ratio is the float 0.5
Option[i64].Some(2) Option[i64].Nothing
{"name":"Ada","tags":["x","y"],"age":36,"ratio":0.5}

parse_text returns a Json value, an enum with one variant per JSON type. match is how you find out which one you have. A number written without a fraction or exponent is Json.Int; any other number is Json.Float.

An object is a list[Member] in source order, where each member has a key and a value. find returns the index of the first member with a key. Output is compact: no spaces or line breaks, and members stay in the order of the list.

Every call takes Limits. limits() gives the defaults: 64 levels of nesting, 64 MiB of input, 64 MiB of output, and duplicate keys rejected.

Errors say where

A JsonError carries a fault, the byte offset, and a line and column that both start at 1.

import std.json
from std.json import Limits

def attempt(text: &str, bounds: &Limits) -> Result[(), AllocError | IoError | std.json.JsonError]:
    match std.json.parse_text(text, bounds):
        case Ok(value):
            print("ok", std.json.encode_text(&value, bounds)?)?
        case Err(error):
            print(error.fault, "at byte", error.offset, "line", error.line, "column", error.column)?
    Ok(())

def main() -> Result[(), AllocError | IoError | std.json.JsonError]:
    mut bounds = std.json.limits()
    attempt("[1, 2,\n  oops]", &bounds)?
    attempt("[1, 2", &bounds)?
    attempt("{\"a\": 1, \"a\": 2}", &bounds)?
    attempt("[9223372036854775808]", &bounds)?
    attempt("[1] [2]", &bounds)?
    bounds.max_depth = 2
    attempt("[[1]]", &bounds)?
    attempt("[[[1]]]", &bounds)?
    Ok(())
std.json.JsonFault.UnexpectedByte at byte 9 line 2 column 3
std.json.JsonFault.UnexpectedEnd at byte 5 line 1 column 6
std.json.JsonFault.DuplicateKey at byte 9 line 1 column 10
std.json.JsonFault.NumberOutOfRange at byte 1 line 1 column 2
std.json.JsonFault.TrailingData at byte 4 line 1 column 5
ok [[1]]
std.json.JsonFault.DepthExceeded at byte 2 line 1 column 3

The decoder accepts standard JSON and nothing else: no comments, no trailing commas, one document per input. An integer that does not fit i64 is an error in the tree, never a rounded float. max_depth counts open arrays and objects, so the limit of 2 accepts [[1]] and stops at the third [.

Stream events instead of building a tree

A Decoder reports one Event at a time, and an Encoder accepts one call per event. Neither holds the document, so memory stays bounded however long the input is.

import std.bytes
import std.io
import std.json
from std.json import Event

def main() -> Result[(), IoError | AllocError | ParseError | std.json.JsonError | std.bytes.BytesError | OverflowError[i64]]:
    bounds = std.json.limits()
    mut reader = std.io.cursor(std.bytes.from_text("[{\"id\": 7, \"ok\": true}, {\"id\": 18446744073709551615}]")?)
    mut decoder = std.json.decoder(&bounds)?
    mut ids = 0
    mut expecting_id = False
    while True:
        event = std.json.read_event(&mut decoder, &mut reader)?
        match &event:
            case Event.End:
                break
            case Event.Key(name):
                expecting_id = name == "id"
            case Event.Number(text):
                if expecting_id:
                    ids += 1
                    print("id", u64.parse(text)?)?
            case _:
                pass
    print(ids, "ids in", decoder.offset(), "bytes")?

    mut encoder = std.json.encoder(&bounds)?
    encoder.start_object()?
    encoder.key("ids")?
    encoder.integer(ids)?
    encoder.key("note")?
    encoder.text("line one\nline two")?
    encoder.end_object()?
    mut out = std.io.cursor(std.bytes.empty()?)
    std.json.write_pending(&mut encoder, &mut out)?
    print(out.data.to_text()?)?
    match encoder.null():
        case Ok(_):
            print("written")?
        case Err(error):
            print(error.fault)?
    Ok(())
id 7
id 18446744073709551615
2 ids in 53 bytes
{"ids":2,"note":"line one\nline two"}
std.json.JsonFault.InvalidState

read_event pulls bytes from any std.io reader as the decoder needs them and returns Event.End after the document. Event.Number holds the number’s text exactly as written, so the program chooses the type: here u64, which holds a value the tree’s i64 cannot.

The encoder checks call order. A key outside an object, a value where a key belongs, or a second top-level value is InvalidState, so an encoder cannot produce malformed JSON. write_pending hands the bytes produced so far to a writer; call it between events to keep the buffer small.

The std.json reference lists every fault, limit, and cost.

Rendering text templates

Use std.template when the layout of a page or message is clearer as text than as a sequence of string concatenations. Register a layout once and pass values for each render. Templates cannot call your program’s functions or read files.

import std.template
from std.template import TemplateError
from std.json import Json, Member

def main() -> Result[(), TemplateError | AllocError | IoError]:
    bounds = std.template.limits()
    mut pages = std.template.engine(&bounds)?
    pages.add("layout", "<h1>{% block title %}Report{% endblock %}</h1>{% block body %}{% endblock %}")?
    pages.add("report", "{% extends 'layout' %}{% block title %}{{ title }}{% endblock %}{% block body %}<ul>{% for name in names %}<li>{{ name }}</li>{% else %}<li>No entries</li>{% endfor %}</ul>{% endblock %}")?
    values = Json.Object([
        Member("title", Json.Text("Team & friends")),
        Member("names", Json.Array([Json.Text("Ada"), Json.Text("Grace")]?)),
    ]?)
    print(pages.render("report", &values)?)?
    Ok(())
<h1>Team &amp; friends</h1><ul><li>Ada</li><li>Grace</li></ul>

The context uses the JSON value model explicitly; no class-to-template conversion happens automatically. Fields such as {{ person.name }} look up object members. A missing field returns an error rather than silently producing an empty string. An empty array takes the loop’s else branch, which is useful for empty reports.

Interpolated text is HTML-escaped by default. Keep user text in HTML text or quoted ordinary attributes. HTML escaping does not validate links or JavaScript; do not place user values in scripts, styles or tag names. For a plain-text message, call pages.render("report", &values, html=False) with a plain-text template. The safe filter deliberately disables escaping for trusted markup.

Set limits for your application before constructing the engine: for example, lower max_output for a short notification. Registration and rendering are fallible, and the example propagates those failures to the caller. Keep the engine and render it again with different values to reuse its parsed templates.

For a small fragment assembled in your program, use std.template.escape rather than a chain of string replacements. Choose HtmlContext.Text for text children or the quote-specific attribute mode for each attribute value. For example, quotes should stay literal in text but must be escaped inside a double-quoted attribute:

from std.template import escape, HtmlContext, TemplateError

def main() -> Result[(), TemplateError | AllocError | IoError]:
    label = "Say \"hello\" & smile"
    attribute = escape(&label, HtmlContext.DoubleQuotedAttribute, max_bytes=256)?
    child = escape(&label, HtmlContext.Text, max_bytes=256)?
    print(f"<button title=\"{attribute}\">{child}</button>"?)?
    Ok(())
<button title="Say &quot;hello&quot; &amp; smile">Say "hello" &amp; smile</button>

max_bytes includes the expanded entities. The helper returns ordinary text: passing its output into a template would escape it again. Keep raw values in Json.Text when using Engine.render, and use these helpers once per original value when assembling HTML directly. The context is your choice; the library does not detect HTML positions or validate URLs, scripts, styles or names.

See the std.template reference for the supported syntax, filters, inheritance rules and precise limits.

Solve systems and factor matrices

std.linalg gives you dense matrices and column vectors of f64 values. It uses faer’s numerical algorithms through an optional native engine. Install the linear algebra engine before running these examples. Matrix construction and element access work without it.

Solve a linear system

Pass matrix entries in row order. Use solve to find x in A x = b:

import std.linalg as la

def main() -> Result[(), AllocError | IoError | la.LinalgError]:
    a = la.matrix(2, 2, &[3.0, 1.0, 1.0, 2.0]?)?
    b = la.vector(&[9.0, 8.0]?)?
    x = a.solve(&b)?
    print(x.get(0, 0)?, x.get(1, 0)?)?
    predicted = (a * x)?
    residual = (predicted - b)?
    print(residual.to_list()?)?
    Ok(())
2.0 3.0
[0.0, 0.0]

Both inputs are preserved. The result is a two-row, one-column matrix. For several right-hand sides with the same a, compute factor = a.lu()? once, then call factor.solve(&b)? for each. This reuses the expensive factorization.

a * x is a matrix product, so it also applies a matrix to a column vector. Addition and subtraction combine corresponding entries of equal-shaped matrices. The parentheses in (predicted - b)? propagate the operation’s error. A small residual is useful evidence about a solve; for general floating-point inputs, compare it with a tolerance appropriate to your data.

Scale with (2.0 * x)? or x.scale(2.0)?, and negate with (-x)?. Scalar multiplication currently puts the scalar on the left: x * 2.0 is unsupported. These operations preserve their inputs and allocate new results. There is no automatic fusion of a chain or fallible +=; use an explicit assignment such as x = (x + adjustment)? with a mutable binding.

Inspect a low-rank approximation

SVD separates a matrix into two orthogonal factors and a list of singular values. Small singular values indicate directions that contribute little:

import std.linalg as la

def main() -> Result[(), AllocError | IoError | la.LinalgError]:
    a = la.diagonal(&[4.0, 2.0, 0.01]?)?
    (u, values, v) = a.svd()?
    print(values)?
    print(u.rows(), u.cols(), v.rows(), v.cols())?
    Ok(())
[4.0, 2.0, 0.01]
3 3 3 3

To reconstruct, create d = la.diagonal(&values)? and vt = v.transpose()?, then compute scaled = (u * d)? and reconstructed = (scaled * vt)?. For rectangular inputs the factors have only min(rows, cols) columns. qr, cholesky, eigh, and eig provide other decompositions; their contracts are in the linear algebra reference.

All matrix-building operations are fallible. Check shapes and handle errors normally with ?. Floating-point calculations are approximate: compare residuals with a tolerance appropriate to your data, not exact factor entries. A successful solve does not guarantee accuracy for a nearly singular matrix.

Hash and sign bytes

std.crypto computes BLAKE3, SHA-256 and SHA-512 digests and keyed tags over std.bytes buffers, and compares secrets in constant time. For AEAD encryption and public-key signatures, see Encrypt and authenticate bytes.

Digests

A digest is a fixed-size fingerprint of some bytes. The one-shot functions take a whole buffer; a Digest takes input in pieces, which gives the same result as hashing the pieces joined together:

import std.bytes
import std.crypto
import std.encoding
from std.crypto import Algorithm

type CryptoBufferError = std.encoding.EncodingError | std.crypto.CryptoError | std.bytes.BytesError

def main() -> Result[(), AllocError | IoError | CryptoBufferError]:
    data = std.bytes.from_text("abc")?
    digest = std.crypto.sha256(&data)?
    print(digest.len())?
    print(std.encoding.hex_text(&digest)?)?
    mut hasher = std.crypto.digest(Algorithm.Sha256)?
    hasher.update_text("a")?
    hasher.update_text("bc")?
    print(std.crypto.equal(&hasher.finish()?, &digest))?
    print(str.repr(hasher.update_text("more"))?)?
    Ok(())
32
ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad
True
Result[(), std.crypto.CryptoError].Err(std.crypto.CryptoError.Finished)

finish ends a digest. Feeding it afterwards is the error Finished, not a new hash; make a new Digest instead. update(data, start, stop) takes a range of a buffer, so a large input can be hashed while it is read, without first collecting it in memory.

BLAKE3

For BLAKE3, choose Algorithm.Blake3 or the one-shot blake3 function:

import std.bytes
import std.crypto
from std.crypto import Algorithm

def main() -> Result[(), IoError | AllocError | std.bytes.BytesError | std.crypto.CryptoError]:
    data = std.bytes.from_text("hello world")?
    expected = std.crypto.blake3(&data)?
    mut hasher = std.crypto.digest(Algorithm.Blake3)?
    hasher.update_text("hello ")?
    hasher.update_text("world")?
    print(expected.len(), std.crypto.equal(&hasher.finish()?, &expected))?
    Ok(())
32 True

BLAKE3 also has a native keyed hash with exactly 32 key bytes and a key derivation mode. Use blake3_keyed_hasher(key) for a Mac with the same update and verify methods used for HMAC below. It is a distinct algorithm: hmac(Algorithm.Blake3, key) reports UnsupportedAlgorithm. See the BLAKE3 reference for key derivation contexts, allocation costs and limits.

Signing with HMAC

A digest proves nothing about who made it: anyone can hash a message. An HMAC tag mixes in a secret key, so only a holder of the key can produce a tag that verifies:

import std.bytes
import std.crypto
import std.encoding
from std.bytes import Bytes
from std.crypto import Algorithm

def sign(key: &Bytes, message: &str) -> Result[Bytes, AllocError | IoError | std.crypto.CryptoError]:
    mut mac = std.crypto.hmac(Algorithm.Sha256, key)?
    mac.update_text(message)?
    Ok(mac.finish()?)

def check(key: &Bytes, message: &str, tag: &Bytes) -> Result[bool, AllocError | IoError | std.crypto.CryptoError]:
    mut mac = std.crypto.hmac(Algorithm.Sha256, key)?
    mac.update_text(message)?
    Ok(mac.verify(tag)?)

type CryptoBufferError = std.encoding.EncodingError | std.crypto.CryptoError | std.bytes.BytesError

def main() -> Result[(), AllocError | IoError | CryptoBufferError]:
    key = std.bytes.from_text("server key")?
    tag = sign(&key, "user=ada")?
    print(std.encoding.hex_text(&tag[:8])?)?
    print(check(&key, "user=ada", &tag)?)?
    print(check(&key, "user=bob", &tag)?)?
    other = std.bytes.from_text("another key")?
    print(check(&other, "user=ada", &tag)?)?
    Ok(())
8f60a639ef523af7
True
False
False

Check a tag with verify, never by finishing and comparing with ==. == stops at the first differing byte, so how long it takes reveals how much of a forged tag was right. verify and std.crypto.equal take the same time wherever the bytes differ.

A real key should be secret and random: make one with std.secrets.token_bytes(32) from Randomness and secrets rather than from text.

See the crypto reference for the guarantees, costs, and what the module does not provide.

Encrypt and authenticate bytes

std.crypto supplies authenticated encryption and public-key operations over Bytes. Use a defined protocol for exchanged messages: encryption alone does not manage identities, replays, message framing, or keys.

Encrypt one message

AEAD encryption protects both the message and associated data such as a protocol header. The associated data stays readable, but changing it makes verification fail. A recipient must have the same secret key, nonce, and associated data.

This example creates a fresh key for one message. When reusing a key, the application must ensure it never repeats a nonce, including across restarts. The API does not track that for you.

import std.bytes
import std.crypto
import std.secrets
from std.crypto import AeadAlgorithm

type BufferError = std.bytes.BytesError | std.crypto.AdvancedError | std.secrets.SecretsError

def main() -> Result[(), IoError | AllocError | BufferError]:
    key = std.secrets.token_bytes(32)?
    nonce = std.secrets.token_bytes(12)?
    header = std.bytes.from_text("example-v1")?
    mut message = std.bytes.from_text("meet at noon")?
    std.crypto.seal_in_place(AeadAlgorithm.ChaCha20Poly1305, &key, &nonce, &header, &mut message)?
    # The buffer now holds ciphertext followed by a 16-byte tag.
    print(message.len())?
    std.crypto.open_in_place(AeadAlgorithm.ChaCha20Poly1305, &key, &nonce, &header, &mut message)?
    print(message.to_text()?)?
    Ok(())
28
meet at noon

Opening returns an error and zeros the buffer if authentication fails; do not use its contents after an error. The ? above propagates that error. Reserve 16 extra bytes before sealing when the buffer needs to be reused without allocation. Opening never allocates.

Sign with a private seed

An Ed25519 signature allows someone with the public key to check a message without knowing the signing seed. The seed remains secret. A verifier also needs an authenticated source for the public key: accepting any key supplied alongside the signature proves no identity.

import std.bytes
import std.crypto
import std.secrets

type SigningError = std.bytes.BytesError | std.crypto.AdvancedError | std.secrets.SecretsError

def main() -> Result[(), IoError | AllocError | SigningError]:
    seed = std.secrets.token_bytes(32)?
    public_key = std.crypto.ed25519_public_key(&seed)?
    message = std.bytes.from_text("release manifest")?
    signature = std.crypto.ed25519_sign(&seed, &message)?
    print(std.crypto.ed25519_verify(&public_key, &message, &signature)?)?
    changed = std.bytes.from_text("modified manifest")?
    print(std.crypto.ed25519_verify(&public_key, &changed, &signature)?)?
    Ok(())
True
False

Signing can reuse its seed. Keys and signatures have fixed lengths; malformed lengths are errors, while a well-sized invalid signature returns False. A public key that anyone could sign for, one of small order, is the error SmallOrderKey.

Derive a key after an exchange

Ephemeral X25519 keys can each perform one agreement. Exchange their public keys first, authenticate the exchange through the application’s protocol, and derive an application key with HKDF. The example runs both peers locally; it demonstrates agreement and key derivation, without providing a network authentication protocol.

import std.bytes
import std.crypto

def main() -> Result[(), IoError | AllocError | std.bytes.BytesError | std.crypto.AdvancedError]:
    mut alice = std.crypto.x25519()?
    mut bob = std.crypto.x25519()?
    alice_public = alice.public_key()?
    bob_public = bob.public_key()?
    alice_shared = alice.agree(&bob_public)?
    bob_shared = bob.agree(&alice_public)?
    salt = std.bytes.empty()?
    context = std.bytes.from_text("example-v1 message key")?
    alice_key = std.crypto.hkdf_sha256(&salt, &alice_shared, &context, 32)?
    bob_key = std.crypto.hkdf_sha256(&salt, &bob_shared, &context, 32)?
    print(std.crypto.equal(&alice_key, &bob_key))?
    Ok(())
True

An agreement attempt consumes the private key even if the peer is rejected; length and allocation errors leave it usable. HKDF separates key material by purpose using context; it is not a password hashing function. None of these byte buffers is automatically scrubbed when dropped. See the key material and allocation contracts before storing secrets or designing a protocol.

Log structured events

std.log records events: a level, a message, and typed fields. Where the events go is up to the program. There is no global logger; you build one around a sink and pass it to the code that logs.

A logger and a sink

A text sink writes each event as one line of key=value pairs to any std.io writer. This one writes to standard output so the lines appear below; a command-line tool would use std.io.stderr() instead:

import std.io
import std.log
from std.log import Level, Clock

def main() -> Result[(), AllocError | IoError | std.log.LogError]:
    sink = std.log.text_sink(std.io.stdout(), 1024)?
    mut log = std.log.logger(sink, Level.Info, Clock.Off)
    log.info("listening", [std.log.integer("port", 8080), std.log.text("host", "db.local")]?)?
    log.debug("not shown", []?)?
    log.warn("slow request", [std.log.floating("seconds", 1.5), std.log.boolean("retried", True)]?)?
    print(log.emitted(), log.filtered())?
    Ok(())
level=info msg="listening" port=8080 host="db.local"
level=warn msg="slow request" seconds=1.5e0 retried=true
2 1

The threshold Level.Info drops the debug event before it reaches the sink. Text values are quoted and escaped, so one event is always one line. Clock.Off leaves out the time; Clock.Wall stamps each event with the current wall-clock time.

The fields of a dropped event were still built. When a field is costly to compute, ask first with log.enabled(&Level.Debug).

Passing a logger to a library

Code that logs takes the logger as a parameter. Making it generic over the sink lets each caller pick the destination:

import std.log
from std.log import Level, Clock, Logger

def connect[S](log: &mut Logger[S], attempts: i64) -> Result[(), AllocError | std.log.LogError | OverflowError[i64]]:
    for attempt in range(attempts):
        log.warn("retrying", [std.log.integer("attempt", attempt + 1)]?)?
    log.info("connected", []?)?
    Ok(())

def main() -> Result[(), AllocError | IoError | std.log.LogError | AllocError | OverflowError[i64]]:
    mut log = std.log.logger(std.log.memory_sink(10)?, Level.Trace, Clock.Off)
    connect(&mut log, 2)?
    for event in &log.sink.events:
        print(std.log.level_name(&event.level), event.message, len(event.fields))?
    Ok(())
warn retrying 1
warn retrying 1
info connected 0

A memory sink keeps events in a list, which suits tests. It holds at most the capacity it was given; after that, logging returns the error Full and the event is dropped.

Threads

Threads do not share one logger. Each builds its own around a channel sink, and one thread writes the queued events out. A channel sink never waits: when the queue is full, the event is dropped with Full, so slow output cannot stall the rest of the program. See the log reference for channel_sink, share, and drain.

Read and write TOML

std.toml parses TOML 1.0 documents, such as configuration files and project manifests, into a value tree, and writes a tree back as TOML text. Every call takes explicit limits.

Parse a configuration file

import std.toml
from std.toml import Entry, Toml

def show(entries: &list[Entry], indent: &str) -> Result[(), AllocError | IoError]:
    for entry in entries:
        name = (indent + entry.key)?
        match &entry.value:
            case Toml.Table(inner):
                print((name + ":")?)?
                show(inner, &(indent + "  ")?)?
            case Toml.Text(text):
                print(name, "is the text", text)?
            case Toml.Int(number):
                print(name, "is the integer", *number)?
            case Toml.Array(items):
                print(name, "has", len(items), "items")?
            case Toml.DateTime(moment):
                print(name, "is a date in", moment.year)?
            case _:
                print(name, "is something else")?
    Ok(())

def main() -> Result[(), AllocError | IoError | std.toml.TomlError]:
    bounds = std.toml.limits()
    text = "# Service settings\ntitle = \"cache\"\n\n[server]\nhost = \"0.0.0.0\"\nport = 8080\n\n[server.tls]\nenabled = true\n\n[[backend]]\nurl = \"http://a.internal\"\n\n[[backend]]\nurl = \"http://b.internal\"\n\n[release]\ndate = 2026-10-01\n"
    value = std.toml.parse_text(text, &bounds)?
    match &value:
        case Toml.Table(root):
            show(root, "")?
            print(std.toml.find(root, "server"), std.toml.find(root, "client"))?
        case _:
            print("a document is always a table")?
    Ok(())
title is the text cache
server:
  host is the text 0.0.0.0
  port is the integer 8080
  tls:
    enabled is something else
backend has 2 items
release:
  date is a date in 2026
Option[i64].Some(1) Option[i64].Nothing

A document is always a Toml.Table: a list[Entry] in source order, where each entry has a key and a value. [server] and [server.tls] become nested tables, and each [[backend]] adds one table to the array backend. find returns the index of the first entry with a key.

Toml has one variant per TOML type: Bool, Int, Float, Text, DateTime, Array, and Table. Integers are i64. A DateTime holds the date and time fields, its offset from UTC in minutes, and a kind that says which parts the text gave: an offset date-time, a local date-time, a local date, or a local time.

Errors say where

A TomlError carries a fault, the byte offset, and a line and column that both start at 1.

import std.toml
from std.toml import Limits

def attempt(text: &str, bounds: &Limits) -> Result[(), IoError]:
    match std.toml.parse_text(text, bounds):
        case Ok(_):
            print("ok")?
        case Err(error):
            print(error.fault, "at line", error.line, "column", error.column)?
    Ok(())

def main() -> Result[(), AllocError | IoError]:
    bounds = std.toml.limits()
    attempt("name = \"Tom\"\nname = \"Pradyun\"", &bounds)?
    attempt("[fruit]\na = 1\n[fruit]\nb = 2", &bounds)?
    attempt("point = { x = 1 }\npoint.y = 2", &bounds)?
    attempt("port = 08080", &bounds)?
    attempt("day = 2026-02-29", &bounds)?
    attempt("key = \"unterminated", &bounds)?
    Ok(())
std.toml.TomlFault.DuplicateKey at line 2 column 1
std.toml.TomlFault.DuplicateTable at line 3 column 1
std.toml.TomlFault.KeyConflict at line 2 column 1
std.toml.TomlFault.InvalidNumber at line 1 column 8
std.toml.TomlFault.InvalidDateTime at line 1 column 7
std.toml.TomlFault.UnexpectedEnd at line 1 column 20

TOML forbids defining a key or table twice, and an inline table such as { x = 1 } is complete as written, so point.y cannot add to it. Integers may not have leading zeros, and dates must exist. An error about a key or a table points at the start of the line that repeats it.

Write a tree as TOML

import std.toml
from std.toml import Entry, Toml

def main() -> Result[(), AllocError | IoError | std.toml.TomlError]:
    bounds = std.toml.limits()
    mut package = list[Entry]()?
    package.append(Entry("name", Toml.Text("player")))?
    package.append(Entry("version", Toml.Text("0.1.0")))?
    mut authors = list[Toml]()?
    authors.append(Toml.Text("Ada"))?
    package.append(Entry("authors", Toml.Array(authors)))?
    mut root = list[Entry]()?
    root.append(Entry("package", Toml.Table(package)))?
    root.append(Entry("manifest-version", Toml.Int(1)))?
    text = std.toml.encode_text(&Toml.Table(root), &bounds)?
    print(text.rstrip()?)?
    Ok(())
manifest-version = 1

[package]
name = "player"
version = "0.1.0"
authors = ["Ada"]

The writer puts each table’s plain keys first and its tables after them as [section] headers, because a key written after a header would belong to that header’s table. Parsing the output gives back the same values. The writer does not keep comments or the original layout, so it suits generated files rather than editing a file a person maintains.

Update a manifest while keeping comments

For a file maintained by a person, use a Document. Replacing a value keeps the surrounding spelling and comments:

import std.toml
from std.toml import Toml

def main() -> Result[(), AllocError | IoError | std.toml.TomlError]:
    source = "# release settings\n[package]\nversion = '0.1.0' # public version\n"
    mut document = std.toml.document_text(source, &std.toml.limits())?
    document.set(&["package", "version"]?, &Toml.Text("0.2.0"))?
    document.insert(&["package", "publish"]?, &Toml.Bool(False))?
    print(document.text()?.rstrip()?)?
    Ok(())
# release settings
[package]
version = "0.2.0" # public version
publish = false

Paths list decoded keys, so a key containing a dot is one list element. get reads an owned value; remove deletes a key or a whole section. Failed edits leave the document unchanged. Arrays and inline tables are replaced as whole values; this API does not edit individual array rows or inline fields. See the editing reference for the exact preservation rules and costs.

The std.toml reference lists every fault, limit, and cost.

Compress bytes and streams

std.compress reads and writes gzip, zlib, raw DEFLATE and Brotli data. Whole buffers go through compress and decompress; streams go through a Compressor writer and a Decompressor reader from std.io.

Whole buffers

compress takes a format, the data, and a level from 0 (fastest) to 9 (smallest). decompress takes a limit on the output size and fails rather than produce more:

import std.bytes
import std.compress
from std.compress import Format

def main() -> Result[(), AllocError | IoError | std.compress.CompressError | std.bytes.BytesError]:
    mut text = std.bytes.empty()?
    for _ in range(0, 100):
        text.extend(&std.bytes.from_text("to be or not to be ")?)?
    packed = std.compress.compress(Format.Gzip, &text, 6)?
    print(text.len())?
    print(packed.len())?
    plain = std.compress.decompress(Format.Gzip, &packed, 10000)?
    print(plain == text)?
    print(str.repr(std.compress.decompress(Format.Gzip, &packed, 1000))?)?
    Ok(())
1900
59
True
Result[std.bytes.Bytes, std.compress.CompressError].Err(std.compress.CompressError.LimitExceeded)

Repetitive text shrinks a lot. That is also the danger in decompressing: a small input can expand a thousandfold. Always choose the limit from the memory the program can spend, never from the size of the input.

Brotli

Use Format.Brotli for Brotli files or HTTP br bodies. It has the same functions and stream wrappers, with quality levels 0–11 instead of 0–9. Quality 5 is a useful starting point. Always choose the decoded-output limit from your application’s budget, independently of the compressed size.

import std.bytes
import std.compress
from std.compress import Format

def main() -> Result[(), IoError | AllocError | std.bytes.BytesError | std.compress.CompressError]:
    source = std.bytes.from_text("a small Brotli message")?
    packed = std.compress.compress(Format.Brotli, &source, 5)?
    restored = std.compress.decompress(Format.Brotli, &packed, 1024)?
    print(restored.to_text()?)?
    Ok(())
a small Brotli message

Brotli allocates working memory during streaming and has no checksum; the output limit does not cap all codec memory. For HTTP, check the actual Content-Encoding before decoding. The Brotli reference explains these differences and HTTP handling.

Streams

A Compressor wraps any writer and is itself a writer, so write_all and copy_all work with it. finish writes the end of the stream; without it the output is incomplete. A Decompressor wraps any reader and is itself a reader, decoding as it goes in fixed-size buffers:

import std.bytes
import std.compress
import std.io
from std.compress import Format

type CompressedStreamError = std.io.StreamError | std.io.Partial | std.compress.CompressError | std.bytes.BytesError

def main() -> Result[(), IoError | AllocError | CompressedStreamError | OverflowError[i64]]:
    line = std.bytes.from_text("one line of a log\n")?
    mut writer = std.compress.compressor(Format.Gzip, std.io.cursor(std.bytes.empty()?), 6)?
    for _ in range(0, 1000):
        std.io.write_all(&mut writer, &line)?
    writer.finish()?
    packed = writer.inner.data.copy()?
    print(packed.len())?
    mut reader = std.compress.decompressor(Format.Gzip, std.io.cursor(packed), 1000000)?
    mut buffer = std.bytes.with_capacity(4096)?
    buffer.resize(4096, 0u8)?
    mut lines = 0
    while True:
        count = reader.read(&mut buffer, 0, 4096)?
        if count == 0:
            break
        lines += buffer.count(10u8, 0, count)
    print(lines)?
    Ok(())
185
1000

The reader counts 18,000 bytes of lines while holding at most 4096 of them, so the same loop handles a file much larger than memory. To compress a file, wrap std.io.open_file(path, "w")? instead of a cursor.

Damaged input

gzip and zlib end with a checksum of the original data, so a damaged stream is caught. decompress returns only an error. A Decompressor reports it through the Reader protocol as a StreamError, and failure names the exact cause:

import std.bytes
import std.compress
import std.io
from std.compress import Format

def main() -> Result[(), AllocError | IoError | std.compress.CompressError | std.bytes.BytesError | OverflowError[i64] | OverflowError[u8]]:
    data = std.bytes.from_text("checked at the end")?
    mut packed = std.compress.compress(Format.Gzip, &data, 6)?
    last = packed.len() - 1
    packed.set(last, packed.get(last) + 1u8)
    print(str.repr(std.compress.decompress(Format.Gzip, &packed, 100))?)?
    mut reader = std.compress.decompressor(Format.Gzip, std.io.cursor(packed), 100)?
    mut out = std.bytes.empty()?
    match std.io.read_up_to(&mut reader, &mut out, 100):
        case Ok(_):
            print("read it all")?
        case Err(partial):
            print(str.repr(partial.cause)?)?
            print(str.repr(reader.failure())?)?
    Ok(())
Result[std.bytes.Bytes, std.compress.CompressError].Err(std.compress.CompressError.Checksum)
std.io.StreamError.Io(IoError.InvalidInput)
Option[std.compress.CompressError].Some(std.compress.CompressError.Checksum)

The checksum comes last, so a reader may already have returned some bytes before it finds the damage. Treat streamed output as unverified until read returns 0.

See the std.compress reference for every error, the rules for input after a stream, and memory costs.

Read and write archives

std.archive reads and writes TAR and ZIP archives over std.io streams, and extracts them safely into a new directory.

Write and list a TAR archive

A TarWriter wraps any writer. Each add_ call writes one entry, and finish writes the end of the archive. A TarReader returns the entries in order; between calls to next, it is a reader of the current entry’s content:

import std.archive
import std.bytes
import std.io
import std.path

def main() -> Result[(), IoError | AllocError | std.io.Partial | std.archive.ArchiveError | std.bytes.BytesError]:
    mut writer = std.archive.tar_writer(std.io.cursor(std.bytes.empty()?))?
    writer.add_dir(&std.path.from_text("notes")?, 493, 1700000000)?
    writer.add_file(&std.path.from_text("notes/todo.txt")?, &std.bytes.from_text("buy milk")?, 420, 1700000000)?
    writer.finish()?
    packed = writer.inner.data.copy()?
    print(packed.len())?
    mut reader = std.archive.tar_reader(std.io.cursor(packed))?
    while True:
        match reader.next()?:
            case Some(entry):
                mut content = std.bytes.empty()?
                drop(std.io.read_up_to(&mut reader, &mut content, 1000)?)
                print(decode_utf8(entry.name.as_bytes())?, entry.size)?
                if content.len() > 0:
                    print(content.to_text()?)?
            case Nothing:
                break
    Ok(())
2560
notes/ 0
notes/todo.txt 8
buy milk

TAR stores each header and each file’s content in 512-byte blocks and ends with two empty blocks, so even a tiny archive takes a few kilobytes. To compress it, wrap the writer in a std.compress Compressor.

ZIP archives

A ZIP archive keeps a directory of its entries at the end, so a ZipReader needs a stream it can seek: a file or a cursor. entries lists everything; open selects one entry to read:

import std.archive
import std.bytes
import std.io
import std.path
from std.archive import Method

def main() -> Result[(), IoError | AllocError | std.io.Partial | std.archive.ArchiveError | std.bytes.BytesError]:
    mut writer = std.archive.zip_writer(std.io.cursor(std.bytes.empty()?))?
    mut text = std.bytes.empty()?
    for _ in range(0, 50):
        text.extend(&std.bytes.from_text("all work and no play ")?)?
    writer.add_file(&std.path.from_text("story.txt")?, &text, Method.Deflate, 420, 1700000000)?
    writer.add_file(&std.path.from_text("short.txt")?, &std.bytes.from_text("hi")?, Method.Stored, 420, 1700000000)?
    writer.finish()?
    mut reader = std.archive.zip_reader(std.io.cursor(writer.inner.data.copy()?), 100)?
    entries = reader.entries()
    for entry in entries:
        print(decode_utf8(entry.name.as_bytes())?, entry.size, entry.compressed)?
    reader.open(1)?
    mut content = std.bytes.empty()?
    drop(std.io.read_up_to(&mut reader, &mut content, 100)?)
    print(content.to_text()?)?
    Ok(())
story.txt 1050 35
short.txt 2 2
hi

The reader checks each entry’s size and CRC-32 when its content ends.

Extract safely

An archive is untrusted input. Its names may point outside the target (../../.bashrc), its links may lead elsewhere, and a small archive may expand into terabytes. extract_tar and extract_zip check every entry against a Policy, write into a hidden staging directory, and rename it to the destination only when everything succeeded:

import std.archive
import std.bytes
import std.fs
import std.io
import std.path
from std.sync import CancellationToken

type ArchiveOperationError = std.archive.ArchiveError | std.bytes.BytesError | std.fs.FsError

def main() -> Result[(), AllocError | IoError | ArchiveOperationError]:
    parent = std.fs.root(&std.path.from_text(".")?)?
    mut scratch = std.fs.temporary_dir(&parent, &std.path.from_text("archive-lesson-")?, 8)?
    mut writer = std.archive.tar_writer(std.io.cursor(std.bytes.empty()?))?
    writer.add_file(&std.path.from_text("site/index.html")?, &std.bytes.from_text("<h1>hi</h1>")?, 420, 0)?
    writer.add_symlink(&std.path.from_text("site/secret")?, &std.path.from_text("/etc/passwd")?, 0)?
    writer.finish()?
    packed = writer.inner.data.copy()?

    cancel = CancellationToken()?
    mut policy = std.archive.policy()
    mut first = std.archive.tar_reader(std.io.cursor(packed.copy()?))?
    match std.archive.extract_tar(&mut first, scratch.root(), &std.path.from_text("out")?, &policy, &cancel):
        case Ok(_):
            print("extracted")?
        case Err(error):
            print(error)?
    policy.skip_links = True
    mut second = std.archive.tar_reader(std.io.cursor(packed))?
    summary = std.archive.extract_tar(&mut second, scratch.root(), &std.path.from_text("out")?, &policy, &cancel)?
    print(summary.files, summary.bytes, summary.skipped)?

    std.fs.remove_tree(scratch.root(), &std.path.from_text("out")?, 10, 4, &cancel)?
    scratch.cleanup()?
    Ok(())
std.archive.ArchiveError.Link
1 11 1

The first attempt fails on the link and leaves nothing behind. The second skips it. Extracted files get private permissions (0600); stored modes and times are not applied.

The default policy allows 10000 entries, 1 GiB per file, 4 GiB in total, and a compression ratio of 100 for large ZIP entries. Lower the limits to what your program expects.

See the std.archive reference for the format rules, every error, and memory costs.

Build a package with dependencies

A package is a directory with a plenty.toml manifest and sources under src/. plenty new hello creates one; plenty new shapes --lib creates a library. Here is a library package in shapes/:

manifest-version = 1

[package]
name = "shapes"
version = "0.1.0"

src/lib.plenty is the library: the module that other packages import.

pub class Square:
    pub side: i64

    pub def area(self) -> Result[i64, OverflowError[i64]]:
        Ok(self.side * self.side)

An application beside it declares the library as a dependency. The key under [dependencies] is the name that application code imports, and path is relative to this manifest:

manifest-version = 1

[package]
name = "app"
version = "0.1.0"

[dependencies]
geometry = { path = "../shapes" }

src/main.plenty is the executable, named after the package:

import geometry
from report import show

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    square = geometry.Square(4)
    show(square.area()?)

Other files under src/ are the package’s own modules, imported as before:

pub def show(area: i64) -> Result[(), IoError]:
    print("area:")?
    print(area)?
    Ok(())

Run plenty run anywhere inside app/:

plenty run
area:
16

plenty run compiles into target/optimized/app and runs it. Progress messages go to standard error; program arguments follow --, as in plenty run -- a b. Builds optimize by default. For shorter edit-run cycles in large projects, use plenty run --fast-compile or plenty build --fast-compile; these write to target/fast-compile/ and keep the same overflow, bounds and ownership checks. Use the default profile when measuring runtime performance. A later build with no changed source reports Fresh and reuses the executable.

The first build also writes plenty.lock, which records the resolved packages. Commit it with the manifest. --locked makes a build fail instead of changing it.

A dependency can also come from a Git repository, at a tag or a commit. The official time-zone package is one:

[dependencies]
tz = { git = "https://github.com/plentylang/tz.git", tag = "v0.3.0", version = "0.3.0" }

The first build fetches the repository into .pkg/ and records the exact commit in plenty.lock. Later builds use that commit even if the tag moves upstream, until plenty update resolves it again. The optional version checks that the dependency’s own version is compatible. plenty add writes these entries and updates the lockfile:

plenty add geometry --path ../shapes
plenty add tz --git https://github.com/plentylang/tz.git --tag v0.3.0

Several related packages can share one lockfile and target/ as a workspace.

A dependency’s name belongs only to the package that declares it. app cannot import what shapes depends on without declaring it too, and a local module named geometry would be an error. Two packages may each define Square; those are distinct types.

The dependency’s name also reaches its other modules. If shapes had src/circles.plenty, import geometry.circles would bind it beside import geometry, unless the library itself declares something named circles; modules and imports gives the rule.

Programs in tests/ are integration tests. Each one sees the package’s library under the package name, and passes when main returns exit status zero:

from shapes import Square

def main() -> Result[i32, OverflowError[i64]]:
    if Square(3).area()? == 9:
        Ok(0i32)
    else:
        Ok(1i32)
plenty test
running 1 test
test area ... ok

test result: ok. 1 passed; 0 failed

A later lesson shows test functions and std.test. plenty check type-checks every target without linking. It reaches only the files that targets import, so it warns about any other .plenty file under src/ and ends with a summary such as Checked app v0.1.0: 1 executable; 2 of 2 files in src/. Keep test fixtures and other files that are not package source outside src/. plenty clean removes target/. Single files still run with plenty FILE; the compiler is also available alone as plentyc.

Match text with regular expressions

std.regex compiles a pattern once into a Regex, then searches text with it. Matching takes time proportional to the length of the text, whatever the pattern: there is no backtracking, so no pattern can make a search run for hours. compile takes Limits, which also bound the pattern itself, so a pattern from an untrusted source is safe to compile.

Keep the compiled Regex when searching repeatedly. The library selects prefix filters and faster matching engines where the pattern permits them; captures, match ordering and resource limits keep the same meaning.

Find matches

find returns the first match as byte offsets; text copies the matched part. find_all returns every match and fails rather than return more than its limit:

import std.regex

def main() -> Result[(), AllocError | IoError | std.regex.RegexError]:
    limits = std.regex.limits()
    year = std.regex.compile("\\b(19|20)\\d\\d\\b", &limits)?
    text = "Released 1999, revised 2024, not 12345."
    match year.find(text)?:
        case Some(found):
            print(found.start, found.end, found.text(text)?)?
        case Nothing:
            print("no year")?
    print(len(year.find_all(text, 10)?))?
    print(year.is_match("in 1850")?)?
    print(str.repr(year.find_all(text, 1))?)?
    Ok(())
9 13 1999
2
False
Result[list[std.regex.Match], std.regex.RegexError].Err(std.regex.RegexError.LimitExceeded(std.regex.Limit.Matches))

A Plenty string needs \\ for each backslash in the pattern. \d, \w, \s and \b are ASCII-only; for other scripts, use properties such as \p{Alphabetic}.

Groups and replacement

Parentheses capture. (?P<name>...) names a group. captures returns every group of the first match; a group that did not take part is Nothing. In a replacement template, $1 and ${name} insert a group and $$ inserts a dollar sign:

import std.regex

def main() -> Result[(), AllocError | IoError | std.regex.RegexError]:
    limits = std.regex.limits()
    date = std.regex.compile("(?P<year>\\d{4})-(?P<month>\\d{2})", &limits)?
    text = "from 2024-05 to 1999-12"
    match date.captures(text)?:
        case Some(groups):
            match groups.name("year"):
                case Some(group):
                    print(group.text(text)?)?
                case Nothing:
                    print("group did not match")?
            match groups.get(2):
                case Some(group):
                    print(group.text(text)?)?
                case Nothing:
                    print("group did not match")?
        case Nothing:
            pass
    print(date.replace_all(text, "${month}/$1", 100)?)?
    words = std.regex.compile("\\s*,\\s*", &limits)?
    print(" | ".join(words.split("red , green,blue", 10)?)?)?
    Ok(())
2024
05
from 05/2024 to 12/1999
red | green | blue

The last argument of replace_all and split is a limit on the result size or the number of pieces. Write ${1}0 when a group number is followed by a digit. A bare $name is an error: use ${name}.

Errors and limits

A pattern error reports its byte offset. A pattern that would build too large a matcher, and a search that would do too much work, are limit errors:

import std.regex
from std.regex import Limits

def main() -> Result[(), AllocError | IoError | std.regex.RegexError]:
    limits = std.regex.limits()
    print(str.repr(std.regex.compile("total: (\\d+", &limits))?)?
    print(str.repr(std.regex.compile("(a{1000}){1000}", &limits))?)?
    tight = Limits(1000, 16, 100, 1000, 1000000, 100)
    word = std.regex.compile("\\w+!", &tight)?
    print(str.repr(word.find("no exclamation mark in this sentence"))?)?
    Ok(())
Result[std.regex.Regex, std.regex.RegexError].Err(std.regex.RegexError.Syntax(std.regex.SyntaxError(offset=7, message="unclosed group")))
Result[std.regex.Regex, std.regex.RegexError].Err(std.regex.RegexError.LimitExceeded(std.regex.Limit.Instructions))
Result[Option[std.regex.Match], std.regex.RegexError].Err(std.regex.RegexError.LimitExceeded(std.regex.Limit.Work))

The default limits suit patterns written into a program. For patterns from users, lower them to what a request may spend.

See the std.regex reference for the full syntax, every limit, and searching bytes.

Write tests with std.test

plenty test runs a package’s tests. A file in tests/ holds either one program with main, as in the packages lesson, or any number of test_ functions. Each test function runs in its own process, so a trap in one fails only that test.

Here is a small library to test:

manifest-version = 1

[package]
name = "stats"
version = "0.1.0"
from std.bytes import from_text
from std.io import Writer, Partial, write_all
from std.bytes import BytesError

pub def total(values: &list[i64]) -> Result[i64, OverflowError[i64]]:
    mut sum = 0
    for value in values:
        sum += *value
    Ok(sum)

pub def mean(values: &list[i64]) -> Result[f64, OverflowError[i64]]:
    Ok(f64(total(values)?) / f64(len(values)))

pub def save(path: &str, values: &list[i64]) -> Result[(), AllocError | IoError | OverflowError[i64]]:
    write_text(path, &str.from(total(values)?)?)?
    Ok(())

pub def report[W: Writer](out: &mut W, values: &list[i64]) -> Result[(), AllocError | BytesError | Partial | OverflowError[i64]]:
    line = from_text(&str.from(total(values)?)?)?
    write_all(out, &line)?
    Ok(())

A test function takes no parameters and returns () or Result[(), E]. std.test supplies assertions that return Result[(), TestError], so ? stops the test at the first failure:

import std.bytes
import std.test
import std.io
from stats import mean, report, save, total

def test_total() -> Result[(), AllocError | IoError | std.test.TestError | OverflowError[i64]]:
    values = [1, 2, 3]?
    std.test.equal(&total(&values)?, &6)?
    empty: list[i64] = []?
    std.test.equal(&total(&empty)?, &0)?
    Ok(())

def test_mean() -> Result[(), AllocError | IoError | std.test.TestError | OverflowError[i64]]:
    values = [1, 2]?
    std.test.close(mean(&values)?, 1.5, 0.000001)?
    Ok(())

def test_save() -> Result[(), AllocError | IoError | std.test.TestError | OverflowError[i64]]:
    scratch = std.test.scratch()?
    path = scratch.file("total.txt")?
    values = [4, 5]?
    save(&path, &values)?
    std.test.equal(&read_text(&path)?, &"9")?
    Ok(())

def test_order_does_not_matter() -> Result[(), AllocError | IoError | std.test.TestError | OverflowError[i64]]:
    mut rng = std.test.rng()?
    mut values = [1, 2, 3, 4, 5]?
    rng.shuffle(&mut values)
    std.test.equal(&total(&values)?, &15)?
    Ok(())

def test_report_survives_short_writes() -> Result[(), AllocError | std.bytes.BytesError | std.io.Partial | std.test.TestError | OverflowError[i64]]:
    mut out = std.test.fake_stream(std.bytes.empty()?)?
    out.limit(1)
    values = [9, 10]?
    report(&mut out, &values)?
    std.test.equal(&out.output.to_text()?, &"19")?
    Ok(())

def test_save_reports_a_full_disk() -> Result[(), AllocError | IoError | std.test.TestError | OverflowError[i64]]:
    scratch = std.test.scratch()?
    path = scratch.file("total.txt")?
    values = [4, 5]?
    std.test.fail_files_after(0u32, 28i32)?
    saved = save(&path, &values)
    std.test.restore_files()?
    std.test.is_err(&saved)?
    Ok(())

std.test.scratch() makes an empty directory under $TMPDIR that is removed with everything in it when the Scratch value is dropped. std.test.rng() is a generator seeded from plenty test --seed N, or from 0, so a run repeats exactly.

report takes any std.io Writer, so the test hands it a fake stream that accepts one byte per call. A file rarely takes a write in pieces, but a pipe or socket does; the fake makes that case happen every time. fail_after makes a chosen call fail, and std.test.fake_clock stands in for the clock in the same way.

A real file fails rarely, so test_save_reports_a_full_disk makes it fail: after fail_files_after(0u32, 28i32), every file call fails with error 28, the system’s code for a full disk, until restore_files. fail_allocations_after does the same for memory. plenty test enables these hooks automatically. For an integration test that launches the ordinary application, explicitly use plenty build --test-hooks and arm the budget at the operation being tested. Default builds contain no hooks. See failure injection and memory-limited runs.

plenty test
running 6 tests
test sums::test_total ... ok
test sums::test_mean ... ok
test sums::test_save ... ok
test sums::test_order_does_not_matter ... ok
test sums::test_report_survives_short_writes ... ok
test sums::test_save_reports_a_full_disk ... ok

test result: ok. 6 passed; 0 failed

Tests run in parallel, one per CPU, but are listed in the order they appear.

Each test is named FILE::FUNCTION. plenty test mean runs only the tests whose names contain mean, and plenty test --list prints the names without running anything.

A test’s output is kept and shown only if it fails. If total added one too many, plenty test test_total would print:

running 1 test
test sums::test_total ... FAILED (returned Err)

failures:

---- sums::test_total output ----
sums.plenty:7:5: assertion failed: left == right
  left: 7
 right: 6

test result: FAILED. 0 passed; 1 failed; 5 filtered out

The command exits with status 1.

The assertion writes where it was called and what it found as it fails, and returned Err says the test stopped there. A trap would read exit status 1 and show the trap’s message. plenty test --no-capture lets output through as it happens, and --timeout 10 fails any test still running after ten seconds.

A file defines either main or test_ functions, not both. Helpers that are not tests need a name without the test_ prefix, or a module in a subdirectory of tests/. See std.test for every assertion.

Compare text with std.diff

std.diff finds what changed between two versions of a text and reports it as edit operations: runs that stayed equal, were deleted, or were inserted.

Show a unified diff

std.diff.lines compares line by line. unified formats the result the way diff -u does, with the given number of unchanged lines around each change:

import std.diff

def main() -> Result[(), AllocError | IoError]:
    old = "name = \"stats\"\nversion = \"0.1.0\"\nauthors = []\nlicense = \"MIT\"\n"
    new = "name = \"stats\"\nversion = \"0.2.0\"\nauthors = []\nlicense = \"MIT\"\nreadme = \"README.md\"\n"
    changes = std.diff.lines(old, new)?
    print(changes.has_changes())?
    print(changes.unified(1, "plenty.toml", "plenty.toml (edited)")?)?
    Ok(())
True
--- plenty.toml
+++ plenty.toml (edited)
@@ -1,4 +1,5 @@
 name = "stats"
-version = "0.1.0"
+version = "0.2.0"
 authors = []
 license = "MIT"
+readme = "README.md"

A hunk header @@ -1,4 +1,5 @@ says that four lines starting at line 1 of the old text became five lines starting at line 1 of the new one. Equal texts give empty text, so check has_changes first when that matters.

Walk the operations

words splits text into runs of whitespace and runs of other characters, and chars into single characters. Each Op gives a range of old tokens and a range of new tokens; a deletion has an empty new range and an insertion an empty old range:

import std.diff
from std.diff import Kind

def main() -> Result[(), AllocError | IoError]:
    changes = std.diff.words("the quick brown fox", "the quick red fox jumps")?
    old = changes.old_tokens()
    new = changes.new_tokens()
    for op in changes.ops():
        match op.kind:
            case Kind.Equal:
                pass
            case Kind.Delete:
                for index in range(op.old_start, op.old_end):
                    print("removed", str.repr(old[index])?)?
            case Kind.Insert:
                for index in range(op.new_start, op.new_end):
                    print("added", str.repr(new[index])?)?
    Ok(())
removed "brown"
added "red"
added " "
added "jumps"

ops returns a reference, so bind the TextDiff to a name before calling it. std.diff.diff compares any two lists whose elements can be dictionary keys, and std.diff.diff_bytes compares Bytes.

The default algorithm finds the fewest deletions and insertions. Passing Algorithm.Patience as the last argument anchors the comparison on lines that occur once in each text, which often reads better for source code.

Finding the fewest edits takes time quadratic in the input size when two large inputs have little in common. For large or untrusted input, pass a cost limit such as std.diff.lines(old, new, max_cost=Some(512)): the work stays roughly linear and the operations still turn the old text into the new one, but may include more deletions and insertions than necessary. See the std.diff reference for both algorithms, the cost limit, and their costs.

Render Markdown as HTML

std.markdown reads CommonMark, the common core of Markdown, and writes HTML. Between the two it offers a stream of events, so a program can inspect or change a document before it is rendered.

Convert a document

to_html renders a whole document. Like other parsers in the standard library, it takes limits on input size, nesting depth, and output size:

import std.markdown
from std.markdown import MarkdownError

def main() -> Result[(), AllocError | IoError | MarkdownError]:
    bounds = std.markdown.limits()
    text = "# Shopping\n\n- *fresh* bread\n- [cheese](https://example.com/cheese)\n"
    print(std.markdown.to_html(text, &bounds)?)?
    Ok(())
<h1>Shopping</h1>
<ul>
<li><em>fresh</em> bread</li>
<li><a href="https://example.com/cheese">cheese</a></li>
</ul>

The HTML ends with a line ending, so print leaves a blank line after it.

Walk the events

A parser yields Start and End events around each block and inline element, with Text and other events between them. This program prints the text of every second-level heading:

import std.markdown
from std.markdown import Event, MarkdownError, Tag

def main() -> Result[(), AllocError | IoError | MarkdownError]:
    bounds = std.markdown.limits()
    text = "# Plenty\n\n## Install\n\nRun `plenty FILE`.\n\n## Learn\n\nRead the *tutorial*.\n"
    mut events = std.markdown.parser_text(text, &bounds)?
    mut in_section = False
    while True:
        event = events.next()?
        match event:
            case Event.Start(tag):
                match tag:
                    case Tag.Heading(level):
                        in_section = level == 2
                    case _:
                        pass
            case Event.End(_):
                in_section = False
            case Event.Text(words):
                if in_section:
                    print("-", words)?
            case Event.Done:
                break
            case _:
                pass
    Ok(())
- Install
- Learn

Text holds the characters to show, with escapes and entities such as &amp; already resolved. The last event is Done.

Change events before rendering

Markdown lets raw HTML through unchanged. When the text comes from someone you do not trust, render the events yourself and turn raw HTML into text, which the renderer escapes:

import std.bytes
import std.markdown
from std.markdown import Event, MarkdownError

def main() -> Result[(), AllocError | IoError | std.bytes.BytesError | MarkdownError]:
    bounds = std.markdown.limits()
    text = "Hi <b onclick=\"steal()\">there</b>!\n\n<script>steal()</script>\n"
    mut events = std.markdown.parser_text(text, &bounds)?
    mut html = std.markdown.renderer(&bounds)?
    while True:
        event = events.next()?
        match event:
            case Event.Html(raw):
                html.event(&Event.Text(raw))?
            case Event.InlineHtml(raw):
                html.event(&Event.Text(raw))?
            case Event.Done:
                break
            case _ as other:
                html.event(&other)?
    print(html.take()?.to_text()?)?
    Ok(())
<p>Hi &lt;b onclick=&quot;steal()&quot;&gt;there&lt;/b&gt;!</p>
&lt;script&gt;steal()&lt;/script&gt;

Link destinations are not checked either: [x](javascript:steal()) is a valid link. A program that publishes untrusted Markdown should also inspect Tag.Link and Tag.Image destinations.

std.markdown.write_html renders a parser’s events straight to a writer, such as a file, without holding the whole page. See the std.markdown reference for the events, limits, errors, and exact HTML output.

Useful programs and resources

Read arguments and text, format results, and use standard streams and files. Combine these operations into a command-line application.

Command-line arguments

args() returns Result[list[str], IoError]. Index zero is the executable’s invocation name, followed by the supplied arguments. Pass arguments to a compiled binary, for example ./program first "two words"; quoted spaces remain within one argument. Each call creates an independent list and reports allocation or UTF-8 errors.

def count_arguments() -> Result[bool, IoError]:
    values = args()?
    Ok(len(values) >= 1)

def main() -> Result[(), AllocError | IoError]:
    print(count_arguments()?)?
    Ok(())
True

Parse typed arguments

Use std.cli when a tool needs options, positionals, generated help, and useful usage errors. The schema explicitly chooses each argument’s type and whether it is required or repeatable. Path values retain native bytes.

This small input-selection tool accepts a row limit and any number of paths. The fixture models finder --limit -2 first.csv -- --last.csv. Real programs use raw = args_bytes()? and parse(&schema, &raw, 1) to skip the executable name.

import std.cli

from std.cli import Spec, Kind, Value, parse

def main() -> Result[(), AllocError | IoError | std.cli.CliError]:
    schema = [
        Spec("--limit", Kind.Integer, False, False, False, "maximum rows"),
        Spec("FILE", Kind.Path, True, True, True, "input files")
    ]?
    raw = [encode_utf8("--limit")?, encode_utf8("-2")?,
           encode_utf8("first.csv")?, encode_utf8("--")?,
           encode_utf8("--last.csv")?]?
    parsed = parse(&schema, &raw, 0)?
    for entry in &parsed.entries:
        match &entry.value:
            case Value.Integer(limit):
                print("limit:", limit)?
            case Value.Path(path):
                print("file:", path.to_string()?)?
            case _:
                pass
    Ok(())
limit: -2
file: first.csv
file: --last.csv

Long options use separate values: --limit 12. A value following an option is always consumed, even if it starts with -. The standalone -- ends option recognition; subsequent tokens are positional. Missing required arguments, unknown options, duplicate nonrepeatable options, and invalid integers are typed errors. Optional specs produce no entry when absent, leaving defaults to you.

Handle help before running the tool. It is a successful result, not an error. The following complete program uses a synthetic request so the lesson can run without shell arguments. Replace raw and the start index as above in a binary.

from std.cli import Spec, Kind, parse, help, describe_for, exit_status
from std.error import Context, Subject

def run() -> Result[i32, AllocError | IoError]:
    schema = [Spec("FILE", Kind.Path, True, True, False, "input file")]?
    raw = [encode_utf8("--help")?]?
    status = match parse(&schema, &raw, 0):
        case Ok(parsed):
            if parsed.help_requested:
                match help(&schema, "finder"):
                    case Ok(text):
                        write_stdout(text)?
                    case Err(error):
                        Context("help", Subject.Absent, error).report()
                        return Ok(1i32)
            0i32
        case Err(error):
            status = exit_status(&error)
            context = Context("arguments", Subject.Absent, error)
            context.report_with(describe_for(&context.cause, &schema))
            status
    Ok(status)

def main() -> i32:
    match run():
        case Ok(status):
            status
        case Err(error):
            Context("argument setup", Subject.Absent, error).report()
            1i32
Usage: finder [--help]
  FILE <path> (required)  input file
  --help  Show this help

--help bypasses missing-required checks, but an earlier bad argument still fails. After --, or as the next token for a text/path option, --help is data. The parser validates the schema before processing either form.

The run helper propagates setup and output failures. main reports those failures and returns a nonzero exit status, just as it does for argument and help errors. std.cli lists all errors, costs, and grammar limits.

Show progress

Use std.cli.progress when a command does enough work that users need feedback. Advance a bar after each successful unit of work and finish it explicitly. The bar uses stderr only when it is a terminal, so redirected output remains clean. The library limits redraws; an update on every item does not redraw on every item.

import std.cli.progress as progress
from std.cli.progress import ProgressError

def main() -> Result[(), ProgressError | IoError | OverflowError[i64]]:
    mut bar = progress.bar(5u64)
    mut total = 0
    for value in range(1, 6):
        total += value * value
        bar.advance()?
    bar.finish()?
    print("sum of squares:", total)?
    Ok(())
sum of squares: 55

The calculation is deliberately short to keep the example runnable; a real program might count imported records or processed files. For an unknown total, use progress.spinner(): advance() counts completed units, and tick() requests an animation step without changing the count. There is no background thread, so a spinner only moves when your program updates it.

Propagate display errors as above unless your application deliberately treats progress as optional. An early failure does not automatically finish the display; finish before returning or writing diagnostics when a clean terminal line is needed. Avoid mixing another progress bar or concurrent stderr logging with the same display. The initial renderer assumes an ANSI terminal at least 80 columns wide and uses compact approximate counts; bar.position() gives the exact count.

The reference describes rates, ETA, deterministic clock inputs and plain-text rendering.

Reading integers from text

Use a sized type’s parse method for external text. It borrows the string and returns an allocation-free Result. Decimal ASCII digits and an optional sign are accepted, with surrounding Unicode whitespace ignored. Numeric prefixes, underscores, and non-ASCII digits are not accepted.

def main() -> Result[(), AllocError | IoError]:
    print(str.repr(u8.parse(" 255 "))?)?
    print(str.repr(u8.parse("256"))?)?
    print(str.repr(i64.parse("hello"))?)?
    Ok(())
Result[u8, ParseError].Ok(255)
Result[u8, ParseError].Err(ParseError.OutOfRange)
Result[i64, ParseError].Err(ParseError.Invalid)

Functions returning the same error type can propagate parsing failures with ?. try_from reports a value that does not fit with RangeError.OutOfRange. Match it explicitly to combine parsing, calculation, and narrowing in a function returning ParseError:

def shade(percent: &str) -> Result[u8, ParseError]:
    number = i64.parse(percent)?
    scaled_result = checked:
        number * 255
    product = match scaled_result:
        case Ok(value):
            value
        case Err(_):
            return Err(ParseError.OutOfRange)
    hundred: Positive[i64] = 100
    scaled = product // hundred
    match u8.try_from(scaled):
        case Ok(value):
            Ok(value)
        case Err(error):
            match error:
                case RangeError.OutOfRange:
                    Err(ParseError.OutOfRange)

def main() -> Result[(), AllocError | IoError]:
    print(str.repr(shade("50"))?)?
    print(str.repr(shade("150"))?)?
    print(str.repr(shade("half"))?)?
    Ok(())
Result[u8, ParseError].Ok(127)
Result[u8, ParseError].Err(ParseError.OutOfRange)
Result[u8, ParseError].Err(ParseError.Invalid)

The local checked multiplication maps overflow to ParseError.OutOfRange, keeping this conversion helper’s public error contract. The positive divisor proves division cannot fail. No unchecked arithmetic or lossy cast is needed.

Reading floating-point text

Floating-point parsing accepts decimal/exponent forms and surrounding whitespace. Overflow returns OutOfRange; underflow can round to zero. Explicit inf, infinity, and nan are accepted case-insensitively, with optional signs.

def main() -> Result[(), AllocError | IoError]:
    print(str.repr(f32.parse("1.25e2"))?)?
    print(str.repr(f32.parse("1e100"))?)?
    print(str.repr(f64.parse("-0.0"))?)?
    Ok(())
Result[f32, ParseError].Ok(125.0)
Result[f32, ParseError].Err(ParseError.OutOfRange)
Result[f64, ParseError].Ok(-0.0)

Convert and format values

str.from converts numbers or booleans to text, returning Result[str, AllocError]. Floats use the shortest text that parses back to the same value.

def main() -> Result[(), AllocError | IoError]:
    print(str.from(42)?)?
    print(str.from(-0.0f32)?)?
    print(str.from(True)?)?
    Ok(())
42
-0.0
True

Inspect a value with str.repr

str.repr formats a borrowed value; print writes a value and newline, reporting formatting allocation failures and output errors:

def main() -> Result[(), AllocError | IoError]:
    values = [1, 2]?
    print(str.repr(values)?)?
    print(values)?
    Ok(())
[1, 2]
[1, 2]

Formatting failure writes nothing. An output error can leave a partial write. The source remains usable; neither call copies a mutable collection.

The inner ? handles formatting; the outer ? handles printing. To inspect a Result wrapper itself, pass it to str.repr:

def main() -> Result[(), AllocError | IoError]:
    result: Result[i64, str] = Err("not ready")
    print(str.repr(result)?)?
    Ok(())
Result[i64, str].Err("not ready")

Split text into fields

Use text.split(separator) to split on an explicit, nonempty string. It returns Result[list[str], AllocError]:

def main() -> Result[(), AllocError | IoError]:
    line = "Ada::Bea::::"
    parts = line.split("::")?
    print(parts)?
    print(str.repr(" / ".join(parts)?)?)?
    print(line)?
    print("".split(",")?)?
    Ok(())
["Ada", "Bea", "", ""]
"Ada / Bea /  / "
Ada::Bea::::
[""]

Adjacent and trailing separators preserve empty fields. Matches do not overlap: "aaaaa".split("aa") succeeds with ["", "", "a"]. A separator that does not occur produces a single piece containing the whole input.

Each piece owns independent storage. Splitting borrows both inputs and leaves them unchanged, including on allocation failure.

This program cannot produce its fields if splitting fails, so ? returns the error from main. The same policy handles failures from joining, formatting, and printing. Use a match instead when you have a useful recovery action.

An empty separator causes a runtime trap, not AllocError. Check len(separator) > 0 for user-supplied separators. Whitespace splitting without a separator and a maximum-split argument are unsupported.

Splitting text that is already in memory

splitlines() returns an independent list without line endings. Pass True to keep the original endings. A final newline does not add an extra empty line, and empty text produces an empty list. Unlike file reads, this method also recognizes Unicode line/paragraph separators and the other Python-style text line boundaries.

def main() -> Result[(), AllocError | IoError]:
    print("first\r\n\nlast\n".splitlines()?)?
    print("first\r\nlast".splitlines(True)?)?
    print("".splitlines()?)?
    Ok(())
["first", "", "last"]
["first\r\n", "last"]
[]

Writing text with recoverable errors

write_stdout(text) borrows a string and writes exactly its contents, without a newline. Success returns the number of Unicode characters. An I/O error can follow a partial write, so retrying the whole string can duplicate output.

def main() -> Result[(), AllocError | IoError]:
    result = write_stdout("Hello!\n")
    print(result?)?
    Ok(())
Hello!
7

IoError.System(code) carries a native OS error code. Decoding, allocation, and stream-state errors have separate variants; the text I/O reference lists them all.

Flushing output and reporting diagnostics

Use write_stderr(text) for diagnostics. It returns the same character-count result as write_stdout. flush_stdout() and flush_stderr() return Result[(), IoError]. Flush when the caller needs to observe buffered-write errors or when displaying a prompt; a flush does not promise disk durability.

def report() -> Result[(), IoError]:
    write_stdout("ready")?
    flush_stdout()?
    Ok(())

def main() -> Result[(), IoError]:
    report()?
    print("")?
    Ok(())
ready

Reading a line

input() returns Result[Option[str], IoError]. It removes a trailing LF or CRLF. Some("") means an empty line; Nothing means end-of-file. UTF-8 errors and allocation failures return Err. A failure can consume input, so retrying does not promise to repeat the same line. Input currently requires a Unix host. For a prompt, call write_stdout and flush_stdout before input().

Call echo_line()? repeatedly until it returns False to process stdin. This example’s main does not read input.

def echo_line() -> Result[bool, IoError]:
    match input()?:
        case Some(line):
            write_stdout(line)?
            write_stdout("\n")?
            Ok(True)
        case Nothing:
            Ok(False)

def main() -> Result[(), IoError]:
    print("echo_line is ready")?
    Ok(())
echo_line is ready

Reading a UTF-8 file

read_text(path) opens, reads, and closes a file, returning Result[str, IoError]. It requires Linux and strict UTF-8, translating CRLF and bare CR to LF. Paths are relative to the process working directory. The handle closes even when reading or allocating fails.

The &str parameter borrows literals and bindings implicitly; configuration(&path) is also valid.

def configuration(path: &str) -> Result[str, IoError]:
    read_text(path)

def main() -> Result[(), IoError]:
    write_text("configuration.txt", "mode=fast\n")?
    print(configuration("configuration.txt")?)?
    path = "configuration.txt"
    print(configuration(path)?)?
    Ok(())
mode=fast

mode=fast

Writing a UTF-8 file

write_text(path, text) creates or replaces a file and returns the number of Unicode characters written. It writes exact UTF-8 bytes without translating newlines and closes the handle before returning. OS failures can leave truncated or partially written contents; this is not an atomic save.

Run this in a scratch directory: it replaces plenty-example.txt.

def save_and_read() -> Result[str, IoError]:
    write_text("plenty-example.txt", "Hello, é!\n")?
    read_text("plenty-example.txt")

def main() -> Result[(), AllocError | IoError]:
    print(str.repr(save_and_read())?)?
    Ok(())
Result[str, IoError].Ok("Hello, é!\n")

Appending text

append_text(path, text) adds exact UTF-8 bytes to the end of a file, creating it if necessary. It returns the number of characters appended. Errors can leave partial output; concurrent appends are not guaranteed to be atomic records.

Run this in a scratch directory: it replaces plenty-log.txt before appending.

def log_example() -> Result[str, IoError]:
    write_text("plenty-log.txt", "started\n")?
    append_text("plenty-log.txt", "finished\n")?
    read_text("plenty-log.txt")

def main() -> Result[(), AllocError | IoError]:
    print(str.repr(log_example())?)?
    Ok(())
Result[str, IoError].Ok("started\nfinished\n")

Open a file for a block

with open(path)? as file makes a stream available inside the indented block. It closes on exit, including on return, ?, break, or continue. The stream reference cannot escape the block.

open(path) opens an existing file for reading. open(path, "w") creates or truncates a file, and open(path, "a") creates or appends. Each returns Result[File, IoError]. Files move on assignment and cannot be copied.

read() returns text from the current position through EOF, translating CRLF and bare CR to LF and validating UTF-8. At EOF it returns an empty string.

Run this in a scratch directory: it replaces plenty-reading.txt.

def read_example() -> Result[str, IoError]:
    write_text("plenty-reading.txt", "hello\r\nworld")?
    with open("plenty-reading.txt")? as file:
        return Ok(file.read()?)

def main() -> Result[(), IoError]:
    print(read_example()?)?
    Ok(())
hello
world

The returned text owns its storage and survives closing the file. Read errors may consume input before failing. File I/O requires Linux.

Reading lines

readline() keeps the terminating newline, translating LF, CRLF, and bare CR to "\n". An empty line is "\n"; EOF is "". The last line need not have a terminating newline. Unlike input(), line reading retains that terminator.

Run this in a scratch directory: it replaces plenty-lines.txt.

def count_lines() -> Result[i64, IoError | OverflowError[i64]]:
    write_text("plenty-lines.txt", "first\r\n\rfinal")?
    mut count = 0
    with open("plenty-lines.txt")? as file:
        while True:
            line = file.readline()?
            if line == "":
                break
            count = count + 1
    Ok(count)

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    print(count_lines()?)?
    Ok(())
3

readline() and read() share newline state. Their errors may consume input; retrying is not a rollback. Direct for line in file iteration is unsupported. For size-limited reads and saved positions, see the text I/O reference.

Write through a file

Use a writable stream when several writes belong to one open file. write(text) returns the number of Unicode characters written and preserves exact UTF-8 bytes, including newlines. Mode "w" replaces a file; "a" appends to it.

Run this in a scratch directory: it replaces plenty-stream.txt.

def main() -> Result[(), IoError]:
    with open("plenty-stream.txt", "w")? as file:
        print(file.write("hello\n")?)?
        file.write("world\n")?
        file.close()?
    with open("plenty-stream.txt", "a")? as file:
        file.write("goodbye\n")?
    print(read_text("plenty-stream.txt")?)?
    Ok(())
6
hello
world
goodbye

Call close()? to report a close error; automatic cleanup cannot report it. Exiting the block can then close the already closed file harmlessly.

For flushing, disk synchronization, and other detailed contracts, see the text I/O reference.

Context managers

Define __enter__ and __exit__ to make a class usable with with. Entry supplies the as value; exit takes only a mutable receiver and returns (). No inheritance or protocol declaration is needed.

class Message:
    text: str
    def __enter__(self: &mut Message) -> str:
        drop(print("enter"))
        self.text
    def __exit__(self: &mut Message) -> ():
        drop(print("exit"))

def main() -> Result[(), IoError]:
    with Message("hello") as text:
        print(text)?
    print("after")?
    Ok(())
enter
hello
exit
after

The manager moves into the block. Body locals and the entry value clean up before __exit__; the manager drops afterward, including on early exits. An owned entry value can move out, for example through return. The as binding is local to the body; omit it for unit entry methods.

with acquire()?: propagates acquisition failure without entering the context.

The lifecycle trace uses drop(print(...)) deliberately: cleanup cannot return an error, and a failed diagnostic write must not interrupt cleanup. For output that must succeed, provide an explicit fallible operation and propagate its result before leaving the scope.

Borrow and combine context managers

To keep a manager after the block, borrow it explicitly. The original binding is exclusively borrowed until exit completes.

class Stage:
    marker: i64
    def __enter__(self: &mut Stage) -> i64:
        self.marker = 1
        self.marker
    def __exit__(self: &mut Stage) -> ():
        self.marker = 11

def main() -> Result[(), IoError]:
    mut stage = Stage(0)
    with &mut stage as n:
        print(n)?
    print(stage.marker)?
    Ok(())
1
11

Stage records lifecycle markers, not an accumulating counter: enter assigns 1 and exit assigns 11, including on repeated uses. Cleanup performs no fallible arithmetic.

Several managers can share one with statement. They enter from left to right and exit from right to left, just like nested blocks. Later manager expressions can use earlier as bindings.

class Label:
    text: str
    def __enter__(self: &mut Label) -> str:
        drop(print(self.text))
        self.text
    def __exit__(self: &mut Label) -> ():
        drop(print(self.text))

def main() -> Result[(), IoError]:
    with Label("first") as first, Label("second") as second:
        print("body")?
    Ok(())
first
second
body
second
first

Exit is infallible and cannot suppress errors. Check fallible writes or flushes explicitly. Fatal traps do not run exits, and yield inside with is not yet supported.

__enter__ can return a reference to the manager, giving the body access to its methods and fields while with keeps ownership. The reference cannot escape.

class Counter:
    count: i64
    def __enter__(self: &mut Counter) -> &mut Counter:
        &mut self
    def __exit__(self: &mut Counter) -> ():
        drop(print(self.count))

def main() -> Result[(), IoError]:
    with Counter(1) as counter:
        counter.count = 7
    Ok(())
7

The lifecycle trace uses drop(print(...)) deliberately: cleanup cannot return an error, and a failed diagnostic write must not interrupt cleanup. For output that must succeed, provide an explicit fallible operation and propagate its result before leaving the scope.

Build a small command-line application

Put one integer on each line of an input file. This program ignores blank lines, adds the integers, and writes a short report to an output file. Its two arguments are the input and output paths. Running it without arguments prints usage.

Test for EOF before stripping whitespace: a blank line should be ignored, not mistaken for the end of the file.

Save this module as report.plenty:

pub def summarize(path: &str) -> Result[str, AllocError | IoError | ParseError | OverflowError[i64]]:
    mut count = 0
    mut total = 0
    with open(path)? as file:
        while True:
            line = file.readline()?
            if line == "":
                break
            cleaned = line.strip()?
            if cleaned == "":
                continue
            total = total + i64.parse(cleaned)?
            count = count + 1
    Ok(f"count={count} total={total}\n"?)

Save this application beside it as main.plenty:

from report import summarize

enum UsageError:
    Arguments

type AppError = AllocError | IoError | ParseError | OverflowError[i64] | UsageError

def run() -> Result[(), AppError]:
    arguments = args()?
    if len(arguments) == 1:
        print("usage: number-report INPUT OUTPUT")?
        return Ok(())
    if len(arguments) != 3:
        write_stderr("usage: number-report INPUT OUTPUT\n")?
        return Err(UsageError.Arguments)
    report = summarize(arguments[1])?
    write_text(arguments[2], report)?
    print("saved report")?
    Ok(())

def main() -> Result[(), AppError]:
    run()
usage: number-report INPUT OUTPUT

Run the application with its arguments:

plenty script main.plenty numbers.txt report.txt

To make it directly executable on Unix, add #!/usr/bin/env -S plenty script as its first line, then run chmod +x main.plenty. Runner options precede the filename; everything after it belongs to your application. See single-file scripts for optional dependency metadata and locks.

For a retained native binary instead:

plenty --compile main.plenty -o number-report
./number-report numbers.txt report.txt

For an input containing 10, a blank line, and 20, the output file contains count=2 total=30 followed by a newline. write_text replaces the output file; choose a path whose contents may be replaced.

AppError preserves usage, parsing, I/O, allocation, and arithmetic errors. Returning the error from main reports its cause on standard error and exits unsuccessfully. run prints "saved report" only after writing succeeds. It propagates failures to main so a failed operation cannot accidentally become a successful exit. Values and their total must fit i64; overflow propagates as OverflowError[i64]. The output file is not opened until the whole input has been summarized successfully.

Checking the argument count directly keeps this two-path interface simple. For named options and typed argument values, use the typed command-line parser. The explicit readline loop shows EOF handling for one file. For buffered line iteration across multiple files or stdin, use std.cli.fileinput; each loop item is a Result that you handle with line?.

See the text I/O reference for file contracts and limits.

Check the application processing module

Test the processing module without command-line arguments. Save its companion as report_demo.plenty:

pub def summarize(path: &str) -> Result[str, AllocError | IoError | ParseError | OverflowError[i64]]:
    mut count = 0
    mut total = 0
    with open(path)? as file:
        while True:
            line = file.readline()?
            if line == "":
                break
            cleaned = line.strip()?
            if cleaned == "":
                continue
            total = total + i64.parse(cleaned)?
            count = count + 1
    Ok(f"count={count} total={total}\n"?)

Run this application in a scratch directory; it replaces numbers.txt and report.txt:

from report_demo import summarize

def main() -> Result[(), AllocError | IoError | ParseError | OverflowError[i64]]:
    write_text("numbers.txt", "10\r\n\r\n20\r\n")?
    report = summarize("numbers.txt")?
    write_text("report.txt", report)?
    write_stdout(read_text("report.txt")?)?
    Ok(())
count=2 total=30

Keep bytes separate from text

Use list[u8] when data may contain invalid UTF-8. Encoding text copies its exact UTF-8 bytes; decoding validates every byte. Both operations observe their input, so you can keep the original data.

def main() -> Result[(), IoError]:
    text = "é\r\n"
    bytes = encode_utf8(text)?
    print(len(bytes))?
    print(decode_utf8(&bytes)? == text)?
    print(len(bytes))?
    invalid = encode_utf8("x")?
    mut raw = invalid
    raw[0] = 255u8
    match decode_utf8(&raw):
        case Ok(_):
            print("unexpected text")?
        case Err(error):
            print(error)?
    Ok(())
4
True
4
IoError.Data(DataError.InvalidUtf8)

On Unix, filenames and executable arguments can contain arbitrary bytes except NUL. Use args_bytes() to obtain those arguments, then pass a path byte list to read_bytes(path) or write_bytes(path, contents). These helpers preserve file contents and names without text decoding or newline conversion. They currently support Linux. write_bytes creates or replaces its destination.

For supplied paths on stdin, use input_bytes(0u8). It returns one NUL-delimited record as Some(record) or Nothing at EOF. Write records with write_stdout_bytes(&record) followed by write_stdout_bytes(b"\0"). Finish with flush_stdout()?. Empty records are distinct from EOF; a final nonempty unterminated record is accepted.

CR, LF, and invalid UTF-8 remain part of the path. Keep the original bytes when opening files, even if you separately decode or format them for display. The finder experiment includes a working --null mode. Its globs match bytes in that mode, so ? matches one byte.

See the binary I/O reference for partial effects, allocation failures, and remaining library limits.

Enumerate and classify paths

Use std.path to represent a native path and std.fs to enumerate it lazily. Each generator item is a Result because advancement can fail after opening:

import std.fs as fs
import std.path as path

def main() -> Result[(), AllocError | IoError]:
    write_text("sample.txt", "abc")?
    root = path.from_text(".")?
    for result in fs.entries(fs.read_dir(&root)?):
        entry = result?
        name = entry.file_name()?
        if name.to_text()? == "sample.txt":
            print(entry.file_type()? == fs.FileType.File)?
            print(entry.metadata()?.size())?
    Ok(())
True
3

entries consumes the directory owner. Exhaustion closes it; an early break, return or propagated error drops it. Keep a named mutable ReadDir when you want manual next() calls or explicit close(). Entries own their data and remain valid after the directory closes. Enumeration includes dot-prefixed names and has unspecified order.

entry.file_type() and entry.metadata() inspect a symlink itself. fs.metadata(&path) follows its target; fs.symlink_metadata(&path) does not. A dangling link can therefore be inspected without requiring its target. Types distinguish regular files, directories, links and other native objects.

Path text conversion is checked. For arbitrary Unix filenames, use name.as_bytes() and NUL-framed transport, avoiding text decoding. These filesystem APIs currently require 64-bit Linux and statx for metadata. See the directory reference.

Use fs.symlink to create a link and fs.canonicalize when you need its resolved absolute name. Relative link targets are relative to the link’s parent:

import std.fs as fs
import std.path as path

def main() -> Result[(), IoError]:
    write_text("original.txt", "data")?
    target = path.from_text("original.txt")?
    link = path.from_text("alias.txt")?
    fs.symlink(&target, &link)?
    print(fs.read_link(&link)?.to_text()?)?
    resolved = fs.canonicalize(&link)?
    print(resolved.is_absolute(), fs.metadata(&resolved)?.size())?
    Ok(())
original.txt
True 4

Canonicalization requires existing components and can fail for dangling links or loops. Its result can become stale immediately: it does not secure a later open against replacement. For operations through a held directory, use rooted filesystem helpers. The metadata and links reference documents native path limits and error behavior.

Use temporary files and directories

Use std.tempfile when a task needs scratch storage. A with statement owns the resource until its body ends and then cleans it up, including on an early return or an error propagated with ?.

For a tool that needs a filename, use named_file. You can write through the owner and pass its path to other code while the owner is alive:

import std.tempfile
import std.io
import std.fs

def main() -> Result[(), AllocError | IoError | std.fs.FsError | std.io.Partial]:
    with std.tempfile.named_file(suffix=".txt")? as temporary:
        std.io.write_all(temporary, &b"hello")?
        print(read_text(&temporary.path().to_text()?)?)?
        temporary.close()?
    Ok(())
hello

The explicit close()? reports cleanup failures. Without it, leaving with still attempts cleanup, but cannot report an error from that attempt. For a stream with no filename, with std.tempfile.file()? as stream: lends a std.io.FileStream instead.

A temporary directory can contain an entire working tree. Cleanup removes its contents recursively; symlinks do not cause traversal outside that tree:

import std.tempfile
import std.path
import std.fs

def main() -> Result[(), AllocError | IoError | std.fs.FsError]:
    with std.tempfile.directory()? as scratch:
        note = scratch.path().join(&std.path.from_text("note.txt")?)?
        write_text(&note.to_text()?, "working copy")?
        print(read_text(&note.to_text()?)?)?
        scratch.cleanup()?
    Ok(())
working copy

To preserve a named file or directory, call keep()? before cleanup. It returns an owned path and transfers responsibility for removing it to your code. Use the parent keyword argument to choose a directory explicitly. Otherwise, the module uses the first nonempty TMPDIR, TEMP, or TMP, falling back to /tmp when none is set.

See the reference for cleanup budgets, Linux filesystem requirements, and the trusted-parent requirement. For atomic publication of a completed file, continue with filesystem staging.

Stage files and clean temporary directories

Open a std.fs.Root to anchor operations to a directory. Child names are single-component lossless paths: no slash, NUL, . or ... Child opens reject symlinks. The initial root’s ancestors remain the caller’s trust decision.

An anonymous temporary file has no name until publish succeeds. Publication refuses an existing destination. Returning early before publication closes the file and releases its partial contents without deleting a named path.

This example uses a random private directory inside the working directory:

import std.io

import std.fs as fs
import std.path as path
from std.sync import CancellationToken

def main() -> Result[(), AllocError | IoError | std.io.StreamError | fs.FsError]:
    current = path.from_text(".")?
    parent = fs.root(&current)?
    prefix = path.from_text("plenty-example-")?
    mut directory = fs.temporary_dir(&parent, &prefix, 8)?
    original = path.from_text("original.txt")?
    copied = path.from_text("copied.txt")?
    mut staged = fs.temporary_file(directory.root())?
    staged.stream.file.write("hello")?
    staged.publish(&original)?
    staged.close()?

    cancel = CancellationToken()?
    print(fs.copy_file(directory.root(), &original, &copied, 1024, &cancel, preserve=fs.CopyMetadata.PermissionsAndTimes)?)?
    pattern = path.from_text("*.txt")?
    names = fs.glob(directory.root(), &pattern, 10, 10, 1000)?
    print(len(names))?
    directory.root().remove_file(&original)?
    directory.root().remove_file(&copied)?
    directory.cleanup()?
    Ok(())
5
2

These operations require Linux, a filesystem supporting O_TMPFILE, and /proc/self/fd for publication. Unsupported environments return errors rather than using a weaker fallback. Single-file copying uses bounded scratch storage, returns errors for byte limits/cancellation, and leaves no partial destination name. To preserve ordinary permission bits and access/modification times, pass preserve=fs.CopyMetadata.PermissionsAndTimes to copy_file; ownership, ACLs, xattrs and special permission bits are excluded.

Temporary-directory cleanup removes only an empty directory. Its destructor is best-effort; call cleanup() when you need to observe failure. Other actors must not rename entries in the parent while the temporary directory is live. An identity check detects existing replacements but is not atomic with rmdir. Use explicit bounded remove_tree only when intentionally deleting contents; it can leave partial progress on cancellation or errors.

See the reference for root contracts, temporary-resource limits, and copy, deletion and glob budgets.

For a directory tree, use copy_tree with explicit entry, depth, byte and handle budgets from copy_limits(). Its default copies contents only. The destination must be new, symlinks and special files are rejected, and a failure after creation leaves partial output for the caller to inspect or remove. The recursive-copy example and recovery contract show the complete call and explain TreeCopyError.Setup versus partial-output Partial/Entry errors.

For cooperating processes that must not operate on a resource simultaneously, use std.fs.lock. Keep the returned owner alive for the protected work. acquire_now reports contention immediately; acquire_before takes a total deadline and a cancellation token. Use a stable lockfile in a trusted directory and leave that file in place after releasing the lock: removing it while another process owns it can split coordination across different inodes. Locks close automatically, and normal subprocesses do not inherit them across exec.

Create a private named file

When a file should be visible immediately and must not already exist, use Root.create_file. Its default 0600 mode prevents group and other access from creation; create_dir defaults to 0700. Both are restricted by the process umask. An existing file, directory or symlink causes an error.

import std.io
import std.fs as fs
import std.path as path

def main() -> Result[(), AllocError | IoError | std.io.StreamError]:
    parent = fs.root(&path.from_text(".")?)?
    name = path.from_text("private-state")?
    parent.create_dir(&name)?
    directory = parent.open_dir(&name)?
    token = path.from_text("token")?
    mut file = directory.create_file(&token)?
    file.file.write("secret")?
    fs.set_file_unix_mode(&file, 0o400u32)?
    file.close()?
    directory.remove_file(&token)?
    parent.remove_dir(&name)?
    Ok(())

set_file_unix_mode changes the held file, so a replacement at its old path cannot redirect the update. Root.set_unix_mode does the same for a directory. Setting a mode does not close the handle or revoke existing access. Unlike staging, a named file remains after a write failure; use the staging pattern when readers must see only complete output. See the permission contracts for custom modes, ACLs, umask and errors.

Watch a directory for changes

Use std.fs.watch when a tool should notice that files changed: a local preview, an indexer, or a configuration reloader. Watch the parent directory when another program might replace a file by renaming a new version over it.

This example watches a temporary directory, writes a file, then inspects the queued batch. It handles overflow by requesting a full rescan instead of assuming every change was delivered:

import std.fs
import std.fs.watch
import std.path
import std.tempfile

def main() -> Result[(), AllocError | IoError | std.fs.FsError | std.fs.watch.WatchError]:
    with std.tempfile.directory()? as scratch:
        mut watcher = std.fs.watch.watcher()?
        watcher.add(scratch.path())?
        note = scratch.path().join(&std.path.from_text("note.txt")?)?
        write_text(&note.to_text()?, "hello")?
        count = watcher.poll()?
        for index in range(count):
            event = watcher.event(index)?
            if event.overflowed():
                print("rescan the directory")?
            elif event.created():
                print("created", event.name.to_string()?)?
        watcher.close()?
        scratch.cleanup()?
    Ok(())
created note.txt

poll() never waits. For a long-running program, use wait(&deadline, &cancel) with a std.time deadline and cancellation token, or register the watcher with a std.ready.Selector. Consume the batch before the next call: event(index) copies its name, so saved events remain valid. The default buffer bounds each read to 4 KiB, and polling does not allocate.

A directory watch covers its immediate children, not an entire tree. Events may be coalesced and an event’s path may have changed again before you read it. Treat notifications as a reason to check the current filesystem state, rather than as a complete history. Names retain native bytes; the example uses display() so a non-UTF-8 name does not prevent reporting the change.

See the reference for rename cookies, watch lifecycle events, overflow recovery, and resource limits.

Prune before descending

The small companion files provide two fixture directories. Run the program from an empty work/ child of the directory containing these files; the root below uses .. to reach the fixture without changing the working directory:

pub const value: i64 = 1
pub const value: i64 = 2

With a top-down walker, edit its directory list before asking for another row:

import std.fs as fs
import std.path as path

def main() -> Result[(), AllocError | IoError | fs.WalkError | OverflowError[i64]]:
    root = path.from_text("../tree")?
    mut walker = fs.walk(&root, True)?
    while walker.next()?:
        print(walker.current.path.to_text()?)?
        mut index = len(walker.current.directories) - 1
        while index >= 0:
            if walker.current.directories[index].to_text()?.endswith("/pruned"):
                drop(walker.current.directories.pop(index))
            index = index - 1
    walker.close()
    print(walker.next()?)?
    Ok(())
../tree
../tree/kept
False

False passed to fs.walk requests bottom-up order. Editing a bottom-up row cannot undo descent. Child directory links appear in current.files, and are never followed; a link to an ancestor cannot make this walker loop. An explicit root may itself resolve through a link.

To continue siblings after an error, match walker.next() instead of using ?:

import std.fs as fs
import std.path as path

def main() -> Result[(), AllocError | IoError | fs.WalkError]:
    root = path.from_text("missing-root")?
    mut walker = fs.walk(&root, True)?
    match walker.next():
        case Ok(found):
            print(found)?
        case Err(error):
            match error:
                case fs.WalkError.Io(at, operation, cause):
                    print(at.to_text()?)?
                    print(operation)?
                    print(cause)?
                case fs.WalkError.Allocation(cause):
                    print(cause)?
    print(walker.next()?)?
    Ok(())
missing-root
read_dir
IoError.System(2)
False

The failed subtree is already skipped. Another call continues any pending siblings. Propagating the error stops traversal with normal cleanup. Only the current scan owns a native directory stream; retaining results does not retain directory handles. See walking contracts.

Say which operation failed

IoError.System(2) says a file was missing, but not which file or which step wanted it. std.error wraps an error in a Context holding an operation label and, optionally, a path. The original error stays inside as cause, with its type.

from std.error import Context, Subject, context, path_context, native_code
from std.path import Path, from_text

def read_limit(path: &Path) -> Result[list[u8], Context[IoError]]:
    path_context(read_bytes(path.as_bytes()), "read limit", path)

def parse_limit(text: &str) -> Result[i64, Context[ParseError]]:
    context(i64.parse(text), "parse limit")

def flush_output() -> Result[(), Context[IoError]]:
    context(flush_stdout(), "flush output")

def limit(path: &Path) -> Result[i64, AllocError | IoError | Context[ParseError]]:
    match read_limit(path):
        case Ok(bytes):
            Ok(parse_limit(&decode_utf8(bytes)?)?)
        case Err(error):
            print(error.describe()?)?
            print(native_code(error.cause))?
            Ok(10)

def run() -> Result[(), AllocError | IoError | Context[IoError] | Context[ParseError]]:
    path = from_text("limit.txt")?
    print(limit(&path)?)?
    match parse_limit("ten"):
        case Ok(value):
            print(value)?
        case Err(error):
            print(error.describe()?)?
            print(error.cause == ParseError.Invalid)?
    flush_output()?
    Ok(())

def main() -> i32:
    match run():
        case Ok(_):
            0i32
        case Err(cause):
            Context("run", Subject.Absent, cause).report()
            1i32
read limit "limit.txt": IoError.System(2)
Option[i32].Some(2)
10
parse limit: ParseError.Invalid
True

context(result, label) and path_context(result, label, &path) pass an Ok through and wrap an Err, including operations whose success is (), as flush_output shows. main uses the direct form, Context(label, Subject.Absent, cause). Nothing is wrapped implicitly: ? needs a compatible declared error, so read_limit returns Context[IoError] and callers can include that context in an error sum.

describe() builds one line: the label, the quoted path if there is one, and the cause. It allocates, so it returns a Result. error.cause is the untouched typed error; match it, compare it, or ask native_code for the OS number.

report() is for the edge of the program. It flushes standard output and writes error: plus the description to standard error. It returns nothing and does not choose an exit status, so main returns i32 and decides. If memory runs out while describing, report still writes the label with a fixed message, and a failed path copy in path_context keeps the label and cause.

Here run preserves its allocation, I/O, and contextual errors in a sum. The outer context adds the operation label without discarding the cause. Return a Context or your own enum when a failure needs more explanation. See the module reference for nesting contexts and the rules standard modules follow.

For a human-readable cause, give its type the public std.report formatting method. Contexts then format their operation labels and delegate to that cause:

from std.error import Context, Subject
from std.fmt import Formatter, FormatError
from std.report import to_string

pub enum SettingsError:
    Invalid

    pub def format[W](self, output: &mut Formatter[W]) -> Result[(), FormatError]:
        output.text("invalid setting")

def main() -> Result[(), FormatError | IoError]:
    inner = Context("parse", Subject.Absent, SettingsError.Invalid)
    outer = Context("load settings", Subject.Absent, inner)
    print(to_string(&outer, 120, 8)?)?
    print(to_string(&outer, 120, 8)?)?
    Ok(())
load settings: parse: invalid setting
load settings: parse: invalid setting

Formatting borrows both contexts and includes the cause once. Returning an Err with this context type from main selects the same wording automatically, with the entrypoint’s prefix and newline. A context whose cause has no formatting method still uses structural entrypoint reporting. The existing describe, describe_with, report and report_with methods keep their behavior.

Without a path, context formatting adds no allocation. A present path still uses fallible Path.display; failure leaves the typed cause intact and may leave part of the report already written. See bounded reports for limits and failure handling.

Pack bytes in a compact buffer

For a binary record, frame, or file body, std.bytes.Bytes provides checked byte-range operations and works with byte-oriented I/O. Its contiguous buffer stores one byte per element. A list[u8] also packs its elements into bytes; use its ordinary list operations and element references when those suit the task.

import std.bytes
from std.bytes import Bytes, BytesError

def checksum(view: &Bytes, start: i64, stop: i64) -> Result[i64, OverflowError[i64]]:
    mut total = 0
    for index in range(start, stop):
        total += i64(view.get(index))
    Ok(total)

def main() -> Result[(), AllocError | IoError | BytesError | OverflowError[i64]]:
    mut frame = std.bytes.with_capacity(8)?
    frame.push(2u8)?
    frame.extend(&std.bytes.from_text("hi")?)?
    frame.push(0u8)?
    frame.push(255u8)?
    print(frame.len(), frame.capacity(), frame)?
    print(checksum(&frame, 1, 3)?)?
    payload = frame.slice(1, 3)?
    print(payload.to_text()?)?
    print(str.repr(frame.to_text())?)?
    Ok(())
5 8 b"\x02hi\x00\xff"
209
hi
Result[str, std.bytes.BytesError].Err(std.bytes.BytesError.InvalidUtf8)

Growth and copies can fail to allocate, so they return AllocError; to_text returns BytesError. A range is start up to but not including stop, and must lie inside the buffer: there are no negative offsets and nothing is clamped. A range outside the buffer, such as checksum(&frame, 3, 9) on five bytes, is a bug in the caller, so it stops the program with a trap report naming the method, Bytes.get: index is outside 0..len(), and your calling line. Check a range that comes from input first and return your own error. slice and copy make independent buffers; nothing is copied unless you ask. to_text succeeds only for valid UTF-8, and a zero byte is ordinary data. Printing a buffer spells it as a byte literal, with \xhh for bytes that are not printable ASCII; for hex output, see printing bytes.

checksum shows how to work on part of a buffer: borrow it and pass the range. While that borrow is in use the owner cannot change, so growth can never move bytes out from under a reader:

import std.bytes

def main() -> Result[(), AllocError | IoError | std.bytes.BytesError]:
    mut frame = std.bytes.from_text("hi")?
    view = &frame
    frame.push(0u8)?
    print(view.len())?
    Ok(())
conflicting borrow: cannot modify or exclusively borrow `frame` while it is borrowed

File-reading and argument builtins still return list[u8]; from_list converts one with an explicit copy. The output builtins write_stdout_bytes, write_stderr_bytes and write_bytes take a &[u8] view, so pass a buffer or a range of it with no conversion:

import std.bytes

def main() -> Result[(), AllocError | IoError]:
    mut record = std.bytes.from_list(&encode_utf8("name=")?)?
    record.extend(b"\xff\0")?
    write_stdout_bytes(&record[0:5])?
    print(record.len(), record.find(61u8, 0, record.len()))?
    Ok(())
name=7 Option[i64].Some(4)

A parser should not call get once per byte: each call costs far more than the comparison it feeds. The scans search a whole range in one call and return an absolute index. find_in stops at any byte of a set, find_not_in at the end of a run, and matches_text compares in place without copying. to_text_range copies one range out as checked text, with no intermediate Bytes:

import std.bytes

def main() -> Result[(), AllocError | IoError | std.bytes.BytesError | OverflowError[i64]]:
    line = std.bytes.from_text("name=Ada;  role=engineer")?
    stops = std.bytes.from_text("=;")?
    blank = std.bytes.from_text(" ")?
    mut at = 0
    while at < line.len():
        match line.find_in(&stops, at, line.len()):
            case Some(index):
                print(index, line.to_text_range(at, index)?)?
                at = index + 1
            case Nothing:
                print("rest", line.to_text_range(at, line.len())?)?
                at = line.len()
    print(line.find_not_in(&blank, 9, line.len()))?
    print(line.matches_text(11, "role"), line.count(32u8, 0, line.len()))?
    Ok(())
4 name
8 Ada
15   role
rest engineer
Option[i64].Some(11)
True 2

Index syntax reads and writes single bytes, with the same rules as a list index: a negative index counts from the end, and a missing one stops the program with index out of bounds. Iterating over &buffer reads each byte without consuming the buffer:

import std.bytes

def main() -> Result[(), AllocError | IoError | std.bytes.BytesError | OverflowError[u8] | OverflowError[i64]]:
    mut word = std.bytes.from_text("plenty")?
    word[0] = 80u8
    word[-1] -= 32u8
    print(word.to_text()?, word[1])?
    mut vowels = 0
    for b in &word:
        if b == 101u8 or b == 89u8:
            vowels += 1
    print(vowels)?
    word.fill(1, 5, 46u8)
    print(word.to_text()?)?
    Ok(())
PlentY 108
2
P....Y

get and set take the same checked index without counting from the end. for b in &mut word yields &mut u8 references into the buffer, so *b = 0u8 in its body clears every byte in place. A slice such as word[1:3]? copies with the list rules, clamping its bounds. The slice method keeps the range rule instead: frame.slice(1, 3)? copies exactly that range, and a range outside the buffer traps. Both report only AllocError.

Fixed binary data, such as a file signature or a protocol delimiter, is written as a byte literal: b"..." holds ASCII characters, the string escapes, and \xHH for any other byte. Its type is bytes, an immutable value in the program’s read-only data, so it never allocates. It has a length, indexing, and iteration:

import std.bytes

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    magic = b"\x89PNG"
    print(magic, len(magic), magic[1])?
    mut total = 0
    for byte in magic:
        total += i64(byte)
    print(total)?
    Ok(())
b"\x89PNG" 4 80
366

Operations that only read another sequence of bytes, such as find_bytes, matches, find_in, and extend, take a literal as readily as a buffer. Their parameter is a &[u8] slice view: a literal, &buffer, or part of either, &buffer[a:b], converts to it, so each call reads the bytes where they are and nothing is copied. Your own functions can accept every form the same way:

import std.bytes

def starts_png(data: &[u8]) -> bool:
    len(data) >= 4 and data[0] == 137u8 and data[1] == 80u8

def main() -> Result[(), AllocError | IoError]:
    magic = b"\x89PNG"
    mut header = std.bytes.from_bytes(&magic)?
    header.extend(b"\r\n\x1a\n")?
    print(header.find_bytes(b"\r\n", 0, header.len()))?
    print(header.matches(0, &magic[:4]), header.find_in(b"\n\r", 0, header.len()))?
    print(starts_png(&header), starts_png(&header[4:]), starts_png(b"GIF8"))?
    Ok(())
Option[i64].Some(4)
True Option[i64].Some(4)
True False False

The literal is never converted into a Bytes behind your back. To change bytes, or to call a Bytes method such as find on them, copy any view with from_bytes, as header does; the copy allocates, so it returns a Result. A slice of a literal, such as b"GET /index"[4:]?, is also an owned Bytes. Stream writes in std.io accept a literal the same way, as the next lessons show.

A byte literal accepts only ASCII characters, so b"é" is an error; use from_text for the UTF-8 encoding of text. See byte literals for the full rules.

See the module reference for every operation, its costs, and what views cannot do yet.

Build and normalize bounded text

std.text offers a reusable UTF-8 builder. Its limit counts bytes: é takes two bytes and 🙂 takes four. A failed append leaves the old contents intact.

import std.text
import std.unicode

def main() -> Result[(), AllocError | IoError | std.text.TextError]:
    mut message = std.text.builder(32)?
    message.append("Hello, ")?
    message.append("é🙂")?
    print(message.to_text()?, message.byte_len())?
    message.clear()
    message.append("again")?
    print(message.finish()?)?
    words = ["one", "two", "three"]?
    print(std.text.join(&words, " / ", 32)?)?
    print(str.repr(std.text.hard_wrap("é🙂abc", 2, 32)?)?)?
    print(std.unicode.version())?
    print(std.unicode.nfc("é", 16)? == "é")?
    print(std.unicode.nfkc("ffi", 16)?)?
    Ok(())
Hello, é🙂 13
again
one / two / three
"é🙂\nab\nc"
17.0.0
True
ffi

Use to_text() for a snapshot when keeping the builder, and finish() when done. finish consumes it even if allocating the final string fails; to_text keeps it available for retry. Both currently copy the output bytes.

hard_wrap counts Unicode scalars, preserves LF, and can split a combining sequence; it does not measure terminal columns or find word boundaries. Normalization has a separate scalar limit covering the input and decomposed intermediate. For example, decomposing é needs room for two scalars even though NFC can compose them back to one. Every operation remains fallible: use ? or match TextError to handle allocation, invalid limits, and output that exceeds them.

Normalization never happens automatically. NFC treats canonically equivalent spellings consistently; NFKC also removes compatibility distinctions such as the ligature above. Choose it only when that loss is intended. Byte decoding still rejects malformed UTF-8 before normalization; embedded NUL is ordinary text. See std.text and std.unicode for costs and complete contracts.

For prose, use fill to wrap at word boundaries. shorten collapses whitespace and truncates to one line; dedent removes a common space/tab margin. These helpers use default input and output caps of 1 MiB, which you can change with max_input_bytes and max_bytes.

import std.text

def main() -> Result[(), IoError | std.text.TextError]:
    print(std.text.fill("one two three four", 9)?)?
    print(std.text.shorten("one two three four", 13)?)?
    print(std.text.indent(&std.text.dedent("    first\n      second")?, "> ")?)?
    Ok(())
one two
three
four
one two [...]
> first
>   second

Widths count scalars rather than display columns. Use a reusable TextWrapper when you need indentation, tab options, or a maximum number of lines. Wrap each paragraph separately: the default settings treat input newlines as spaces.

Read and write streams in pieces

A file, a pipe, or a terminal rarely moves all the bytes you ask for in one call. std.io makes that visible: one read or write reports how many bytes it moved, and helpers repeat the call until a range is complete.

A stream is any pub class with the right methods. This one hands out its text at most three bytes at a time, as a slow pipe would:

import std.bytes
import std.io
from std.bytes import Bytes
from std.io import StreamError

pub class Trickle:
    pub data: Bytes
    pub at: i64

    pub def read(self: &mut Trickle, buffer: &mut Bytes, start: i64, stop: i64) -> Result[i64, StreamError]:
        if start < 0 or start > stop or stop > buffer.len():
            return Err(StreamError.OutOfRange)
        if self.at < 0 or self.at > self.data.len():
            return Err(StreamError.OutOfRange)
        # Valid ranges give nonnegative differences <= i64.MAX.
        # count <= stop-start and <= data.len()-at, so both ends stay in range.
        unchecked:
            mut count = stop - start
            if count > 3:
                count = 3
            if count > self.data.len() - self.at:
                count = self.data.len() - self.at
            buffer.overwrite(start, &self.data[self.at:self.at + count])
            self.at += count
            Ok(count)

def main() -> Result[(), AllocError | IoError | StreamError | std.bytes.BytesError]:
    mut source = Trickle(std.bytes.from_text("0123456789")?, 0)
    mut buffer = std.bytes.with_capacity(8)?
    buffer.resize(8, 46u8)?
    print(source.read(&mut buffer, 0, 8)?, buffer.to_text()?)?
    print(str.repr(std.io.read_exact(&mut source, &mut buffer, 0, 4))?, buffer.to_text()?)?
    print(str.repr(std.io.read_exact(&mut source, &mut buffer, 0, 8))?)?
    print(source.read(&mut buffer, 0, 8)?)?
    Ok(())
3 012.....
Result[(), std.io.Partial].Ok(()) 3456....
Result[(), std.io.Partial].Err(std.io.Partial(completed=3, cause=std.io.StreamError.UnexpectedEof))
0

read fills part of buffer[start:stop] and returns the count. A reader checks the range itself and reports OutOfRange, as the stream contract requires; overwrite would otherwise trap on it. Three of eight bytes is a short read, not an error; zero means end of input. read_exact calls read until the range is full. When the input ends first, the error says how many bytes arrived, and they are in the buffer.

Writing has the same shape. write may accept only part of a range, and write_all continues until all of it is delivered. A BufWriter collects small writes in a fixed-size buffer and passes them on in larger pieces:

import std.bytes
import std.io

def main() -> Result[(), IoError | AllocError | std.io.Partial | std.io.StreamError | std.bytes.BytesError | OverflowError[i64]]:
    mut input = std.io.buffered_reader(std.io.cursor(std.bytes.from_text("red\ngreen\nblue")?), 4)?
    mut output = std.io.buffered_writer(std.io.stdout(), 16)?
    mut line = std.bytes.empty()?
    mut count = 0
    while input.read_until(10u8, &mut line, 64)? > 0:
        count += 1
        std.io.write_all(&mut output, b"line: ")?
        std.io.write_all(&mut output, &line)?
        line.clear()
    std.io.write_all(&mut output, b"\n")?
    output.flush()?
    print(count, output.pending(), output.capacity())?
    Ok(())
line: red
line: green
line: blue
3 0 16

write_all takes the byte literal b"line: " as readily as the buffer line: a stream’s write reads a &[u8] view in place, so it also takes part of a buffer, &line[:3]. A class of your own becomes a writer with a flush method and a write method taking source: &[u8] that returns how many bytes it accepted; the module reference shows one.

cursor turns a Bytes into a stream, which is how to test stream code without a file. read_until appends one line, including its newline, and never more than the limit of 64 bytes, so a file with no newline cannot exhaust memory. Both buffers are allocated once, with the capacity you pass, and never grow. Output reaches the terminal when the buffer fills and at flush, so flush before printing anything that must come after it.

Bytes held by a BufWriter are not written when it goes out of scope. Call flush, or close_buffered for a stream that also needs closing:

import std.bytes
import std.io
from std.io import SeekFrom

type ByteTransferError = std.io.Partial | std.bytes.BytesError | std.io.StreamError

def main() -> Result[(), AllocError | IoError | ByteTransferError]:
    mut file = std.io.buffered_writer(std.io.open_file("streams-lesson.bin", "w+")?, 8)?
    record = std.bytes.from_list(&[1u8, 0u8, 255u8, 10u8, 7u8]?)?
    std.io.write_all(&mut file, &record[:5])?
    print(file.pending(), std.io.seek_writer(&mut file, SeekFrom.Start(2))?, file.pending())?
    mut back = std.bytes.with_capacity(3)?
    back.resize(3, 0u8)?
    std.io.read_exact(&mut file.inner, &mut back, 0, 3)?
    print(back.to_list()?)?
    print(str.repr(std.io.close_buffered(&mut file))?)?
    print(str.repr(file.flush())?)?
    Ok(())
5 2 0
[255, 10, 7]
Result[(), std.io.CloseError].Ok(())
Result[(), std.io.StreamError].Err(std.io.StreamError.Io(IoError.Closed))

Five bytes wait in the buffer until seek_writer delivers them and moves the file to byte 2. File streams carry raw bytes: 255 and the newline byte come back unchanged. close_buffered flushes, closes even when the flush fails, and reports which of the two went wrong.

See the module reference for every error, the copy helper, and the standard stream adapters.

A text filter over files or stdin

For a command-line text filter, let std.cli.fileinput handle the buffered reads, UTF-8 decoding, and owned files:

import std.cli.fileinput as fileinput
from std.cli.fileinput import FileInputError

def main() -> Result[(), IoError | AllocError | FileInputError]:
    for line in fileinput.lines()?:
        write_stdout(line?)?
    flush_stdout()?
    Ok(())

After compiling, pass filenames in order, or pipe text into the executable with no arguments. A filename - selects stdin. The outer ? handles argument-list construction; line? propagates an error while opening, reading, or decoding. Use write_stdout, not print, because lines retain their endings. The final line of a file need not have a newline.

For applications that parse flags, pass the chosen list[Path] to fileinput.files(paths) instead: lines() does not parse options. Both APIs use chunked buffering, not a system call for each character, and decode strict UTF-8. Breaking out closes owned files, never stdin; any bytes read ahead into the iterator’s buffer are discarded, so do not interleave other stdin readers.

For fixed-width in-memory fields, use the binary packet cursor instead of manually tracking offsets or expecting stream reads to be atomic.

A read range must satisfy 0 <= start <= stop <= buffer.len(). Invalid buffer ranges trap before reading; see the stream contract.

Read and write binary packets

A Cursor keeps the byte position for a packet. Its typed operations read or write one complete field, with explicit byte order and no manual offset bookkeeping.

import std.bytes
import std.io
from std.encoding import Endian
from std.io import SeekFrom

def main() -> Result[(), AllocError | IoError | RangeError | std.bytes.BytesError | std.io.BinaryError | std.io.StreamError]:
    payload = b"hello"
    mut packet = std.io.cursor(std.bytes.empty()?)
    packet.write_u8(1u8)?
    # Check that a payload size fits its wire field before converting it.
    packet.write_u32(u32.try_from(len(payload))?, Endian.Big)?
    packet.write_f32(1.5f32, Endian.Big)?
    packet.write_bytes(&payload, 64)?
    drop(packet.seek(SeekFrom.Start(0))?)

    version = packet.read_u8()?
    length = i64(packet.read_u32(Endian.Big)?)
    scale = packet.read_f32(Endian.Big)?
    # A hostile length is rejected before allocating its owned payload copy.
    body = packet.read_bytes(length, 64)?
    packet.require_end()?
    print(version, scale, body.to_text()?, packet.position())?
    Ok(())
1 1.5 hello 14

The checked u32 conversion propagates RangeError if a payload length cannot fit the wire field. read_bytes returns an owned copy because borrowed views cannot escape a method. Scalar fields allocate nothing when storage already has capacity.

If a field is truncated, BinaryError.Truncated reports its position, requested width, and available bytes. The failed operation leaves the cursor in place; previous fields remain consumed. require_end detects unexpected trailing bytes. Raw stream reads and writes and seek use the same position, so they can be interleaved with typed fields when the format calls for them.

Read and write CSV records

std.csv reads a table one record at a time, so a large file never has to fit in memory. A reader sits on any byte stream and fills the same Record on every call:

import std.bytes
import std.csv
import std.io

type CsvInputError = ParseError | std.csv.CsvError | std.bytes.BytesError

def main() -> Result[(), AllocError | IoError | CsvInputError | OverflowError[i64]]:
    text = "item,price,note\r\npen,3,\"blue, fine\"\r\nbook,12,\"says \"\"read me\"\"\"\r\ncard,2,\"two\nlines\"\r\n"
    mut input = std.csv.reader(std.io.cursor(std.bytes.from_text(text)?), &std.csv.rfc4180(), 16)?
    mut record = std.csv.record()?
    input.read(&mut record)?
    print(str.repr(record.texts()?)?)?
    mut total = 0
    while input.read(&mut record)?:
        total += i64.parse(&record.text(1)?)?
        print(record.number(), record.line(), record.text(0)?, str.repr(record.text(2)?)?)?
    print(total, input.records())?
    Ok(())
["item", "price", "note"]
2 2 pen "blue, fine"
3 3 book "says \"read me\""
4 4 card "two\nlines"
17 4

read returns True while there is another record and False at the end of the input. The reader’s buffer is the 16 bytes passed to reader; records longer than that are assembled across several fills. Quoted fields may hold the delimiter, line breaks, and doubled quotes, and arrive without their quoting. The first record is not special, so the program reads the header itself.

Fields are bytes. text(index) checks one field as UTF-8 and returns a copy you own; texts() copies them all. The record itself is overwritten by the next read, which is why read takes it as &mut.

rfc4180() is the usual dialect: comma, double quote, and every record as wide as the first. A record of another width is reported, but it has been read and the reader carries on. Malformed input stops the reader for good:

import std.bytes
import std.csv
import std.io
from std.csv import CsvError

def main() -> Result[(), AllocError | IoError | CsvError | std.bytes.BytesError]:
    text = "a,b\nc\nd,e\nf,g\"h\ni,j\n"
    mut input = std.csv.reader(std.io.cursor(std.bytes.from_text(text)?), &std.csv.rfc4180(), 16)?
    mut record = std.csv.record()?
    mut more = True
    while more:
        outcome = input.read(&mut record)
        match outcome:
            case Ok(found):
                if found:
                    print(str.repr(record.texts()?)?)?
                else:
                    more = False
            case Err(cause):
                print(str.repr(&cause)?)?
                match cause:
                    case CsvError.Width(_, _, _):
                        print("skipped", str.repr(record.texts()?)?)?
                    case _:
                        more = False
    print(str.repr(input.read(&mut record))?)?
    Ok(())
["a", "b"]
std.csv.CsvError.Width(2, 2, 1)
skipped ["c"]
["d", "e"]
std.csv.CsvError.Invalid(std.csv.Fault.BareQuote, std.csv.Position(record=4, field=1, line=4, offset=13))
Result[bool, std.csv.CsvError].Err(std.csv.CsvError.Stopped)

Width(2, 2, 1) says record 2 should have two fields and has one. The Position names the record, the field index, the line, and the byte offset of the stray quote. Record i,j is never reached: after a broken quote there is no trustworthy place to resume.

A writer builds one record from fields and delivers it at end_record. texts does both for a list. Change the dialect’s public fields before creating the writer:

import std.bytes
import std.csv
import std.io
from std.csv import Newline

def main() -> Result[(), AllocError | IoError | std.bytes.BytesError | std.csv.CsvError]:
    mut dialect = std.csv.rfc4180()
    dialect.newline = Newline.Lf
    dialect.max_field = 12
    mut output = std.csv.writer(std.io.stdout(), &dialect)?
    output.texts(&["item", "note"]?)?
    output.texts(&["pen", "blue, fine"]?)?
    output.text("book")?
    output.text("says \"hi\"")?
    output.end_record()?
    print(str.repr(output.texts(&["card"]?))?)?
    print(str.repr(output.text("much too long for a field"))?)?
    raw = std.bytes.from_text("card:plain")?
    output.field(&raw[:4])?
    output.field(&raw[5:10])?
    output.end_record()?
    output.flush()?
    print(output.records())?
    Ok(())
item,note
pen,"blue, fine"
book,"says ""hi"""
Result[(), std.csv.CsvError].Err(std.csv.CsvError.Width(4, 2, 1))
Result[(), std.csv.CsvError].Err(std.csv.CsvError.Rejected(std.csv.Fault.FieldTooLong, 4, 0))
card,plain
4

Only fields that need quotes get them. The one-field record is refused by the width policy and the oversized text by max_field; neither writes anything. field adds a byte range without building a string first. Call flush when you are done: a writer does not flush or close its stream when it goes out of scope.

Limits apply to reading too. A field over max_field bytes or a record over max_record is an error at the byte that does not fit, so a file with a missing closing quote cannot consume all memory.

See the module reference for the exact grammar, every error, the borrowed data() view, and what allocates.

Process XML events

std.xml reads one bounded event at a time from a byte stream. Event text has already been checked as UTF-8 and entity references have been decoded. Match the events your application needs, and keep reading through Eof to check the complete document:

import std.bytes
import std.io
import std.xml
from std.xml import Event

def main() -> Result[(), AllocError | IoError | std.bytes.BytesError | std.xml.XmlError]:
    source = "<items><item>Tea &amp; cake</item><item>Milk</item></items>"
    mut input = std.xml.reader(std.io.cursor(std.bytes.from_text(source)?), &std.xml.limits())?
    mut output = std.xml.writer(std.io.cursor(std.bytes.empty()?), &std.xml.limits())?
    mut done = False
    while not done:
        event = input.next()?
        match &event:
            case Event.Text(text):
                print(*text)?
            case Event.Eof:
                done = True
            case _:
                pass
        output.write(&event)?
    print(output.inner.data.to_text()?)?
    Ok(())
Tea & cake
Milk
<items><item>Tea &amp; cake</item><item>Milk</item></items>

The writer escapes values and checks the element structure. Passing Eof checks that the root has closed and flushes its stream. When constructing events yourself, call finish() after the final element. Events own their strings, so retaining an event also retains its memory.

Choose limits for the input you accept. max_token bounds a complete tag, comment, or text run; max_depth bounds nesting. The reference describes all limits and errors. This example lets errors propagate so malformed input and stream failures remain visible. A failed file writer may have emitted partial output; applications that need replacement only on success should manage a temporary destination.

This reader accepts UTF-8 XML 1.0 without a DTD. It never loads external entities. Namespace prefixes remain literal names, so applications needing namespace URI resolution must supply that policy themselves.

Read the environment and run other programs

std.env reads the variables a program was started with. std.process starts another program from a list of arguments and collects what it prints:

import std.bytes

import std.io

import std.env
import std.process
from std.process import Stdio

def main() -> Result[(), AllocError | IoError | std.bytes.BytesError | std.io.StreamError]:
    print(std.env.get("PLENTY_LESSON_UNSET")?)?
    mut command = std.process.command("printf")?
    command.arg("%s\n")?
    command.arg("two words; echo $HOME | wc")?
    command.stdout(Stdio.Piped)
    mut child = command.spawn()?
    output = child.output(4096)?
    print(output.stdout.to_text()?)?
    print(output.status.success(), output.status.code())?
    Ok(())
Option[str].Nothing
two words; echo $HOME | wc

True Option[i32].Some(0)

An unset variable is Nothing, not an error. Each arg is one argument, delivered byte for byte: the semicolon, the $HOME, and the | are printed, not run, because no shell reads them. Stdio.Piped connects the child’s output to a pipe, and output collects at most 4096 bytes of it and waits for the child to end.

If a configuration supplies arguments as one string, use std.shlex.split to recognize quoted words, then pass them individually to arg. Do not quote arguments passed to arg itself; those quotes would become literal data.

import std.shlex
from std.shlex import ShlexError

def main() -> Result[(), ShlexError | AllocError | IoError]:
    arguments = std.shlex.split("--title 'Two words' --literal '$HOME'")?
    for argument in &arguments:
        print(argument)?
    Ok(())
--title
Two words
--literal
$HOME

split removes word quoting without expanding $HOME or interpreting pipes. std.shlex also provides quote for the cases where one argument must be written into a POSIX shell command string.

To find where to keep settings or caches, ask std.fs.dirs rather than reading HOME yourself: config_dir() and cache_dir() apply the XDG rules and return a Path, or Nothing when the environment names no directory.

When shell syntax is what you want, ask for it by name. shell runs /bin/sh -c with your script. communicate also feeds the child’s input:

import std.io

import std.bytes
import std.process
from std.process import Stdio

def main() -> Result[(), AllocError | IoError | std.io.StreamError | std.bytes.BytesError]:
    mut command = std.process.shell("tr a-z A-Z; echo \"mode: $MODE\" >&2; exit 3")?
    command.env("MODE", "loud")?
    command.stdin(Stdio.Piped)
    command.stdout(Stdio.Piped)
    command.stderr(Stdio.Piped)
    mut child = command.spawn()?
    output = child.communicate(b"quiet words", 4096)?
    print(output.stdout.to_text()?)?
    print(output.stderr.to_text()?)?
    print(output.status.success(), output.status.code())?
    Ok(())
QUIET WORDS
mode: loud

False Option[i32].Some(3)

env sets a variable for this child only. communicate writes the input, reads both outputs as they arrive so that no pipe fills up, and returns the exit status. A nonzero exit code is a normal result: the program ran. Never put text you did not write into a shell script; pass it as an argument or a variable.

A program that cannot be started is an error, and a child that runs too long can be stopped:

import std.time
import std.process

def main() -> Result[(), AllocError | IoError | std.process.WaitError]:
    mut missing = std.process.command("plenty-no-such-program")?
    match missing.spawn():
        case Ok(_):
            print("started")?
        case Err(error):
            print(error)?
    mut sleeper = std.process.command("sleep")?
    sleeper.arg("60")?
    mut child = sleeper.spawn()?
    cancel = CancellationToken()?
    match child.wait_timeout(&std.time.milliseconds(50u32), &cancel)?:
        case Some(_):
            print("ended")?
        case Nothing:
            print("still running")?
    child.kill()?
    status = child.wait()?
    print(status.success(), status.code(), status.signal())?
    Ok(())
IoError.System(2)
still running
False Option[i32].Nothing Option[i32].Some(9)

System(2) is the operating system’s “no such file”. wait_timeout gives up after 50 milliseconds and leaves the child running. kill stops it, and wait collects its status: ended by signal 9, with no exit code. Always wait for a child you started, even after killing it; a dropped Child is not cleaned up.

See the references for std.env and std.process, including pipes you read and write yourself.

To inspect CPU, memory, disk or process information without launching a command, continue to system information.

Read system information

std.system reads Linux’s /proc files into typed values. Open the source once, then ask for the information needed. Each query is fresh and fallible:

import std.system
from std.system import SystemError

def main() -> Result[(), IoError | SystemError]:
    source = std.system.local()?
    cpus = source.cpu_count()?
    memory = source.memory()?
    uptime = source.uptime()?
    print("CPU information available:", cpus > 0)?
    print("Memory information available:", memory.total_bytes > 0u64)?
    print("Uptime available:", uptime.seconds >= 0.0)?
    Ok(())
CPU information available: True
Memory information available: True
Uptime available: True

The example prints checks so its output stays the same on different machines. A monitor can print cpus, memory.total_bytes and uptime.seconds directly. There is no timer or background thread: another call reads another snapshot.

For memory headroom, prefer available_bytes to free_bytes: the kernel’s available estimate includes memory it expects to reclaim. The field is optional because not every source provides it. Here is how to keep absence distinct from zero, using sample values so both paths are visible:

import std.system

def show_available(memory: &std.system.Memory) -> Result[(), IoError]:
    match memory.available_bytes:
        case Some(bytes):
            print("available bytes:", bytes)?
        case Nothing:
            print("available memory is not reported")?
    Ok(())

def main() -> Result[(), IoError]:
    reported = std.system.Memory(8192u64, 1024u64, Some(4096u64), 0u64, 0u64)
    absent = std.system.Memory(8192u64, 1024u64, Nothing, 0u64, 0u64)
    show_available(&reported)?
    show_available(&absent)?
    Ok(())
available bytes: 4096
available memory is not reported

Reads have default byte limits; override them with a named argument such as source.memory(max_bytes=65536). A file that exceeds the limit produces an error, not a truncated record. open_procfs(&path) reads a different procfs mount or captured directory, useful when testing a monitoring tool.

process_ids() returns a bounded, sorted list of visible process IDs. process(pid) reads one status record. Processes can exit between those calls, and access can be denied; handle the SystemError.Io cause rather than treating the whole list as guaranteed live. Use std.process to manage children your program starts.

block_devices() reports device and partition sizes; disk_io() reports counters. Avoid summing whole disks with their partitions. Neither gives free filesystem space. CPU and memory values can describe the host even inside a container; they are not cgroup quota measurements or a recommendation for how many workers to start.

See std.system for fields, units, error variants and snapshot limitations.

Store records in SQLite

For a dedicated file of byte-keyed records without SQL, the next lesson’s std.db.kv supplies a smaller interface over this same engine. This lesson uses SQL to describe and query structured rows.

std.db.sqlite keeps data in a SQLite database: a single file, or memory. Values travel only through prepared statements, so a value is never mistaken for SQL.

Insert and query

execute runs fixed SQL such as a schema. prepare compiles one statement with ? parameters; bind each by number, from 1, and run it. A query is read a row at a time with step, and each column, numbered from 0, is read with the method for its type:

import std.db.sqlite

def main() -> Result[(), AllocError | IoError | std.db.sqlite.DbError]:
    mut db = std.db.sqlite.memory(65536)?
    db.execute("CREATE TABLE books(title TEXT NOT NULL, year INTEGER, rating REAL)")?
    mut insert = db.prepare("INSERT INTO books VALUES (?, ?, ?)")?
    insert.bind_text(1, "Dune")?
    insert.bind_integer(2, 1965)?
    insert.bind_real(3, 4.3)?
    insert.run()?
    insert.bind_text(1, "x'); DROP TABLE books; --")?
    insert.bind_null(2)?
    insert.bind_real(3, 1.0)?
    insert.run()?
    mut query = db.prepare("SELECT title, year FROM books WHERE rating > ? ORDER BY rating")?
    query.bind_real(1, 0.5)?
    while query.step()?:
        print(query.text(0)?, query.optional_integer(1)?)?
    Ok(())
x'); DROP TABLE books; -- Option[i64].Nothing
Dune Option[i64].Some(1965)

The second title looks like an attack but is stored as plain text. Reads copy each value out, so a str from text stays valid after the next step. memory(65536) sets the largest text, blob or row the connection will store or read; anything larger is TooLarge.

Types are checked

Each read requires the column’s stored type. A NULL needs optional_integer or another optional read, and a number stored as text is not an integer:

import std.db.sqlite

def main() -> Result[(), AllocError | IoError | std.db.sqlite.DbError]:
    mut db = std.db.sqlite.memory(65536)?
    mut query = db.prepare("SELECT '42', NULL")?
    query.step()?
    print(str.repr(query.integer(0))?)?
    print(str.repr(query.integer(1))?)?
    print(str.repr(query.kind(0)?)?)?
    Ok(())
Result[i64, std.db.sqlite.DbError].Err(std.db.sqlite.DbError.TypeMismatch(std.db.sqlite.ValueType.Text))
Result[i64, std.db.sqlite.DbError].Err(std.db.sqlite.DbError.TypeMismatch(std.db.sqlite.ValueType.Null))
std.db.sqlite.ValueType.Text

Transactions

begin returns a Transaction. commit keeps its changes; dropping it without commit, including by returning early with ?, rolls them back. SQLite failures carry SQLite’s own code and message:

import std.db.sqlite
from std.db.sqlite import Begin, Connection, DbError

def transfer(db: &mut Connection, amount: i64) -> Result[(), DbError | OverflowError[i64]]:
    mut tx = db.begin(Begin.Immediate)?
    mut change = db.prepare("UPDATE accounts SET balance = balance + ? WHERE name = ?")?
    change.bind_integer(1, 0 - amount)?
    change.bind_text(2, "alice")?
    change.run()?
    change.bind_integer(1, amount)?
    change.bind_text(2, "bob")?
    change.run()?
    tx.commit()

def main() -> Result[(), AllocError | IoError | DbError | OverflowError[i64]]:
    mut db = std.db.sqlite.memory(65536)?
    db.execute("CREATE TABLE accounts(name TEXT PRIMARY KEY, balance INTEGER CHECK (balance >= 0))")?
    db.execute("INSERT INTO accounts VALUES ('alice', 10), ('bob', 0)")?
    transfer(&mut db, 4)?
    match transfer(&mut db, 50):
        case Ok(_):
            print("moved")?
        case Err(DbError.Sqlite(error)):
            print(error.primary() == std.db.sqlite.constraint, error.message)?
        case Err(_):
            print("other")?
    mut query = db.prepare("SELECT name, balance FROM accounts ORDER BY name")?
    while query.step()?:
        print(query.text(0)?, query.integer(1)?)?
    Ok(())
True CHECK constraint failed: balance >= 0
alice 6
bob 4

The failed transfer left no trace: its transaction rolled back when transfer returned the error.

For a file, use std.db.sqlite.connect(path, Mode.Create, limit). Another connection holding a write lock makes a write fail at once with SQLite’s busy code; set_busy_timeout makes it wait instead. See the std.db.sqlite reference for every error, the busy policy, and how connections are shared between threads.

SQLite’s FTS5 full-text search is built in. Applications that need a native extension, such as vector search, can load a trusted shared library with Connection.load_extension(path, entrypoint=""). This runs native code in the application; SQL cannot enable extension loading. See native extensions and full-text search for the loading contract and connection lifetime.

Store byte-keyed records

Use std.db.kv when a program needs named maps of byte keys and values, and ordered scans, without writing SQL. Use std.db.sqlite when SQL queries, joins or your own database schema are part of the application.

Batch related writes in a transaction. A failed operation propagated by ? drops the unfinished transaction and rolls the batch back:

import std.db.kv as kv
import std.bytes as buffers
from std.bytes import BytesError

def main() -> Result[(), AllocError | kv.KvError | BytesError | IoError]:
    mut store = kv.memory()?
    key = buffers.from_text("user:42")?
    value = buffers.from_text("Ada")?
    mut transaction = store.begin()?
    store.put("names", &key, &value)?
    transaction.commit()?
    drop(transaction)

    match store.get("names", &key)?:
        case Some(name):
            print(name.to_text()?)?
        case Nothing:
            print("not found")?

    mut rows = store.scan_all("names", max_rows=20)?
    while True:
        match rows.next()?:
            case Some(entry):
                print(entry.key.to_text()?, entry.value.to_text()?)?
            case Nothing:
                break
    drop(rows)
    store.close()?
    Ok(())
Ada
user:42 Ada

This example uses a private in-memory store. For persistence, replace kv.memory() with kv.connect("records.db"); the default creates a new store or opens an existing KV file. It rejects a database belonging to another schema. Mode.ReadOnly opens an existing store without allowing writes.

Keys and values need not be text; the example converts them because names are UTF-8. Every returned value owns its buffers. Scans require an explicit row limit and copy one row at a time. Finish/drop a scan before modifying the same Store or ending its transaction. Use scan(table, start, stop, max_rows) for a byte range with inclusive start and exclusive stop.

The reference explains file compatibility, per-row size limits, lock conflicts and scan lifetimes. The engine comparison explains why this API uses SQLite rather than introducing a redb dependency.

Read and write HTTP messages

An HTTP/1.1 connection carries one message after another. Each message is a head, which ends at a blank line, and an optional body. A reader must find where each body ends, or it reads the next request as part of the last one. std.http does that work over any std.io stream, so you can test it with bytes in memory before any socket is involved.

import std.bytes
import std.http
import std.io

type MessageStreamError = std.http.HttpError | std.io.StreamError | std.bytes.BytesError

def main() -> Result[(), AllocError | IoError | MessageStreamError]:
    wire = "POST /notes HTTP/1.1\r\nHost: example.com\r\nContent-Length: 5\r\n\r\nhelloGET /notes HTTP/1.1\r\nHost: example.com\r\nConnection: close\r\n\r\n"
    limits = std.http.limits()
    mut conn = std.io.buffered_reader(std.io.cursor(std.bytes.from_text(wire)?), 1024)?
    while True:
        match std.http.read_request(&mut conn, &limits)?:
            case Nothing:
                break
            case Some(head):
                framing = std.http.request_framing(&head)?
                mut body = std.http.body_reader(conn, &framing, &limits)?
                mut data = std.bytes.empty()?
                std.http.read_to_end(&mut body, &mut data)?
                print(head.method, head.target, str.repr(&framing)?, str.repr(data.to_text()?)?, std.http.request_keeps_alive(&head))?
                conn = std.http.finish(&mut body)?
    Ok(())
POST /notes std.http.Framing.Length(5) "hello" True
GET /notes std.http.Framing.Empty "" False

read_request reads one head and returns Nothing when the input ends between messages. request_framing says how the body is delimited: here by Content-Length, then not at all. The body reader takes the connection, so nothing else can read from it until finish hands it back, positioned at the next request. finish fails if the body was not read to its end. Every read is bounded by limits(): line length, head size, header count, and body size.

Writing goes the other way. A chunked body is sent in pieces whose total size the writer need not know in advance:

import std.bytes
import std.http
import std.io
from std.http import Header

type MessageStreamError = std.io.StreamError | std.http.HttpError | std.bytes.BytesError

def main() -> Result[(), AllocError | IoError | MessageStreamError]:
    mut sink = std.io.cursor(std.bytes.empty()?)
    mut reply = std.http.response(200, "OK")?
    reply.add("Transfer-Encoding", "chunked")?
    std.http.write_response(&mut sink, &reply)?
    text = std.bytes.from_text("first second")?
    std.http.write_chunk(&mut sink, &text[:6])?
    std.http.write_chunk(&mut sink, &text[6:12])?
    std.http.write_last_chunk(&mut sink, &list[Header].new()?)?
    print(str.repr(sink.data.to_text()?)?)?
    mut forged = std.http.response(200, "OK")?
    forged.add("Location", "/a\r\nSet-Cookie: x=1")?
    print(str.repr(std.http.write_response(&mut sink, &forged))?)?
    smuggled = "POST / HTTP/1.1\r\nContent-Length: 4\r\nTransfer-Encoding: chunked\r\n\r\n"
    mut conn = std.io.buffered_reader(std.io.cursor(std.bytes.from_text(smuggled)?), 1024)?
    match std.http.read_request(&mut conn, &std.http.limits())?:
        case Some(head):
            print(str.repr(std.http.request_framing(&head))?)?
        case Nothing:
            print("connection ended before a request")?
    Ok(())
"HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\n\r\n6\r\nfirst \r\n6\r\nsecond\r\n0\r\n\r\n"
Result[(), std.http.HttpError].Err(std.http.HttpError.InvalidHeader)
Result[std.http.Framing, std.http.HttpError].Err(std.http.HttpError.ConflictingFraming)

The writer refuses a header value with a line break, which would let text from a request add headers of its own. The reader refuses a message with both Content-Length and Transfer-Encoding: two servers could disagree about where its body ends, which is how request smuggling works. Neither guesses.

The same functions work over a std.net stream wrapped in a BufReader. Deadlines stay with the socket calls. See the module reference for every error, response framing rules, and the body reader’s use as a std.io stream.

Formatting into a reusable writer

An f-string is the short way to build a str; it compiles to the calls below. Use std.fmt directly when output must go to a chosen byte writer, reuse storage, or stay within a byte budget. Numeric widths count bytes and include the sign. Every call is fallible, and the byte budget includes punctuation and padding.

import std.fmt

def main() -> Result[(), AllocError | IoError | std.fmt.FormatError]:
    mut out = std.fmt.string_formatter(80, 8)?
    out.text("count=")?
    out.integer(12, 4)?
    out.text(", label=")?
    out.debug_text("one\ntwo")?
    print(std.fmt.finish(&out)?)?
    Ok(())
count=  12, label="one\ntwo"

Your type chooses separate display and debug methods. Both can be generic over the output writer; callers need no trait object or format-string parser. print and f"{x}" call display; str.repr, f"{x!r}" and the elements of a printed list or other aggregate call debug. std.fmt.to_string and to_debug call the same methods within a byte budget and depth you choose.

import std.fmt
from std.fmt import Formatter

pub class Label:
    name: str

    pub def display[W](self, out: &mut Formatter[W]) -> Result[(), FormatError]:
        out.text(&self.name)

    pub def debug[W](self, out: &mut Formatter[W]) -> Result[(), FormatError]:
        out.text("Label(")?
        out.debug_text(&self.name)?
        out.text(")")

def main() -> Result[(), AllocError | IoError | FormatError]:
    label = Label("ready")
    print(label)?
    print(str.repr(label)?)?
    print([Label("a"), Label("b")]?)?
    print(std.fmt.to_debug(&label, 80, 8)?)?
    Ok(())
ready
Label("ready")
[Label("a"), Label("b")]
Label("ready")

Because these methods can fail, printing such a value returns Result[(), IoError | FormatError], and str.repr and an f-string with such a field add FormatError to AllocError. A type without display prints with its debug; one without either prints its fields.

Enums use the same methods. Match the shared receiver to format a payload without consuming it:

import std.fmt
from std.fmt import Formatter

pub enum ReadError:
    Missing
    Invalid(i64)

    pub def display[W](self, out: &mut Formatter[W]) -> Result[(), FormatError]:
        match self:
            case ReadError.Missing:
                out.text("no reading")
            case ReadError.Invalid(line):
                out.text("invalid reading on line ")?
                out.integer(*line, 0)

def main() -> Result[(), FormatError | IoError]:
    error = ReadError.Invalid(12)
    print(error)?
    print(std.fmt.to_string(&error, 80, 8)?)?
    Ok(())
invalid reading on line 12
invalid reading on line 12

Returning the enum from main does not call the method yet: the report still uses the structural entrypoint report.

Inside such a method, out.format(f"...") writes an f-string straight to the formatter, with no intermediate str. It returns Result[(), FormatError], and the formatter’s byte budget and depth limit apply:

import std.fmt
from std.fmt import Formatter

pub class Reading:
    line: i64
    value: f64

    pub def display[W](self, out: &mut Formatter[W]) -> Result[(), FormatError]:
        out.format(f"line {self.line}: {self.value:.1f}")

def main() -> Result[(), AllocError | IoError | FormatError]:
    reading = Reading(7, 21.25)
    print(f"[{reading}] [{reading:>14}]"?)?
    mut out = std.fmt.string_formatter(10, 8)?
    match out.format(f"{reading}"):
        case Err(FormatError.LimitExceeded):
            print("kept", std.fmt.finish(&out)?)?
        case _:
            print("fits")?
    Ok(())
[line 7: 21.2] [  line 7: 21.2]
kept line 7: 21

To stream, construct std.fmt.formatter(writer, scratch, max_bytes, max_depth) with a std.io writer and initialized Bytes scratch of at least 32 bytes. Provision scratch and sink capacity before entering an allocation-free section. Formatting itself then reuses that memory; user methods remain responsible for their own allocations. Flush the public out.writer explicitly when needed. On failure, out.completed() tells you how many bytes were accepted; do not blindly restart a partially written record.

The reference describes aggregate callbacks, errors, escaping, depth limits and exact costs.

Formatting a bounded report

Use std.report to give a value a human-readable spelling through a writer-generic formatter. The method borrows its receiver, so an error can be formatted again or examined afterward.

import std.fmt
import std.report
from std.fmt import Formatter, FormatError

pub enum ReadingError:
    Invalid(i64)

    pub def format[W](self, output: &mut Formatter[W]) -> Result[(), FormatError]:
        match self:
            case ReadingError.Invalid(line):
                output.text("invalid reading on line ")?
                output.integer(*line)

def main() -> Result[(), FormatError | IoError]:
    error = ReadingError.Invalid(12)
    print(std.report.to_string(&error, 80, 8)?)?
    print(std.report.to_string(&error, 80, 8)?)?
    Ok(())
invalid reading on line 12
invalid reading on line 12

The method’s [W] lets it write to a string, stderr, or another writer. The formatter supplies that type, so callers do not pass a type argument. The public enum and its public format method satisfy the structural std.fmt.Formattable protocol. A class uses the same visibility and method signature. std.report.render(&value, &mut output) counts a nesting level and restores it on success or error. Use it when a formatting method delegates to another value.

An anonymous sum supports the same reporting API when every member satisfies std.fmt.Formattable. You can keep distinct error types and let the compiler select the active member’s method:

import std.fmt
import std.report
from std.fmt import Formatter, FormatError

pub enum ReadingError:
    Invalid(i64)

    pub def format[W](self, output: &mut Formatter[W]) -> Result[(), FormatError]:
        match self:
            case ReadingError.Invalid(line):
                output.text("invalid reading on line ")?
                output.integer(*line)

pub enum ConnectionError:
    Closed

    pub def format[W](self, output: &mut Formatter[W]) -> Result[(), FormatError]:
        output.text("connection closed")

def show(error: &(ReadingError | ConnectionError)) -> Result[(), FormatError | IoError]:
    print(std.report.to_string(error, 80, 1)?)?
    Ok(())

def main() -> Result[(), FormatError | IoError]:
    reading: ReadingError | ConnectionError = ReadingError.Invalid(12)
    connection: ReadingError | ConnectionError = ConnectionError.Closed
    show(&reading)?
    show(&connection)?
    show(&reading)?
    Ok(())
invalid reading on line 12
connection closed
invalid reading on line 12

Dispatch borrows the payload without allocating; string collection still allocates. The original sum remains usable afterward. render counts one depth level for the sum, with no extra level for selecting its member. Calling error.format(&mut output) directly also dispatches, but does not count depth. All members must have the required public method, even if the value currently holds a different member. Other protocols are not automatically forwarded.

For stderr, construct std.fmt.stderr_formatter(max_bytes, max_depth) and call render explicitly. Construction and scalar writes allocate no heap memory, so simple report methods remain usable when allocation is unavailable. Add any desired "error: " prefix and final "\n" with output.text yourself; they count toward the byte limit. User methods can still allocate, so this guarantee depends on what those methods do.

The collector has the same formatting contract but allocates fallibly:

import std.fmt
import std.report
from std.fmt import FormatError

def main() -> Result[(), FormatError | IoError]:
    mut output = std.fmt.string_formatter(4, 0)?
    output.text("a")?
    match output.text("é🙂"):
        case Err(_):
            print("too long", output.completed())?
        case Ok(_):
            pass
    output.text("é")?
    print(std.fmt.finish(&output)?)?
    Ok(())
too long 1
aé

Text checks its whole UTF-8 byte length before writing. Numeric output streams digits, so a limit can leave part of a number. On stderr failure, inspect completed() and has_failed(); subsequent writes return OutputClosed without writing again. A limit error is recoverable, as the example shows.

When main returns Err, its concrete error type selects the reporting path. A type with this public format method produces a human-readable report with the prefix error: main returned and a final newline. A sum selects this path only when all its members support it; other types retain structural reporting. The automatic reporter allows 16,384 bytes and 64 nesting levels. If formatting fails after writing part of the message, it appends [report unavailable] and a newline without trying the method again. Stderr failures stop output.

Use std.fmt for floats, widths, debug escaping or an arbitrary byte writer. The reference lists the complete reporting API.

Generic code and protocols

Use type parameters when several types need the same algorithm or data shape. These lessons build on core functions, borrowing, classes, and enums. Recursive data is an optional extension at the end.

Replace duplicated functions with a type parameter

Suppose the same operation is needed for two integer widths. Separate functions work, but repeat both the implementation and its interface:

def keep_small(value: u8) -> u8:
    value

def keep_large(value: i64) -> i64:
    value

def main() -> Result[(), IoError]:
    print(keep_small(7u8))?
    print(keep_large(9))?
    Ok(())
7
9

A type parameter lets one declaration accept either width while keeping the argument and result types the same.

def identity[T](value: T) -> T:
    value

def sum_to[T: IntType](stop: T) -> Result[T, OverflowError[T]]:
    mut total: T = 0
    for n in range[T](stop):
        total = total + n
    Ok(total)

def first[T](values: &list[T]) -> &T:
    &values[0]

def main() -> Result[(), AllocError | IoError | OverflowError[u16]]:
    print(sum_to(5u16)?)?
    values = identity([3, 4]?)
    print(first(&values))?
    print(values)?
    Ok(())
10
3
[3, 4]

Declare type parameters after the function name. Calls infer them from the argument types, or you can supply them explicitly before the call. Ownership still follows the concrete type: identity(values) transfers the list, while a function taking &T borrows its argument.

IntType restricts a parameter to integer types, and FloatType to f32 and f64. CopyType accepts any type whose values copy, such as numbers, str, and tuples of them, and rejects lists and other values that move; see CopyType. Unconstrained T is useful when the body only moves, borrows, or uses operations supported by the chosen concrete type. Each specialization is checked against those operations.

A type parameter can also stand for unit () inside a Result or Option. This lets one helper handle both operations that return a value and operations that only succeed or fail:

def keep_result[T, E](result: Result[T, E]) -> Result[T, E]:
    result

def main() -> Result[(), AllocError | IoError]:
    written = print("saved")
    keep_result(written)?
    Ok(())
saved

Here T is inferred as (). You can also write an explicit unit type argument, such as keep_result[(), IoError](written). Unit still cannot be a direct function parameter, so identity[()](()) is rejected.

Require collection keys

Use K: KeyType when a generic algorithm needs dictionary keys or set members. It accepts integers, booleans, strings, bytes, and supported tuples, classes, enums and anonymous sums, including custom cleanup. The compiler checks all enum payloads; no hashing method is required.

enum Label:
    Id(i64)
    Name(str)

def frequencies[K: KeyType](values: list[K]) -> Result[dict[K, i64], AllocError | OverflowError[i64]]:
    mut counts: dict[K, i64] = {}?
    for value in values:
        *counts.get_or_insert(value, 0)? += 1
    Ok(counts)

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    values = [Label.Id(7), Label.Name("7"), Label.Id(7)]?
    counts = frequencies(values)?
    print(counts[Label.Id(7)], counts[Label.Name("7")])?
    Ok(())
2 1

The loop consumes the list and moves each key into get_or_insert, which drops it when the key is already present. This also works for keys with custom cleanup, which cannot be copied. Dictionary insertion and the count’s addition remain fallible. Floats do not satisfy KeyType; see the constraint reference.

Infer or choose a specialization

Inference matches the signature against the argument types. In first(&values), the parameter is &list[T] and the argument is &list[i64], so T is i64. When multiple arguments determine the same parameter, their types must agree. Unsuffixed literals in generic positions use their normal defaults (i64 or f64); another argument does not change them. Keep explicit type arguments when you want them to guide literals: sum_to[u16](5) is equivalent to sum_to(5u16).

Types must be determined from arguments, not the expected return type. For example, this function needs empty[u8]() because it has no input mentioning T:

def empty[T]() -> Result[list[T], AllocError]:
    []

def main() -> Result[(), AllocError | IoError]:
    values = empty[u8]()?
    print(values)?
    Ok(())
[]

An annotation on the receiving binding does not change that rule:

def empty[T]() -> Result[list[T], AllocError]:
    []

def main() -> Result[(), AllocError | IoError]:
    values: list[u8] = empty()?
    Ok(())
cannot infer type parameter `T` for `empty` from its arguments

Calls with the same concrete types share one compiled specialization, including recursive calls. Independent uses can specialize many different helpers, or the same helper for many types. The compiler separately limits how deeply one new specialization requires another, how many one generic function has, and how many are created in total. These are compile-time resource limits; generic calls add no runtime allocation. See specialization limits for the current bounds.

Requiring methods with a protocol

A protocol names the methods a generic algorithm needs. A class or enum satisfies it by having matching methods; no inheritance or registration is required. Importing a protocol does not add methods to a type.

protocol Readable:
    def read(self) -> str:
        pass

class Message:
    text: str
    def read(self) -> str:
        self.text

def read_message[T: Readable](source: &T) -> str:
    source.read()

def main() -> Result[(), IoError]:
    message = Message("hello")
    print(read_message(&message))?
    Ok(())
hello

Here &message determines T = Message; the compiler then checks the Readable requirements. Writing read_message[Message](&message) explicitly also works.

self means a shared receiver. Write self: &mut ProtocolName when a required method mutates the receiver. Method parameters, return types, and receiver borrowing must match exactly. All required methods are checked at specialization, including methods the generic function does not happen to use. Across modules, the class methods must be visible to the generic function’s defining module. That means both pub class Message and pub def read, even when the caller is in Message’s own module. The caller does not lend private access to the helper. If only the class is private, the call diagnostic asks you to declare it pub class; private methods still need their own pub def.

For example, save this helper as reader.plenty:

pub protocol Readable:
    def read(self) -> str:
        pass

pub def read_message[T: Readable](source: &T) -> str:
    source.read()

This application needs pub class Message, as well as its public method:

import reader

class Message:
    pub def read(self) -> str:
        "hello"

def main() -> ():
    message = Message()
    drop(reader.read_message(&message))
class `Message` is private to its defining module; declare it `pub class`

Protocols constrain class and enum type arguments; they cannot be stored as values. Use source: &T with T: Readable, not source: Readable.

Parameterize a class

A generic class shares one declaration across concrete field types. Its methods can use the class’s type parameters.

class Cell[T]:
    value: T

    def get(self) -> T:
        self.value

    def replace(self: &mut Cell[T], value: T) -> ():
        self.value = value

def read[T](cell: &Cell[T]) -> T:
    cell.get()

def main() -> Result[(), IoError]:
    mut count = Cell(7u8)
    count.replace(9)
    print(read(&count))?
    text = Cell("hello")
    print(text.get())?
    Ok(())
9
hello

Constructors infer their parameters: Cell(7u8) and Cell[u8](7) have the same type. With Cell(7), the unsuffixed integer defaults to i64.

This get method works for integers and immutable strings, which can be read from a shared field. An owned list needs an explicit copy or a reference: returning it would move it out of a borrowed field. A Cell can hold a list, but its get cannot be called:

class Cell[T]:
    value: T

    def get(self) -> T:
        self.value

def main() -> Result[(), AllocError | IoError]:
    numbers = Cell([1, 2]?)
    first = numbers.get()
    Ok(())
lesson.plenty:9:13: `Cell[list[i64]].get` copies `T`, but `list[i64]` cannot be copied
lesson.plenty:5:9: note: in instantiation `Cell[list[i64]].get`: cannot move out of a field

A generic method is checked once for each set of type arguments it is used with, not where it is written. Without the get call, the program is accepted. The error is reported at the line of your code that first required Cell[list[i64]].get, here the method call, and names the type argument that the method cannot accept. The note below it gives the line in the definition and the reason.

That wording follows the value: here self.value is declared as T. An unrelated list created inside the method does not become a T value just because the caller also chooses a list for T. For failures without a clear parameter origin, the compiler instead reports that the instance’s definition is rejected, followed by the operation’s error. See instance diagnostics for the attribution rules.

When you write a generic class for others to use, plenty --check --check-all-methods FILE also checks the methods your program never calls, for every instance it creates of a generic type it declares. The generics reference describes the check.

Parameterize an enum

Use a type parameter when the same alternatives should carry different kinds of values. Choose the concrete payload type in brackets. Here, T: IntType accepts any supported sized integer. Omit the constraint to allow other stored types.

enum Choice[T: IntType]:
    Empty
    Value(T)

def main() -> Result[(), IoError]:
    selected = Choice[u8].Value(7)
    match selected:
        case Choice.Empty:
            print("empty")?
        case Choice.Value(value):
            print(value)?
    Ok(())
7

A constructor names the instance, as in Choice[u8].Value(7). A pattern names only the enum, as in case Choice.Value(value): the matched value already fixes T, so a pattern takes no type arguments.

Parameterize a protocol

A protocol can describe the same operation for several result types. Classes satisfy it by providing matching methods.

protocol Readable[T]:
    def read(self) -> T:
        pass

class Cell[T]:
    value: T
    def read(self) -> T:
        self.value

def read[T, R: Readable[T]](source: &R) -> T:
    source.read()

def main() -> Result[(), IoError]:
    cell = Cell(7u8)
    print(read(&cell))?
    Ok(())
7

The argument determines R = Cell[u8]; that class’s read signature determines T = u8. Its return type and receiver borrowing must match the protocol. Explicit read[u8, Cell[u8]](&cell) also works. Conflicting type evidence is an error, rather than a reason to widen integers.

Store different implementations of a protocol

An enum wrapper works well when your program knows all the alternatives. A protocol box lets implementations be added independently: the collection only names the operations it needs.

protocol Counter:
    def current(self) -> i64:
        pass

    def increment(self: &mut Counter) -> Result[(), OverflowError[i64]]:
        pass

class Count:
    value: i64

    def current(self) -> i64:
        self.value

    def increment(self: &mut Count) -> Result[(), OverflowError[i64]]:
        self.value += 1
        Ok(())

enum Flag:
    Off
    On

    def current(self) -> i64:
        match self:
            case Flag.Off:
                0
            case Flag.On:
                1

    def increment(self: &mut Flag) -> Result[(), OverflowError[i64]]:
        *self = Flag.On
        Ok(())

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    mut items: list[Box[Counter]] = []?
    items.append(Box[Counter](Count(10))?)?
    items.append(Box[Counter](Flag.Off)?)?
    for item in &mut items:
        item.increment()?
        print(item.current())?
    Ok(())
11
1

Each Box[Counter](value)? moves its input into one heap allocation. The list stores owners with a common interface. Its mutable loop borrows each owner, so increment and current operate on the same instance. No match or cast is needed. Use &items for a loop that only calls shared methods.

The protocol fixes each method’s contract. Count.increment can overflow, so increment returns Result[(), OverflowError[i64]], and Flag declares the same result even though it never fails.

The Box boundary is explicit: list[Counter] is not a stored value type, and Box(Count(10))? preserves the concrete Count type. Allocation failure drops the consumed input. Appending can also fail and consumes its new box under the ordinary list contract.

Dictionary values and generic readers

The same owner type works as a dictionary value and through a protocol bound:

protocol Label:
    def label(self) -> str:
        pass

class Message:
    text: str

    def label(self) -> str:
        self.text

enum Status:
    Ready

    def label(self) -> str:
        return "ready"

def read[T: Label](value: &T) -> str:
    value.label()

def main() -> Result[(), AllocError | IoError]:
    items: dict[str, Box[Label]] = {
        "message": Box[Label](Message("hello"))?,
        "status": Box[Label](Status.Ready)?,
    }?
    for name, item in items.items():
        print(name, read(item))?
    Ok(())
message hello
status ready

Borrowing avoids copying the boxes. Their concrete payloads are destroyed when their owners leave scope, including any custom cleanup.

A box exposes only its protocol’s methods. This initial interface supports shared and mutable receivers, fixed protocol arguments, ordinary parameters, and owned results. Methods with their own type parameters, consuming receivers, or returned borrows cannot be boxed. Protocol boxes cannot be copied, used as set elements or dictionary keys, or transferred to worker threads. Payloads containing channel senders or receivers also need concrete owners: erasing them would hide endpoints from the compiler’s checks for waits that could deadlock. See the reference for the complete limits.

Give a method its own type parameters

A method may declare type parameters of its own, in addition to those on its class. Arguments determine them just as they do for generic functions. A parameter that appears only in the result must be chosen explicitly:

class Number:
    value: i64

    def convert[T: IntType](self) -> T:
        T(self.value)

def main() -> Result[(), IoError]:
    number = Number(7)
    small: u8 = number.convert[u8]()
    print(small)?
    Ok(())
7

Method parameters cannot shadow class parameters. Constructors and destructors use only the class parameters. See the functions-as-values path for a generic method that accepts a callback.

Keep a numeric bound in the type

A configuration value often needs a fixed interval. Name that interval with Bounded, then validate computed input once before passing it around:

type RetryCount = Bounded[i64, 0, 8]

def describe(retries: RetryCount) -> Result[(), IoError]:
    print("Retries:", retries.get())

def main() -> Result[(), RangeError | IoError]:
    describe(3)?
    configured = 5
    retries = RetryCount.try_from(configured)?
    describe(retries)?
    print(RetryCount.try_from(9).is_err())?
    Ok(())
Retries: 3
Retries: 5
True

Both ends are included. The literal 3 is checked at compile time because the parameter expects RetryCount. A computed integer goes through try_from, which returns RangeError.OutOfRange when it does not fit. .get() extracts the integer for calculations. A literal 9 passed directly to describe would be a compile error.

Abstract over the bounds

The arguments 0 and 8 are compile-time values, not types. A generic function can keep them as const parameters:

def interval[T: IntType, const lo: T, const hi: T](value: &Bounded[T, lo, hi]) -> (T, T):
    (lo, hi)

def main() -> Result[(), IoError]:
    retries: Bounded[i64, 0, 8] = 3
    percentage: Bounded[u8, 0, 100] = 75
    print(interval(&retries), interval(&percentage))?
    Ok(())
(0, 8) (0, 100)

The argument’s type supplies T, lo and hi; no runtime integer becomes a specialization argument. const lo: T uses the earlier T: IntType, so a u8 interval has u8 bounds. The values are available inside the body without storing them alongside each integer.

User classes, enums and methods can also declare const parameters. Use them when the value belongs in type identity. A setting read from a file normally stays a runtime value: it cannot select a new type at runtime. The value-parameter reference gives argument restrictions, and bounded integers describes conversion and value behavior.

Calculate with quantities

Use Quantity[T, D] for a physical value whose dimension should be checked in arithmetic. T is its numeric storage type and D is a compile-time dimension. The std.units module provides the SI dimensions, units and constants:

import std.units as u
from std.units import Velocity, m, km, s, minute

def double[D: Dimension](value: Quantity[f64, D]) -> Quantity[f64, D]:
    value + value

def main() -> Result[(), IoError]:
    distance = 1500.0 * m
    elapsed = 2.0 * minute
    speed: Quantity[f64, Velocity] = distance / elapsed
    print(double(distance).to(km), "km")?
    print(speed, speed.to(km / u.hour), "km/h")?
    Ok(())
3.0 km
12.5 m/s 45.0 km/h

u.Length and u.Time are distinct base dimensions. Velocity is an alias for Length / Time. Dividing the distance by the elapsed time computes this dimension, so the speed annotation checks successfully. Adding a time to a length would be a compile error. When a dimension error involves Velocity, the diagnostic includes a note naming Velocity and its normal form, std.units.Length * Pow[std.units.Time, -1]. This connects the normalized dimension to the name used in your program.

Units are constants. km is a length whose stored value is 1000.0, and 2.0 * minute stores 120.0 seconds. Multiplying a number by a unit constructs a quantity, and dividing one unit by another, as in km / u.hour, gives another unit.

double infers its dimension parameter from its argument. The same function works for a length, a time or a velocity. Dimension arguments add no runtime fields: a quantity uses its numeric storage type’s native representation. Construction and get add no numeric conversion, and float to performs the same division as plain numbers. A dimension itself has no values: length: u.Length is invalid.

.to(km) divides by the unit’s stored value and returns a plain f64. .get() reads the coherent value directly. Both methods remove the dimension from the result, so keep quantities until the boundary where you need a number. Printing a quantity includes its coherent-unit symbol. The explicit "km" label accompanies the plain number returned by .to(km); printing double(distance) directly would show 3000.0 m.

Use an f-string to choose numeric precision: f"{speed:.2f}"? produces "12.50 m/s". As with other f-strings, include AllocError in the function’s error type when propagating its allocation failure.

Scalars only scale quantities. Write 0.0 * m for a zero length; `distance

  • 0.0anddistance > 0.0are errors. Dividing a length by a length producesQuantity[f64, One]; use .get()` to obtain its dimensionless number.

std.units also has dimension-aware functions. u.sqrt of an area returns a length, u.sum adds a list of quantities, and u.sin takes an angle such as 30.0 * u.degree. Temperatures in Celsius or Fahrenheit convert at the boundary with u.from_celsius and u.to_celsius, because their zero is offset.

Declare your own dimensions

Declare a base dimension for a quantity outside SI, or to keep two kinds of value apart that SI would give the same dimension. Give each base its coherent-unit symbol:

dimension Pixel("px")
dimension Point("pt")
dimension PixelDensity = Pixel / Point

const px: Quantity[f64, Pixel] = Quantity[f64, Pixel](1.0)
const pt: Quantity[f64, Point] = Quantity[f64, Point](1.0)
const retina: Quantity[f64, PixelDensity] = 2.0 * px / pt

def main() -> Result[(), IoError]:
    width = 320.0 * pt
    print(width * retina)?
    Ok(())
640.0 px

Unit constants can be shared across functions or exported from a module with pub const, as std.units does. Their initializers are evaluated at compile time, including unit scaling and arithmetic between constants. The constructor takes the value in the coherent unit.

Dimension expressions also support products, integer powers and exact roots. Root[Pow[Length, 2], 2] is Length; fractional dimensions are rejected. In a generic function’s parameter types, use concrete dimensions or bare parameters. Expressions using parameters belong in return types or bodies.

See std.units for its units, constants and functions, dimensions for algebra and visibility, and quantities for integer/domain storage, conversions, checked arithmetic and current limits.

Build recursive data

Chains and trees contain values of their own type. A directly embedded value would have no finite size, so put the recursive field in a Box or collection. A box owns one value on the heap.

This chain ends with Empty. Each Link owns a value and a box holding the rest of the chain:

enum Chain[T]:
    Empty
    Link(T, Box[Chain[T]])

def total(chain: Chain[i64]) -> Result[i64, OverflowError[i64]]:
    mut remaining = chain
    mut answer = 0
    while True:
        match remaining:
            case Chain.Empty:
                break
            case Chain.Link(value, rest):
                answer = answer + value
                remaining = rest
    Ok(answer)

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    mut chain = Chain[i64].Empty
    for value in range(1, 4):
        chain = Chain[i64].Link(value, Box(chain)?)
    print(total(chain)?)?
    Ok(())
6

Box(value) moves the value onto the heap and returns an allocation result. Constructing the Link itself allocates nothing.

When a value of type T is required, a Box[T] moves its content out and frees the box, without allocating. Here, remaining = rest expects Chain[i64], so it unboxes rest. The conversion also works for arguments, returns, annotated bindings, constructor fields, and collection elements. total consumes its chain; the caller cannot use it afterward.

A boxed class’s fields and methods are reached as b.field and b.method(), and where a function expects &T, you can pass a &Box[T]. Where no type is required, a box stays a box: other = b moves the box itself. Operators and conditions do not convert either. Write *b there to move the content out, or &*b and &mut *b to borrow it.

Without the box, the compiler rejects the declaration:

enum Chain:
    Empty
    Link(i64, Chain)

def main() -> ():
    pass
`Chain` contains itself and would have infinite size; store the recursive field in a `Box`, list, set, or dict

Use values through a box

Call collection and string methods through a box, including nested boxes. Methods borrow its content; mutation requires a mut binding or an exclusive reference:

def main() -> Result[(), AllocError | IoError]:
    mut items = Box(Box([1, 2]?)?)?
    items.append(3)?
    match items.get(2):
        case Some(value):
            print(value)?
        case Nothing:
            print("missing item")?
    text = Box("  hello  ")?
    print(text.strip()?)?
    print(text.startswith("  "))?
    Ok(())
3
hello
True

Read-only methods also work on temporary boxes. Bind a box with mut before calling a mutating method.

The recursive-data lesson explains when a box transfers its contents and when it keeps ownership of them.

Store recursive children in a list

A list of children provides the indirection a recursive class needs; each child needs no additional box. Save this module as forest.plenty:

pub class Node[T]:
    pub value: T
    pub children: list[Node[T]]
import forest

def main() -> Result[(), AllocError | IoError]:
    mut root = forest.Node[i64](1, [forest.Node[i64](2, []?)]?)
    root.children[0].value = 9
    print(root.value)?
    print(root.children[0].value)?
    Ok(())
1
9

An empty children list terminates the tree. For a single optional child, use a field such as next: Option[Box[Node]] and terminate it with Nothing.

Remove a child and clean up a recursive structure

To take an owned child out of a mutable class, leave a replacement behind. replace(node.next, Nothing) returns the old tail without cloning or allocating:

class Link:
    value: i64
    next: Option[Box[Link]]

def main() -> Result[(), AllocError | IoError]:
    mut chain = Some(Box(Link(1, Some(Box(Link(2, Nothing))?)))?)
    while True:
        match chain:
            case Some(link):
                mut node = link
                print(node.value)?
                chain = replace(node.next, Nothing)
            case Nothing:
                break
    Ok(())
1
2

Replacing node.next with Nothing leaves that field empty when node drops. Use another tail as the replacement to relink nodes. replace requires a mutable field and evaluates the replacement before any destination indices.

Automatic cleanup can drop a deep boxed chain without growing the native stack. A recursive traversal you write can still overflow it; use a loop for unbounded depth.

To inspect a structure while keeping it, traverse it through a borrow. Moving individual owned fields out without replacing them remains unsupported. Automatic copy, equality, print, and str.repr also reject recursive data, including containers holding it. Print selected scalar fields as above, or write an explicit traversal.

Optional: traverse recursive data

Traverse a chain without consuming it by matching a borrow. Some(next) below binds next as &Box[links.Node[i64]], which can be passed where a reference to the node is expected.

pub class Node[T]:
    pub value: T
    pub next: Option[Box[Node[T]]]
import links

def total(node: &links.Node[i64]) -> Result[i64, OverflowError[i64]]:
    match &node.next:
        case Some(next):
            Ok(node.value + total(next)?)
        case Nothing:
            Ok(node.value)

def increment(node: &mut links.Node[i64]) -> Result[(), OverflowError[i64]]:
    node.value = node.value + 1
    match &mut node.next:
        case Some(next):
            return increment(next)
        case Nothing:
            pass
    Ok(())

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    mut chain = links.Node[i64](3, Some(Box(links.Node[i64](6, Nothing))?))
    print(total(&chain)?)?
    increment(&mut chain)?
    print(total(&chain)?)?
    print(chain.value)?
    Ok(())
9
11
4

match &mut value binds mutable payload references, allowing increment to update each node while its caller keeps the chain.

increment ends with a tail call through a reference borrowed from its parameter, so its stack does not grow. total adds after its recursive call returns, so it uses native stack space for every node.

A loop avoids that. Declare a reference binding with mut and assign it the next node. Such a binding may only be assigned a reference borrowed from itself, which next is: it comes from match &node.next.

class Node:
    value: i64
    next: Option[Box[Node]]

def total(head: &Node) -> Result[i64, OverflowError[i64]]:
    mut sum = 0
    mut node = head
    while True:
        sum = sum + node.value
        match &node.next:
            case Some(next):
                node = next
            case Nothing:
                break
    Ok(sum)

def increment(head: &mut Node) -> Result[(), OverflowError[i64]]:
    mut node = head
    while True:
        node.value = node.value + 1
        match &mut node.next:
            case Some(next):
                node = next
            case Nothing:
                break
    Ok(())

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    mut chain = Node(3, Some(Box(Node(6, Nothing))?))
    increment(&mut chain)?
    print(total(&chain)?)?
    Ok(())
11

The chain stays borrowed while node is used. The loop’s stack usage is independent of chain length.

Walk recursive data with a reference

A reference binding declared with mut can advance through a recursive structure by borrowing each next step from itself:

class Folder:
    name: str
    inside: list[Folder]

def innermost(top: &Folder) -> str:
    mut folder = top
    while len(folder.inside) > 0:
        folder = &folder.inside[0]
    folder.name

def main() -> Result[(), AllocError | IoError]:
    notes = Folder("notes", []?)
    work = Folder("work", [notes]?)
    home = Folder("home", [work]?)
    print(innermost(&home))?
    Ok(())
notes

folder starts at top. Each &folder.inside[0] is borrowed from folder, so the assignment is allowed. home stays borrowed until folder’s last use.

The binding can also be pointed at another place within the value it first borrowed, without going through itself. This retargets it: the old loan ends at the assignment. A search can remember its best element this way, without keeping an index:

def longest(words: &list[str]) -> &str:
    mut best = &words[0]
    for i in range(1, len(words)):
        if words[i].byte_len() > best.byte_len():
            best = &words[i]
    best

def main() -> Result[(), AllocError | IoError]:
    words = ["fig", "banana", "kiwi"]?
    print(longest(&words))?
    Ok(())
banana

best first borrows from words, so &words[i] is a valid new target. See reference binding reassignment for the full rule. Pointing it at a different value is rejected, even one of the same type:

def main() -> Result[(), AllocError | IoError]:
    first = [1, 2]?
    second = [3]?
    mut view = &first
    view = &second
    print(view)?
    Ok(())
reference binding `view` can be reassigned only to a reference into the owner it first borrowed

Matching borrowed values also leaves the owner in place while inspecting its contents.

The same descendant-cursor pattern can lend exclusive access while updating each node. See the complete mutable chain walk. For branching traversal, a stack of references is currently unsupported; an arena with a worklist of IDs is an alternative.

Preserve a chain when insertion fails

A borrowed collection should usually keep its existing contents when insertion fails. Allocate the new node before detaching the old head:

class Node:
    value: i64
    next: Option[Box[Node]]

class Chain:
    head: Option[Box[Node]]

def push(chain: &mut Chain, value: i64) -> Result[(), AllocError]:
    mut node = Box(Node(value, Nothing))?
    node.next = replace(chain.head, Nothing)
    chain.head = Some(node)
    Ok(())

def main() -> Result[(), AllocError | IoError]:
    mut chain = Chain(Nothing)
    push(&mut chain, 10)?
    push(&mut chain, 20)?
    match &chain.head:
        case Some(head):
            print(head.value)?
        case Nothing:
            print("empty")?
    Ok(())
20

The only allocation is Box(...). After it succeeds, replace transfers the old head and leaves Nothing behind; linking the nodes cannot fail or allocate. If allocation fails, the existing chain has not changed.

Putting replace(chain.head, Nothing) inside the Box argument would detach the chain first. Failed box allocation destroys its input, including that old chain. Memory safety does not imply that a failed operation preserves data. For a generic payload, also decide whether failure should destroy the incoming value or return it to the caller.

The earlier consuming construction example abandons the whole local computation on failure. This example has a stronger contract because the caller retains the collection. See the complete linked-stack operations.

Represent parent links with node IDs

When children need to identify their parent, one list can own all the nodes. Links then contain indices, not references or owning boxes. This representation is often called an arena. Moving the list’s storage during growth does not change an index.

This append-only example stores children as IDs too:

class Node:
    name: str
    parent: Option[i64]
    children: list[i64]

def main() -> Result[(), AllocError | IoError]:
    mut nodes: list[Node] = []?
    nodes.append(Node("root", Nothing, []?))?
    nodes.append(Node("leaf", Some(0), []?))?
    nodes[0].children.append(1)?

    child = nodes[0].children[0]
    match nodes[child].parent:
        case Some(parent):
            print(nodes[parent].name)?
        case Nothing:
            print("no parent")?

    mut pending = [0]?
    while True:
        match pending.pop():
            case Some(id):
                print(nodes[id].name)?
                for child_id in &nodes[id].children:
                    pending.append(*child_id)?
            case Nothing:
                break
    Ok(())
root
root
leaf

The explicit worklist avoids recursive function calls. It owns integers, so it does not need to store references, which Plenty currently rejects. Its growth is fallible and its memory use depends on the tree’s shape. A graph with cycles needs visited-node tracking as well.

Keep this list append-only while IDs are live. Removing an element from the middle, sorting, or swapping elements changes which node an index denotes. Removing a graph edge does not destroy its target: the arena owns that node until explicitly removed or the whole arena is dropped. If IDs come from outside this controlled example, check their range before indexing.

For reusable slots, tombstones preserve indices and generation checks can reject stale IDs after reuse. Those invariants belong in a reusable abstraction; raw integers do not enforce them. The arena proposal contains a tested prototype, not an available std.arena module.

This example exits and drops the arena if construction fails. An API that mutates a caller’s existing graph must reserve needed storage before changing parent/child links, or explicitly roll back partial changes. See failure-preserving insertion.

Functions as values

Pass a function when its caller should choose an operation, or store one for later. The first lessons need only the core path; generic signatures and records also use type parameters and constraints.

Pass functions as values

Use a function’s name without parentheses to pass it as a value. A Callable annotation specifies its parameter types and return type. The value stores only a code address. To accept a callback with captured state too, use a generic callback parameter.

def double(value: i64) -> Result[i64, OverflowError[i64]]:
    Ok(value * 2)

def apply(operation: Callable[[i64], Result[i64, OverflowError[i64]]], value: i64) -> Result[i64, OverflowError[i64]]:
    operation(value)

def main() -> Result[(), IoError | OverflowError[i64]]:
    operation = double
    print(operation(6)?)?
    print(apply(double, 9)?)?
    Ok(())
12
18

The callback’s own return type includes OverflowError[i64]; apply forwards that public Result and main propagates it with ?. A caller does not change a callback’s arithmetic contract. See checked arithmetic.

Function values can also be returned and reassigned through mut bindings. Copying a function value copies its code address; no allocation is needed.

You can call a value selected from a collection directly:

def increment(value: i64) -> Result[i64, OverflowError[i64]]:
    Ok(value + 1)

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    handlers = [increment]?
    index = 0
    print(handlers[index](41)?)?
    Ok(())
42

A callable can also borrow an argument and return a fallible unit result:

def increment(value: &mut i64) -> Result[(), OverflowError[i64]]:
    *value = *value + 1
    Ok(())

def main() -> Result[(), IoError | OverflowError[i64]]:
    update: Callable[[&mut i64], Result[(), OverflowError[i64]]] = increment
    mut value = 41
    update(&mut value)?
    print(value)?
    Ok(())
42

Class constructors and payload-bearing enum variants can be passed the same way. Use the constructor directly when a callback only wraps a value:

class Reading:
    value: i64

def main() -> Result[(), AllocError | IoError]:
    readings: list[Reading] = iter(range(3)).map(Reading).collect()?
    wrap: Callable[[i64], Option[i64]] = Some
    print(readings, wrap(7))?
    Ok(())
[Reading(value=0), Reading(value=1), Reading(value=2)] Option[i64].Some(7)

For a generic constructor, an annotation supplies the instance, as for wrap above. Without that context, name it explicitly: Option[i64].Some. Constructor values take every positional argument and retain any initializer failure in their result. See constructor rules.

Anonymous functions

Use def without a name to create a function value. Parameters and the return value need explicit types. Inside brackets, such as a call’s arguments, the body is one expression and its value is returned:

def apply(f: Callable[[i64], Result[i64, OverflowError[i64]]], value: i64) -> Result[i64, OverflowError[i64]]:
    f(value)

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    print(apply(def(x: i64) -> Result[i64, OverflowError[i64]]: Ok(x * 2), 4)?)?
    print(apply(
        def(x: i64) -> Result[i64, OverflowError[i64]]: Ok(x + 1 if x > 0 else 0),
        4,
    )?)?
    Ok(())
8
5

The body ends at the comma or closing bracket that ends the argument. Each function has type Callable[[i64], Result[i64, OverflowError[i64]]] and can be stored, returned, or passed to a callback parameter.

Outside brackets, the body is an indented block with the same rules as a named function:

def main() -> Result[(), IoError | OverflowError[i64]]:
    transform = def(value: i64) -> Result[i64, OverflowError[i64]]:
        adjusted = value * 2
        if adjusted > 10:
            return Ok(10)
        Ok(adjusted + 1)
    print(transform(3)?)?
    print(transform(8)?)?
    Ok(())
7
10

The position decides the form. A function with statements cannot be written inside brackets; give it an indented body and a name first, as transform has:

def apply(f: Callable[[i64], Result[i64, OverflowError[i64]]], value: i64) -> Result[i64, OverflowError[i64]]:
    f(value)
def main() -> ():
    apply(def(x: i64) -> Result[i64, OverflowError[i64]]: return Ok(x * 2), 4)
inside brackets an anonymous function body is one expression, and its value is returned; remove `return`

Parentheses are brackets, so a one-expression function can still be bound: double = (def(x: i64) -> Result[i64, OverflowError[i64]]: Ok(x * 2)). Python’s lambda is not available; the compiler names the def spelling instead.

Capture data and private state

When an operation needs more than its call arguments, let its function value own that extra data. Use an explicit list to capture surrounding bindings:

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    offset = 4
    values = [10, 20]?
    adjust = def [offset, values](index: i64) -> Result[i64, OverflowError[i64]]:
        Ok(values[index] + offset)
    print(adjust(0)?)?
    print(adjust(1)?)?
    Ok(())
14
24

values moves into the closure; the scalar offset copies. Each call borrows the captures, so the list remains available for subsequent calls and drops with the closure. Its environment is stored inline and has a different type from the code-only Callable. A reusable closure cannot move captures out; borrow or explicitly copy them when an independent owner is needed.

Use mut in the capture list for private state. A stateful closure also needs a mutable binding to call:

def main() -> Result[(), IoError | OverflowError[i64]]:
    count = 0
    mut next = def [mut count]() -> Result[i64, OverflowError[i64]]:
        count = count + 1
        Ok(count)
    print(next()?)?
    print(next()?)?
    print(count)?
    Ok(())
1
2
0

The captured scalar changes independently of the original count.

Accept functions and closures through one API

Constrain a type parameter by its call signature to accept named functions and captured closures through one parameter. Borrow the callback when its caller should keep it:

def apply[T, E, F: Callable[[T], Result[T, E]]](f: &F, value: T) -> Result[T, E]:
    f(value)

def double(value: i64) -> Result[i64, OverflowError[i64]]:
    Ok(value * 2)

def main() -> Result[(), IoError | OverflowError[i64] | OverflowError[u8]]:
    offset = 2u8
    add = def [offset](value: u8) -> Result[u8, OverflowError[u8]]:
        Ok(offset + value)
    print(apply(&double, 3)?)?
    print(apply(&add, 3u8)?)?
    Ok(())
6
5

A short callback can be written in the call. An anonymous function needs parentheses to be the operand of &:

def apply[T, E, F: Callable[[T], Result[T, E]]](f: &F, value: T) -> Result[T, E]:
    f(value)

def main() -> Result[(), IoError | OverflowError[i64]]:
    offset = 2
    print(apply(&(def [offset](value: i64) -> Result[i64, OverflowError[i64]]: Ok(offset + value)), 3)?)?
    print(apply(&(def(value: i64) -> Result[i64, OverflowError[i64]]: Ok(value * value)), 3)?)?
    Ok(())
5
9

F preserves the callback’s concrete environment type. Both arguments agree on T, the input and successful output type; E preserves the callback’s error type. The caller handles the returned Result normally. Use &mut F for a callback that changes its captured state. Borrowed captures stay protected throughout the call.

Borrow a named function directly with &double, or with &mut double for a &mut F parameter; a separate binding is optional. A function has no state, so each &mut borrow takes its own copy of the code address.

A concrete parameter such as f: Callable[[i64], i64] holds only a function’s code address. The constraint F: Callable[[i64], i64] instead checks whether the actual type can be called with that signature; its captured state remains inline. Adding & to the concrete Callable type does not make it accept closures. Closure[...] describes a concrete environment, chiefly when returning one from a factory.

Passing captured state to a code-only parameter explains the mismatch and suggests the generic form:

def apply(f: Callable[[i64], Result[i64, OverflowError[i64]]], value: i64) -> Result[i64, OverflowError[i64]]:
    f(value)
def main() -> ():
    offset = 3
    add = def [offset](value: i64) -> Result[i64, OverflowError[i64]]:
        Ok(value + offset)
    apply(add, 4)
for a generic callback API, use `F: Callable[[i64], Result[i64, OverflowError[i64]]]` with `callback: &F`

Let the callback determine a separate result type

The first helpers require the input and successful output to have the same type T. Give the complete result its own parameter U to allow other transformations, including infallible callbacks. The callback’s concrete signature determines U, including when it does not appear in another argument. This example uses both forms of callback through that shared interface:

def apply[T, U, F: Callable[[T], U]](callback: &F, value: T) -> U:
    callback(value)

def double(value: i64) -> Result[i64, OverflowError[i64]]:
    Ok(value * 2)

def main() -> Result[(), IoError | OverflowError[i64]]:
    offset = 7
    add = def [offset](value: i64) -> Result[i64, OverflowError[i64]]:
        Ok(offset + value)
    print(apply(&double, 10)?)?
    print(apply(&add, 10)?)?
    Ok(())
20
17

Borrow in a closure

A local callback may need to inspect data that its caller will use again. Use & in a capture list to keep ownership outside the closure:

def main() -> Result[(), AllocError | IoError]:
    mut values = [4]?
    read = def [&values]() -> i64:
        values[0]
    print(read())?
    moved = read
    print(moved())?
    values.append(5)?
    print(values)?
    Ok(())
4
4
[4, 5]

The borrow follows the closure into moved and ends after its last call. Changing the list before a later call would conflict with that borrow. Borrowing closures cannot be returned from their creating function or stored in collections or sum types. Pass them to other functions through reference parameters:

def invoke(f: &Closure[[], i64]) -> i64:
    f()

def main() -> Result[(), IoError]:
    value = 7
    read = def [&value]() -> i64:
        value
    print(invoke(read))?
    Ok(())
7

The &Closure parameter borrows read implicitly; invoke(&read) is also valid. The capture remains borrowed throughout invoke, including while its other arguments are being evaluated. A closure can also explicitly borrow another closure; the same rule protects all of their captured owners.

Use &mut to modify an existing binding rather than moving it into private state:

def main() -> Result[(), IoError | OverflowError[i64]]:
    mut count = 0
    mut increment = def [&mut count]() -> Result[(), OverflowError[i64]]:
        count = count + 1
        Ok(())
    increment()?
    increment()?
    print(count)?
    Ok(())
2

Both the source and closure bindings need mut. The original count becomes available again after the last closure call. Reading it before another call would conflict with the exclusive capture.

Return a closure

Return an owned closure to keep its captured data after its creating function returns. Closure describes the returned call signature:

def counter(start: i64) -> Closure[[], Result[i64, OverflowError[i64]]]:
    def [mut start]() -> Result[i64, OverflowError[i64]]:
        start = start + 1
        Ok(start)

def apply(f: &mut Closure[[], Result[i64, OverflowError[i64]]]) -> Result[i64, OverflowError[i64]]:
    f()

def main() -> Result[(), IoError | OverflowError[i64]]:
    mut first = counter(10)
    mut second = counter(30)
    print(apply(&mut first)?)?
    print(second()?)?
    print(first()?)?
    Ok(())
11
31
12

apply needs an exclusive borrow because calling the counter changes its state. For a read-only closure, use a shared parameter such as &Closure[[i64], Result[i64, OverflowError[i64]]].

Different anonymous expressions have different concrete environment types, even with identical signatures. A factory must return one concrete type on every path. Borrowing closures cannot escape their creating function. To return different owned shared-call environments, explicitly allocate a BoxedCallable.

An owned closure can capture another closure or be wrapped in Option or Result:

def make(offset: i64) -> Option[Closure[[i64], Result[i64, OverflowError[i64]]]]:
    inner = def [offset](value: i64) -> Result[i64, OverflowError[i64]]:
        Ok(offset + value)
    outer = def [inner](value: i64) -> Result[i64, OverflowError[i64]]:
        Ok(inner(value)? * 2)
    Some(outer)

def main() -> Result[(), IoError | OverflowError[i64]]:
    match make(3):
        case Some(calculate):
            print(calculate(4)?)?
        case Nothing:
            print("no callback")?
    Ok(())
14

Captured resources drop in reverse capture order, including nested environments.

You can call a returned closure immediately, for example counter(10)(). Its environment remains alive through the complete expression and is then dropped. ? in a later argument also cleans up that temporary. Match an optional closure before calling it, as in the example above.

Generic function values

A higher-order function can use a type parameter inside its callable signature. The compiler infers the type from the function value and the other arguments:

def increment(value: u8) -> Result[u8, OverflowError[u8]]:
    Ok(value + 1)

def apply[T, U](operation: Callable[[T], U], value: T) -> U:
    operation(value)

def main() -> Result[(), IoError | OverflowError[u8]]:
    print(apply(increment, 41u8)?)?
    Ok(())
42

Here T is u8. Passing 41i64 would conflict with increment’s signature.

To choose a generic function’s own type arguments, use brackets without calling it:

def identity[T](value: T) -> T:
    value

def main() -> Result[(), IoError]:
    keep = identity[u8]
    print(keep(42))?
    Ok(())
42

identity[u8] is a function value; identity[u8](42) calls that specialization.

A concrete callable annotation can also supply those type arguments:

def identity[T](value: T) -> T:
    value

def main() -> Result[(), IoError]:
    keep: Callable[[u8], u8] = identity
    print(keep(42))?
    Ok(())
42

This also works when a parameter or return type supplies the concrete signature. Without that context, use explicit brackets such as identity[u8].

Transform a cell with a generic method

Give a method its own type parameters to let a callback transform Cell[T] into Cell[U]:

class Cell[T]:
    value: T

    def map[U, F: Callable[[T], U]](self, transform: &F) -> Cell[U]:
        Cell(transform(self.value))

def double(value: u8) -> u16:
    # u8 is at most 255, so the doubled value is at most 510 in u16.
    unchecked:
        u16(value) * 2

def main() -> Result[(), IoError]:
    original = Cell(7u8)
    transform = double
    changed = original.map(&transform)
    print(changed.value)?
    print(original.value)?
    Ok(())
14
7

map borrows the original cell, so passing its field to the callback requires a copyable T. Here, double determines the result type U = u16.

Store stateful callbacks

Pair explicit state with a named function when a registry needs different callback bodies with the same state type and call signature:

class Handler[State]:
    state: State
    callback: Callable[[&mut State, i64], Result[i64, OverflowError[i64]]]
    def call(self: &mut Handler[State], value: i64) -> Result[i64, OverflowError[i64]]:
        self.callback(&mut self.state, value)

def accumulate(total: &mut i64, value: i64) -> Result[i64, OverflowError[i64]]:
    *total = *total + value
    Ok(*total)

def subtract(total: &mut i64, value: i64) -> Result[i64, OverflowError[i64]]:
    *total = *total - value
    Ok(*total)

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    mut handlers = [Handler(10, accumulate), Handler(100, subtract)]?
    for handler in &mut handlers:
        print(handler.call(3)?)?
    print(handlers[0].call(2)?)?
    Ok(())
13
97
15

Each record owns its state; call borrows it exclusively. Constructing the records needs no allocation, but storing them in a list does.

For different state shapes, use an enum and handle its variants in the callback. Owned concrete closures can also be stored when their producer types agree.

See examples/callback_registry.plenty for a registry example.

Store owned closures

A generic record can own a callback’s captured state. You do not need to name the closure’s concrete type: the constructor infers it.

class Handler[F: Callable[[i64], Result[i64, OverflowError[i64]]]]:
    callback: F
    def call(self: &mut Handler[F], amount: i64) -> Result[i64, OverflowError[i64]]:
        self.callback(amount)

def counter(total: i64) -> Closure[[i64], Result[i64, OverflowError[i64]]]:
    def [mut total](amount: i64) -> Result[i64, OverflowError[i64]]:
        total = total + amount
        Ok(total)

def main() -> Result[(), IoError | OverflowError[i64]]:
    mut first = Handler(counter(10))
    mut second = Handler(counter(100))
    print(first.call(3)?)?
    print(second.call(7)?)?
    print(first.callback(2)?)?
    Ok(())
13
107
15

The Callable constraint checks the signature without erasing the concrete environment type. Each record stores its environment inline.

Keep captured state owned when storing a callback. A closure borrowing a local cannot enter a record. Moving the record transfers ownership; dropping it cleans up captured resources. Closures do not support copy or equality.

Reduce a borrowed collection with a pipeline

iter pipelines also accept functions, so they can transform items without making an intermediate collection. Borrow the source when the caller should keep it.

class Payment:
    name: str
    cents: i64

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    payments = [Payment("bob", 120), Payment("ada", 300), Payment("bob", 80)]?
    selected = iter(&payments).filter(def(p: &Payment) -> bool: p.name == "bob")
    total = selected.map(def(p: &Payment) -> i64: p.cents).sum()?
    largest = iter(&payments).map(def(p: &Payment) -> i64: p.cents).max()
    refund = iter(&payments).any(def(p: &Payment) -> bool: p.cents < 0)
    print(total, largest, refund, len(payments))?
    Ok(())
200 Option[i64].Some(300) False 3

selected holds a loan of payments. The filter and map run when sum consumes the pipeline, in source order. A closure written directly in the chain, like the filter and map here, is compiled into that loop rather than called. The payments remain available afterwards. Trying to consume selected again is an ownership error.

sum and max work as in Chain iteration steps with iter; here they reduce mapped values, since a Payment is neither a number nor a str. Likewise, any and all take a predicate when the items are not bool. Use try_fold for a fallible accumulation, or fold for an infallible one.

A method returning a borrowed list can be used directly as the source. Its owner remains borrowed while the pipeline is in use:

class Readings:
    values: list[i64]

    def samples(self) -> &list[i64]:
        &self.values

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    readings = Readings([10, 20, 30]?)
    product = iter(readings.samples()).try_fold(1, def(acc: i64, n: &i64) -> Result[i64, OverflowError[i64]]: Ok(acc * *n))?
    print(product)?
    Ok(())
6000

try_fold starts with its first argument and passes the accumulator and each item to the callback. An Ok supplies the next accumulator; the first Err stops iteration and is returned to the caller.

Predicates borrow each item; map receives whatever the source yields. A borrowed list yields references, so the map above takes &Payment. A range yields integers by value:

def main() -> Result[(), IoError | OverflowError[i64]]:
    for square in iter(range(10)).skip(2).take(3).map(def(n: i64) -> Result[i64, OverflowError[i64]]: Ok(n * n)):
        print(square?)?
    Ok(())
4
9
16

Use try_collect when a transformation can fail and the desired result is an owned collection. The annotation chooses the collection, and ? preserves the first parsing or allocation error:

def main() -> Result[(), AllocError | ParseError | IoError]:
    lines = ["10", "20", "30"]?
    values: list[i64] = iter(&lines).map(def(line: &str) -> Result[i64, ParseError]: i64.parse(line)).try_collect()?
    print(values, len(lines))?
    Ok(())
[10, 20, 30] 3

For an infallible transformation, use collect() with the same annotation. try_fold and try_for_each propagate errors returned by their callbacks; they do not automatically unwrap Result items.

After enumerate or zip, callbacks take two item parameters. Predicates borrow both components; mapping receives the index by value and the borrowed item:

def main() -> Result[(), AllocError | IoError]:
    names = ["Ada", "Bob", "Cal"]?
    indexed = iter(&names).enumerate(1)
    selected = indexed.filter(def(i: &i64, name: &str) -> bool: *i != 2)
    labels: dict[i64, str] = selected.map(def(i: i64, name: &str) -> Result[(i64, str), AllocError]: Ok((i, copy(name)?))).try_collect()?
    print(labels[1], labels[3], len(names))?
    Ok(())
Ada Cal 3

Copying a name can fail, so try_collect propagates that error as well as dictionary allocation errors. The successful owned pairs become dictionary entries. Repeated keys keep the last value. A pair fold takes the accumulator followed by both item components; a tuple accumulator stays a single parameter. Zip stops at the shorter input, pulling the left first, so it may consume one extra left item.

Use chain to continue with a second source of the same item type, and flat_map when each item expands into a source of its own:

class Batch:
    first: i64
    stop: i64

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    morning = [3, 1]?
    evening = [4]?
    total = iter(&morning).chain(&evening).sum()?
    print(total)?
    batches = [Batch(100, 102), Batch(200, 201)]?
    for ticket in iter(&batches).flat_map(def(b: &Batch) -> range[i64]: range(b.first, b.stop)):
        print(ticket)?
    Ok(())
8
100
101
200

Both lists stay borrowed until the sum ends. The flat_map callback runs once per batch, only after the previous batch’s tickets are used up, and returns a range, so nothing is allocated. Batch stores start and stop endpoints directly; constructing the range needs no endpoint arithmetic. A callback may return any owned iterable, such as a list, range or generator; returning a list allocates one list per item.

Lend a pipeline whose callbacks borrow

A pipeline lent to a function can use a closure that borrows, when the closure is written inside the adapter call:

def first(p: &mut Iter[i64]) -> Option[i64]:
    next(p)

def main() -> Result[(), IoError]:
    limit = 3
    mut above = iter(range(9)).filter(def [&limit](x: &i64) -> bool: *x > limit)
    print(first(&mut above))?
    print(above.count())?
    Ok(())
Option[i64].Some(4)
4

Storing the same closure in a variable first and passing the variable makes the pipeline impossible to lend:

def first(p: &mut Iter[i64]) -> Option[i64]:
    next(p)

def main() -> Result[(), IoError]:
    limit = 3
    keep = def [&limit](x: &i64) -> bool:
        *x > limit
    mut above = iter(range(9)).filter(keep)
    print(first(&mut above))?
    Ok(())
it holds closure `keep`, which is stored in a variable and borrows its captures; write the closure literal inside the adapter call instead

A closure written in the chain is compiled into the pipeline, so its borrow of limit becomes one of the pipeline’s own loans, which the caller keeps while first runs. A stored closure is a value holding that reference inside it, and a function receiving a lent pipeline cannot take such a value. Without the lend, both forms work.

Keep a plain loop for several statements or error propagation, and use a comprehension when building a list is the clearest form. Pipelines are local: Transform a stream of values applies them to generators and shows how to return a stream from a function. See the pipeline reference for terminals and the current limits.

Transform a stream of values

A generator has the same pipeline methods as iter(...), so a stream of values can be filtered, mapped and reduced without an intermediate list:

def numbers(stop: i64) -> Generator[i64]:
    for n in range(stop):
        yield n

def main() -> Result[(), IoError | OverflowError[i64]]:
    two: NonZero[i64] = 2
    evens = numbers(10).filter(def [two](n: &i64) -> bool: *n % two == 0)
    for square in evens.map(def(n: i64) -> Result[i64, OverflowError[i64]]: Ok(n * n)):
        n = square?
        print(n)?
        if n > 3:
            break
    Ok(())
0
4

The generator body and callbacks run only when the loop asks for a value, and break drops the rest of the chain: numbers never makes 3. The adapters are fused into the loop, so they add no allocation or generator frame of their own.

take limits how many items are requested from a generator. Callbacks can capture values, as with any closure:

def naturals() -> Generator[i64]:
    for n in range(i64.MAX):
        yield n

def main() -> Result[(), IoError | OverflowError[i64]]:
    offset = 100
    for n in naturals().map(def [offset](n: i64) -> Result[i64, OverflowError[i64]]: Ok(n + offset)).take(3):
        print(n?)?
    Ok(())
100
101
102

naturals is bounded by the representable range rather than overflowing a manual counter. take(3) requests only its first three values. The map returns ordinary public Result items, which the loop propagates before printing.

Return a stream

A pipeline stays in the function that builds it. When a chain needs to be returned or passed to another function, end it with .generator(). This explicitly moves the source and captures into one resumable generator, with the adapters still fused inside it:

def squares(stop: i64) -> Generator[Result[i64, OverflowError[i64]]]:
    iter(range(stop)).map(def(n: i64) -> Result[i64, OverflowError[i64]]: Ok(n * n)).generator()

def total(values: Generator[Result[i64, OverflowError[i64]]]) -> Result[i64, OverflowError[i64]]:
    values.try_fold(0, def(acc: i64, n: Result[i64, OverflowError[i64]]) -> Result[i64, OverflowError[i64]]: Ok(acc + n?))

def main() -> Result[(), IoError | OverflowError[i64]]:
    print(total(squares(4))?)?
    mut values = squares(3)
    first = next(values).unwrap()?
    second = next(values).unwrap()?
    print(Some(first), Some(second))?
    Ok(())
14
Option[i64].Some(0) Option[i64].Some(1)

The two unwrap calls assert only that squares(3) has at least two items; the arithmetic Result in each item is still propagated with ?. For an unknown-length stream, match next instead. A generator cannot propagate an arithmetic error from its body: yielding Result items lets its consumer decide how to handle failure. try_fold here handles each item as well as its own addition.

Conversion does not visit any items. The returned generator owns its source and captures, so a pipeline over a borrowed list or with borrowed captures cannot be converted, even after mapping to owned values; return an owned collection instead. Dropping the generator early releases its remaining state.

Each next resumes the generator, which costs more than a fused local loop. Consume a pipeline where it is built unless a resumable stream must cross a function boundary. See the pipeline reference for the full list of adapters and terminals.

Keep callbacks in tuples and lists

A tuple can hold callbacks with different concrete types. Indexing invokes a callback in place; unpacking transfers ownership to new local bindings.

def add(offset: i64) -> Closure[[i64], Result[i64, OverflowError[i64]]]:
    def [offset](n: i64) -> Result[i64, OverflowError[i64]]:
        Ok(offset + n)

def main() -> Result[(), IoError | OverflowError[i64]]:
    pair = (add(10), add(100))
    print(pair[0](2)?)?
    first, second = pair
    print(second(3)?)?
    Ok(())
12
103

Calls to one factory also fit in a list. Each callback owns its own state.

def counter(total: i64) -> Closure[[], Result[i64, OverflowError[i64]]]:
    def [mut total]() -> Result[i64, OverflowError[i64]]:
        total = total + 1
        Ok(total)

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    mut callbacks = [counter(n) for n in range(3)]?
    for callback in &mut callbacks:
        print(callback()?)?
    match callbacks.pop(0):
        case Some(callback):
            mut removed = callback
            print(removed()?)?
        case Nothing:
            print("no callback")?
    print(callbacks[0]()?)?
    Ok(())
1
2
3
2
3

pop transfers the callback out of the list. A borrowed callback prevents resizing or removing entries until its borrow ends.

Name stored callbacks

A dictionary can associate names with owned callbacks from one factory. Calling a callback that changes its captured state needs a mutable borrow: use get_mut to keep the callback registered, or pop to take ownership.

def counter(total: i64) -> Closure[[], Result[i64, OverflowError[i64]]]:
    def [mut total]() -> Result[i64, OverflowError[i64]]:
        total = total + 1
        Ok(total)

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    mut callbacks = {"open": counter(10), "close": counter(100)}?
    match callbacks.get_mut("open"):
        case Some(callback):
            print(callback()?)?
        case Nothing:
            print("no open handler")?
    match callbacks.pop("close"):
        case Some(callback):
            mut owned = callback
            print(owned()?)?
        case Nothing:
            print("no close handler")?
    print(len(callbacks))?
    Ok(())
11
101
1

Replacing a registered callback drops the old captured state. Inserting a new key may grow the dictionary and returns Result. A live borrow of any callback protects the dictionary against replacement, removal, and growth.

The callback itself cannot be a dictionary key or a set element: closures do not define equality or hashing. The dictionary’s value type preserves its concrete factory identity, rather than accepting every callback with the same signature. Use explicit boxing to store different shared-call environments behind one signature.

Store different captured callbacks

Concrete closures remember where they were defined. To put different closures behind one signature, explicitly allocate a BoxedCallable. Its constructor consumes the callback and returns Result, so ? handles allocation failure.

def choose(k: i64) -> Result[BoxedCallable[[i64], Result[i64, OverflowError[i64]]], AllocError]:
    if k > 0:
        add = def [k](value: i64) -> Result[i64, OverflowError[i64]]:
            Ok(k + value)
        return BoxedCallable(add)
    span = range(10, 13)
    multiply = def [span](value: i64) -> Result[i64, OverflowError[i64]]:
        Ok(span[1] * value)
    BoxedCallable(multiply)

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    callbacks: dict[str, BoxedCallable[[i64], Result[i64, OverflowError[i64]]]] = {
        "add": choose(3)?,
        "multiply": choose(0)?,
    }?
    print(callbacks["add"](4)?)?
    print(callbacks["multiply"](4)?)?
    borrowed = &callbacks["add"]
    print(borrowed(5)?)?
    Ok(())
7
44
8

The factory’s branches capture different data but return the same boxed signature. Calling borrows the stored owner, so repeated calls keep it registered and require no extra allocation. Ordinary moves transfer ownership, and dropping or replacing the owner drops its captures. Named functions can also use BoxedCallable(name).

This type currently accepts reusable shared-call callbacks with owned captures. Keep mutable, one-shot, borrowed, or generator-containing closures concrete. Callbacks returning references and worker transfer are also unsupported. Boxing does not make a callback copyable or comparable; use names as registry keys.

Put a callback in an enum

An enum can also own a callback. Match a borrow when you want to leave the owner in place, or match the value to transfer its environment into an arm.

enum Job[F: Callable[[], i64]]:
    Run(F)
    Empty

def package[F: Callable[[], i64]](callback: F) -> Job[F]:
    Job[F].Run(callback)

def invoke[F: Callable[[], i64]](job: &Job[F]) -> i64:
    match &job:
        case Job.Run(callback):
            callback()
        case Job.Empty:
            0

def main() -> Result[(), IoError]:
    value = 42
    callback = def [value]() -> i64:
        value
    job = package(callback)
    print(invoke(&job))?
    Ok(())
42

Consume a capture

A reusable closure borrows its captured state on each call. Sometimes the purpose of the callback is to hand over an owned value. Mark that callback once:

def main() -> Result[(), AllocError | IoError]:
    values = [2, 3, 5]?
    take = def once [values]() -> list[i64]:
        values
    result = take()
    print(result)?
    Ok(())
[2, 3, 5]

take() consumes the callback and transfers its captured list to the result. A second call is a use-after-move error. If take is never called, dropping it cleans up the list.

Consume a callback

Use OnceCallable for a function that owns and calls its callback once. It accepts named functions, reusable closures, and one-shot closures.

pub def run[T, F: OnceCallable[[], T]](callback: F) -> T:
    mut owned = callback
    owned()
import callbacks

def defer[T](value: T) -> OnceClosure[[], T]:
    def once [value]() -> T:
        value

def main() -> Result[(), AllocError | IoError]:
    take = defer([3, 5]?)
    print(callbacks.run(take))?
    Ok(())
[3, 5]

OnceClosure describes the concrete value returned by defer. OnceCallable describes what the generic run function can accept. Passing take transfers ownership; calling it again afterward is an error. The mutable local inside run also permits reusable callbacks that update private captured state.

Consume owned data while borrowing local state

For a callback that also borrows local state, invoke it in that local scope:

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    mut count = 0
    values = [8]?
    take = def once [values, &mut count]() -> Result[list[i64], OverflowError[i64]]:
        count = count + 1
        Ok(values)
    result = take()?
    print(count)?
    print(result)?
    Ok(())
1
[8]

The borrow ends after the call. This callback cannot be passed to callbacks.run because a borrowing environment cannot move across function boundaries.

Store consuming jobs

A stored job may need to hand its captured resources to its caller. Use the existing def once form and extract the job before calling it.

def make_job(n: i64) -> Result[OnceClosure[[], list[i64]], AllocError]:
    values = [n]?
    job = def once [values]() -> list[i64]:
        values
    Ok(job)

def main() -> Result[(), AllocError | IoError]:
    mut jobs = [make_job(10)?, make_job(20)?]?
    match jobs.pop(0):
        case Some(job):
            print(job())?
        case Nothing:
            pass
    for job in jobs:
        print(job())?
    Ok(())
[10]
[20]

Calling jobs[0]() would try to consume a borrowed entry and is rejected. Extract an owned callback with pop, consuming iteration, consuming enum matching, or tuple unpacking.

Dropping uncalled jobs cleans up their captures. Borrowed captures and generator-containing environments cannot be stored in these heap owners.

Concurrent and parallel programs

Use scoped threads for work that borrows local data, or an executor for many owned jobs. Captured jobs use the functions-as-values lessons; generic recovery helpers also use type parameters.

Import concurrency explicitly

The concurrency operations in these lessons also have explicit standard-library imports. Module aliases and item aliases name the same facilities and types as the familiar prelude spellings. Existing programs continue to work.

import std.thread as threads
from std.channel import channel as bounded, Receiver as Inbox
from std.concurrent import ThreadPoolExecutor as Pool
from std.sync import CancellationToken as Stop

def receive(inbox: Inbox[str]) -> Result[str, RecvError]:
    inbox.recv()

def stopped(stop: Stop) -> bool:
    stop.wait()
    stop.is_cancelled()

type Errors = AllocError | IoError | SendError[str] | RecvError | SpawnError[()] | PoolError | FutureError | SubmitError[()]

def main() -> Result[(), Errors]:
    sender, receiver = bounded[str](1)?
    sender.send("hello")?
    drop(sender)
    with threads.spawn(receive, receiver).drop_job()? as task:
        print(task.join()?)?
    stop = Stop()?
    with Pool(1, 1)? as pool:
        future = pool.submit(stopped, stop.share()).drop_job()?
        stop.cancel()
        print(future.result()?)?
    Ok(())
hello
True

Dropping the sender before joining permits a receiving worker to finish even when it loops until disconnection. Imported names retain the same loan, worker-effect, queue-bound and hang diagnostics as prelude names. Error types can also be imported: from std.concurrent import FutureError, for example. Importing an existing builtin under its own name is allowed only when it names that exact builtin; unrelated imports still cannot replace it.

The module references list the complete surfaces: std.thread, std.channel, std.concurrent, and std.sync. Imports ship inside the compiler and need no network or separately installed Plenty sources.

A failed submission returns its unstarted job inside SubmitError; drop_job() drops it and leaves SubmitError[()], as the executor lesson explains. A spawn of a named function with owned arguments works the same way: it returns them inside SpawnError, and drop_job() leaves SpawnError[()].

Run scoped threads

Use with spawn(...) to run a function on a native thread. The current thread continues until join() waits for the result:

def squares(n: i64) -> Result[list[i64], AllocError | OverflowError[i64]]:
    [i * i for i in range(n)]

def main() -> Result[(), SpawnError[()] | AllocError | IoError | OverflowError[i64]]:
    with spawn(squares, 4).drop_job()? as first, spawn(squares, 3).drop_job()? as second:
        print(first.join()?)?
        print(second.join()?)?
    Ok(())
[0, 1, 4, 9]
[0, 1, 4]

The ? after spawn(...) handles failure to create the thread. The ? after join() handles the worker’s own Result, here allocation failure or arithmetic overflow while building its list. They are separate failures. drop_job() is explained in error types at thread start. Join consumes the task binding, so it cannot be called twice.

Every started thread finishes before its with block exits. This includes return, ?, break, and continue. If you omit join(), exit waits and drops the unused result, including any application error. Join when the result matters. A task cannot escape its block or be stored for later.

Scopes cannot forcibly cancel workers. Keep their computations finite or give them a shutdown protocol. Fatal traps terminate the process without unwinding.

Error types at thread start

A failed start reports SpawnError[F], which returns the unstarted job: here the function and its arguments. drop_job() drops that job at the start, so the error becomes SpawnError[()], one member for every start. Keep the job when you want to retry it, as owned worker jobs shows. Borrowed workers report SpawnError[()] too.

Like any ?, the start error widens into the enclosing Result’s error sum, so main lists SpawnError[()] beside the worker’s AllocError and OverflowError[i64], and the printing IoError. A sum without the start error is rejected.

Use module-level error aliases to keep longer signatures readable, as in executor jobs. Each alias names that example’s propagated errors; it does not change widening through ? or introduce an error wrapper.

Borrow local data for a worker

Borrow mutable inputs explicitly. Shared readers can coexist; an exclusive borrow prevents the parent from accessing the same data until the block ends. Join explicitly to propagate a worker’s arithmetic error:

def increment(value: &mut i64) -> Result[(), OverflowError[i64]]:
    *value = *value + 1
    Ok(())

def main() -> Result[(), SpawnError[()] | IoError | OverflowError[i64]]:
    mut count = 4
    with spawn(increment, &mut count)? as task:
        task.join()?
    print(count)?
    Ok(())
5

The borrow lasts until the block ends, even after an explicit join. Reading or modifying count inside the block conflicts with it; use a shorter block when you need access back sooner.

Reusable closures work too. Borrow the closure itself so that a failed start leaves the job available to retry:

def main() -> Result[(), SpawnError[()] | IoError | AllocError]:
    mut values = [1, 2]?
    mut job = def [&mut values]() -> Result[i64, AllocError]:
        values.append(3)?
        Ok(len(values))
    with spawn(&mut job)? as task:
        print(task.join()?)?
    print(values)?
    Ok(())
3
[1, 2, 3]

For immutable captures, spawn(&job) allows shared access. Scoped work may borrow or own its inputs, so a named worker can also receive an owned list:

def total(values: list[i64]) -> Result[i64, OverflowError[i64]]:
    mut result = 0
    for value in values:
        result = result + value
    Ok(result)

def main() -> Result[(), SpawnError[()] | IoError | AllocError | OverflowError[i64]]:
    values = [2, 3, 4]?
    with spawn(total, values).drop_job()? as task:
        print(task.join()?)?
    Ok(())
9

values moves and cannot be used again. Because every argument is owned, the start reports SpawnError, like the scoped threads lesson’s starts: without drop_job() a failed start would return values inside the error. A start that borrows reports the job-free SpawnError[()], because an error cannot hold a borrow. If such a start fails, its moved arguments drop on the parent thread.

To preserve creation error details, put the scope in a helper and match its result. SpawnError.Unavailable(job, code) contains the native error code.

The compiler checks worker helpers, captured data, and destructors. Foreign calls, File values, indirect function calls, and generators are not yet eligible. Worker code can use checked print, but output from different threads may arrive in any order.

Count with atomics

A shared borrow lets several workers read the same data, but not change it. An atomic from std.atomic is the exception: its methods update it through a shared reference, one indivisible step at a time:

from std.atomic import Atomic, Ordering

def count(hits: &Atomic[u64], rounds: i64) -> ():
    for _ in range(rounds):
        hits.fetch_add(1, Ordering.Relaxed)

def main() -> Result[(), SpawnError[()] | IoError]:
    hits = Atomic[u64](0)
    with spawn(count, &hits, 100000)?, spawn(count, &hits, 100000)?:
        pass
    print(hits.load(Ordering.Acquire))?
    Ok(())
200000

Atomic[u64](0) stores the counter inline in main’s frame; nothing is allocated. fetch_add returns the value it replaced. It stops the program on overflow, before storing anything.

Every operation names an Ordering. Relaxed only makes the update itself indivisible. Release on a store and Acquire on a load make the writes before the store visible to the thread that loads it. Use SeqCst when unsure. The compiler rejects orderings an operation cannot use:

from std.atomic import Atomic, Ordering

def main() -> ():
    ready = Atomic[bool](False)
    ready.store(True, Ordering.Acquire)
store does not accept Ordering.Acquire; use Ordering.Relaxed, Ordering.Release, or Ordering.SeqCst

A class can hold atomics as fields, so one borrow shares several of them. Atomics are not values to print or compare, so read each one with load:

from std.atomic import Atomic, Ordering

class Stats:
    seen: Atomic[u32]
    largest: Atomic[i64]

def record(stats: &Stats, start: i64) -> Result[(), OverflowError[i64]]:
    for value in range(start, start + 1000):
        stats.seen.fetch_add(1, Ordering.Relaxed)
        stats.largest.fetch_max(value, Ordering.Relaxed)
    Ok(())

def main() -> Result[(), SpawnError[()] | IoError | OverflowError[i64]]:
    stats = Stats(seen=Atomic(0), largest=Atomic(i64.MIN))
    with spawn(record, &stats, 0)? as first, spawn(record, &stats, 5000)? as second:
        first.join()?
        second.join()?
    print(stats.seen.load(Ordering.Acquire))?
    print(stats.largest.load(Ordering.Acquire))?
    Ok(())
2000
5999

compare_exchange replaces a value only if it still equals the one you expect, and returns Err with the actual value otherwise. See std.atomic for every method.

Share mutable state with a mutex

An atomic keeps one value consistent. When several values must change together, put them in a Mutex from std.sync. Only one thread at a time can hold its lock, so no worker sees count updated without total:

from std.sync import Mutex

class Stats:
    count: i64
    total: i64

def record(stats: &Mutex[Stats], value: i64, rounds: i64) -> Result[(), OverflowError[i64]]:
    for _ in range(rounds):
        with stats.lock() as data:
            count = data.count + 1
            total = data.total + value
            data.count = count
            data.total = total
    Ok(())

def main() -> Result[(), SpawnError[()] | IoError | OverflowError[i64]]:
    stats = Mutex[Stats](Stats(0, 0))
    with spawn(record, &stats, 3, 10000)? as first, spawn(record, &stats, 4, 10000)? as second:
        first.join()?
        second.join()?
    final = stats.into_inner()
    print(final.count)?
    print(final.total)?
    Ok(())
20000
70000

stats.lock() waits until no other thread holds the lock, then binds data, a &mut Stats, for the block. Leaving the block unlocks, whether by reaching its end, return, ?, break, or continue. Both new fields are computed before either is stored, so arithmetic failure cannot leave a half-updated pair. Workers share &stats without mut, because the lock, not the borrow, makes the access exclusive. After the workers finish, into_inner() takes the value back without locking. Nothing here allocates.

lock() works only as a with manager, so a lock cannot be left held by mistake. data belongs to the block: it cannot be stored, returned, or used after it.

Waiting under a lock for something that needs the same lock never finishes. The compiler rejects the waits it can see. Relocking a mutex inside its own block is one; joining a worker that can lock the same kind of mutex is another:

from std.sync import Mutex

def bump(counter: &Mutex[i64]) -> Result[(), OverflowError[i64]]:
    with counter.lock() as value:
        *value = *value + 1
    Ok(())

def main() -> Result[(), SpawnError[()]]:
    counter = Mutex(0)
    with counter.lock() as value:
        with spawn(bump, &counter)?:
            pass
    Ok(())
leaving this `with` block joins a task whose worker can lock a `Mutex[i64]` while this function holds a lock on a `Mutex[i64]`
note: the lock is acquired here; end the lock's `with` block before this one, or start the task outside it

Start the task first, then lock inside its block: the lock’s block ends, and unlocks, before the task is joined.

from std.sync import Mutex

def bump(counter: &Mutex[i64]) -> Result[(), OverflowError[i64]]:
    with counter.lock() as value:
        *value = *value + 1
    Ok(())

def main() -> Result[(), SpawnError[()] | IoError | OverflowError[i64]]:
    counter = Mutex(0)
    with spawn(bump, &counter)? as task:
        with counter.lock() as value:
            *value = *value + 10
        task.join()?
    print(counter.into_inner())?
    Ok(())
11

Nesting a lock inside the block of another mutex of the same type is also rejected when either one is reached through a reference or a computed index: two references can name one mutex. Two separately owned mutexes, such as two local names, may nest. The compiler cannot see a relock inside a called function, a join inside a function called while the lock is held, or two threads taking two locks in opposite orders. Keep lock blocks short and take nested locks in one fixed order. See std.sync and mutexes for the full rules.

Choose how workers share state

Prefer giving each worker what it needs and taking results back. Reach for shared mutable state only when workers must see each other’s changes while running.

NeedUse
Inputs a worker reads or ownsShared borrows, or move the values in
Results to collecttask.join(), or send them on a channel
One independent counter or flagAn atomic
Several values that must change togetherA mutex
State that executor jobs shareA Shared handle to an atomic or a mutex
A request to stopA cancellation token

Here each worker sums its own slice of the work and returns the result, so nothing is shared and nothing needs a lock:

def total(start: i64, end: i64) -> Result[i64, OverflowError[i64]]:
    mut sum = 0
    for value in range(start, end):
        sum = sum + value
    Ok(sum)

def main() -> Result[(), SpawnError[()] | IoError | OverflowError[i64]]:
    with spawn(total, 0, 500).drop_job()? as low, spawn(total, 500, 1000).drop_job()? as high:
        print(low.join()? + high.join()?)?
    Ok(())
499500

An atomic is the cheapest shared update, but each operation is separate: two atomics updated one after the other can be observed half-done. A mutex makes a whole block of updates appear at once, at the cost of making other threads wait.

Atomics and mutexes are shared by borrowing, so they work with scoped threads. Executor jobs must own their inputs, so they cannot borrow either one. Give each job its own owner of a Shared[Atomic[T]] or Shared[Mutex[T]] instead, or send jobs their data and collect their results.

Move jobs to workers

An owned closure can move its inputs into a worker. Use a one-shot closure when the worker also moves those inputs into its result.

def main() -> Result[(), AllocError | IoError | SpawnError[OnceClosure[[], list[i64]]]]:
    values = [3, 4]?
    with spawn(def once [values]() -> list[i64]: values)? as task:
        print(task.join())?
    Ok(())
[3, 4]

If thread creation fails, the error is SpawnError[F].Unavailable(job, code), which retains the complete job and the native error code. F is the closure’s own type. The signature writes OnceClosure[[], list[i64]], and ? fills in the concrete closure it propagates, so main returns the unstarted job unchanged.

A signature names one closure there: a second spawn of a different closure literal is rejected. When you do not need the job back, call drop_job() on each start. It drops a failed start’s job right away, and the error becomes SpawnError[()], which names no job, so one member covers every start:

def main() -> Result[(), AllocError | IoError | SpawnError[()]]:
    values = [3, 4]?
    with spawn(def once [values]() -> list[i64]: values).drop_job()? as first:
        print(first.join())?
    with spawn(def once () -> str: "done").drop_job()? as second:
        print(second.join())?
    Ok(())
[3, 4]
done

Dropping the job also drops the channel endpoints it captured, so they cannot keep a later join waiting; avoid waits that hang shows why that matters.

To keep the job, make each one with one factory function, or use a generic helper, which names the job as F. Recover the job to retry or run it locally, as this helper does:

def attempt[F: OnceCallable[[], i64]](job: F) -> Result[i64, SpawnError[F]]:
    with spawn(job)? as task:
        return Ok(task.join())

def run[F: OnceCallable[[], i64]](job: F) -> i64:
    match attempt(job):
        case Ok(value):
            value
        case Err(SpawnError.Unavailable(job, _)):
            job()

def main() -> Result[(), IoError]:
    n = 42
    job = def once [n]() -> i64:
        n
    print(run(job))?
    Ok(())
42

A reusable closure can also be moved into spawn; its captures are dropped after the worker’s single invocation.

Run stored jobs

Stored reusable callbacks can run on scoped threads. Borrow them explicitly, just as you would borrow a local closure. Distinct tuple slots have separate loans.

def counter(total: i64) -> Closure[[], Result[i64, OverflowError[i64]]]:
    def [mut total]() -> Result[i64, OverflowError[i64]]:
        total = total + 1
        Ok(total)

def main() -> Result[(), SpawnError[()] | IoError | OverflowError[i64]]:
    mut jobs = (counter(10), counter(100))
    with spawn(&mut jobs[0])? as first, spawn(&mut jobs[1])? as second:
        print(first.join()?)?
        print(second.join()?)?
    print(jobs[0]()?)?
    Ok(())
11
101
12

A failed start leaves its borrowed callback owned by the caller. Any earlier successful start is joined.

Workers can also return an owned callback through join():

def make(offset: i64) -> Closure[[i64], Result[i64, OverflowError[i64]]]:
    def [offset](n: i64) -> Result[i64, OverflowError[i64]]:
        Ok(offset + n)

def main() -> Result[(), SpawnError[()] | IoError | OverflowError[i64]]:
    with spawn(make, 10).drop_job()? as task:
        callback = task.join()
        print(callback(3)?)?
    Ok(())
13

List and dictionary entries can also supply a borrowed job. Except at distinct literal list indices, their loans protect the whole collection. Even after join(), wait until the with block ends before resizing or replacing that collection. Stored callbacks have the same thread-eligibility restrictions as local ones.

Send values between threads

A channel transfers owned messages from senders to receivers. Give it a fixed capacity to bound queued memory: a sender waits when the queue is full, and a receiver waits when it is empty.

def produce(sender: Sender[i64]) -> Result[(), SendError[i64] | OverflowError[i64]]:
    for n in range(5):
        sender.send(n * n)?
    Ok(())

def drain(inbox: &Receiver[i64]) -> Result[i64, OverflowError[i64]]:
    mut total = 0
    while True:
        match inbox.recv():
            case Ok(value):
                total = total + value
            case Err(_):
                break
    Ok(total)

def main() -> Result[(), SpawnError[()] | AllocError | IoError | OverflowError[i64]]:
    sender, receiver = channel[i64](2)?
    with spawn(produce, sender).drop_job()?, receiver as inbox:
        print(drain(inbox)?)?
    Ok(())
30

This introductory example deliberately leaves the producer result unclaimed to focus on endpoint guards; use the explicit join pattern when worker errors affect the outcome. The receiver’s accumulation error is still propagated.

spawn(produce, sender) moves the only sender into the producer. When the producer returns, its sender drops; the receiver drains the queue, then reports disconnection, which ends the loop. A blocking recv fails only on disconnection, so case Err(_) is enough here.

The receiver guard is the last manager in the with statement. Managers exit in reverse order, so however the body exits, the receiver drops before Plenty joins the producer. A producer waiting for queue space then sees disconnection instead of waiting forever.

Plenty requires this. A join reached while receiver is still alive could wait forever for a producer blocked on a full queue, so the compiler rejects it:

def produce(sender: Sender[i64]) -> Result[(), SendError[i64] | OverflowError[i64]]:
    for n in range(5):
        sender.send(n * n)?
    Ok(())

def main() -> Result[(), SpawnError[()] | AllocError | RecvError | IoError | OverflowError[i64]]:
    sender, receiver = channel[i64](2)?
    with spawn(produce, sender).drop_job()?:
        print(receiver.recv()?)?
    Ok(())
leaving this `with` block joins a task whose worker holds `Sender[i64]` while `receiver` still holds `Receiver[i64]`

The compiler compares message types rather than following each queue, so it also asks for this order in some programs that could not hang. Write the guard anyway; it costs nothing.

Wait for the producer’s result

The automatic join above drops the producer’s result. To check it with task.join(), finish with the receiver first: put its guard inside the task’s block, before any statement that can leave the block, and join after the guard ends.

def produce(sender: Sender[i64]) -> Result[(), SendError[i64] | OverflowError[i64]]:
    for n in range(5):
        sender.send(n * n)?
    Ok(())

def drain(inbox: &Receiver[i64]) -> Result[i64, OverflowError[i64]]:
    mut total = 0
    while True:
        match inbox.recv():
            case Ok(value):
                total = total + value
            case Err(_):
                break
    Ok(total)

def main() -> Result[(), SpawnError[()] | AllocError | IoError | SendError[i64] | OverflowError[i64]]:
    sender, receiver = channel[i64](2)?
    with spawn(produce, sender).drop_job()? as task:
        mut total = 0
        with receiver as inbox:
            total = drain(inbox)?
        task.join()?
        print(total)?
    Ok(())
30

total is declared before the inner guard because a binding made inside a block ends with that block. task.join()? propagates the producer’s SendError[i64] or OverflowError[i64], if any.

A ? before the guard leaves the block with the receiver still alive, so this version is rejected:

def produce(sender: Sender[i64]) -> Result[(), SendError[i64] | OverflowError[i64]]:
    for n in range(5):
        sender.send(n * n)?
    Ok(())

def drain(inbox: &Receiver[i64]) -> Result[i64, OverflowError[i64]]:
    mut total = 0
    while True:
        match inbox.recv():
            case Ok(value):
                total = total + value
            case Err(_):
                break
    Ok(total)

def main() -> Result[(), SpawnError[()] | AllocError | IoError | SendError[i64] | OverflowError[i64]]:
    sender, receiver = channel[i64](2)?
    with spawn(produce, sender).drop_job()? as task:
        print("receiving")?
        mut total = 0
        with receiver as inbox:
            total = drain(inbox)?
        task.join()?
        print(total)?
    Ok(())
leaving this `with` block joins a task whose worker holds `Sender[i64]` while `receiver` still holds `Receiver[i64]`
`?` leaves the block here

Prefer a guard to drop(receiver): a drop protects only the paths that reach it, and an earlier ? does not.

A receiving worker

The same order applies when the worker receives: an empty queue keeps it waiting while any sender survives. Guard the sender last, so every exit disconnects the queue before the join:

def consume(receiver: Receiver[i64]) -> ():
    match receiver.recv():
        case Ok(_):
            pass
        case Err(_):
            pass

def main() -> Result[(), SpawnError[()] | AllocError | SendError[i64]]:
    sender, receiver = channel[i64](1)?
    with spawn(consume, receiver).drop_job()?, sender as outbox:
        outbox.send(42)?
    Ok(())

Leaving sender alive until after this block is rejected, even if the message is already queued. For an explicit task.join(), end an inner sender guard first, as with the receiver above.

Ownership and cost

Create additional handles explicitly with sender.share() or receiver.share(). Multiple receivers compete for messages on the same queue: each message is delivered to one of them. Disconnection occurs only after the last handle on that side drops, so every handle you keep needs a guard or a drop. The wait habits lesson shows how to start several producers.

Channel construction can fail. Its positive capacity fixes the queue’s storage up front; sending and receiving introduce no further allocations. Messages still use their ordinary types: sending a list moves it, and receiving returns its ownership. A failed send returns the unsent value inside its error.

A failed start returns its owned arguments, including endpoints, inside SpawnError, as in owned worker jobs. drop_job() drops them at the start instead, so the error holds no endpoint; wait habits shows why that matters.

Avoid waits that hang

A join waits for a worker. If the worker is blocked on a channel endpoint or a lock that the waiting function still holds, or waits for a cancel that the function has not made yet, the join never returns. Plenty rejects such waits at compile time. These habits keep your programs clear of the whole class:

  1. Move each endpoint into one worker as an argument: spawn(produce, sender).drop_job()?. A failed start then drops it.
  2. Guard each endpoint you keep as the last manager: with spawn(produce, sender).drop_job()?, receiver as inbox:.
  3. To wait inside the block, end the guard first. Open it before any statement that can leave the block, and wait after it ends.
  4. Wait in a statement of its own. An endpoint moved into the same expression is still held while the wait runs.
  5. Call drop_job() on a start you will not retry.
  6. Start workers outside a lock block, and lock inside them.
  7. Cancel a token on every exit when a worker waits for it: with spawn(serve, stop.share()).drop_job()?, stop.cancel_on_exit():.

The channel shutdown lesson showed habits 1 to 3, and the mutex lesson showed habit 6. Cooperative cancellation shows habit 7. Executor pools follow the same habits, as the future wait lesson shows. This lesson covers several producers and failed starts.

Several producers

Each producer needs its own sender, and the receiver must outlive none of them. One guard after several ? starts is not enough: when a later start fails, ? joins the earlier producers before the guard exists.

def produce(sender: Sender[i64]) -> Result[(), SendError[i64] | OverflowError[i64]]:
    for n in range(5):
        sender.send(n * n)?
    Ok(())

def drain(inbox: &Receiver[i64]) -> Result[i64, OverflowError[i64]]:
    mut total = 0
    while True:
        match inbox.recv():
            case Ok(value):
                total = total + value
            case Err(_):
                break
    Ok(total)

def main() -> Result[(), SpawnError[()] | AllocError | IoError | OverflowError[i64]]:
    sender, receiver = channel[i64](2)?
    second = sender.share()
    with spawn(produce, sender).drop_job()?, spawn(produce, second).drop_job()?, receiver as inbox:
        print(drain(inbox)?)?
    Ok(())
leaving this `with` block joins a task whose worker holds `Sender[i64]` while `receiver` still holds `Receiver[i64]`
`?` leaves the block here

Give the receiver to a consuming worker instead, and start it first. Then guard the sender and hand each producer a share of it:

def produce(sender: Sender[i64]) -> Result[(), SendError[i64] | OverflowError[i64]]:
    for n in range(5):
        sender.send(n * n)?
    Ok(())

def total(receiver: Receiver[i64]) -> Result[i64, OverflowError[i64]]:
    mut sum = 0
    while True:
        match receiver.recv():
            case Ok(value):
                sum = sum + value
            case Err(_):
                break
    Ok(sum)

def main() -> Result[(), SpawnError[()] | AllocError | IoError | OverflowError[i64]]:
    sender, receiver = channel[i64](2)?
    with spawn(total, receiver).drop_job()? as consumer:
        with sender as outbox, spawn(produce, outbox.share()).drop_job()?, spawn(produce, outbox.share()).drop_job()?:
            pass
        print(consumer.join()?)?
    Ok(())
60

Here the sender guard comes first: the producers send too, so only the consumer, joined later, needs the sender gone. The inner block joins both producers, then drops the last sender. The consumer sees disconnection and returns, so consumer.join() after the block cannot hang. If a producer fails to start, ? leaves through the same exits in the same order.

Failed starts keep their endpoints

A job that fails to start returns inside SpawnError[F], with every endpoint it captured. ? keeps that error until the function returns, which is after the block’s joins:

def report(receiver: Receiver[i64]) -> Result[(), IoError | OverflowError[i64]]:
    mut total = 0
    while True:
        match receiver.recv():
            case Ok(value):
                total = total + value
            case Err(_):
                break
    print(total)

def produce(sender: Sender[i64], count: i64) -> Result[(), SendError[i64] | OverflowError[i64]]:
    for n in range(count):
        sender.send(n * n)?
    Ok(())

type Errors = AllocError | SpawnError[()] | SpawnError[OnceClosure[[], Result[(), SendError[i64] | OverflowError[i64]]]]

def main() -> Result[(), Errors]:
    sender, receiver = channel[i64](2)?
    count = 5
    job = def once [sender, count]() -> Result[(), SendError[i64] | OverflowError[i64]]:
        produce(sender, count)
    with spawn(report, receiver).drop_job()?, spawn(job)?:
        pass
    Ok(())
leaving this `with` block joins a task whose worker holds `Receiver[i64]` while the error that `?` propagates still holds `Sender[i64]`
call `drop_job()` on the start

When you will not retry the job, call drop_job() on the start. It drops the job, and its sender, before ? leaves, and the error becomes SpawnError[()]:

def report(receiver: Receiver[i64]) -> Result[(), IoError | OverflowError[i64]]:
    mut total = 0
    while True:
        match receiver.recv():
            case Ok(value):
                total = total + value
            case Err(_):
                break
    print(total)

def produce(sender: Sender[i64], count: i64) -> Result[(), SendError[i64] | OverflowError[i64]]:
    for n in range(count):
        sender.send(n * n)?
    Ok(())

type Errors = AllocError | SpawnError[()]

def main() -> Result[(), Errors]:
    sender, receiver = channel[i64](2)?
    count = 5
    job = def once [sender, count]() -> Result[(), SendError[i64] | OverflowError[i64]]:
        produce(sender, count)
    with spawn(report, receiver).drop_job()?, spawn(job).drop_job()?:
        pass
    Ok(())
30

A named start, spawn(produce, sender, count), binds the same kind of job from its arguments, so it needs drop_job() here too.

The channel reference lists every way a function can hold an endpoint, and the programs the check rejects although they could not hang.

Handle channel backpressure

Use send_nowait when a full queue should return control to the caller. Its Full error retains your message, so you can retry it after making space.

type MessageError = RecvError | SendError[list[i64]] | AllocError

def main() -> Result[(), AllocError | IoError | MessageError | OverflowError[i64]]:
    sender, receiver = channel[list[i64]](1)?
    sender.send([10]?)?
    match sender.send_nowait([20, 30]?):
        case Ok(_):
            pass
        case Err(SendError.Full(message)):
            print(receiver.recv()?)?
            sender.send(message)?
        case Err(SendError.Disconnected(message)):
            print(message)?
    print(receiver.recv()?)?
    drop(sender)
    print(str.repr(receiver.recv())?)?
    Ok(())
[10]
[20, 30]
Result[list[i64], RecvError].Err(RecvError.Disconnected)

recv_nowait() similarly reports RecvError.Empty when the channel is connected but has no message ready. Neither nowait operation waits for the queue to change state, though both synchronize access to the queue.

If you need only part of a stream, the receiver guard handles shutdown:

def produce(sender: Sender[i64]) -> Result[(), AllocError | IoError | SendError[i64] | OverflowError[i64]]:
    for n in range(10, 100):
        sender.send(n * n)?
    Ok(())

def first_square() -> Result[i64, SpawnError[()] | AllocError | RecvError]:
    sender, receiver = channel[i64](1)?
    with spawn(produce, sender).drop_job()?, receiver as inbox:
        return Ok(inbox.recv()?)

def main() -> Result[(), SpawnError[()] | AllocError | RecvError | IoError | OverflowError[i64]]:
    print(first_square()?)?
    Ok(())
100

Returning drops the receiver first. The producer’s next send reports disconnection; its ? ends the worker. The task scope then joins it and discards its unclaimed result.

Channel capacity has type Positive[i64]. A positive literal converts at compile time; for configuration read at run time, pass Positive.try_from(capacity)?. Construction then reports only allocation failure. See integer domains.

Bound your waits

Use a timeout when your program needs control back even if no message arrives. Timeouts are u64 milliseconds; zero performs an immediate check.

def main() -> Result[(), IoError | RecvTimeoutError | SendTimeoutError[i64] | AllocError]:
    sender, receiver = channel[i64](1)?
    match receiver.recv_timeout_ms(0):
        case Ok(value):
            print(value)?
        case Err(RecvTimeoutError.TimedOut):
            print("nothing yet")?
        case Err(RecvTimeoutError.Disconnected):
            print("senders finished")?
    sender.send_timeout_ms(42, 0)?
    print(receiver.recv_timeout_ms(0)?)?
    Ok(())
nothing yet
42

send_timeout_ms(value, milliseconds) returns an unsent value in SendTimeoutError[T].TimedOut(value) or .Disconnected(value). You can retry, store, or drop it. A timeout does not discard the message.

Timeouts limit waiting, not total wall-clock execution. Lock contention and scheduling can delay return. A message already available at the check is received even with a zero budget. To bound several waits by one total budget, share a deadline.

Share one deadline across waits

A timeout bounds one wait. When a task makes several waits, give it a Deadline from std.time: one total budget that every wait draws from.

import std.sync
from std.time import Deadline, monotonic_now, deadline_after, milliseconds

def collect(inbox: &Receiver[i64], deadline: &Deadline, cancel: &CancellationToken) -> Result[i64, OverflowError[i64]]:
    mut total = 0
    while True:
        match std.sync.recv_before(inbox, deadline, cancel):
            case Ok(value):
                total += value
            case Err(std.sync.WaitError.TimedOut):
                break
            case Err(std.sync.WaitError.Disconnected) | Err(std.sync.WaitError.Cancelled):
                break
    Ok(total)

def main() -> Result[(), AllocError | IoError | RangeError | SendError[i64] | OverflowError[i64]]:
    sender, receiver = channel[i64](4)?
    sender.send(1)?
    sender.send(2)?
    sender.send(3)?
    deadline = deadline_after(&monotonic_now(), &milliseconds(20))?
    cancel = CancellationToken()?
    print(collect(&receiver, &deadline, &cancel)?)?
    print(deadline.expired(&monotonic_now()))?
    print(deadline.remaining(&monotonic_now()).as_timeout_ms())?
    Ok(())
6
True
0

recv_before checks caller cancellation between bounded waits and shares one deadline. The loop makes four waits against one 20-millisecond budget instead of granting each wait its own. remaining takes the current instant and returns zero once the deadline has passed. A zero budget still receives a message that is already queued, so the three sent values always arrive.

as_timeout_ms() is available for the low-level _timeout_ms methods. It converts a span to the u64 milliseconds that timed waits accept. It rounds up: a wait never returns before its deadline, and a remainder below a millisecond does not turn the loop into a zero-timeout spin.

Sleep that can be cancelled

sleep cannot be interrupted. To pause until either a span passes or someone asks you to stop, wait on a cancellation token with the span as the budget:

from std.time import seconds

def pause(stop: &CancellationToken) -> bool:
    stop.wait_timeout_ms(seconds(3600).as_timeout_ms())

def main() -> Result[(), AllocError | IoError | SpawnError[()]]:
    stop = CancellationToken()?
    with spawn(pause, &stop)? as task:
        stop.cancel()
        print(task.join())?
    Ok(())
True

The worker borrows the token and asks for an hour. It returns as soon as cancel() runs, even if the request comes before the wait starts. The result is true when cancellation ended the wait and false when the span elapsed.

Deadlines use the monotonic clock, so changing the system time neither extends nor shortens them. They limit waiting, not execution: a wait can return late on a busy machine. See the time reference.

Reuse workers with an executor

A thread pool reuses workers for many jobs. Set its worker count and queue capacity at construction:

def square(value: i64) -> Result[i64, OverflowError[i64]]:
    Ok(value * value)

type PoolJobError = ParallelError[OverflowError[i64]] | FutureError | SubmitError[()] | PoolError

def main() -> Result[(), IoError | PoolJobError | OverflowError[i64]]:
    with ThreadPoolExecutor(2, 4)? as pool:
        first = pool.submit(square, 6).drop_job()?
        second = pool.submit(square, 9).drop_job()?
        print(first.result()??)?
        print(second.result()??)?
        print(pool.map_result(square, range(6))?)?
    Ok(())
36
81
[0, 1, 4, 9, 16, 25]

submit returns an owned Future[Result[i64, OverflowError[i64]]]; result() waits and consumes it. The first ? handles the future wait, and the second propagates the worker’s overflow error. A failed submission returns SubmitError, which owns the unstarted job. drop_job() drops that job, as for thread starts, and leaves SubmitError[()], which still says why submission failed. Submit jobs before waiting for their results to let them overlap. Leaving with waits for accepted jobs and joins every worker, including on early return or propagated error.

map_result returns successful values in input order, or the first worker error in input order. map instead keeps every returned value, including any Result wrappers. It accepts an owned list or an integer range and a named function or owned reusable immutable closure, keeping a bounded window of jobs outstanding. Job and result storage can fail to allocate.

def main() -> Result[(), AllocError | IoError | ParallelError[OverflowError[i64]] | PoolError]:
    with ThreadPoolExecutor(2, 2)? as pool:
        factor = 3
        scale = def [factor](value: i64) -> Result[i64, OverflowError[i64]]:
            Ok(value * factor)
        print(pool.map_result(scale, range(4))?)?
    Ok(())
[0, 3, 6, 9]

Mapping consumes scale and shares its one immutable environment until all jobs finish. Mutable, one-shot, or borrowed closures cannot be shared this way. map_result accepts the same workers when they return Result[T, E].

You can submit a closure to move several values into a job explicitly:

type PoolJobError = FutureError | SubmitError[()] | PoolError

def main() -> Result[(), AllocError | IoError | PoolJobError]:
    with ThreadPoolExecutor(2, 2)? as pool:
        values = [3, 6, 9]?
        future = pool.submit(def once [values]() -> list[i64]: values).drop_job()?
        print(future.result()?)?
    Ok(())
[3, 6, 9]

Pool jobs own their inputs; use spawn when a worker needs to borrow local data. A worker may use channels, following the same habits as a task, but cannot hold an executor or future. Keep submission and future waits on the owning thread to avoid pool dependency deadlocks. Files, native effects and foreign calls follow the same rules as for scoped workers.

Dropping a future leaves its job running; an unclaimed result is cleaned up. A future may also outlive the pool’s scope: shutdown finishes the job, and the handle retains the result until you retrieve or drop it.

Keep the job instead when you want to retry it or run it locally; a submission backpressure lesson shows how.

Handle executor outcomes

A job returning Result keeps its application errors separate from submission failure or cancellation. Retrieve its result, then handle the application result:

def divide(value: i64) -> Result[i64, str]:
    divisor = NonZero.try_from(value).map_error(def(error: RangeError) -> str: "zero divisor")?
    # 24 // a nonzero i64 cannot overflow: the dividend is not i64.MIN.
    unchecked:
        Ok(24 // divisor)

type PoolJobError = PoolMapError | FutureError | SubmitError[()] | PoolError

def main() -> Result[(), AllocError | IoError | PoolJobError]:
    with ThreadPoolExecutor(2, 2)? as pool:
        future = pool.submit(divide, 0).drop_job()?
        match future.result()?:
            case Ok(value):
                print(value)?
            case Err(message):
                print(message)?
        print(pool.map(divide, [2, 0, 4]?)?)?
    Ok(())
zero divisor
[Result[i64, str].Ok(12), Result[i64, str].Err("zero divisor"), Result[i64, str].Ok(6)]

The example prints the application error to show it; a program that cannot continue propagates it instead. To propagate both layers from a single future, write (future.result()?)?. Mapping preserves each application outcome in input order.

Share state with pool workers

Scoped threads can borrow an atomic or a mutex, but executor jobs must own their inputs. Shared[T] gives a value several owners. Hand each job its own owner with share():

from std.atomic import Atomic, Ordering
from std.sync import Mutex

class Stats:
    count: i64
    total: i64

def work(hits: Shared[Atomic[u64]], stats: Shared[Mutex[Stats]], value: i64) -> Result[i64, OverflowError[i64]]:
    for _ in range(1000):
        hits.fetch_add(1, Ordering.Relaxed)
        with stats.lock() as data:
            count = data.count + 1
            total = data.total + value
            data.count = count
            data.total = total
    Ok(value)

type PoolJobError = FutureError | SubmitError[()] | PoolError

def main() -> Result[(), AllocError | IoError | PoolJobError | OverflowError[i64]]:
    hits = Shared[Atomic[u64]](Atomic(0))?
    stats = Shared[Mutex[Stats]](Mutex(Stats(0, 0)))?
    with ThreadPoolExecutor(2, 4)? as pool:
        first = pool.submit(work, hits.share(), stats.share(), 3).drop_job()?
        second = pool.submit(work, hits.share(), stats.share(), 4).drop_job()?
        print(first.result()?? + second.result()??)?
    print(hits.load(Ordering.Acquire))?
    with stats.lock() as data:
        print(data.count)?
        print(data.total)?
    Ok(())
7
2000
2000
7000

Shared[T](value) moves the value into one allocation, which can fail. share() returns another owner of the same value without allocating. Each owner is dropped like any other value; the last one to drop, on whichever thread, runs the value’s cleanup and frees it.

The handle lends its content: hits.fetch_add(...) and stats.lock() reach the atomic and the mutex through it, and &*hits borrows the atomic itself. The content is shared, never exclusive, so it changes only through an atomic or a mutex:

class Stats:
    count: i64

def main() -> Result[(), AllocError]:
    stats = Shared(Stats(0))?
    stats.count = 1
    Ok(())
a Shared owner gives shared access only; mutate its content through an Atomic or a Mutex inside it

Handles cannot be compared or printed; read the content instead. Take another owner with share(), not copy(). Copying a value that holds a handle, such as a job’s (str, Shared[Atomic[u64]]) input, shares that handle the same way. A value cannot hold a share of itself through a mutex, since it would never be released: link back to it with an index into a list that owns the values.

The compiler’s lock checks see through handles. Shutting a pool down while holding a lock on a mutex one of its jobs can reach is rejected, as is waiting for such a job’s result. Wait after the lock block ends, as main reads stats only after the pool’s block. Nesting a lock through a handle inside a lock on another Mutex[T] of the same type is rejected too, because two handles can name one mutex. See shared owners for the full rules.

Cancel queued work and bound a future wait

future.cancel() returns True when it prevents a queued job from starting. It cannot interrupt running work. A cancelled future reports FutureError.Cancelled from result(). future.done() observes completion or cancellation without consuming the handle. pool.cancel_pending() cancels every job still queued without waiting, and a following pool.shutdown() waits for running jobs; ordinary scope exit drains all work.

For a future, future.wait_timeout_ms(milliseconds) returns a boolean: true means completed or cancelled. This only observes the future; call result() afterward to take its result. A false return leaves the job running or queued.

Jobs that send

Shutdown and result() both wait for a job, so a job that sends follows the same habits as a task. When you do not wait for its result inside the block, guard the receiver as the last manager of the pool’s with:

def produce(sender: Sender[i64]) -> Result[(), SendError[i64] | OverflowError[i64]]:
    for n in range(5):
        sender.send(n * n)?
    Ok(())

def drain(inbox: &Receiver[i64]) -> Result[i64, OverflowError[i64]]:
    mut total = 0
    while True:
        match inbox.recv():
            case Ok(value):
                total = total + value
            case Err(_):
                break
    Ok(total)

def main() -> Result[(), AllocError | IoError | PoolError | SubmitError[()] | OverflowError[i64]]:
    sender, receiver = channel[i64](2)?
    with ThreadPoolExecutor(1, 1)? as pool, receiver as inbox:
        pool.submit(produce, sender).drop_job()?
        print(drain(inbox)?)?
    Ok(())
30

That guard drops the receiver at shutdown, which is too late for a result() inside the block:

def produce(sender: Sender[i64]) -> Result[(), SendError[i64] | OverflowError[i64]]:
    for n in range(5):
        sender.send(n * n)?
    Ok(())

def drain(inbox: &Receiver[i64]) -> Result[i64, OverflowError[i64]]:
    mut total = 0
    while True:
        match inbox.recv():
            case Ok(value):
                total = total + value
            case Err(_):
                break
    Ok(total)

def main() -> Result[(), AllocError | FutureError | IoError | PoolError | SendError[i64] | SubmitError[()] | OverflowError[i64]]:
    sender, receiver = channel[i64](2)?
    with ThreadPoolExecutor(1, 1)? as pool, receiver as inbox:
        future = pool.submit(produce, sender).drop_job()?
        print(drain(inbox)?)?
        future.result()??
    Ok(())
this result() waits for a job that holds `Sender[i64]` while `receiver` still holds `Receiver[i64]`

To wait for the result, end the receiver’s own guard first:

def produce(sender: Sender[i64]) -> Result[(), SendError[i64] | OverflowError[i64]]:
    for n in range(5):
        sender.send(n * n)?
    Ok(())

def drain(inbox: &Receiver[i64]) -> Result[i64, OverflowError[i64]]:
    mut total = 0
    while True:
        match inbox.recv():
            case Ok(value):
                total = total + value
            case Err(_):
                break
    Ok(total)

def main() -> Result[(), AllocError | FutureError | IoError | PoolError | SendError[i64] | SubmitError[()] | OverflowError[i64]]:
    sender, receiver = channel[i64](2)?
    with ThreadPoolExecutor(1, 1)? as pool:
        future = pool.submit(produce, sender).drop_job()?
        mut total = 0
        with receiver as inbox:
            total = drain(inbox)?
        future.result()??
        print(total)?
    Ok(())
30

A failed submit(...)? has submitted nothing, so it can come before the guard. done() and wait_timeout_ms() do not prove the channel protocol to the checker: the result wait is rejected while a matching receiver lives, even after they report completion.

A worker that may run indefinitely needs a cooperative shutdown protocol: scope exit still waits after a timed wait returns.

Stop work cooperatively

Use a cancellation token to ask a running worker to stop. Share the token explicitly; all handles observe the same request.

def work(stop: CancellationToken, ready: Sender[i64]) -> Result[i64, SendError[i64]]:
    ready.send(1)?
    stop.wait()
    # Release resources and return through ordinary control flow.
    Ok(42)

type CancellableWorkError = SendError[i64] | FutureError | RecvError | SubmitError[()] | AllocError | PoolError

def main() -> Result[(), AllocError | IoError | CancellableWorkError]:
    stop = CancellationToken()?
    ready, started = channel[i64](1)?
    with ThreadPoolExecutor(1, 1)? as pool, stop.cancel_on_exit():
        future = pool.submit(work, stop.share(), ready).drop_job()?
        with started as inbox:
            inbox.recv()?
        stop.cancel()
        print(future.result()??)?
    print(stop.is_cancelled())?
    Ok(())
42
True

The worker reports that it started, then waits for the request. stop.cancel_on_exit() cancels stop however the block ends, including through a ?, so the pool’s shutdown never waits for a job that is still waiting for the request; cancel on every exit shows the mistake it prevents. The explicit stop.cancel() comes before future.result(), which waits inside the block.

The receiver’s own guard ends before future.result() waits, and it also covers the ? on recv(), as in jobs that send. Without the guard, an exit from the pool’s block would reach shutdown with started alive, and Plenty rejects that:

def report(ready: Sender[i64]) -> Result[(), SendError[i64]]:
    ready.send(1)

def main() -> Result[(), AllocError | PoolError | RecvError | SubmitError[()]]:
    ready, started = channel[i64](1)?
    with ThreadPoolExecutor(1, 1)? as pool:
        pool.submit(report, ready).drop_job()?
        started.recv()?
    Ok(())
leaving its scope shuts down this pool after a job that holds `Sender[i64]` was submitted, while `started` still holds `Receiver[i64]`

As with task joins, the compiler compares message types rather than counting messages, so it asks for the guard even though this job’s one send always fits the queue.

Cancel on every exit

A worker that waits for a token keeps a join or pool shutdown waiting until someone cancels it. When the cancel comes after a ?, return, or break, that exit skips it, and Plenty rejects the program:

def serve(stop: CancellationToken) -> ():
    stop.wait()

def prepare() -> Result[(), str]:
    Ok(())

def main() -> Result[(), AllocError | SpawnError[()] | IoError | str]:
    stop = CancellationToken()?
    with spawn(serve, stop.share()).drop_job()?:
        prepare()?
        print("ready")?
        stop.cancel()
    Ok(())
leaving this `with` block joins a task that holds `stop` before this function cancels it
`?` leaves the block here

Add stop.cancel_on_exit() as the last manager, after the task. On every exit it cancels stop before the task is joined, so the explicit cancel() is no longer needed:

def serve(stop: CancellationToken) -> ():
    stop.wait()

def prepare() -> Result[(), str]:
    Ok(())

def main() -> Result[(), AllocError | SpawnError[()] | IoError | str]:
    stop = CancellationToken()?
    with spawn(serve, stop.share()).drop_job()?, stop.cancel_on_exit():
        prepare()?
        print("ready")?
    Ok(())
ready

The guard holds a share of the token and allocates nothing.

The same mistake can happen the other way round. Here the parent waits for the worker to cancel stop when it finishes, but a ? in the worker skips the cancel:

def prepare() -> Result[(), str]:
    Ok(())

def client(stop: CancellationToken) -> Result[(), str]:
    prepare()?
    stop.cancel()
    Ok(())

def main() -> Result[(), AllocError | SpawnError[()] | IoError | str]:
    stop = CancellationToken()?
    with spawn(client, stop.share()).drop_job()? as task:
        stop.wait()
        task.join()?
    Ok(())
`?` leaves this function without cancelling `stop`, which its normal end cancels

Put the worker’s body in with stop.cancel_on_exit():, so the parent’s stop.wait() returns however the worker ends. task.join()? after the wait then reports the worker’s error, if it had one:

def prepare() -> Result[(), str]:
    Ok(())

def client(stop: CancellationToken) -> Result[(), str]:
    with stop.cancel_on_exit():
        prepare()?
    Ok(())

def main() -> Result[(), AllocError | SpawnError[()] | IoError | str]:
    stop = CancellationToken()?
    with spawn(client, stop.share()).drop_job()? as task:
        stop.wait()
        task.join()?
        print("client finished")?
    Ok(())
client finished

The cancellation reference describes the check and its limits.

Checking for cancellation

Check stop.is_cancelled() between work units and return an appropriate result. Cancellation does not interrupt instructions or skip cleanup.

cancel() is safe to repeat. Dropping a token handle does not cancel work. wait_timeout_ms(milliseconds) waits for a request and returns whether it arrived. A token does not wake an ordinary recv() call. Use a shutdown message or a timed receive when a worker must periodically check for cancellation. Automatic joining still waits for that worker to return.

In a receive loop, check is_cancelled() between bounded receives. Avoid a zero-timeout retry loop that consumes a CPU while idle.

Recover a job when submission cannot proceed

submit waits for queue space. submit_nowait instead returns the unstarted job in SubmitError.Full(job) when full. Every submission error owns the job, so a generic helper can run it locally when submission is unavailable:

def run_or_submit[F: OnceCallable[[], i64]](
    pool: &ThreadPoolExecutor, job: F
) -> Result[i64, FutureError]:
    match pool.submit_nowait(job):
        case Ok(future):
            future.result()
        case Err(
            SubmitError.Full(job)
            | SubmitError.Shutdown(job)
            | SubmitError.OutOfMemory(job)
            | SubmitError.CapacityOverflow(job)
        ):
            Ok(job())

def main() -> Result[(), AllocError | IoError | FutureError | PoolError]:
    with ThreadPoolExecutor(1, 1)? as pool:
        pool.shutdown()
        number = 42
        job = def once [number]() -> i64:
            number
        print(run_or_submit(pool, job)?)?
    Ok(())
42

When you do not need the job back, call drop_job() on the submission, as in the executor lesson: the error keeps its variant and names no job. Erasing it into Failure also drops the job but loses the error type; that is a last resort.

Map fallible work in parallel

Use map_result when each worker returns a Result and you want either all the successful values or one error. The error keeps its original type.

def square(n: i64) -> Result[i64, str | OverflowError[i64]]:
    if n < 0:
        Err("negative input")
    else:
        Ok(n * n)

def main() -> Result[(), AllocError | IoError | ParallelError[str | OverflowError[i64]] | PoolError]:
    with ThreadPoolExecutor(2, 2)? as pool:
        print(pool.map_result(square, range(5))?)?
        match pool.map_result(square, [3, -1, -2]?):
            case Ok(values):
                print(values)?
            case Err(ParallelError.Worker(error)):
                match error:
                    case message: str:
                        print(message)?
                    case overflow: OverflowError[i64]:
                        return Err(ParallelError[str | OverflowError[i64]].Worker(overflow))
            case Err(error):
                return Err(error)
    Ok(())
[0, 1, 4, 9, 16]
negative input

The fixed negative inputs deliberately demonstrate a handled worker error; a program that cannot recover should propagate that error too. Allocation failure, arithmetic overflow, and shutdown are not recovered here and leave main with an error.

Outputs keep input order. If several inputs fail, the earliest input’s error wins; the fastest worker does not decide. map_result waits for accepted jobs and cleans up partial results before returning. Later inputs may already have run, so an error cannot undo their effects. Do not make a worker wait for an input that might never be submitted.

Both map and map_result consume input lists and use a bounded job window. Use ordinary map when you want a list containing every worker’s Result, including all errors. Both operations can report allocation failure.

The worker preserves both its input error and arithmetic error in an anonymous sum. Matching the str member prints its message rather than the structural representation of the sum.

Combine values in parallel

reduce_tree combines adjacent pairs on the pool, then combines their results. The function takes two values and returns either one value or a Result of that value type. Give arithmetic callbacks their own fallible contract.

def add[T: IntType](left: T, right: T) -> Result[T, OverflowError[T]]:
    Ok(left + right)

def main() -> Result[(), AllocError | IoError | PoolMapError | PoolError | OverflowError[i64]]:
    with ThreadPoolExecutor(2, 2)? as pool:
        print(pool.reduce_tree(add, range(1, 6))?)?
        print(pool.reduce_tree(add, range(0))?)?
    Ok(())
Option[i64].Some(15)
Option[i64].Nothing

Empty input returns Nothing; otherwise the result is Some(value). The outer Result reports allocation failure, a closed pool, or the callback’s precise error. A fallible callback gives Result[Option[T], PoolMapError | E]. Lists are consumed without copying their elements; ranges remain ordinary copyable values.

The tree is fixed: [a, b, c, d, e] becomes [f(a, b), f(c, d), e], then [f(f(a, b), f(c, d)), e], then one value. More workers do not change that grouping. An odd final element advances unchanged.

Choose an operation whose grouping is suitable for your task. Subtraction and floating-point addition can give different answers from a serial left-to-right loop. Keep a serial loop when that exact order matters. Worker side effects need not follow the tree order. For fallible callbacks, errors are selected by tree level, then by left-to-right pair, not by completion time. Already accepted jobs finish before the error returns; later levels do not run. Fallible callbacks use one synchronization round per tree level rather than the multi-level jobs available to infallible callbacks.

For larger inputs, reduction allocates scratch storage proportional to the input length and only a bounded window of jobs. Empty and singleton reductions allocate nothing. If allocation fails, the operation waits for accepted jobs and cleans up every intermediate value before returning.

Run a pipeline on a pool

.par(&pool) runs the rest of a pipeline on a pool’s workers. Steps after it must be map, filter or filter_value, and the pipeline must end in a terminal that combines the chunks: collect, try_collect, count, any, all, sum, min, max or reduce.

def score(n: i64) -> i64:
    seven: NonZero[i64] = 7
    remainder = n % seven
    # Floor remainder is 0..6, so its square fits even for arbitrary n.
    unchecked:
        remainder * remainder % seven

def positive(n: &i64) -> bool:
    *n > 0

def main() -> Result[(), AllocError | IoError | PoolMapError | PoolError | OverflowError[i64]]:
    with ThreadPoolExecutor(4, 4)? as pool:
        scores = iter(range(0, 10)).par(&pool).map(score).filter(positive).collect()?
        print(scores)?
        print(iter(range(0, 10)).par(&pool).map(score).sum()?)?
    Ok(())
[1, 4, 2, 2, 4, 1, 1, 4]
19

The pool splits the input into one contiguous chunk per worker, and each chunk runs the fused steps as one job. Results come back in input order. Every terminal returns a Result, because submitting work can fail or the pool can be shut down. Integer sum also returns OverflowError[T] for the first serial left-to-right overflowing addition, including its exact wrapped payload. It retains the mapped numbers until every job has finished, then adds them in input order; callbacks run once, not again to recover the error. This costs temporary storage proportional to the surviving items. Caller arithmetic modes do not change this public result.

A list source works the same way. iter(xs) consumes the list; iter(&xs) borrows it, yields &i64 items as a serial pipeline does, and leaves xs usable afterward. The chunks read the list in place instead of copying it.

def big(n: &i64) -> bool:
    *n > 10

def main() -> Result[(), AllocError | IoError | PoolMapError | PoolError | OverflowError[i64]]:
    with ThreadPoolExecutor(4, 4)? as pool:
        xs = [3, 14, 15, 9, 26]?
        print(iter(&xs).par(&pool).filter(big).count()?)?
        print(iter(&xs).par(&pool).sum()?)?
        print(xs)?
    Ok(())
3
67
[3, 14, 15, 9, 26]

Callbacks run concurrently. Use named functions or reusable closures that own their captures and do not mutate captured bindings. Side effects can occur in any chunk order. Put enumerate before .par: the chunk offsets carry the index. An overflowing enumeration index still terminates the process (even in a checked: caller); it is separate from a recoverable sum overflow.

The chunking is fixed by the input length and the pool’s worker count. A floating-point sum adds each chunk, then adds the chunk totals, so it can differ from a serial sum. reduce groups the same way: each chunk, then the chunk results in order. Pass chunks=N to fix the count, for example .par(&pool, chunks=8), when the same answer is needed from pools of any size. chunks has type Positive[i64]; a computed count uses Positive.try_from(count)?, which rejects zero or negative values before the pipeline starts. The reference lists the rules and error types.

Receive from either channel

select_recv waits for a message from either of two channels. Their message types may differ; match the returned Selected value to find which one arrived. Both receivers are borrowed implicitly; explicit & arguments also work.

type ChannelSelectionError = RecvError | SelectError | SendError[i64] | SendError[str] | AllocError

def main() -> Result[(), IoError | ChannelSelectionError]:
    commands, control = channel[str](1)?
    numbers, data = channel[i64](1)?
    commands.send("stop")?
    numbers.send(42)?
    match select_recv(control, data)?:
        case Selected.First(command):
            print(command)?
        case Selected.Second(number):
            print(number)?
    print(data.recv()?)?
    Ok(())
stop
42

When both channels have messages, the first takes priority. The other message stays in its queue. This can be useful for shutdown commands, but constant traffic on the first channel can starve the second.

select_recv_nowait(&first, &second) returns SelectError.Empty if neither has a message. select_recv_timeout_ms(&first, &second, milliseconds) instead waits up to that budget and may return SelectError.TimedOut.

Selection skips a drained channel whose senders have all gone away. It reports SelectError.Disconnected only when both channels are drained and disconnected. The receivers remain yours after every outcome.

Serve a TCP connection

std.net opens TCP and UDP sockets. Every call that waits for the network takes a deadline and a cancellation token, the same as the bounded I/O waits, so a worker serving a connection cannot hang forever.

import std.bytes
from std.bytes import Bytes
from std.net import NetError, Shutdown, TcpListener, connect, listen, loopback_v4
from std.sync import CancellationToken
from std.time import Deadline, deadline_after, milliseconds, monotonic_now

# Echoes one connection until the client stops sending.
def echo(listener: &TcpListener, deadline: &Deadline, cancel: &CancellationToken) -> Result[i64, AllocError | NetError | std.bytes.BytesError | OverflowError[i64]]:
    mut stream = listener.accept(deadline, cancel)?
    mut buffer = std.bytes.with_capacity(64)?
    buffer.resize(64, 0u8)?
    mut total = 0
    while True:
        count = stream.read_before(&mut buffer, 0, 64, deadline, cancel)?
        if count == 0:
            return Ok(total)
        stream.write_all_before(&buffer[:count], deadline, cancel)?
        total = total + count
    Ok(total)

type SocketOperationError = RangeError | NetError | std.bytes.BytesError

def main() -> Result[(), SpawnError[()] | IoError | AllocError | SocketOperationError | OverflowError[i64]]:
    cancel = CancellationToken()?
    deadline = deadline_after(&monotonic_now(), &milliseconds(5000))?
    mut listener = listen(&loopback_v4(0u16), 8)?
    address = listener.local_address()?
    with spawn(echo, &listener, &deadline, &cancel)? as server:
        mut stream = connect(&address, &deadline, &cancel)?
        message = std.bytes.from_text("hello")?
        stream.write_all_before(&message, &deadline, &cancel)?
        stream.shutdown(Shutdown.Write)?
        mut reply = std.bytes.with_capacity(16)?
        reply.resize(16, 0u8)?
        count = stream.read_before(&mut reply, 0, 16, &deadline, &cancel)?
        print(reply.to_text_range(0, count)?)?
        print(server.join()?)?
    listener.close()?
    match connect(&address, &deadline, &cancel):
        case Err(NetError.Refused):
            print("refused")?
        case _:
            print("unexpected")?
    Ok(())
hello
5
refused

Port 0 asks the system for a free port, and local_address reports which one it chose. shutdown(Shutdown.Write) tells the server that no more data is coming, so its read returns 0 and it returns how many bytes it echoed. After the listener is closed, nothing listens on the port, so the last attempt is NetError.Refused: an error value to handle, not a crash.

read_before makes one read, which may return fewer bytes than were sent; a real client loops until it has what it needs.

Wait on many connections

A worker per connection is simple, but a server with many idle connections can wait on all of them in one call with a selector. Each socket is registered under a key you choose, and wait reports which keys are ready:

import std.ready

import std.net

import std.bytes
from std.net import connect, listen, loopback_v4
from std.ready import Interest, selector
from std.sync import CancellationToken
from std.time import deadline_after, milliseconds, monotonic_now

type SocketOperationError = std.bytes.BytesError | std.ready.WaitError | std.net.NetError | RangeError

def main() -> Result[(), AllocError | IoError | SocketOperationError]:
    cancel = CancellationToken()?
    deadline = deadline_after(&monotonic_now(), &milliseconds(5000))?
    listener = listen(&loopback_v4(0u16), 8)?
    address = listener.local_address()?
    quiet = connect(&address, &deadline, &cancel)?
    mut chatty = connect(&address, &deadline, &cancel)?
    first = listener.accept(&deadline, &cancel)?
    mut second = listener.accept(&deadline, &cancel)?
    mut chosen = selector(8)?
    first.watch(&mut chosen, 1u64, Interest.Read)?
    second.watch(&mut chosen, 2u64, Interest.Read)?
    chatty.write_all_before(&b"x", &deadline, &cancel)?
    mut buffer = std.bytes.with_capacity(16)?
    buffer.resize(16, 0u8)?
    while True:
        count = chosen.wait(&deadline, &cancel)?
        match second.read_now(&mut buffer, 0, 16):
            case Ok(received):
                print(count, chosen.event(0)?.key)?
                print(buffer.to_text_range(0, received)?)?
                break
            case Err(std.net.NetError.WouldBlock):
                continue
            case Err(error):
                return Err(error)
    Ok(())
1 2
x

Only the second connection has input, so the wait reports one event, key 2. read_now makes a single nonblocking read without another readiness wait. If readiness has become stale, WouldBlock sends the loop back to the selector. This example has only one possible sender and one byte of input. A server dispatches each event by its key, handles zero from a nonempty read as EOF, and keeps any unsent suffix after a partial write_now. accept_now similarly accepts one queued connection or returns WouldBlock without waiting. Name lookup, UDP datagrams and IPv6 addresses are in the reference.

Encrypt a TCP connection

std.tls wraps a connected TCP stream in TLS. The client checks that the server’s certificate chains to a trust root it chose and that the certificate names the host it meant to reach. Every handshake, read, and write takes the same deadline and cancellation token as the TCP calls.

A server needs a certificate chain and its private key, as PEM text. roots_from_pem turns CA certificates into TrustRoots; real clients usually call std.tls.system_roots() instead, which reads the operating system’s bundle. This lesson embeds a test CA so that it runs offline.

import std.bytes
import std.tls
from std.net import NetError, TcpListener, connect, listen, loopback_v4
from std.sync import CancellationToken
from std.time import Deadline, deadline_after, milliseconds, monotonic_now
from std.tls import ServerConfig, TlsError

# Test-only certificates: a CA, and a leaf it signed for localhost and
# 127.0.0.1. Never use a published key outside an example.
def ca_pem() -> str:
    return "-----BEGIN CERTIFICATE-----\nMIIBmjCCAT+gAwIBAgIULOQ2Qxr8CVZY2bG0uRYKKEdMD3swCgYIKoZIzj0EAwIw\nGTEXMBUGA1UEAwwOUGxlbnR5IFRlc3QgQ0EwIBcNMjQwMTAxMDAwMDAwWhgPMjEy\nNDAxMDEwMDAwMDBaMBkxFzAVBgNVBAMMDlBsZW50eSBUZXN0IENBMFkwEwYHKoZI\nzj0CAQYIKoZIzj0DAQcDQgAEkiB3Ogehrrt6YbQghwDPv6tkhE+z9zTCVih8fvtR\nZ1/691Nfh7/J9IgaNgNSTMJjGTolzGnjaCUDqoARlFyM+6NjMGEwHQYDVR0OBBYE\nFOTFBDgWXHj/qmVJDMlAHYCEZFX5MB8GA1UdIwQYMBaAFOTFBDgWXHj/qmVJDMlA\nHYCEZFX5MA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgIEMAoGCCqGSM49\nBAMCA0kAMEYCIQD41ATQnIjKkKMLQAb9AyoyOFmljYZyz348XJEIbqMgKwIhAL3m\nmWd8On5gvkfytspatH5mDWOcSyW+au7vuIC2gHhO\n-----END CERTIFICATE-----\n"

def leaf_pem() -> str:
    return "-----BEGIN CERTIFICATE-----\nMIIBnDCCAUKgAwIBAgIBATAKBggqhkjOPQQDAjAZMRcwFQYDVQQDDA5QbGVudHkg\nVGVzdCBDQTAgFw0yNDAxMDEwMDAwMDBaGA8yMTI0MDEwMTAwMDAwMFowFDESMBAG\nA1UEAwwJbG9jYWxob3N0MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE8THNQeTi\nJ6Eq0tSxtHW5Q4TCsbIqB3MOtM98ERyKgl6kHeHCeDZYKx8B+VRGT2jkWuTeN24Q\nljwUfh7uh7Iaz6N+MHwwGgYDVR0RBBMwEYIJbG9jYWxob3N0hwR/AAABMAkGA1Ud\nEwQCMAAwEwYDVR0lBAwwCgYIKwYBBQUHAwEwHQYDVR0OBBYEFEIFC2Fyd4cfAKTx\nMiNCMdDkqDNbMB8GA1UdIwQYMBaAFOTFBDgWXHj/qmVJDMlAHYCEZFX5MAoGCCqG\nSM49BAMCA0gAMEUCIAUF3HH43lnirgSN1Vr4twZrFmXKBxlbYNQni/h4ZaxhAiEA\n5rD/yE1B4IsbEGCMlyAeX507leXfGY9+wrHaxikU/w0=\n-----END CERTIFICATE-----\n"

def leaf_key() -> str:
    return "-----BEGIN PRIVATE KEY-----\nMIGHAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBG0wawIBAQQglvSEE3H7v+mInxSF\nAyWFDKYpzpXr20wH8mfFY3bCaB2hRANCAATxMc1B5OInoSrS1LG0dblDhMKxsioH\ncw60z3wRHIqCXqQd4cJ4NlgrHwH5VEZPaORa5N43bhCWPBR+Hu6HshrP\n-----END PRIVATE KEY-----\n"

# Answers one message on one connection.
def echo(listener: &TcpListener, config: &ServerConfig, deadline: &Deadline, cancel: &CancellationToken) -> Result[i64, AllocError | TlsError | std.bytes.BytesError]:
    tcp = listener.accept(deadline, cancel).map_error(TlsError.Net)?
    mut stream = std.tls.accept(tcp, config, deadline, cancel)?
    mut buffer = std.bytes.with_capacity(64)?
    buffer.resize(64, 0u8)?
    count = stream.read_before(&mut buffer, 0, 64, deadline, cancel)?
    stream.write_all_before(&buffer[:count], deadline, cancel)?
    stream.close_before(deadline, cancel)?
    Ok(count)

type TlsConnectionError = RangeError | TlsError | NetError | std.bytes.BytesError

def main() -> Result[(), SpawnError[()] | IoError | AllocError | TlsConnectionError]:
    cancel = CancellationToken()?
    deadline = deadline_after(&monotonic_now(), &milliseconds(5000))?
    roots = std.tls.roots_from_pem(&ca_pem())?
    client = std.tls.client_config(&roots)?
    server = std.tls.server_config(&leaf_pem(), &leaf_key())?
    listener = listen(&loopback_v4(0u16), 8)?
    address = listener.local_address()?
    with spawn(echo, &listener, &server, &deadline, &cancel)? as worker:
        tcp = connect(&address, &deadline, &cancel)?
        mut stream = std.tls.connect(tcp, &client, "localhost", &deadline, &cancel)?
        message = std.bytes.from_text("hello")?
        stream.write_all_before(&message, &deadline, &cancel)?
        mut reply = std.bytes.with_capacity(16)?
        reply.resize(16, 0u8)?
        count = stream.read_before(&mut reply, 0, 16, &deadline, &cancel)?
        print(reply.to_text_range(0, count)?)?
        print(stream.read_before(&mut reply, 0, 16, &deadline, &cancel)?)?
        stream.close_before(&deadline, &cancel)?
        print(worker.join()?)?
    with spawn(echo, &listener, &server, &deadline, &cancel)? as worker:
        tcp = connect(&address, &deadline, &cancel)?
        match std.tls.connect(tcp, &client, "example.test", &deadline, &cancel):
            case Err(error):
                print(error)?
            case Ok(_):
                print("unexpected")?
        match worker.join():
            case Err(error):
                match error:
                    case tls: TlsError:
                        print(tls)?
                    case buffer: std.bytes.BytesError:
                        print(buffer)?
                    case cause: AllocError:
                        print(cause)?
            case Ok(_):
                print("unexpected")?
    Ok(())
hello
0
5
std.tls.TlsError.WrongHost
std.tls.TlsError.Rejected

std.tls.connect takes ownership of the TCP stream and returns only after the handshake, so a TlsStream is always verified. The second read returns 0 because the server sent close_notify before closing. A connection that ends without it is TlsError.Truncated, since an attacker can cut a TCP connection short.

The second client asks for example.test, which the certificate does not name. It fails with WrongHost and tells the server, whose handshake fails with Rejected. An unknown issuer is UntrustedCertificate, and an expired certificate is ExpiredCertificate.

TlsStream also has read, write, flush, and close, so it works with std.io helpers and std.http framing. Those methods wait without a deadline.

See std.tls for the full reference.

Verify clients and choose a protocol

A server can ask the client for a certificate too. This is mutual TLS: both sides prove who they are, so a service can accept only clients holding a key that its own authority signed. Build such a configuration with a builder, which gathers settings and checks them together in build.

The same builders negotiate an application protocol (ALPN), the name a client and server agree on during the handshake, such as h2 for HTTP/2.

import std.bytes
import std.tls
from std.net import NetError, TcpListener, connect, listen, loopback_v4
from std.sync import CancellationToken
from std.time import Deadline, deadline_after, milliseconds, monotonic_now
from std.tls import TlsError, ClientCertificates, ClientConfig, ServerConfig, TlsStream

type ConnectionError = AllocError | IoError | NetError | TlsError | std.bytes.BytesError

# Test-only certificates: a server CA and a localhost leaf it signed, and a
# client CA and a client certificate it signed. Never use a published key
# outside an example.
def ca_pem() -> str:
    return "-----BEGIN CERTIFICATE-----\nMIIBmTCCAT+gAwIBAgIUUResO5RQujUgwbO6wnWnB6CmmckwCgYIKoZIzj0EAwIw\nGTEXMBUGA1UEAwwOUGxlbnR5IFRlc3QgQ0EwIBcNMjQwMTAxMDAwMDAwWhgPMjEy\nNDAxMDEwMDAwMDBaMBkxFzAVBgNVBAMMDlBsZW50eSBUZXN0IENBMFkwEwYHKoZI\nzj0CAQYIKoZIzj0DAQcDQgAEnJV8Ts/yoTQC8df/IwhCPwjYuY+siUFEkwoyRG82\nFbjVeUydEKr7G3fG2hp21P46YwzRUsYal8Y/dZNwDPHKcKNjMGEwHQYDVR0OBBYE\nFIRt2tnXzddX1P2dNz1kPdQrraOuMB8GA1UdIwQYMBaAFIRt2tnXzddX1P2dNz1k\nPdQrraOuMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEGMAoGCCqGSM49\nBAMCA0gAMEUCIQCvPKJicaEPA6BwyR27QUz2f+h4RXSkJ9AX48/WuZ9K/QIgcV5e\nt2woOIP3sGU2PAopPcJVH2eC2068REEAlU7QFWc=\n-----END CERTIFICATE-----\n"

def leaf_pem() -> str:
    return "-----BEGIN CERTIFICATE-----\nMIIBnDCCAUKgAwIBAgIBATAKBggqhkjOPQQDAjAZMRcwFQYDVQQDDA5QbGVudHkg\nVGVzdCBDQTAgFw0yNDAxMDEwMDAwMDBaGA8yMTI0MDEwMTAwMDAwMFowFDESMBAG\nA1UEAwwJbG9jYWxob3N0MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEh9wzUObb\nKhx8QP/mmT2p56niQY0XpFwOiXAC8qThDYNrqtZY+f+gzGFjFamRWJBgegZ7lTWc\n3XACctwTZV7F7aN+MHwwGgYDVR0RBBMwEYIJbG9jYWxob3N0hwR/AAABMBMGA1Ud\nJQQMMAoGCCsGAQUFBwMBMAkGA1UdEwQCMAAwHQYDVR0OBBYEFP2y8OrD2DL5XVhP\ntwRYSPXCUxblMB8GA1UdIwQYMBaAFIRt2tnXzddX1P2dNz1kPdQrraOuMAoGCCqG\nSM49BAMCA0gAMEUCIQD7b3qMvbfg125pcOxRnlgi9d1kA/XO06NcKtspGn/chAIg\nRsvsbrsb0St1++9xCNovakguzYFGDNOUGEecwkoMwRw=\n-----END CERTIFICATE-----\n"

def leaf_key() -> str:
    return "-----BEGIN PRIVATE KEY-----\nMIGHAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBG0wawIBAQQgkVCAyTuCJRiqL7JD\nJEWjEGpE5Jas61PHxV76daBTJaChRANCAASH3DNQ5tsqHHxA/+aZPannqeJBjRek\nXA6JcALypOENg2uq1lj5/6DMYWMVqZFYkGB6BnuVNZzdcAJy3BNlXsXt\n-----END PRIVATE KEY-----\n"

def client_ca_pem() -> str:
    return "-----BEGIN CERTIFICATE-----\nMIIBnTCCAUOgAwIBAgIUdjiICLarpx0Q9y7yY6grXgRasaAwCgYIKoZIzj0EAwIw\nGzEZMBcGA1UEAwwQUGxlbnR5IENsaWVudCBDQTAgFw0yNDAxMDEwMDAwMDBaGA8y\nMTI0MDEwMTAwMDAwMFowGzEZMBcGA1UEAwwQUGxlbnR5IENsaWVudCBDQTBZMBMG\nByqGSM49AgEGCCqGSM49AwEHA0IABOMjuGk6yTyYHlHTSFqdxWo2pgdqU5/oUzGq\naI+Hwz342D5kZPevdjFf1uyS+wa8XAdRdGqcxoTfqMMKaaj5vw+jYzBhMB0GA1Ud\nDgQWBBS89JCv7pReUSMZUl8kRbl5orgjjjAfBgNVHSMEGDAWgBS89JCv7pReUSMZ\nUl8kRbl5orgjjjAPBgNVHRMBAf8EBTADAQH/MA4GA1UdDwEB/wQEAwIBBjAKBggq\nhkjOPQQDAgNIADBFAiEA5H1xwC1vK6Fx1Zn8HqI2Ob8wry5EdjCO3Usej/KfXG8C\nIF3oydaIMmuMWw6yVxTJ6XMLbvMf4TXIxlPt7uZwpBAc\n-----END CERTIFICATE-----\n"

def client_pem() -> str:
    return "-----BEGIN CERTIFICATE-----\nMIIBhjCCASygAwIBAgIBATAKBggqhkjOPQQDAjAbMRkwFwYDVQQDDBBQbGVudHkg\nQ2xpZW50IENBMCAXDTI0MDEwMTAwMDAwMFoYDzIxMjQwMTAxMDAwMDAwWjAYMRYw\nFAYDVQQDDA1QbGVudHkgQ2xpZW50MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE\nT5GBs10j6jCpjnrqvhM72PkArJoUAQHTk30vDaFUfzqLjJLvNnGG3mxcJ6nhmZIH\nl4K0HSop5C/aewN7v67bWaNiMGAwEwYDVR0lBAwwCgYIKwYBBQUHAwIwCQYDVR0T\nBAIwADAdBgNVHQ4EFgQUd3LfI5CS7q5ORpggbgx2rsLnrsQwHwYDVR0jBBgwFoAU\nvPSQr+6UXlEjGVJfJEW5eaK4I44wCgYIKoZIzj0EAwIDSAAwRQIgUAs5ByLBx0fx\nhNbzolmRLRLUgo0jwQJwhc3dINKcY2ECIQC5MGMpqLAhsdUm00Kdjco/wNwFiPq8\nUypR3xWb5//Lkw==\n-----END CERTIFICATE-----\n"

def client_key() -> str:
    return "-----BEGIN PRIVATE KEY-----\nMIGHAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBG0wawIBAQQgIVqyz+J6RT5Qaqpt\ndcpJwQnZR3muh8UkZ4GdOuUB1aShRANCAARPkYGzXSPqMKmOeuq+EzvY+QCsmhQB\nAdOTfS8NoVR/OouMku82cYbebFwnqeGZkgeXgrQdKinkL9p7A3u/rttZ\n-----END PRIVATE KEY-----\n"

# Reports who connected, then answers one message.
def answer(stream: &mut TlsStream, deadline: &Deadline, cancel: &CancellationToken) -> Result[(), ConnectionError]:
    match stream.peer_certificate()?:
        case Some(der):
            print(der.len() > 0)?
        case Nothing:
            print("no certificate")?
    print(stream.application_protocol()?)?
    mut buffer = std.bytes.with_capacity(64)?
    buffer.resize(64, 0u8)?
    count = stream.read_before(&mut buffer, 0, 64, deadline, cancel)?
    stream.write_all_before(&buffer[:count], deadline, cancel)?
    stream.close_before(deadline, cancel)?
    Ok(())

def greet(listener: &TcpListener, config: &ServerConfig, deadline: &Deadline, cancel: &CancellationToken) -> Result[(), ConnectionError]:
    tcp = listener.accept(deadline, cancel)?
    match std.tls.accept(tcp, config, deadline, cancel):
        case Ok(opened):
            mut stream = opened
            answer(&mut stream, deadline, cancel)?
        case Err(error):
            print(error)?
    Ok(())

# The client's outcome is printed after the server's, once the server is done.
def ask(listener: &TcpListener, server: &ServerConfig, client: &ClientConfig, deadline: &Deadline, cancel: &CancellationToken) -> Result[(), ConnectionError | SpawnError[()]]:
    address = listener.local_address()?
    mut reply = std.bytes.with_capacity(16)?
    reply.resize(16, 0u8)?
    with spawn(greet, listener, server, deadline, cancel)? as worker:
        tcp = connect(&address, deadline, cancel)?
        mut stream = std.tls.connect(tcp, client, "localhost", deadline, cancel)?
        message = std.bytes.from_text("hello")?
        stream.write_all_before(&message, deadline, cancel)?
        outcome = stream.read_before(&mut reply, 0, 16, deadline, cancel)
        worker.join()?
        match outcome:
            case Ok(count):
                print(reply.to_text_range(0, count)?)?
            case Err(error):
                print(error)?
    Ok(())

def main() -> Result[(), ConnectionError | SpawnError[()] | RangeError]:
    cancel = CancellationToken()?
    deadline = deadline_after(&monotonic_now(), &milliseconds(5000))?
    mut server = std.tls.server_builder(&leaf_pem(), &leaf_key())?
    server.verify_clients(&std.tls.roots_from_pem(&client_ca_pem())?, ClientCertificates.Required)
    server.application_protocol("greet/1")?
    server_config = server.build()?
    roots = std.tls.roots_from_pem(&ca_pem())?
    mut client = std.tls.client_builder(&roots)?
    client.certificate(&client_pem(), &client_key())?
    client.application_protocol("greet/1")?
    client_config = client.build()?
    listener = listen(&loopback_v4(0u16), 8)?
    ask(&listener, &server_config, &client_config, &deadline, &cancel)?
    ask(&listener, &server_config, &std.tls.client_config(&roots)?, &deadline, &cancel)?
    Ok(())
True
Option[str].Some("greet/1")
hello
std.tls.TlsError.CertificateRequired
std.tls.TlsError.Rejected

verify_clients takes the roots that client certificates must chain to, usually a private authority rather than the system bundle. With ClientCertificates.Optional a client may present none, and the server’s peer_certificate() is then Nothing. peer_certificate() returns the DER bytes of the verified leaf, for code that maps a certificate to an account.

The second client has no certificate. The server fails with CertificateRequired. In TLS 1.3 the client finishes its side of the handshake before the server checks it, so the client learns of the refusal at its first read, as Rejected.

A server picks the first of its protocol names that the client also offers, and application_protocol() reports it on both sides. A client that offers only names the server does not accept fails with Incompatible; one that offers none gets Nothing.

Builders also choose the minimum TLS version and the cipher suites, add revocation lists, and give a server several certificates to choose by the name the client asked for. See std.tls.

ConnectionError preserves the concrete allocation, I/O, networking, TLS, and byte-buffer errors from the connection helpers. ask adds SpawnError[()] for the scoped start, and main adds the range errors from setup.

Serve HTTP requests

std.http.server runs an HTTP/1.1 server on a std.net listener. You give it a table of routes and one handler; it accepts connections, frames requests with std.http, and stops when a cancellation token is cancelled.

Each route maps a method and a path pattern to a value of your own route type, normally an enum. The handler receives that value and matches on it.

For supervised connection heaps and event-driven handlers, see Isolate HTTP connections with actors. The worker server taught here keeps its sequential Exchange interface.

import std.http

import std.bytes
import std.http.server
import std.net
import std.time
from std.http.server import Exchange
from std.net import TcpStream
from std.sync import CancellationToken

pub enum Route:
    Home
    Hello(Greeting)

pub class Greeting:
    pub opening: str

    pub def handle(self, exchange: &mut Exchange[TcpStream]) -> Result[(), AllocError | std.http.HttpError]:
        text = f"{self.opening}{exchange.params[0]}\n"?
        exchange.send_text(200, &text)?
        Ok(())

pub class App:
    pub def handle(self, route: &Route, exchange: &mut Exchange[TcpStream]) -> Result[(), AllocError | std.http.HttpError]:
        match route:
            case Route.Home:
                exchange.send_text(200, "home\n")?
            case Route.Hello(greeting):
                greeting.handle(exchange)?
        Ok(())

type FetchError = AllocError | IoError | std.bytes.BytesError | std.net.NetError | RangeError

def get(port: u16, path: &str) -> Result[str, FetchError]:
    never = CancellationToken()?
    deadline = std.time.deadline_after(&std.time.monotonic_now(), &std.time.seconds(5u32))?
    mut stream = std.net.connect(&std.net.loopback_v4(port), &deadline, &never)?
    request = std.bytes.from_text(&f"GET {path} HTTP/1.1\r\nHost: localhost\r\nConnection: close\r\n\r\n"?)?
    stream.write_all_before(&request, &deadline, &never)?
    mut reply = std.bytes.empty()?
    mut buffer = std.bytes.empty()?
    buffer.resize(512, 0u8)?
    while True:
        count = stream.read_before(&mut buffer, 0, 512, &deadline, &never)?
        if count == 0:
            break
        reply.extend(&buffer[:count])?
    text = reply.to_text()?
    status = text.split("\r\n")?[0]
    body = text.split("\r\n\r\n")?[1].strip()?
    Ok(f"{status} | {body}"?)

def client(port: u16, stop: CancellationToken) -> Result[(), FetchError]:
    with stop.cancel_on_exit():
        print(get(port, "/")?)?
        print(get(port, "/index.html")?)?
        print(get(port, "/hello/Ada%20L")?)?
        print(get(port, "/missing")?)?
    Ok(())

type ServerSetupError = std.http.server.ServerError | std.net.NetError

def main() -> Result[(), SpawnError[()] | IoError | AllocError | ServerSetupError | FetchError]:
    mut routes = std.http.server.router[Route]()?
    routes.add("GET", "/", Route.Home)?
    routes.add("GET", "/index.html", Route.Home)?
    routes.add("GET", "/hello/{name}", Route.Hello(Greeting("hello, ")))?
    listener = std.net.listen(&std.net.loopback_v4(0u16), 16)?
    port = listener.local_address()?.port
    stop = CancellationToken()?
    with spawn(client, port, stop.share()).drop_job()? as task:
        report = std.http.server.serve(listener, &routes, &App(), &std.http.server.config(), &stop)?
        task.join()?
        print(report.requests)?
    Ok(())
HTTP/1.1 200 OK | home
HTTP/1.1 200 OK | home
HTTP/1.1 200 OK | hello, Ada L
HTTP/1.1 404 Not Found | Not Found
4

The client cancels stop when it is done, which stops the server. Its body is in with stop.cancel_on_exit():, so a failed request stops the server too, and task.join()? after serve reports that failure; see cancel on every exit. The later HTTP lessons use the same shape.

FetchError names the client’s errors, shared by get and client. ServerSetupError names this example’s server and listener errors. serve already wraps its network failures in ServerError; the separate NetError member is needed because main opens and inspects the listener directly. The alias composes with allocation, printing and thread-start errors without mapping their values.

router[Route]() makes an empty table for the route type Route. It matches the request path against each pattern in the order the routes were added. {name} captures one path segment, and the handler finds it, percent-decoded, in exchange.params. The value given to add comes back to handle as route, borrowed from the table, so the handler chooses what to do with a match. A route the handler forgets is a compile error:

import std.http
from std.http.server import Exchange
from std.net import TcpStream

pub enum Route:
    Home
    Hello

pub class App:
    pub def handle(self, route: &Route, exchange: &mut Exchange[TcpStream]) -> Result[(), std.http.HttpError]:
        match route:
            case Route.Home:
                exchange.send_text(200, "home\n")?
        Ok(())

def main() -> Result[(), AllocError | IoError]:
    pass
    Ok(())
non-exhaustive match; missing Route.Hello

A variant can carry the route’s own handler and its state, as Hello carries a Greeting here; App only forwards to it. Several patterns may share a variant, such as / and /index.html both adding Route.Home.

A path with no route gets 404 without reaching the handler, and a path whose route has another method gets 405 with an Allow field.

The client sends a Host field because HTTP/1.1 requires exactly one: a request without it, with two, or with a malformed value gets 400 before routing. The handler finds the host the request names in exchange.authority.

The handler is a pub class with a pub def handle, because serve, in another module, calls it, and Route is a pub enum, because handle’s signature names it. Every worker thread borrows the same handler and the same table, so handle takes self and route by shared borrow and cannot change them.

handle returns its own error type, here AllocError | std.http.HttpError from the f-string and send_text. The server answers an Err of any type with 500 and counts it in report.handler_errors; When a handler fails lists the other cases.

serve runs on the calling thread until stop is cancelled. Here the client thread cancels it after its last request. serve then stops accepting, lets the connections it has finish within the drain timeout, joins its workers and returns a Report of what it counted. config() gives the defaults: four workers, a queue of sixteen waiting connections, and a deadline for each phase of a request. A connection that arrives when the queue is full is answered 503 at once instead of waiting.

A worker serves one request at a time, but it does not wait for a client to send one. Between requests, and while a request head is still arriving, the connection waits on the server’s own thread, on a selector, and goes back to the queue once the client has sent a whole head. A few workers can therefore keep many idle clients open, and clients that send slowly cannot hold them all. config.idle_connections caps how many connections may wait.

HTTPS

std.http.tls.serve is serve over TLS. It takes the ServerConfig from the TLS lesson, and the handler’s exchanges carry a TlsTransport instead of a TcpStream:

import std.http.tls
from std.http.tls import TlsTransport

pub class App:
    pub def handle(self, route: &Route, exchange: &mut Exchange[TlsTransport]) -> Result[(), AllocError | std.http.HttpError]:
        ...

server = std.tls.server_config(&leaf_pem(), &leaf_key())?
report = std.http.tls.serve(listener, server, &routes, &App(), &config, &stop)?

Each connection must finish its handshake within config.handshake_timeout of being accepted, so a client that connects and says nothing is closed then. Between the client’s handshake messages the connection waits on the server’s thread too. Routing, deadlines and shutdown work as for plain HTTP.

Stop a server on a signal

A supervisor such as systemd, a container runtime or a shell stops a program with SIGTERM, or SIGINT for Ctrl-C. By default either signal ends a Plenty program at once, cutting off responses in flight. std.signal turns them into an event, so the HTTP server can drain instead:

import std.http
import std.http.server
import std.net
import std.process
import std.ready
import std.signal
from std.http.server import Exchange
from std.net import TcpStream
from std.signal import Signal
from std.sync import CancellationToken

pub enum Route:
    Home

pub class Hello:
    pub def handle(self, route: &Route, exchange: &mut Exchange[TcpStream]) -> Result[(), std.http.HttpError]:
        exchange.send_text(200, "hello\n")?
        Ok(())

# Plays the supervisor: a shell sends SIGTERM to its parent, this program.
def supervisor() -> Result[(), AllocError | IoError]:
    mut command = std.process.shell("kill -TERM $PPID")?
    command.spawn()?.wait()?
    Ok(())

type ServerShutdownError = std.signal.SignalError | std.http.server.ServerError | std.net.NetError | std.ready.WaitError

def main() -> Result[(), SpawnError[()] | IoError | AllocError | ServerShutdownError]:
    signals = std.signal.listen()?
    mut routes = std.http.server.router[Route]()?
    routes.add("GET", "/", Route.Home)?
    listener = std.net.listen(&std.net.loopback_v4(0u16), 16)?
    stop = CancellationToken()?
    with spawn(std.signal.cancel_on, signals, stop.share()).drop_job()? as watcher, stop.cancel_on_exit(), spawn(supervisor).drop_job()? as stopper:
        served = std.http.server.serve(listener, &routes, &Hello(), &std.http.server.config(), &stop)
        stop.cancel()
        stopper.join()?
        match watcher.join()?:
            case Some(Signal.Terminate):
                print("stopping on SIGTERM")?
            case Some(Signal.Interrupt):
                print("stopping on SIGINT")?
            case Nothing:
                print("stopped without a signal")?
        print(served?.requests)?
    Ok(())
stopping on SIGTERM
0

std.signal.listen() catches both signals for the whole process and returns a Signals handle on which they arrive. Call it on the main thread, before the work it should stop; a worker may not call it, and a process listens only once.

cancel_on runs on its own thread. It waits for a signal and cancels stop, which begins the server’s shutdown: serve stops accepting, finishes the requests it has within the drain timeout, and returns its Report. If serve returns for another reason, stop.cancel() ends the watcher instead, and it returns Nothing.

The watcher waits until a signal arrives or stop is cancelled, and the block joins it as it exits. stop.cancel_on_exit() directly after the watcher cancels stop before that join on every exit, including a failed spawn(supervisor).drop_job()?; see cancel on every exit.

After the first signal both signals have their default action again, so a second Ctrl-C or SIGTERM ends a slow drain at once. Closing or dropping the Signals handle restores the default action too. Child processes started by std.process keep the default action.

A program that is not a server can wait on the handle directly with signals.wait(&deadline, &cancel), or register it on a selector with signals.watch(&mut selector, key) and take the signal with signals.received().

Read forms, cookies and forwarded clients

A handler for std.http.server gets the raw request. Three modules read the common parts of it safely: std.http.body reads forms, JSON and uploads within limits, std.http.cookie reads and writes cookies, and std.http.proxy finds the client behind a reverse proxy.

import std.bytes
import std.http
import std.http.body
import std.http.cookie
import std.http.proxy
import std.http.server
import std.net
import std.time
from std.http.body import FormLimits
from std.http.proxy import Forwarding, Proxies
from std.http.server import Exchange
from std.net import TcpStream
from std.sync import CancellationToken

pub enum Route:
    Login
    Me

pub class Site:
    pub forms: FormLimits
    pub proxies: Proxies

    pub def handle(self, route: &Route, exchange: &mut Exchange[TcpStream]) -> Result[(), AllocError | std.http.HttpError | std.bytes.BytesError | std.http.cookie.CookieError | std.http.proxy.ProxyError | std.net.NetError]:
        match route:
            case Route.Login:
                match std.http.body.read_form(exchange, &self.forms):
                    case Ok(fields):
                        match std.http.body.find(&fields, "name"):
                            case Some(index):
                                name = &fields[index].value
                                mut head = std.http.response(200, "OK")?
                                std.http.cookie.add(&mut head, &std.http.cookie.set_cookie("user", name)?)?
                                head.add("Content-Type", "text/plain")?
                                mut text = f"welcome {name}"?
                                match std.http.body.find(&fields, "city"):
                                    case Some(city):
                                        text = f"{text} from {fields[city].value}"?
                                    case Nothing:
                                        pass
                                exchange.respond(head, &std.bytes.from_text(&text)?)?
                            case Nothing:
                                exchange.send_text(400, "name is required")?
                    case Err(error):
                        std.http.body.reject(exchange, &error)?
            case Route.Me:
                cookies = std.http.cookie.cookies(&exchange.request.headers, 32)?
                client = std.http.proxy.client_of(exchange, &self.proxies)?
                address = std.net.format_ip(client.address)?
                mut who = "nobody"
                match std.http.cookie.find(&cookies, "user"):
                    case Some(index):
                        who = cookies[index].value
                    case Nothing:
                        pass
                exchange.send_text(200, &f"{address} is {who}"?)?
        Ok(())

def fetch(port: u16, request: &str) -> Result[str, AllocError | IoError | std.bytes.BytesError | std.net.NetError | RangeError]:
    never = CancellationToken()?
    deadline = std.time.deadline_after(&std.time.monotonic_now(), &std.time.seconds(5u32))?
    mut stream = std.net.connect(&std.net.loopback_v4(port), &deadline, &never)?
    data = std.bytes.from_text(request)?
    stream.write_all_before(&data, &deadline, &never)?
    mut reply = std.bytes.empty()?
    mut buffer = std.bytes.empty()?
    buffer.resize(512, 0u8)?
    while True:
        count = stream.read_before(&mut buffer, 0, 512, &deadline, &never)?
        if count == 0:
            break
        reply.extend(&buffer[:count])?
    text = reply.to_text()?
    lines = text.split("\r\n")?
    mut parts = [lines[0]]?
    for line in lines:
        if line.startswith("Set-Cookie"):
            parts.append(line)?
    parts.append(text.split("\r\n\r\n")?[1].strip()?)?
    Ok(" | ".join(&parts)?)

def client(port: u16, stop: CancellationToken) -> Result[(), AllocError | IoError | RangeError | std.bytes.BytesError | std.net.NetError]:
    with stop.cancel_on_exit():
        form = "POST /login HTTP/1.1\r\nHost: localhost\r\nContent-Type: application/x-www-form-urlencoded\r\nContent-Length: 28\r\nConnection: close\r\n\r\nname=Ada&city=S%C3%A3o+Paulo"
        print(fetch(port, form)?)?
        print(fetch(port, "POST /login HTTP/1.1\r\nHost: localhost\r\nContent-Type: text/plain\r\nContent-Length: 3\r\nConnection: close\r\n\r\nAda")?)?
        print(fetch(port, "GET /me HTTP/1.1\r\nHost: localhost\r\nCookie: theme=dark; user=Ada\r\nX-Forwarded-For: 198.51.100.4, 203.0.113.7\r\nConnection: close\r\n\r\n")?)?
    Ok(())

type ProxySetupError = std.http.server.ServerError | std.http.proxy.ProxyError | std.net.NetError

def main() -> Result[(), SpawnError[()] | AllocError | ProxySetupError | IoError | RangeError | std.bytes.BytesError | std.net.NetError]:
    mut routes = std.http.server.router[Route]()?
    routes.add("POST", "/login", Route.Login)?
    routes.add("GET", "/me", Route.Me)?
    mut proxies = std.http.proxy.proxies(Forwarding.XForwardedFor)?
    proxies.trust("127.0.0.1")?
    site = Site(std.http.body.form_limits(), proxies)
    listener = std.net.listen(&std.net.loopback_v4(0u16), 16)?
    port = listener.local_address()?.port
    stop = CancellationToken()?
    with spawn(client, port, stop.share()).drop_job()? as task:
        std.http.server.serve(listener, &routes, &site, &std.http.server.config(), &stop)?
        task.join()?
    Ok(())
HTTP/1.1 200 OK | Set-Cookie: user=Ada; Path=/; Secure; HttpOnly; SameSite=Lax | welcome Ada from São Paulo
HTTP/1.1 415 Unsupported Media Type | Unsupported Media Type
HTTP/1.1 200 OK | 203.0.113.7 is Ada

read_form checks that the request says it carries a form, refuses a body over forms.max_bytes (64 KiB by default) or with more than forms.max_fields fields, and decodes each name=value pair: + is a space and %C3%A3 is the UTF-8 encoding of ã. It returns the pairs in order; find gives the index of the first with a name. When it fails, reject answers with the status that fits: 415 for the wrong type, 413 for too much, 400 for a malformed body.

set_cookie checks the name and value and gives the cookie safe defaults: Secure, so the browser sends it only over HTTPS, HttpOnly, so page scripts cannot read it, and SameSite=Lax, so other sites cannot send it with a form post. No value can add an attribute or another field: a name with a space, such as Ada L, is an error rather than a broken header, so encode free text before storing it in a cookie.

cookies reads every Cookie field. It skips pairs that are not valid cookies instead of failing, because the field carries the cookies of every application on the host.

The server sees its peer, 127.0.0.1. When a reverse proxy forwards requests, the peer is the proxy and the client’s address is in X-Forwarded-For, which anyone can write. client_of believes the header only because the program trusts 127.0.0.1, and reads it from the right: 203.0.113.7 is the first address that is not a trusted proxy, so it is the client. 198.51.100.4 came from the client itself and is ignored. With no trust call, the header is ignored and the client is the peer.

Serve files and sign cookies

std.http.files answers a request with a file from one directory, and std.http.cookie can sign a cookie so the server can tell whether a client changed it.

import std.bytes
import std.fs
import std.http
import std.http.cookie
import std.http.files
import std.io
import std.http.server
import std.net
import std.path
import std.secrets
import std.time
from std.bytes import Bytes
from std.fs import Root
from std.http.server import Exchange
from std.net import TcpStream
from std.sync import CancellationToken

pub enum Route:
    Files
    Login
    Me

pub class Site:
    pub public: Root
    pub keys: list[Bytes]

    pub def handle(self, route: &Route, exchange: &mut Exchange[TcpStream]) -> Result[(), AllocError | std.http.HttpError | std.bytes.BytesError | std.http.files.FilesError | std.http.cookie.CookieError]:
        match route:
            case Route.Files:
                path = exchange.params[0]
                std.http.files.serve(exchange, &self.public, &path)?
            case Route.Login:
                mut head = std.http.response(204, "No Content")?
                cookie = std.http.cookie.signed_cookie(&self.keys, "user", "Ada")?
                std.http.cookie.add(&mut head, &cookie)?
                exchange.respond(head, &std.bytes.empty()?)?
            case Route.Me:
                cookies = std.http.cookie.cookies(&exchange.request.headers, 32)?
                match std.http.cookie.find_signed(&cookies, &self.keys, "user")?:
                    case Some(name):
                        exchange.send_text(200, &"hello ".concat(&name)?)?
                    case Nothing:
                        exchange.send_text(401, "who are you?")?
        Ok(())

type FetchError = AllocError | IoError | std.bytes.BytesError | std.net.NetError | RangeError

def fetch(port: u16, request: &str) -> Result[str, FetchError]:
    never = CancellationToken()?
    deadline = std.time.deadline_after(&std.time.monotonic_now(), &std.time.seconds(5u32))?
    mut stream = std.net.connect(&std.net.loopback_v4(port), &deadline, &never)?
    data = std.bytes.from_text(request)?
    stream.write_all_before(&data, &deadline, &never)?
    mut reply = std.bytes.empty()?
    mut buffer = std.bytes.empty()?
    buffer.resize(512, 0u8)?
    while True:
        count = stream.read_before(&mut buffer, 0, 512, &deadline, &never)?
        if count == 0:
            break
        reply.extend(&buffer[:count])?
    Ok(reply.to_text()?)

# The value of the field `name` in a response, or "".
def field(reply: &str, name: &str) -> Result[str, AllocError | IoError]:
    for line in reply.split("\r\n")?:
        if line.startswith(name):
            return Ok(line.split(": ")?[1])
    Ok("")

def get(path: &str, extra: &str) -> Result[str, AllocError]:
    Ok(f"GET {path} HTTP/1.1\r\nHost: localhost\r\nConnection: close\r\n{extra}\r\n"?)

def client(port: u16, stop: CancellationToken) -> Result[(), FetchError]:
    with stop.cancel_on_exit():
        page = fetch(port, &get("/files/index.html", "")?)?
        print(page.split("\r\n")?[0])?
        print(field(&page, "Content-Type")?)?
        print(page.split("\r\n\r\n")?[1].strip()?)?
        tag = field(&page, "ETag")?
        again = fetch(port, &get("/files/index.html", &f"If-None-Match: {tag}\r\n"?)?)?
        print(again.split("\r\n")?[0])?
        for path in ["/files/../secret.txt", "/files/%2e%2e/secret.txt", "/files/.env"]?:
            print(fetch(port, &get(&path, "")?)?.split("\r\n")?[0])?

        login = fetch(port, "POST /login HTTP/1.1\r\nHost: localhost\r\nContent-Length: 0\r\nConnection: close\r\n\r\n")?
        cookie = field(&login, "Set-Cookie")?.split(";")?[0]
        print(cookie.startswith("user=Ada."))?
        print(fetch(port, &get("/me", &f"Cookie: {cookie}\r\n"?)?)?.split("\r\n\r\n")?[1])?
        forged = cookie.replace("Ada", "Eve")?
        print(fetch(port, &get("/me", &f"Cookie: {forged}\r\n"?)?)?.split("\r\n\r\n")?[1])?
    Ok(())

type FileSiteError = std.fs.FsError | std.io.StreamError | std.http.server.ServerError | std.secrets.SecretsError | std.net.NetError

def main() -> Result[(), SpawnError[()] | IoError | AllocError | FileSiteError | FetchError]:
    parent = std.fs.root(&std.path.from_text(".")?)?
    mut directory = std.fs.temporary_dir(&parent, &std.path.from_text("plenty-site-")?, 8)?
    for name in ["index.html", ".env"]?:
        mut staged = std.fs.temporary_file(directory.root())?
        staged.stream.file.write("<h1>Welcome</h1>\n")?
        staged.publish(&std.path.from_text(name)?)?
        staged.close()?

    mut routes = std.http.server.router[Route]()?
    routes.add("GET", "/files/{path...}", Route.Files)?
    routes.add("POST", "/login", Route.Login)?
    routes.add("GET", "/me", Route.Me)?
    site = Site(parent.open_dir(directory.name())?, [std.secrets.token_bytes(32)?]?)
    listener = std.net.listen(&std.net.loopback_v4(0u16), 16)?
    port = listener.local_address()?.port
    stop = CancellationToken()?
    with spawn(client, port, stop.share()).drop_job()? as task:
        std.http.server.serve(listener, &routes, &site, &std.http.server.config(), &stop)?
        task.join()?
    for name in ["index.html", ".env"]?:
        directory.root().remove_file(&std.path.from_text(name)?)?
    directory.cleanup()?
    Ok(())
HTTP/1.1 200 OK
text/html; charset=utf-8
<h1>Welcome</h1>
HTTP/1.1 304 Not Modified
HTTP/1.1 404 Not Found
HTTP/1.1 404 Not Found
HTTP/1.1 404 Not Found
True
hello Ada
who are you?

serve takes the path the route captured, already percent-decoded, and opens it one segment at a time below public. A segment that is empty or starts with . is refused, so .. cannot climb out and hidden files such as .env stay hidden, and a symbolic link is never followed, so a link inside the directory cannot lead out of it either. Every refusal is a plain 404, which says nothing about what lies outside.

The type comes from the file’s extension, and the response carries an ETag and Last-Modified built from the file’s size and modification time. A browser that sends the tag back in If-None-Match gets 304 Not Modified with no body. The file itself is streamed in 16 KiB pieces, so a large file does not need a large buffer.

signed_cookie appends a tag to the value: an HMAC of the cookie’s name and value under the first key, so user=Ada.<tag>. find_signed checks the tag and gives back Ada; a client that changes the value to Eve cannot compute a matching tag without the key, so the forged cookie is treated as missing. The value is still readable by the client: signing stops changes, not reading.

Keys are at least 32 random bytes, kept secret and stable across restarts; token_bytes here makes a new one per run, which signs out every client when the program restarts. To replace a key, put the new one first and keep the old one after it until old cookies have expired: find_signed accepts any listed key, and signed_cookie always signs with the first.

Send HTTP requests

std.http.client sends HTTP/1.1 requests and keeps finished connections open for the next request to the same server. Every call takes a deadline and a cancellation token, so a slow server cannot hold the caller longer than it allows.

This program runs a small server on a loopback port and sends it three requests from another thread:

import std.bytes
import std.http
import std.http.client
import std.http.server
import std.net
import std.time
from std.http.server import Exchange
from std.net import TcpStream
from std.sync import CancellationToken

pub enum Route:
    Home
    Old
    Echo

pub class Site:
    pub def handle(self, route: &Route, exchange: &mut Exchange[TcpStream]) -> Result[(), AllocError | std.http.HttpError | std.bytes.BytesError]:
        match route:
            case Route.Home:
                exchange.send_text(200, "home\n")?
            case Route.Old:
                mut head = std.http.response(301, "Moved Permanently")?
                head.add("Location", "/")?
                exchange.respond(head, &std.bytes.empty()?)?
            case Route.Echo:
                mut body = std.bytes.empty()?
                exchange.read_all(&mut body)?
                exchange.send(200, "text/plain", &body)?
        Ok(())

type VisitError = AllocError | IoError | std.bytes.BytesError | std.http.client.ClientError | RangeError

def visit(port: u16, stop: CancellationToken) -> Result[(), VisitError]:
    with stop.cancel_on_exit():
        never = CancellationToken()?
        deadline = std.time.deadline_after(&std.time.monotonic_now(), &std.time.seconds(5u32))?
        base = f"http://127.0.0.1:{port}"?
        client = std.http.client.client(std.http.client.config()?)?

        home = client.get(&f"{base}/"?, &deadline, &never)?
        print(home.status, home.body.to_text()?.strip()?)?

        moved = client.get(&f"{base}/old"?, &deadline, &never)?
        print(moved.status, moved.redirects, moved.url.endswith("/"))?

        mut post = std.http.client.request("POST", &f"{base}/echo"?)?
        post.add("Content-Type", "text/plain")?
        post.body = std.bytes.from_text("ping")?
        echoed = client.fetch(post, &deadline, &never)?
        print(echoed.body.to_text()?)?

        stats = client.stats()
        print(stats.opened, stats.reused)?
    Ok(())

def main() -> Result[(), SpawnError[()] | std.http.server.ServerError | std.net.NetError | AllocError | VisitError]:
    mut routes = std.http.server.router[Route]()?
    routes.add("GET", "/", Route.Home)?
    routes.add("GET", "/old", Route.Old)?
    routes.add("POST", "/echo", Route.Echo)?
    listener = std.net.listen(&std.net.loopback_v4(0u16), 16)?
    port = listener.local_address()?.port
    stop = CancellationToken()?
    with spawn(visit, port, stop.share()).drop_job()? as task:
        drop(std.http.server.serve(listener, &routes, &Site(), &std.http.server.config(), &stop)?)
        task.join()?
    Ok(())
200 home
200 1 True
ping
1 3

client(config()) makes a client for plain http URLs with the default limits and timeouts. get and fetch send a request, read the whole response body into body, and put the connection back in the client’s pool. All four requests here, counting the redirect, used one connection: opened is 1 and reused is 3.

The client follows the 301 itself. redirects counts the hops and url is the address that finally answered. When a redirect leaves the original server, the client drops Authorization and Cookie so that credentials meant for one server are not sent to another.

A request holds its method, URL, extra header fields and body. The client writes Host and Content-Length itself and refuses requests that try to set them.

deadline covers the whole exchange: connecting, sending, waiting for the answer and reading the body. The config adds a limit for each phase, such as head_timeout for the wait between sending a request and its response starting. If either runs out, the call returns Net(TimedOut).

A request that fails because a reused connection was closed by the server is sent again on a new connection, but only for methods that are safe to repeat, such as GET and PUT. A POST is never sent twice; its error is returned so the program can decide what to do.

For a body that should be processed incrementally, send returns a response after its head arrives. Use read_body to read chunks, or read_all_progress to append into your own buffer while retaining the count appended before an error. Both preserve the full ClientError, including cancellation and TLS failures. response.total() counts successfully read payload bytes; response.trailers() is complete after response.is_done(). Give a completed response to client.finish to allow reuse. The typed stream lesson explains how to compose it with transfers through typed_response without a stored-error side channel.

Share one HTTP client between workers

A client keeps finished connections in a pool for the next request to the same server. Every client method borrows the client shared, so jobs on an executor can use one client through Shared handles and reuse each other’s connections.

This program serves a loopback port from one thread and sends it requests from three pool jobs that share a client:

import std.http
import std.http.client
import std.http.server
import std.net
import std.time
from std.concurrent import ThreadPoolExecutor, Future, SubmitError, PoolError, FutureError
from std.http.client import Client, ClientError, Plain, TcpConnector
from std.http.server import Exchange
from std.net import TcpListener, TcpStream
from std.sync import CancellationToken

pub enum Route:
    Home

pub class Site:
    pub def handle(self, route: &Route, exchange: &mut Exchange[TcpStream]) -> Result[(), std.http.HttpError]:
        exchange.send_text(200, "home\n")?
        Ok(())

def serve(listener: TcpListener, stop: CancellationToken) -> Result[(), AllocError | std.http.server.ServerError]:
    mut routes = std.http.server.router[Route]()?
    routes.add("GET", "/", Route.Home)?
    drop(std.http.server.serve(listener, &routes, &Site(), &std.http.server.config(), &stop)?)
    Ok(())

def fetch_all(client: Shared[Client[Plain, TcpConnector]], url: str, count: i64) -> Result[i64, AllocError | ClientError | RangeError | OverflowError[i64]]:
    never = CancellationToken()?
    mut ok = 0
    for _ in range(count):
        deadline = std.time.deadline_after(&std.time.monotonic_now(), &std.time.seconds(5u32))?
        if client.get(&url, &deadline, &never)?.status == 200:
            ok += 1
    Ok(ok)

type PoolJobError = PoolError | FutureError | SubmitError[()]
type ClientRunError = OverflowError[i64] | OverflowError[u64] | AllocError | IoError | SpawnError[()] | RangeError | ClientError | std.net.NetError | std.http.server.ServerError | PoolJobError

def main() -> Result[(), ClientRunError]:
    listener = std.net.listen(&std.net.loopback_v4(0u16), 16)?
    port = listener.local_address()?.port
    stop = CancellationToken()?
    with spawn(serve, listener, stop.share()).drop_job()? as server, stop.cancel_on_exit():
        url = f"http://127.0.0.1:{port}/"?
        client = Shared(std.http.client.client(std.http.client.config()?)?)?
        mut answered = 0
        with ThreadPoolExecutor(3, 8)? as pool:
            a = pool.submit(fetch_all, client.share(), url, 10).drop_job()?
            b = pool.submit(fetch_all, client.share(), url, 10).drop_job()?
            c = pool.submit(fetch_all, client.share(), url, 10).drop_job()?
            answered = a.result()?? + b.result()?? + c.result()??
        stats = client.stats()
        print(answered, stats.opened + stats.reused, stats.opened <= 3)?
        print(stats.opened == u64(client.idle()) + stats.discarded)?
        stop.cancel()
        server.join()?
    Ok(())
30 30 True
True

stop.cancel_on_exit() stops the server however the block ends, so a failed request cannot leave the block’s join waiting for it. On success, stop.cancel() ends the server before server.join()? checks its result.

Shared(client(config)?)? moves the client into one shared allocation; client.share() gives each job its own owner, and client.get reaches the client through the handle. Thirty requests went out, and every one either opened a connection or reused one: opened + reused is 30. A job holds one connection at a time and opens one only when none is idle, and a connection one job parks is taken by whichever job asks next. Three jobs ran at once and none of their connections closed, so at most three were opened.

The client holds its pool lock only while taking a connection out or putting one back. Connecting, sending, and waiting for a response happen with the lock released, so a job stuck on a slow server does not delay the others. Each request still has its own deadline and cancellation token.

stats() and idle() work through the handle too. These observation counters saturate at u64.MAX; the small fixed workload here stays far below that limit. Before saturation, every connection the client opened is idle, in use, or counted as discarded once it has closed, whatever closed it; with no request or response holding one, opened is idle() plus discarded, as the last line shows. The pool limits max_idle and max_idle_per_origin and the idle_timeout apply to the shared pool as a whole; see the client reference.

Send HTTPS requests

std.http.https gives the HTTP client https URLs. It runs TLS on each connection, checking that the server’s certificate chains to a trusted root and names the host in the URL. Everything else is the same client: pooled connections, deadlines, redirects and replays.

This program serves HTTPS on a loopback port with std.http.tls and sends it two requests from another thread. The second streams its body from a reader instead of holding it in the request. Like the TLS lesson, it embeds a test CA so that it runs offline.

import std.bytes
import std.http
import std.http.client
import std.http.https
import std.http.server
import std.http.tls
import std.io
import std.net
import std.time
import std.tls
from std.http.server import Exchange
from std.http.tls import TlsTransport
from std.sync import CancellationToken

# Test-only certificates: a CA, and a leaf it signed for localhost and
# 127.0.0.1. Never use a published key outside an example.
def ca_pem() -> str:
    return "-----BEGIN CERTIFICATE-----\nMIIBmjCCAT+gAwIBAgIULOQ2Qxr8CVZY2bG0uRYKKEdMD3swCgYIKoZIzj0EAwIw\nGTEXMBUGA1UEAwwOUGxlbnR5IFRlc3QgQ0EwIBcNMjQwMTAxMDAwMDAwWhgPMjEy\nNDAxMDEwMDAwMDBaMBkxFzAVBgNVBAMMDlBsZW50eSBUZXN0IENBMFkwEwYHKoZI\nzj0CAQYIKoZIzj0DAQcDQgAEkiB3Ogehrrt6YbQghwDPv6tkhE+z9zTCVih8fvtR\nZ1/691Nfh7/J9IgaNgNSTMJjGTolzGnjaCUDqoARlFyM+6NjMGEwHQYDVR0OBBYE\nFOTFBDgWXHj/qmVJDMlAHYCEZFX5MB8GA1UdIwQYMBaAFOTFBDgWXHj/qmVJDMlA\nHYCEZFX5MA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgIEMAoGCCqGSM49\nBAMCA0kAMEYCIQD41ATQnIjKkKMLQAb9AyoyOFmljYZyz348XJEIbqMgKwIhAL3m\nmWd8On5gvkfytspatH5mDWOcSyW+au7vuIC2gHhO\n-----END CERTIFICATE-----\n"

def leaf_pem() -> str:
    return "-----BEGIN CERTIFICATE-----\nMIIBnDCCAUKgAwIBAgIBATAKBggqhkjOPQQDAjAZMRcwFQYDVQQDDA5QbGVudHkg\nVGVzdCBDQTAgFw0yNDAxMDEwMDAwMDBaGA8yMTI0MDEwMTAwMDAwMFowFDESMBAG\nA1UEAwwJbG9jYWxob3N0MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE8THNQeTi\nJ6Eq0tSxtHW5Q4TCsbIqB3MOtM98ERyKgl6kHeHCeDZYKx8B+VRGT2jkWuTeN24Q\nljwUfh7uh7Iaz6N+MHwwGgYDVR0RBBMwEYIJbG9jYWxob3N0hwR/AAABMAkGA1Ud\nEwQCMAAwEwYDVR0lBAwwCgYIKwYBBQUHAwEwHQYDVR0OBBYEFEIFC2Fyd4cfAKTx\nMiNCMdDkqDNbMB8GA1UdIwQYMBaAFOTFBDgWXHj/qmVJDMlAHYCEZFX5MAoGCCqG\nSM49BAMCA0gAMEUCIAUF3HH43lnirgSN1Vr4twZrFmXKBxlbYNQni/h4ZaxhAiEA\n5rD/yE1B4IsbEGCMlyAeX507leXfGY9+wrHaxikU/w0=\n-----END CERTIFICATE-----\n"

def leaf_key() -> str:
    return "-----BEGIN PRIVATE KEY-----\nMIGHAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBG0wawIBAQQglvSEE3H7v+mInxSF\nAyWFDKYpzpXr20wH8mfFY3bCaB2hRANCAATxMc1B5OInoSrS1LG0dblDhMKxsioH\ncw60z3wRHIqCXqQd4cJ4NlgrHwH5VEZPaORa5N43bhCWPBR+Hu6HshrP\n-----END PRIVATE KEY-----\n"

pub enum Route:
    Count

pub class Site:
    pub def handle(self, route: &Route, exchange: &mut Exchange[TlsTransport]) -> Result[(), AllocError | std.http.HttpError | std.bytes.BytesError]:
        mut body = std.bytes.empty()?
        exchange.read_all(&mut body)?
        exchange.send_text(200, &f"{body.len()} bytes"?)?
        Ok(())

type VisitError = AllocError | IoError | std.bytes.BytesError | std.http.client.ClientError | std.tls.TlsError | RangeError

def visit(port: u16, stop: CancellationToken) -> Result[(), VisitError]:
    with stop.cancel_on_exit():
        never = CancellationToken()?
        deadline = std.time.deadline_after(&std.time.monotonic_now(), &std.time.seconds(5u32))?
        base = f"https://localhost:{port}"?
        roots = std.tls.roots_from_pem(&ca_pem())?
        tls = std.tls.client_config(&roots)?
        client = std.http.https.client(std.http.client.config()?, tls)?

        home = client.get(&f"{base}/"?, &deadline, &never)?
        print(home.status, home.body.to_text()?)?

        mut source = std.io.cursor(std.bytes.from_text("a body read in pieces")?)
        request = std.http.client.request("PUT", &f"{base}/upload"?)?
        mut response = client.send_from(request, &mut source, &std.http.client.chunked(), &deadline, &never)?
        mut answer = std.bytes.empty()?
        response.read_all(&mut answer)?
        print(response.status, answer.to_text()?)?
        client.finish(response)

        stats = client.stats()
        print(stats.opened, stats.reused)?
    Ok(())

type HttpsSetupError = std.http.server.ServerError | std.net.NetError | std.tls.TlsError

def main() -> Result[(), SpawnError[()] | AllocError | HttpsSetupError | VisitError]:
    mut routes = std.http.server.router[Route]()?
    routes.add("GET", "/", Route.Count)?
    routes.add("PUT", "/upload", Route.Count)?
    listener = std.net.listen(&std.net.loopback_v4(0u16), 16)?
    port = listener.local_address()?.port
    stop = CancellationToken()?
    server = std.tls.server_config(&leaf_pem(), &leaf_key())?
    with spawn(visit, port, stop.share()).drop_job()? as task:
        drop(std.http.tls.serve(listener, server, &routes, &Site(), &std.http.server.config(), &stop)?)
        task.join()?
    Ok(())
200 0 bytes
200 21 bytes
1 1

std.http.https.client takes the client config and a TLS ClientConfig. The client uses the URL’s host, here localhost, as the name the certificate must carry. Verification cannot be turned off: to trust a private server, build the ClientConfig from its CA as this program does, and use std.http.https.system_client for public servers. A certificate from an unknown CA fails with Tls(UntrustedCertificate), and one for another host with Tls(WrongHost). Both requests used one TLS connection.

send_from reads the body from any std.io reader in 16 KiB pieces, so a large upload never sits in memory whole. chunked() sends it in chunks of unknown total length; sized(n) sends Content-Length: n and fails with WrongBodyLength if the reader yields a different amount. Setting expect_continue on the upload asks the server to approve the request before the body is sent.

A streamed body can be read only once. The client still replays a request on a fresh connection when a pooled one was closed before any of the body was read, but it returns a redirect that would need the body again rather than following it.

With config.proxy set, an https request asks the proxy to open a tunnel with CONNECT, then runs TLS to the server through it, so the proxy sees neither the request nor the response.

The HTTPS support is a separate module: a program that imports only std.http.client does not include the TLS library.

Isolate HTTP connections with actors

std.actor.http.server gives each connection its own handler and heap. A contained trap loses that connection; other connections can continue. Its qualification record states which deployment checks have passed.

Start with a complete, bounded request handler. This endpoint counts requests on one connection. The ordinary HTTP client runs on a separate native thread; blocking client operations must not run inside an actor callback.

import std.actor.http.server
import std.bytes
import std.http
import std.http.client
import std.net
import std.time
from std.actor.http.server import Request, Response, ServerError
from std.sync import CancellationToken

type HandlerError = OverflowError[i64] | AllocError | std.http.HttpError | ServerError

pub class Counter:
    pub requests: i64

    pub def handle(self: &mut Counter, request: Request, response: &mut Response) -> Result[(), HandlerError]:
        if request.target.path != "/":
            response.send_text(404, "Not Found")?
            return Ok(())
        if request.head.method != "GET" and request.head.method != "HEAD":
            mut head = std.http.response(405, "Method Not Allowed")?
            head.add("Allow", "GET, HEAD")?
            response.respond(head, b"")?
            return Ok(())
        self.requests += 1
        response.send_text(200, &str.from(self.requests)?)?
        Ok(())

def make_counter() -> Result[Counter, HandlerError]:
    Ok(Counter(0))

type VisitError = AllocError | IoError | std.bytes.BytesError | std.http.client.ClientError | RangeError

def visit(port: u16, stop: CancellationToken) -> Result[(), VisitError]:
    with stop.cancel_on_exit():
        client = std.http.client.client(std.http.client.config()?)?
        url = f"http://127.0.0.1:{port}/"?
        deadline = std.time.deadline_after(&std.time.monotonic_now(), &std.time.seconds(5u32))?
        never = CancellationToken()?
        for _ in range(2):
            reply = client.get(&url, &deadline, &never)?
            print(reply.status, reply.body.to_text()?)?
    Ok(())

def main() -> Result[(), SpawnError[()] | std.net.NetError | HandlerError | VisitError]:
    listener = std.net.listen(&std.net.loopback_v4(0u16), 16)?
    port = listener.local_address()?.port
    stop = CancellationToken()?
    with spawn(visit, port, stop.share()).drop_job()? as task:
        report = std.actor.http.server.serve(listener, make_counter, &std.actor.http.server.config(), &stop)?
        task.join()?
        print(report.requests)?
    Ok(())
200 1
200 2
2

make_counter runs inside the new connection’s heap. Two connections get different counters; the client here reuses one connection. The mutable handle receiver updates only that connection’s state. Store longer-lived text as owned Bytes; the request’s str fields are available during the callback but cannot become fields of the actor’s handler.

The handler checks the method as well as the path, includes Allow in its 405 response, and propagates response errors. HEAD follows the GET handler but does not send body bytes. The client cancels the server even if a request fails, and task.join()? preserves that failure rather than printing success.

The complete request body is bounded by protocol limits and the actor heap. A whole response must fit the configured output queue. Use the event interface for larger streams or multi-step nonblocking work: retain progress, handle short writes, and resume when output becomes writable.

Use the worker server when sequential handlers and its routing/application helpers suit the service. Use the actor server when connection isolation justifies an event-driven application model. Neither choice turns an infinite loop, stack overflow or foreign-library failure into a safely recoverable request; consult the reliability boundary.

Talk over WebSockets

A WebSocket starts as an HTTP request and then stays open, carrying messages both ways. std.http.websocket does both halves: accept turns a server exchange into a WebSocket, and connect opens one as a client.

This program serves /chat on a loopback port. The handler echoes each text message in upper case until the client closes. The client, on another thread, sends two messages and then closes the connection.

import std.time
import std.bytes
import std.http.server
import std.http.websocket
import std.net
from std.http.server import Exchange
from std.http.websocket import Kind, Message, WebSocketError
from std.net import TcpStream
from std.sync import CancellationToken
from std.time import Deadline

type ChatError = WebSocketError | std.bytes.BytesError | RangeError | AllocError

def soon() -> Result[Deadline, RangeError]:
    Ok(std.time.deadline_after(&std.time.monotonic_now(), &std.time.seconds(5u32))?)

pub enum Route:
    Chat

pub class Echo:
    pub def handle(self, route: &Route, exchange: &mut Exchange[TcpStream]) -> Result[(), ChatError]:
        wait_cancel = exchange.cancel_token()
        mut socket = std.http.websocket.accept(exchange, &std.http.websocket.config())?
        mut message = std.bytes.empty()?
        while True:
            message.clear()
            match socket.receive(&mut message, &soon()?, &wait_cancel)?:
                case Message.Text:
                    shout = message.to_text()?.upper()?
                    socket.send_text(&shout, &soon()?, &wait_cancel)?
                case Message.Binary:
                    socket.close(1003, "text only", &soon()?, &wait_cancel)?
                    return Ok(())
                case Message.Close(code):
                    return Ok(())
        Ok(())

def chat(port: u16, stop: CancellationToken) -> Result[(), ChatError | IoError]:
    with stop.cancel_on_exit():
        never = CancellationToken()?
        url = f"ws://localhost:{port}/chat"?
        mut socket = std.http.websocket.connect(&url, &std.http.websocket.config(), &soon()?, &never)?
        mut reply = std.bytes.empty()?
        for line in ["hello", "still there?"]?:
            socket.send_text(&line, &soon()?, &never)?
            reply.clear()
            match socket.receive(&mut reply, &soon()?, &never)?:
                case Message.Text:
                    print(reply.to_text()?)?
                case _ as other:
                    print(other)?
        socket.close(1000, "bye", &soon()?, &never)?
    Ok(())

def main() -> Result[(), SpawnError[()] | AllocError | std.http.server.ServerError | std.net.NetError | ChatError | IoError]:
    mut routes = std.http.server.router[Route]()?
    routes.add("GET", "/chat", Route.Chat)?
    listener = std.net.listen(&std.net.loopback_v4(0u16), 16)?
    port = listener.local_address()?.port
    stop = CancellationToken()?
    with spawn(chat, port, stop.share()).drop_job()? as task:
        drop(std.http.server.serve(listener, &routes, &Echo(), &std.http.server.config(), &stop)?)
        task.join()?
    Ok(())
HELLO
STILL THERE?

receive appends one whole message to a buffer you own and says what kind it was. A message sent in several frames arrives joined; pings are answered on the way. Config bounds each frame and each message, 64 KiB and 1 MiB by default, and a peer that sends more is closed with code 1009 and the error TooLarge. Text that is not UTF-8 closes the connection with 1007.

Every call takes a deadline. A handler waiting in receive also wakes when the server begins to shut down, with Stopping; it should then close with 1001, “going away”. Each open WebSocket holds a server worker until its handler returns, so set config.workers for the connections you expect.

close sends a close code and reason, waits for the other side’s close and then shuts the connection down. When the other side closes first, receive answers for you and returns Close with its code.

For wss URLs, std.http.wss.connect takes a TLS ClientConfig as the HTTPS client does; on the server, accept works the same on exchanges from std.http.tls.serve.

Stream events to browsers

Server-Sent Events keep one HTTP response open and send it a line-based event each time something happens. Browsers read them with EventSource, which reconnects by itself when the connection drops. std.http.sse turns a server exchange into an event stream.

This program serves /clock on a loopback port. The handler sends three tick events a short interval apart, then an end event, and closes the stream. The client, on another thread, reads the whole response with the HTTP client and prints it without its final blank line.

import std.time
import std.bytes
import std.http
import std.http.client
import std.http.server
import std.http.sse
import std.net
from std.http.server import Exchange
from std.http.sse import SseError
from std.net import TcpStream
from std.sync import CancellationToken
from std.time import Deadline

def after(milliseconds: u32) -> Result[Deadline, RangeError]:
    Ok(std.time.deadline_after(&std.time.monotonic_now(), &std.time.milliseconds(milliseconds))?)

pub enum Route:
    Clock

pub class Clock:
    pub def handle(self, route: &Route, exchange: &mut Exchange[TcpStream]) -> Result[(), SseError | AllocError | RangeError]:
        wait_cancel = exchange.cancel_token()
        mut events = std.http.sse.accept(exchange)?
        events.retry(&std.time.milliseconds(2000u32), &after(1000)?, &wait_cancel)?
        for n in range(3):
            match events.wait(&after(20)?, &wait_cancel):
                case Ok(_):
                    pass
                case Err(SseError.Stopping):
                    events.send("server stopping", &after(1000)?, &wait_cancel, event="end")?
                    return Ok(())
                case Err(error):
                    return Err(error)
            id = f"{n}"?
            events.send(&f"tick {n}\nall well"?, &after(1000)?, &wait_cancel, event="tick", id=Some(id))?
        events.send("done", &after(1000)?, &wait_cancel, event="end")?
        events.close()
        Ok(())

type WatchError = AllocError | IoError | std.bytes.BytesError | std.http.client.ClientError | RangeError

def watch(port: u16, stop: CancellationToken) -> Result[(), WatchError]:
    with stop.cancel_on_exit():
        never = CancellationToken()?
        client = std.http.client.client(std.http.client.config()?)?
        reply = client.get(&f"http://127.0.0.1:{port}/clock"?, &after(5000)?, &never)?
        match std.http.find(&reply.head.headers, "content-type"):
            case Some(index):
                print(reply.status, reply.head.headers[index].value)?
            case Nothing:
                print(reply.status)?
        print(reply.body.to_text()?.strip()?)?
    Ok(())

def main() -> Result[(), SpawnError[()] | std.http.server.ServerError | std.net.NetError | AllocError | WatchError]:
    mut routes = std.http.server.router[Route]()?
    routes.add("GET", "/clock", Route.Clock)?
    listener = std.net.listen(&std.net.loopback_v4(0u16), 16)?
    port = listener.local_address()?.port
    stop = CancellationToken()?
    with spawn(watch, port, stop.share()).drop_job()? as task:
        drop(std.http.server.serve(listener, &routes, &Clock(), &std.http.server.config(), &stop)?)
        task.join()?
    Ok(())
200 text/event-stream
retry: 2000

event: tick
id: 0
data: tick 0
data: all well

event: tick
id: 1
data: tick 1
data: all well

event: tick
id: 2
data: tick 2
data: all well

event: end
data: done

Each event is an optional event name, an optional id, and one data line for each line of its text, ended by an empty line. A browser’s EventSource joins the data lines back together and calls the listener for the event’s name, or message when it has none. retry tells it how long to wait before reconnecting. Every call writes and flushes whole events before it returns, within its deadline.

The response has no length: it ends when the handler closes the stream, so it is not cut off by the server’s response_timeout. A client that reconnects sends the last id it received, which the handler reads as events.last_event_id to resume after it.

wait paces the stream. It returns Ok at its deadline, Disconnected as soon as the client goes away, and Stopping when the server begins to shut down; the stream is still open then, so the handler can say goodbye. A send to a client that has gone away also fails with Disconnected. Each open stream holds a server worker until its handler returns, so set config.workers for the streams you expect.

Wait for I/O with a deadline

A worker can own or borrow a File like any other value. To keep it from blocking forever on a pipe that never delivers, wait with std.ready: every wait has a deadline, and a cancellable wait checks its token at least every 10 milliseconds.

import std.bytes
import std.io
import std.process
from std.io import FileStream
from std.process import Stdio
from std.ready import WaitError, read_before
from std.sync import CancellationToken
from std.time import Deadline, deadline_after, milliseconds, monotonic_now, sleep

def first_bytes(stream: &mut FileStream, deadline: &Deadline, stop: &CancellationToken) -> Result[str, std.bytes.BytesError | AllocError]:
    mut buffer = std.bytes.with_capacity(16)?
    buffer.resize(16, 0u8)?
    text = match read_before(stream, &mut buffer, 0, 16, deadline, stop):
        case Ok(count):
            str.from(count)?
        case Err(WaitError.TimedOut):
            "timed out"
        case Err(WaitError.Cancelled):
            "cancelled"
        case Err(WaitError.Stream(_)):
            "failed"
    Ok(text)

type PipeWaitError = str | RangeError | std.bytes.BytesError

def main() -> Result[(), SpawnError[()] | IoError | AllocError | PipeWaitError]:
    mut command = std.process.shell("sleep 10")?
    command.stdout(Stdio.Piped)
    mut child = command.spawn()?
    pipe: Result[File, str] = child.take_stdout().ok_or("missing piped stdout")
    mut quiet = std.io.from_file(pipe?)
    never = CancellationToken()?
    soon = deadline_after(&monotonic_now(), &milliseconds(20))?
    with spawn(first_bytes, &mut quiet, &soon, &never)? as task:
        print(task.join()?)?
    stop = CancellationToken()?
    later = deadline_after(&monotonic_now(), &milliseconds(60000))?
    observer = stop.share()
    with spawn(first_bytes, &mut quiet, &later, &observer)? as task:
        sleep(&milliseconds(20))
        stop.cancel()
        print(task.join()?)?
    child.kill()?
    child.wait()?
    Ok(())
timed out
cancelled

The child writes nothing, so the first worker’s 20-millisecond deadline passes. The second worker has a minute, but the parent cancels it, and the cancellation wakes its wait at once. Either way the worker returns normally and the scope joins it; the pipe and buffer are unchanged.

A worker may not change the working directory, because every thread resolves relative paths against it:

import std.env
import std.path

def worker() -> Result[(), AllocError | IoError]:
    std.env.set_current_dir(&std.path.from_text("/")?)

def main() -> Result[(), SpawnError[()]]:
    with spawn(worker).drop_job()?:
        pass
    Ok(())
changing the working directory affects every thread's relative paths

Change it on the main thread before starting workers, or give a child its own directory with Command.current_dir. See native effects on worker threads for the rules each native operation follows.

Working with C libraries

These lessons require core modules, text, borrowing, resource owners, and Result. They are independent of the generic, callback, and concurrency paths.

Call C libraries

Declare C functions in a .plentyi interface and import it as a module. C’s abs uses int, which is i32 on Plenty’s supported target. The wrapper rejects the minimum integer, which C cannot negate.

Save this as c_math.plentyi:

extern def abs_raw(value: i32) -> i32 = "abs"

pub def absolute(value: i32) -> Option[i32]:
    if value == -2147483648i32:
        return Nothing
    Some(abs_raw(value))
import c_math

def main() -> Result[(), IoError]:
    print(c_math.absolute(-42i32))?
    print(c_math.absolute(-2147483648i32))?
    Ok(())
Option[i32].Some(42)
Option[i32].Nothing

The C standard library is already linked. For another library, pass its archive or driver options explicitly, for example --link-arg /path/to/libexample.a or --link-arg -L/path/to/libs --link-arg -lexample. A shared library also needs to be discoverable by the system loader when the application runs.

The interface author must check the C header and document pointer lifetimes, ownership, and errors. Plenty does not verify the native implementation; application code should use typed wrappers. pub exposes a name to Plenty imports; it does not export a C symbol. See the C interface reference for the supported ABI.

Pass text to C

Choose the text adapter to match C’s signature. as utf8 lends a pointer and byte length without allocating. as c_string makes a terminated copy and reports embedded NULs or allocation failure through CStrError.

Save this interface as c_text.plentyi:

# strlen reads only during the call and does not retain or return its input.
# C signature: size_t strlen(const char *text).
pub extern def byte_length(text: &str as c_string) -> Result[u64, CStrError] = "strlen"
import c_text

def main() -> Result[(), AllocError | IoError | CStrError]:
    text = "é"
    print(c_text.byte_length(&text)?)?
    with_nul = "a\0b"
    match c_text.byte_length(&with_nul):
        case Ok(size):
            print(size)?
        case Err(error):
            print(error)?
    print(text)?
    Ok(())
2
CStrError.EmbeddedNul
é

The C call returns a plain size_t, which is u64 on this target. The generated adapter returns Result[u64, CStrError] because preparing the argument may fail. The temporary buffer is freed after the call; the original text is unchanged. CStrError.Allocation(error) reports a failed buffer allocation. These conversion errors do not replace any error status that the C function itself returns.

An interface for uint64_t checksum(const uint8_t *data, size_t size) would instead declare text: &str as utf8. It receives all bytes, including embedded NULs, and needs no temporary copy. Neither adapter lets C retain or modify the buffer. Use a binding with an explicit ownership contract for those cases.

Own a foreign handle

Own a C resource with a private opaque pointer and a destructor. This interface wraps a temporary C file, keeping raw operations private and reporting acquisition failure through its public factory.

Save this as c_scratch.plentyi:

opaque RawFile
extern def create_raw() -> RawFile = "tmpfile"
extern def close_raw(file: RawFile) -> i32 = "fclose"
extern def position_raw(file: RawFile) -> i64 = "ftell"

# fwrite borrows the bytes and file only for the call; it retains neither.
# The adapter expands to (data, byte_len), followed by count and file.
extern def write_raw(text: &str as utf8, count: u64, file: RawFile) -> u64 = "fwrite"

pub enum ScratchError:
    Create
    Write
    Position

pub class Scratch:
    raw: RawFile
    def __init__(self) -> ():
        self.raw = RawFile.null()
    def __del__(self) -> ():
        if not self.raw.is_null():
            drop(close_raw(self.raw))
    pub def write(self: &mut Scratch, text: &str) -> Result[(), ScratchError]:
        if len(text) == 0:
            return Ok(())
        if write_raw(text, 1u64, self.raw) != 1u64:
            return Err(ScratchError.Write)
        Ok(())
    pub def position(self: &mut Scratch) -> Result[i64, ScratchError]:
        value = position_raw(self.raw)
        if value < 0:
            return Err(ScratchError.Position)
        Ok(value)

pub def create() -> Result[Scratch, ScratchError]:
    mut owner = Scratch()
    owner.raw = create_raw()
    if owner.raw.is_null():
        return Err(ScratchError.Create)
    Ok(owner)
import c_scratch

def main() -> Result[(), c_scratch.ScratchError | IoError]:
    mut file = c_scratch.create()?
    text = "é\0A"
    file.write(&text)?
    print(file.position()?)?
    Ok(())
4

Initialize the owner to null before acquiring the resource, so cleanup is valid even if acquisition fails. After success, the destructor closes the file through its originating library, including on early exit. The owner cannot be copied.

The destructor discards close errors. Offer an explicit fallible close method when callers need to handle them. Native ownership transfer needs its own wrapper: clear the field when C consumes the handle, and retain or return the owner when C leaves ownership with the caller.

Build a C library

Use export def to name the C symbol for a function. This calc.plenty module can also be imported and called by an ordinary Plenty program:

pub enum CalcError:
    Overflow

pub export def add(a: i32, b: i32) -> Result[i32, CalcError] = "calc_add":
    "Add two checked integers."
    result = checked:
        a + b
    result.map_error(def(error: OverflowError[i32]) -> CalcError: CalcError.Overflow)

pub export def increment(value: &mut i32) -> Result[(), CalcError] = "calc_increment":
    result = checked:
        *value + 1
    *value = result.map_error(def(error: OverflowError[i32]) -> CalcError: CalcError.Overflow)?
    Ok(())
import calc

def main() -> Result[(), IoError | calc.CalcError]:
    print(calc.add(20, 22)?)?
    mut value = 10i32
    calc.increment(&mut value)?
    print(value)?
    mut maximum = i32.MAX
    match calc.increment(&mut maximum):
        case Err(calc.CalcError.Overflow):
            print("overflow", maximum == i32.MAX)?
        case Ok(_):
            print("unexpected success")?
    Ok(())
42
11
overflow True

Arithmetic errors are mapped locally to a payload-free error enum supported by the C ABI. increment computes before assigning, so overflow leaves its borrowed value unchanged. Builtin OverflowError[T] carries a payload and cannot be exported directly. See exported error contracts.

pub makes the function accessible through source imports. export independently creates a C entry point in a library build. Libraries do not need main.

Save the module as calc.plenty, create a build directory, and run:

plenty --shared-library calc.plenty --library-name calc -o build/libcalc.so

The output includes calc.h, with a status-returning calc_add declaration and generated contract comments, and calc.plentyi, which Plenty consumers can import. Copy that generated interface into the consuming project’s source root and link libcalc.so using --link-arg; it replaces the source module there. C consumers include calc.h and link the library.

If you only have the binary, recover its Plenty interface without loading it:

plenty --extract-interface build/libcalc.so --library-name calc -o calc.plentyi
plenty --verify-interface build/libcalc.so calc.plentyi

For a static archive, use --static-library instead. Pass the native arguments listed in calc.link-args.txt after the archive when linking a C program.

For C callers, increment takes an int32_t * to an initialized value, borrowed exclusively for the call. Plenty callers get this borrowing contract from the generated interface.

Exports support numeric scalars, borrows, unit, supported Result values, and owned class handles. Read the generated header’s requirements even for simple APIs: runtime traps can terminate the process, and current libraries require serialized calls on one caller thread. See the library reference for supported export types and packaging details.

Report errors through an exported function

Exports can return a numeric error through Result. Plenty callers use the ordinary result; C callers receive a status and separate output pointers:

export def positive(value: i32) -> Result[i32, i32] = "calc_positive":
    if value < 0:
        Err(1)
    else:
        Ok(value)

def main() -> Result[(), AllocError | IoError | i32]:
    print(positive(42)?)?
    print(str.repr(positive(-1))?)?
    Ok(())
42
Result[i32, i32].Err(1)

The generated C declaration is uint32_t calc_positive(int32_t p0, int32_t *out_ok, int32_t *out_error). Status 0 writes out_ok; status 1 writes out_error. Its header explains output storage and borrowing requirements.

Name errors with an enum

A numeric code means little to a C caller. Return an enum whose variants have no payloads instead:

enum CalcError:
    Empty
    Negative

export def checked(value: i32) -> Result[i32, CalcError] = "calc_checked":
    if value == 0:
        Err(CalcError.Empty)
    elif value < 0:
        Err(CalcError.Negative)
    else:
        Ok(value)

def main() -> Result[(), AllocError | IoError | CalcError]:
    print(checked(42)?)?
    print(str.repr(checked(-1))?)?
    Ok(())
42
Result[i32, CalcError].Err(CalcError.Negative)

C receives a uint32_t code, the variant’s position in the declaration. The header names each code:

#define calc_CalcError_Empty UINT32_C(0)
#define calc_CalcError_Negative UINT32_C(1)
uint32_t calc_checked(int32_t p0, int32_t *out_ok, uint32_t *out_error);

The generated calc.plentyi declares its own CalcError, so Plenty consumers match calc.CalcError variants. Enums with payloads and sums such as AllocError | CalcError cannot cross the C boundary.

Export owned objects

C receives an exported class as an opaque handle. The generated header names its destroy function and ownership rules.

pub enum CalcError:
    Overflow

pub class Counter:
    value: i64
    def __del__(self) -> ():
        drop(print(self.value))

pub export def create(value: i64) -> Result[Counter, AllocError] = "counter_create":
    Ok(Counter(value))

pub export def read(owner: &Counter) -> i64 = "counter_read":
    owner.value

pub export def increment(owner: &mut Counter) -> Result[(), CalcError] = "counter_increment":
    result = checked:
        owner.value + 1
    owner.value = result.map_error(def(error: OverflowError[i64]) -> CalcError: CalcError.Overflow)?
    Ok(())

pub export def finish(owner: Counter) -> () = "counter_finish":
    drop(owner)
import counter

def main() -> Result[(), AllocError | IoError | counter.CalcError]:
    mut owner = counter.create(42)?
    counter.increment(&mut owner)?
    print(counter.read(&owner))?
    print("ready")?
    counter.finish(owner)
    Ok(())
43
ready
43

Build counter.plenty with --shared-library --library-name counter to get a C factory returning a status and writing a counter_Counter * output on success. A class returned to C must be declared as Result[Class, AllocError], because allocating its handle can fail. The library allocates the handle before running the function, so a failure has no other effects. Release the output exactly once with counter_Counter_destroy.

counter_read borrows a const counter_Counter *; counter_increment borrows a counter_Counter * exclusively. Neither transfers ownership.

counter_finish consumes its handle. A C caller must not use or destroy that handle again; a Plenty caller gets a moved-value error if it tries to reuse owner. A consuming function keeps this rule even if it returns an error.

Plenty consumers get automatic cleanup through the generated counter.plentyi. Keep the originating library loaded for every owner’s lifetime.

The lifecycle trace uses drop(print(...)) deliberately: cleanup cannot return an error, and a failed diagnostic write must not interrupt cleanup. For output that must succeed, provide an explicit fallible operation and propagate its result before leaving the scope.

The fallible increment maps arithmetic overflow to the payload-free CalcError.Overflow at the ABI boundary. It leaves the object unchanged on failure; callers must handle its status before continuing.

Load a library at runtime

A program can choose a native library while it runs. Plenty still compiles the function signatures ahead of time. Start with a small library, saved as counter.plenty:

enum CalcError:
    Overflow

class Counter:
    value: i64
    def __del__(self) -> ():
        drop(print("released"))

export def create(value: i64) -> Result[Counter, AllocError] = "counter_create":
    Ok(Counter(value))

export def read(counter: &Counter) -> i64 = "counter_read":
    counter.value

export def increment(counter: &mut Counter) -> Result[(), CalcError] = "counter_increment":
    result = checked:
        counter.value + 1
    counter.value = result.map_error(def(error: OverflowError[i64]) -> CalcError: CalcError.Overflow)?
    Ok(())

Build the shared library and its runtime-loading module:

plenty --shared-library counter.plenty --library-name counter -o libcounter.so
plenty --runtime-interface counter.plenty --library-name counter -o plugin.plentyi

Now save this consumer beside plugin.plentyi and run it from the directory containing libcounter.so. No library link arguments are needed.

import plugin

def main() -> Result[(), AllocError | IoError | LoadError | plugin.CalcError]:
    path = "./libcounter.so"
    library = plugin.load(&path)?
    mut counter = library.create(42)?
    library.increment(&mut counter)?
    print(library.read(&counter))?
    mut maximum = library.create(i64.MAX)?
    match library.increment(&mut maximum):
        case Err(plugin.CalcError.Overflow):
            print("overflow", library.read(&maximum) == i64.MAX)?
        case Ok(_):
            print("unexpected success")?
    drop(maximum)
    drop(library)
    drop(counter)

    missing = "./missing-library.so"
    match plugin.load(&missing):
        case Ok(_):
            print("unexpected library")?
        case Err(error):
            print(error)?
    Ok(())
43
overflow True
released
released
LoadError.OpenFailed

Loading checks the library’s exact interface contract and every required export. A changed contract reports LoadError.IncompatibleContract; a missing export reports LoadError.MissingSymbol. Imports alone load nothing.

The loaded methods preserve the exports’ signatures, borrows, and errors. Dropping library does not invalidate its objects: counter still uses the matching destructor, and native code stays loaded until process exit.

Keep objects with the library instance that created them. Passing an object to a different loaded instance stops with a diagnostic before making the native call. Loading trusts native code: metadata checks compatibility, and library constructors may run before a later check reports an error.

The lifecycle trace uses drop(print(...)) deliberately: cleanup cannot return an error, and a failed diagnostic write must not interrupt cleanup. For output that must succeed, provide an explicit fallible operation and propagate its result before leaving the scope.

The maximum-value object exercises the loaded error-status path and is released separately. The fallible increment maps arithmetic overflow to the payload-free CalcError.Overflow at the ABI boundary. It leaves the object unchanged on failure; callers must handle its status before continuing.

Keep exploring

Choose a small program of your own: a report, a file converter, or a command-line checker. Start with the core features and add an optional path when you need it.

Use the reference for exact contracts and the implementation status to check feature support. The backlog records planned work.

Reference

The reference describes implemented behavior and current restrictions. Start with the language overview for the connecting ideas, or use the support index to find a contract.

Learn Plenty teaches the language through executable examples. Supplementary examples cover additional operations. Design alternatives live in Proposals; future work lives in the backlog.

Language contract

These pages define source syntax and what programs mean. Read the overview, then follow source and modules, types, expressions, ownership, functions, generics, errors and resource scopes, and iteration.

Operation signatures and their effects belong to built-in facilities; compiler/runtime mechanisms belong to implementation.

Language overview

Plenty is a statically typed, ahead-of-time compiled language with Python-shaped syntax and Rust-influenced behaviours. Its design favors fast compilation, explicit interfaces, and predictable memory use.

Every function declares its parameter and return types. Local expressions can infer types, while generics reuse definitions at concrete types. Classes and enums have fixed declarations rather than dynamic attributes or inheritance.

Ownership makes copies, moves, shared access, and exclusive mutation explicit. Owned resources have deterministic cleanup. Recoverable errors are values; operations that allocate can report allocation failure without exceptions.

The language contract describes source organization, types, expressions, callable values, and iteration. Built-in operation contracts belong to facilities; building and executing programs belongs to the toolchain.

Source files and declarations

Spaces delimit indented suites. Tabs are rejected, and indentation must return to an existing level. Blank lines and comment-only lines do not affect indentation. Parentheses permit multiline expressions. Comments start with #.

Identifiers contain ASCII letters, digits, and underscores and cannot start with a digit. Keywords and names beginning __plenty_ are reserved. Diagnostics use one-based line and column positions.

Strings accept single, double, or triple quotes; triple quotes permit physical newlines. The first standalone string in an indented function body is its documentation. Use return "text" when that first string should be the returned value. A b prefix makes a byte literal, b"...", which holds ASCII characters and \xHH escapes. Strings accept the escapes \0, \n, \r, \t, \\, \', \", and \u{H...} for a scalar by code point. Integer literals are decimal, hexadecimal (0xFF), octal (0o17), or binary (0b1010), with optional _ digit separators and width suffixes.

Module scope contains functions, classes, enums, protocols, type aliases, constants, static assertions, imports, and target conditionals. Executable statements and mutable bindings belong inside functions; there are no global variables. Named functions belong at module scope; classes and enums can declare methods, and protocols declare method requirements. Anonymous functions can appear in function bodies; see functions.

A line holding only @name annotates the declaration below it. The set of annotations is closed and compiler-known: @copy and @newtype on classes, and @redacted on classes and enums. See annotations.

Names follow lexical scope and prelude rules. Modules use explicit imports, with visibility controlled by pub. An application has a main entrypoint.

Keywords

A keyword cannot name a binding, parameter, field, function, type, or module. These keywords have a meaning in Plenty:

and, as, break, case, class, const, continue, def, dimension, elif, else, enum, export, extern, False, for, from, if, import, in, match, mut, not, opaque, or, pass, protocol, pub, return, True, type, while, with, yield

These are reserved but have no meaning, so a Python or Rust habit is rejected instead of becoming a name:

async, await, lambda, None, struct, trait, try

None, try, and lambda have diagnostics that name the replacement: Nothing, a Result, and an anonymous function. self and is are not keywords. except, finally, and raise are not reserved either, but a statement that begins like Python exception handling is rejected.

Names and scopes

Module declarations are visible throughout their module. Functions can call later functions or recurse mutually; aliases and data declarations can refer forward. Duplicate declarations and redefinitions are rejected.

Function parameters and local bindings occupy lexical scopes. A declaration in a branch does not escape that branch: using the name afterwards is an unknown binding, and the diagnostic says that the earlier binding ended with its block. Bind a mut local to a value before the block and assign it inside instead. Assigning an existing mutable local changes that local across a branch join. Pattern bindings belong to their arm and may shadow outer bindings. A new binding cannot read itself in its initializer.

Type lookup is separate from local expression names. A local can shadow a numeric cast name without changing type annotations. A binding shadowing an enum type qualifier is diagnosed rather than selecting a different constructor.

The built-in prelude is available without an import. It provides primitive and container type names (including Box and Shared), standard sums and variants (Option, Result, Some, Nothing, Ok, Err), callable and generator types, and facility names for files, errors, threads, channels, executors, and cancellation. FormatError, the error of user formatting methods and of f-strings that run them, names std.fmt.FormatError; a module that declares or imports its own FormatError, such as std.report’s, uses that one instead. Built-in operations include copy, drop, len, range, text I/O, worker submission, and channel construction and selection. Their contracts appear under facilities and the relevant language section. Top-level declarations cannot redefine built-ins. Locals can shadow expression names; unqualified standard variant patterns still denote the standard variants.

Imported names are explicit private bindings. Importing a module or protocol does not activate methods or introduce transitive names.

An import binds its alias, or else its item or module path. That name cannot be a built-in, a module declaration, or a name an earlier import bound. Importing a standard declaration that is itself the built-in, such as from std.concurrent import ThreadPoolExecutor, is allowed. The diagnostic points at the bound name, says whether it conflicts with a prelude built-in or with a declaration or import on a given line, and suggests renaming, binding with as, or removing a redundant import. Function-local bindings may still shadow imported names, as math = 3 inside a function can after import std.math as math.

Modules and imports

std is a reserved toolchain namespace. The embedded modules std.math, std.path and std.fs are available without local sources. Unknown std modules never fall back to the source root. Local files/directories and declarations named std, and imports binding a non-standard module as std, are rejected.

import package.module
import package.module as module_alias
from package.module import Name as Alias

Comma-separated imports are supported. Imports occur only at module scope and execute no initializers. Relative imports, wildcards, re-exports, and implicit transitive imports are unsupported. Directories are namespaces, without __init__ execution.

The source root defaults to the entry file’s canonical directory; --module-root DIR selects another root. In a package, the root is src/, and the first component of an import may instead be a declared dependency alias, which names that dependency’s library; an alias may not match a local top-level module. a.b resolves to <root>/a/b.plenty or the trusted interface <root>/a/b.plentyi. Having both is ambiguous. Imported canonical filenames and directories must have identifier components and cannot escape the root. File/directory name collisions and import cycles are errors; a cycle diagnostic names the actual dependency chain.

import a.b binds the path a.b, and a.b.f names a member of that module. A binding cannot reuse a name the file declares or imports with from, and an alias cannot enclose another binding: import x as a with import a.b is rejected. Because a.plenty beside a/ is ambiguous, only a dependency alias names a module with other modules beneath it: tz names the library, such as src/lib.plenty, and tz.zones names src/zones.plenty of that dependency. A file can import both under their own paths, in either order:

import tz
import tz.zones

tz.f then names the library’s f, and tz.zones.f the f of tz.zones. If the library declares a member zones, public or private, tz.zones would name both, so the later import is rejected with a diagnostic naming both imports; import tz.zones as zones binds the module without the clash. The same holds for a deeper path such as tz.rules.dst and a library member rules. from tz import zones imports only a declaration of the library, never the module beside it, since imports do not re-export. When the library declares no zones and tz.zones is a module, the diagnostic says that from imports declarations and suggests import tz.zones or import tz.zones as zones. The same applies to a local namespace directory (from util import helpers) and to std (from std import json).

A module path is not a value. Using tz.zones (or tz) in an expression or type reports that it is a module and names one of its declarations, such as tz.zones.offset; a path that encloses an imported module, such as tz.rules with import tz.rules.dst, is reported as a module namespace. A missing member of an imported module, such as tz.missing, reports that it is private or is not a declared export.

Canonical paths identify modules, so imports through different aliases or diamond dependencies preserve the same nominal types. Private declarations remain private to their defining module, regardless of directory ancestry. All reachable modules are checked before execution. Imported modules need no entrypoint, and their main functions are ordinary functions.

The concurrency standard modules export existing builtin identities. An explicit import such as from std.thread import spawn may bind the matching builtin name; an import cannot replace a builtin with a different symbol. Item and module aliases retain the same types and compiler safety checks.

Constants

const name: T = value declares an immutable value at module scope; pub const exports it. T must be a concrete integer or float type, an integer domain such as Positive[i64], a Quantity[T, D], bool, str, or bytes. The compiler evaluates the initializer, a constant expression, and each read uses the result:

const KIB: i64 = 1 << 10
const MIB: i64 = KIB * KIB
const MASK: u8 = ~(1u8 << 3)
const RATIO: f32 = f32(MIB) / 3.0
const NAME: str = "plenty" + "-" + "lang"
const WIDE: bool = MIB > KIB and not (KIB == 0)

def main() -> Result[(), IoError]:
    print(MIB)?
    print(MASK)?
    print(RATIO)?
    print(NAME)?
    print(WIDE)?
    Ok(())
1048576
247
349525.34
plenty-lang
True

A constant expression contains literals, other constants (from this module or imported, in any declaration order), numeric type constants such as u8.MAX or f64.PI, numeric casts such as f32(x) and u8(x), parentheses, the unary operators -, ~, and not, the arithmetic, bitwise, shift, comparison, and logical operators, x if c else y, and + on two str values. Quantity construction, quantity operators, .get() and floating .to(unit) also fold. User function calls, f-strings, collections, and other expressions are rejected with “not a constant expression”.

Evaluation follows the numeric rules of the same expression at run time, so the folded value is the value the program would compute: // floors, % takes the divisor’s sign, / divides floats only, << discards shifted-out bits, >> is arithmetic for signed types, and operands must share a type. Unsuffixed literals take the type the context expects. Where run time would trap, evaluation is a compile error instead: overflow, division by zero, a shift count outside 0..width, and an out-of-range float-to-integer cast. As at run time, x if c else y evaluates only the selected branch, and and and or evaluate their right operand only when it decides the result; the other operand is still type-checked, so 1 if D == 0 else 100 // D is valid when D is zero. A cycle is reported with its path, for example constant A depends on itself: A -> B -> A.

An integer domain constant (NonZero[T], Positive[T] or NonNegative[T]) evaluates its initializer as T, and a value outside the domain is a compile error. Each read is the domain value, as if the literal were written there. Another initializer reads its integer with .get(), or names it alone as the whole initializer of a constant of the same type:

const MAX_URL: NonNegative[i64] = 8192
const PAGE: Positive[i64] = MAX_URL.get() // 64
const FIRST: Positive[i64] = PAGE
assert PAGE.get() <= 1024

def pages(size: Positive[i64] = PAGE) -> i64:
    MAX_URL.get() // size

def main() -> Result[(), IoError]:
    print(pages(), pages(FIRST), MAX_URL)?
    Ok(())
64 64 8192

Static assertions

assert CONDITION or assert CONDITION, "message" at module scope checks a constant expression during compilation. A false condition fails the build with static assertion failed, followed by the message when given. The message is itself a constant str expression. An assertion declares no name and produces no code.

const PAGE: i64 = 4096
assert PAGE % 64 == 0, "pages hold whole cache lines"

def main() -> Result[(), IoError | OverflowError[i64] | ZeroDivisionError]:
    print(PAGE // 64)?
    Ok(())
64

Module-scope assert uses the spelling of a Python assert statement, which the compiler checks once instead of at run time. Plenty has no run-time assert statement; inside a function, assert is an ordinary name.

Included files

include_str("path") initializes a str constant and include_bytes("path") a bytes constant with a file’s contents:

const SCHEMA: str = include_str("sql/schema.sql")
const ICON: bytes = include_bytes("assets/icon.png")

The argument is one string literal: a path relative to the directory of the source file that contains it. Absolute paths and paths that resolve outside the package’s source root, including through .. or symbolic links, are errors. include_str rejects a file that is not valid UTF-8 and names the first invalid byte. Both forms initialize a const only and may appear in other constant expressions, for example "> " + SCHEMA. They require a source file, so source passed without a path, such as an editor buffer, cannot use them.

The contents become read-only data in the executable; reads allocate nothing. An included file is an input of the program cache: editing it rebuilds the program.

Binding and representation

Imports and aliases bind constants as values: import std.math as math permits math.pi, while from std.math import pi permits pi. Constants are not callables or mutable places; brackets on a constant index it, as in TABLE[n] or tables.TABLE[n:]. Private constants follow the same visibility rules as private functions. A local binding may shadow an imported constant. Constant names share the declaration namespace with functions and types.

Constants require no module initialization or allocation. Each read of a numeric or bool constant compiles to an immediate of the constant’s declared type (an integer domain constant’s immediate is converted as a literal is), and each read of a str or bytes constant to a reference to read-only data, in debug and release builds; there is no getter function, global storage, or code run at import time. See call-site substitution.

Visibility

pub exposes top-level functions, aliases, classes, enums, protocols, and constants. Other declarations are private to their module. Class fields and methods require their own pub; public enums expose every variant, but enum methods need their own pub.

Generic functions check access in their defining module, including protocol requirements and calls on a type parameter. Calling a generic function does not lend it the caller’s private access. To let a generic helper in another module (such as std.io.read_exact) call a method, mark both the type pub class or pub enum and the method pub def; a public method alone does not expose a private type. Private fields and other private methods remain inaccessible. This keeps generic code under the same lexical visibility rules as ordinary functions. A failing protocol check is reported at the user’s call; nested instantiations retain the outer call location and an instantiation note. Code within the type’s own module can still use its private members through a locally defined generic function.

A generated field constructor is public only when its class and every field are public. An explicit __init__ controls its own visibility. A public factory may use private constructors inside their defining module. Construction through an alias obeys the same access checks.

Public signatures, aliases, fields, and enum payloads cannot expose private nominal types, including through aliases or nested containers. Public generic contracts also require public arguments and constraint types; see generics.

Access checks apply to field reads, writes, borrows, methods, and construction. Automatic destruction is unaffected by destructor privacy; direct lifecycle calls are prohibited. Structural printing and equality include private fields, so privacy does not hide data. pub does not specify a C ABI or binary export.

Target conditionals

if const selects code by facts about the compilation target. It may appear at module scope, around declarations and imports, and in any function body, around statements:

if const target.os == "windows":
    import windows_console
    def separator() -> str:
        return "\\"
else:
    def separator() -> str:
        return "/"

def main() -> Result[(), IoError]:
    if const target.pointer_width == 64:
        limit = 1 << 40
    else:
        limit = 1 << 20
    print(separator())?
    print(limit)?
    Ok(())

On a 64-bit Linux or macOS target, it prints:

/
1099511627776

The compiler decides the condition while parsing and keeps only the first arm whose condition holds, or the else arm. Each continuation is written elif const; an elif without const is an error. Unselected arms must parse, but are not resolved or checked: they may import modules, call functions, and name declarations that exist only on another target. An unselected arm is not checked on the current target, so build on each target a program supports.

In a function body, the selected statements take the conditional’s place in the enclosing block; they open no new scope. A binding in the selected arm, such as limit above, is visible after the conditional. The last statement of a selected arm can be the function’s result.

Facts

FactKnown values
target.os"linux", "macos", "windows", "freebsd"
target.arch"x86_64", "aarch64"
target.pointer_width32, 64
target.endian"little", "big"

The set is closed. An unknown fact, such as target.cpu, is an error, and so is a comparison with a value outside the known values, such as "linx", so a misspelling cannot silently select the else arm. The compiler currently builds natively for GNU/Linux on x86-64 and AArch64: target.os is "linux", target.arch is "x86_64" or "aarch64", pointer width is 64, and endian is "little". The facts describe the packaged runtime, and an installed compiler cannot select the other architecture with --target; see platform support.

Conditions

A condition compares one fact with a string or integer literal using == or !=, and combines comparisons with and, or, not, parentheses, True, and False. Any other expression is an error. A condition cannot name a module constant: conditions are decided while parsing, before names are resolved, so that unselected arms can name declarations that do not exist. A constant can instead be defined differently in each arm.

Application entrypoint

A binary application requires exactly one module-level main in its entry module, with no parameters and one of these return types:

Return typeProcess result
()Status zero
i32The returned status
Result[(), E]Ok(()) gives zero; Err gives one
Result[i32, E]Ok(status) gives that status; Err gives one

Transparent aliases are accepted. The wrapper calls main once. Small nonnegative statuses are portable; an operating system may truncate others.

A returned Err is reported once on standard error after flushing standard output. Its payload is destroyed exactly once, and reporting failure does not change status one. Intermediate error propagation prints nothing. See entrypoint reporting for the exact format and fallback behavior.

Checking and compilation diagnose missing or invalid entrypoints before running code or producing an output artifact. Both diagnostics list all four accepted return types. Compilation errors execute nothing; runtime errors can leave effects that already occurred. Library checking does not require main.

Types and data definitions

Primitive types are i8, i16, i32, i64, u8, u16, u32, u64, f32, f64, bool, str, and bytes, the type of byte literals such as b"\x89PNG". There is no built-in int; use a width or a transparent alias. Numbers have local literal inference and explicit conversions rather than implicit widening.

The unit type () denotes completion without data. It can be an expression, function result, or enum or tuple component, including Option[()] and Result[(), E]. Standalone unit bindings, parameters, collection elements, and class fields are unsupported. A reference to unit, &() or &mut (), is an ordinary reference: borrowed unit payloads have this type, and a &() parameter lends a unit argument like any other temporary. There is no None type or value.

Tuples are structural products. Classes and enums are nominal: identical fields or payloads do not make separately declared types interchangeable. Import aliases preserve a declaration’s identity; transparent aliases create none. Generic instances with different arguments have different nominal identities.

Dimensions are compile-time types with no values. Nominal bases and normalized products, quotients, powers and exact roots can be arguments to generic parameters constrained by Dimension.

Type inference stays within local expressions and calls; every function interface declares its types. Collection, reference, callable, and generator types follow their respective facilities, ownership, function, and iteration contracts.

Data definitions may refer forward and recur through permitted storage boundaries. References cannot be stored in classes, enums, or tuples. Generators can occur inside standard Option and Result wrappers, but not class fields, user enum payloads, or tuple components.

Quantities combine numeric storage with a compile-time dimension and check that arithmetic combines those dimensions correctly.

Numeric types and literals

For a fixed inclusive integer interval, use Bounded[T, lo, hi].

Unsuffixed integers default to i64; decimal or exponent literals default to f64. Width suffixes include 7u8, 1f32, 1.5f32, and 1e-3f64. Decimal forms such as .5 and 1. are accepted. Literals must fit their selected width.

Integer literals may also be hexadecimal (0x1F1E6), octal (0o755), or binary (0b1010). The prefix letter and hexadecimal digits may be either case. These forms are always integers: they take the integer suffixes and range checks of decimal literals, as in 0xFFu8 and -0x80i8, and have no fraction, exponent, or float suffix, so 0x1f32 is the integer 7986. Any literal may separate digits with single underscores, as in 1_000_000 and 0xffff_ffff; an underscore cannot lead, trail, repeat, or follow the prefix.

SourceDiagnostic
0x`0x` needs at least one hexadecimal digit
0b102invalid digit `2` in a binary literal
0b1f32invalid digit `f` in a binary literal
0x1u7invalid integer suffix
0x_FFinvalid integer separator
0x1FFu8integer literal out of range for u8

Annotations, parameters, returns, and typed arithmetic operands guide unsuffixed literals within an expression. Already typed values and suffixed literals must match exactly; no numeric widening occurs. Operands of different numeric types are rejected with the cast that reconciles them, for example cannot combine i64 with f64; convert with `f64(n)` . The suggested cast never fails: an unsigned and a signed integer convert to the narrowest signed type wider than the unsigned one, both operands if neither has that type (i8 with u16 suggests i32 for each). u64 with a signed type has no such type, so the message names no cast. Ordinary generic calls have argument inference rules that do not infer one argument’s literal width from another.

Explicit casts use numeric type names, for example u8(200). A cast to an integer type requires the target to represent the value. An integer keeps its mathematical value; one outside the target’s range traps, unlike the default recoverable integer arithmetic errors. This covers narrowing, a negative value cast to an unsigned type, and an unsigned value above a signed type’s maximum. A float is truncated toward zero first: u8(255.9) is 255 and u8(-0.9) is 0. A float whose truncated value is outside the target’s range, an infinity, or NaN is the same runtime error. The message is integer cast out of range.

A cast whose operand is a numeric literal is checked at compile time, so u8(300) and u8(300.7) are compile-time errors. A cast between integer types where the target represents every source value, such as i64(small_u8), emits no check. A failing check calls the runtime; a passing one does not, and no cast allocates. A cast whose operand compiles to a constant, such as i64(u16.MAX) or u8(i64(200)) in a function body, emits the converted constant with no conversion or check, as the literal 65535 would. Such an operand outside the target’s range, as in u8(i64(300)), is still the runtime error.

Casts to a float type never fail. Integer-to-float and narrowing float casts may round, and an f64 beyond f32’s range becomes an infinity. Widening f32 to f64 is exact.

Two other forms convert between integer types. T.try_from(value) returns a Result instead of stopping the program. T.wrapping_from(value) keeps the low bits and never fails.

All three conversions take a number, not a reference. Borrowed iteration such as for b in &buffer yields &u8, so i64(b) is a compile-time error that names the fix: numeric casts require a number, not a reference; read it with `*b` . A borrow written in the operand, as in i64(&n), is reported with remove the `&` instead.

Unary minus requires a signed integer or a float; an unsigned operand is a compile-time error, including a sign directly before a literal. Both -3u8 and -0u8 report unsigned negation, as does value: u8 = -0 when the literal gets its type from context. Signed minimum literals such as -128i8 remain valid, and floating-point negative zero is preserved.

Integer arithmetic failures return Err from the enclosing function by default. Its declared Result error type must include the possible errors; successful arithmetic still produces ordinary integers. Use an explicit checked: block for local recovery, or unchecked: where the programmer guarantees that the arithmetic cannot fail. // floors signed quotients, including negative operands; % is integer-only. / accepts matching-width floats. Float arithmetic follows IEEE behavior for rounding, signed zero, infinities, NaN, and underflow; arithmetic overflow may yield infinity.

Integers support bitwise &, |, ^, and unary ~, and shifts << and >>. Operands must have the same integer type; a typed left operand guides an unsuffixed count literal. Counts must be nonnegative and less than the integer width; invalid counts trap before shifting. Left shifts discard high bits. Signed right shifts extend the sign bit; unsigned right shifts fill with zero. From lower to higher precedence: |, ^, &, shifts, addition/subtraction, multiplication/division/modulo, unary operators. Comparisons bind below |. Prefix & and &mut still create borrows.

value.wrapping_add(other), wrapping_sub, and wrapping_mul require matching integer types and compute modulo two to the integer width. They allocate nothing. Ordinary +, -, and * keep their overflow checks.

value.rotate_left(count) and value.rotate_right(count) rotate the bits of an integer: bits leaving one end enter at the other, so none are lost. The count has the receiver’s type, and a typed receiver guides an unsuffixed count literal. Because no count loses bits, rotation never traps: the count is reduced modulo the integer width, so rotating by the width is the identity and a negative signed count rotates the other way. Signed values rotate their two’s-complement bit pattern. Rotation allocates nothing. These built-ins are selected for integer receivers; classes may define methods with the same names and their own signatures.

The numeric facilities provide the type constants T.MIN, T.MAX, T.PI, and T.E; pow on every numeric type; and the float methods sqrt, exp, log, sin, cos, floor, ceil, and fmod. A two-operand method requires one type, as the operators do, and an integer result outside its type is the same overflow error.

Comparisons require matching types; ordering accepts integers, floats, strings, and tuples of those. For scalar floats, NaN is unequal to every value, including itself, and all ordered comparisons with it are false. These rules also apply to structural equality and membership. Floats cannot be dictionary keys or set elements.

Function arithmetic contracts

def scale(value: i64, factor: i64) -> Result[i64, OverflowError[i64]]:
    Ok(value * factor)

def main() -> Result[(), IoError]:
    print(scale(3, 4).unwrap(), scale(i64.MAX, 2).is_err())?
    Ok(())
12 True

Each named or anonymous function has its own boundary, independent of its definition or call site. The declared error type must contain every escaping arithmetic error; a plain or Option return cannot implicitly propagate them. Explicit Ok, Err, returns and ? keep their ordinary meaning. Operators do not produce individual Result wrappers or implicitly unwrap calls.

The operation/error table under checked blocks also applies to the function default. Error contracts follow operators and operand types, not range analysis. Locally handled errors do not have to appear in the function’s error type. Generators cannot propagate function-level arithmetic errors: handle a local checked result or explicitly opt out.

Coordinated repository adoption under B14 remains in progress. Public contracts are explicit: do not assume that an arbitrary callback consumer accepts a Result-returning callback.

Unchecked blocks

unchecked: states that integer arithmetic written in its lexical suite does not overflow or divide by zero; that is the programmer’s precondition, not the compiler’s. It yields the last expression’s ordinary type, without wrapping or unwrapping a Result. It is also allowed as a statement suite, including an empty pass suite and suites whose paths exit. Locals remain block scoped and reference-valued suites are not supported.

def scale_in_bounds(value: i64, factor: i64) -> i64:
    unchecked:
        value * factor

The --unchecked MODE build setting selects what a plain unchecked: suite does with its preconditions. unchecked trap: always checks them:

Suite--unchecked trap (default)--unchecked assume
unchecked:Check; trap on a violationOmit the checks; a violation is undefined behavior
unchecked trap:Check; trap on a violationCheck; trap on a violation

A trap terminates the process; it never wraps or reaches an outer checked boundary. In assume mode, overflow is not wrapping: a violation may produce a silently wrong result or worse, and no crash is guaranteed. Use explicit wrapping operations for modular arithmetic, and unchecked trap: or ordinary validation when termination on bad input is part of the contract. The default does not change with optimization or profile.

def checked_index(row: i64, width: i64, column: i64) -> i64:
    unchecked trap:
        row * width + column

The preconditions are representable results of +, -, *, negation and pow, nonzero divisors and representable quotients for // and %, valid shift counts, and in-range integer conversions. i64.MIN % -1 stays 0, and a negative pow exponent still traps in either mode. Floating-point arithmetic, allocation and explicit wrapping, saturating and checked methods keep their contracts.

In assume mode, list, slice-view and Bytes indexing written inside a plain unchecked: suite also requires 0 <= index < len(collection). Reads, writes, augmented assignments and element borrows omit both bounds checks and negative-index normalization. A negative or out-of-range index is undefined behavior, even when the negative index would normally count from the end. Default/trap builds retain normal negative indexing and out-of-range traps. This does not change slices, collection methods such as swap, or indexing of strings, immutable bytes, dictionaries, tuples or ranges.

Modes apply lexically. Nested checked: restores local recoverability; a nested plain unchecked: follows the build setting, even inside unchecked trap:; leaving any suite restores the outer mode. Named functions and closures keep their own contracts wherever they are defined or called. Calls still return their declared results, and ?, return, break and continue keep their ordinary targets.

std always checks its unchecked: suites, so the setting cannot change library behavior before its preconditions are audited. The setting is part of the compile cache identity; there is no plenty.toml key yet. Cranelift has no no-overflow instruction flags, so assume mode removes checks but cannot give the backend further overflow facts.

unchecked and trap are contextual: only unchecked: or unchecked trap: followed by a newline opens a suite; unchecked: i64 = 0 remains a typed binding. Suites support binding, assignment, return, function-tail and match-arm-tail value positions.

Checked blocks

A checked: block turns the integer failures written inside it into a value. Its body is an indented suite whose last expression is the block’s value T. The block is an expression of type Result[T, E]:

def mean(total: i64, count: i64) -> Result[i64, OverflowError[i64] | ZeroDivisionError]:
    checked:
        total * 2 // count

If no operation fails, the block’s value is Ok(T). The first failing operation exits the block with Err, and the rest of the block does not run. E is the anonymous sum of the errors that the block’s operators can produce:

Operation inside the blockError
+, -, *, unary - and pow on TOverflowError[T]
// on a signed T with a T or NonZero[T] divisor (T.MIN // -1)OverflowError[T]
// and % with a plain integer divisor that may be zeroZeroDivisionError

OverflowError[T] has one field, wrapped: T, the exact result modulo 2N for an N-bit T. ZeroDivisionError has no fields. Both are builtin classes; their names cannot be redefined. T.MIN % -1 is 0, inside or outside a block. Shifts, casts, and float arithmetic are unchanged: they neither add an error nor exit the block. A negative pow exponent still traps, since it has no integer result. Float a.div_checked(b) returns Result[F, ZeroDivisionError], failing for either signed zero divisor; it behaves as a one-operation block and may appear anywhere.

A class pattern names each error and binds the wrapped result:

def halve_sum(a: i64, b: i64) -> i64:
    result = checked:
        (a + b) // 2
    match result:
        case Ok(half):
            half
        case Err(OverflowError(wrapped)):
            # A positive divisor rules out zero and MIN // -1.
            two: Positive[i64] = 2
            wrapped // two
        case Err(ZeroDivisionError()):
            0

The error type follows the operators, not their operand values. n // 4 inside a block still adds ZeroDivisionError and OverflowError[i64]; the code generator omits the branches a constant divisor rules out. A declared Result whose error includes every possible failure is kept, so a block can be the body of a function that returns Result[i64, OverflowError[i64] | ParseError].

The rule is lexical within one function. Compound assignment such as x += 1, comprehensions, and operators that @newtype classes forward to their field report the field’s arithmetic error types. The nearest explicit arithmetic mode controls failure routing.

Functions and closures defined inside the block keep their own declared contracts. There is no private callback error channel or checked-block lifetime restriction; ordinary ownership and borrow restrictions still apply. A callee’s Result requires explicit handling, and ? exits the function rather than being caught by the block.

Integer std.math reductions and serial pipeline sum() return public overflow results; float reductions stay infallible. sort_by_key accepts ordinary fallible key callbacks and returns their error. Other callback consumers require their own explicitly fallible contracts. Do not assume a call inherits local checking or that an infallible callback signature can accept recoverable arithmetic.

Control flow is otherwise unchanged. ? inside the block exits the function, and return, break and continue behave as they do outside it. A failure releases the values and locals the block owns, as ? does. It does not roll back assignments the block already made to outer variables or elements, and an assignment whose operation failed does not happen.

A block may appear wherever a match block can produce a value: as the value of a binding or assignment, after return, or as a function’s or case’s final expression. checked is not reserved: it starts a block only when : and a newline follow it, so checked: i64 = 0 still declares a variable. A block whose value is discarded is rejected like any unused Result. A block with no operation that can fail is a compile-time error, as is one whose every path exits. A block cannot produce a reference.

Module constants are evaluated at compile time, where overflow is a compile-time error. Arithmetic in a block is evaluated at run time even when its operands are constants, so checked: around i8.MAX + 1i8 produces Err(OverflowError[i8](wrapped=-128)).

Arithmetic uses plain values and a shared cold error exit at its nearest recoverable boundary. A trapping unchecked: operation uses an inline trap guard instead; assume mode emits the plain instruction. See the design note.

Nonzero integers

NonZero[T], for any integer type T, holds a T that is not zero. Where one is expected, so that the program cannot hold a zero, an integer literal converts at compile time, including a negative one; 0 is a compile-time error. Any other value is converted with NonZero.try_from(value), which returns Result[NonZero[T], RangeError] and fails with RangeError.OutOfRange for zero. T is the value’s type; NonZero[T].try_from(value) names it and requires a T. get() reads the integer back.

def per_group(total: i64, groups: NonZero[i64]) -> Result[i64, OverflowError[i64]]:
    Ok(total // groups)

def main() -> Result[(), IoError | RangeError | OverflowError[i64]]:
    three: NonZero[i64] = 3
    print(per_group(12, three)?)?
    print(per_group(12, -4)?)?
    count = 5
    five = NonZero.try_from(count)?
    print(five * three, five + three, five > three)?
    match NonZero.try_from(count - 5):
        case Ok(n):
            print(n)?
        case Err(RangeError.OutOfRange):
            print("zero")?
    Ok(())
4
-3
15 8 True
zero

A value has the same representation as its integer and allocates nothing. The constructor is private: NonZero(3) is a compile-time error that names the two conversions. NonZero[T] copies, compares with ==, !=, <, <=, >, and >=, prints and formats as its integer, and is a valid dictionary key and set element.

Arithmetic keeps a result nonzero only where that follows from nonzero operands. Unary - and +, and * between two NonZero[T] values, return NonZero[T]; *= updates one. + and - between two values return T, since their result can be zero. Overflow returns Err at the nearest recoverable boundary, and violates the precondition of unchecked blocks, so a product cannot wrap to zero. Other operators between two NonZero values, including //, %, bitwise operators, and shifts, are compile-time errors that suggest .get(). An integer literal beside a NonZero[T] operand is a NonZero[T] too: step * 2 is a NonZero[i64], and step == 0 is a compile-time error. Operands must otherwise have the same type; mixing NonZero[T] with T is otherwise a type error whose hint shows try_from.

A plain T dividend accepts a NonZero[T] divisor: total // groups and total % groups return T, and //= and %= update a T destination. These forms omit the divide-by-zero check. Signed floor division still checks T.MIN // -1 for overflow; T.MIN % -1 returns zero. In a checked: block, signed division by NonZero[T] contributes only OverflowError[T]; modulo and unsigned division contribute no failures. A block containing only operations that cannot fail is rejected, as with other checked arithmetic.

A range step is NonZero[i64]. NonZero is a builtin name and cannot be redefined.

Positive and non-negative integers

Positive[T] holds an integer greater than zero; NonNegative[T] holds an integer greater than or equal to zero. T must be i8, i16, i32 or i64. For unsigned integers use NonZero[uN] or plain uN, respectively. Floats and other non-integer types are rejected without a replacement suggestion. Both are builtin names and cannot be redefined.

An unsuffixed integer literal converts where the domain type is expected, including parameters, defaults, fields, returns and collection elements. An out-of-domain literal is a compile-time error. A module constant may have a domain type; its value is checked the same way. A computed integer converts with Positive.try_from(value) or NonNegative.try_from(value), returning Result[Domain[T], RangeError]; invalid values give RangeError.OutOfRange. The qualified forms Positive[T].try_from and NonNegative[T].try_from require exactly T. Constructors and fields are private; .get() reads T. There is no implicit conversion between a domain and its integer or another domain.

def configured(value: i64) -> Result[Positive[i64], RangeError]:
    Positive.try_from(value)

def main() -> Result[(), IoError | RangeError | OverflowError[i64]]:
    size: Positive[i64] = 3
    limit: NonNegative[i64] = 0
    print(size + 2, size * 2, size - size, -size, limit)?
    stride = NonZero.from(size)
    print(list(range(0, 8, stride)).unwrap())?
    print(NonNegative.from(configured(4)?))?
    print(configured(0).is_err())?
    Ok(())
5 6 0 -3 0
[0, 3, 6]
4
True

+, *, += and *= preserve the domain; subtraction and unary - return T, and unary + preserves it. Overflow returns Err by default and violates the precondition of unchecked blocks, so the domain cannot be violated by wrapping. Comparisons accept the same domain type or a literal in its domain. Other operators between domain values, including division, modulo, bitwise operations and shifts, require .get().

A plain T dividend accepts a Positive[T] divisor directly with //, %, //= and %=. The result is T. These forms omit both the zero check and the signed MIN // -1 check, since a positive divisor can be neither zero nor -1. They contribute no failures inside a checked: block. A NonNegative[T] divisor still needs .get() or explicit conversion to a nonzero domain, since zero is valid for it.

NonZero.from(positive) and NonNegative.from(positive) explicitly widen a Positive[T]; they cannot fail or allocate. Qualified forms name T. Conversion from a NonNegative to a Positive or NonZero uses try_from(value.get()), since it may be zero.

Both domains have T’s representation, copy without allocation, print and format as the integer, and work as dictionary keys and set elements. len() continues to return i64; convert a computed length explicitly when a domain is needed.

Bounded integers

Bounded[T, lo, hi] stores an integer T in the inclusive interval lo <= value <= hi. T is any of the eight integer types. The two bounds are value arguments of exactly that type. Bounds outside T or a lower bound above the upper bound are compile errors, including when an instance is used only in an alias, field or signature. Equal bounds are valid.

Use a type alias to name a domain:

type Percent = Bounded[u8, 0, 100]

def set_percent(value: Percent = 50) -> u8:
    value.get()

def main() -> Result[(), IoError | RangeError]:
    literal: Percent = 25
    configured = 75u8
    checked = Percent.try_from(configured)?
    print(set_percent(literal), set_percent(100), checked.get())?
    print(Percent.try_from(101).is_err())?
    Ok(())
25 100 75
True

Construction and extraction

OperationResult
Integer literal where Bounded[T, lo, hi] is expectedChecked at compile time; no allocation or Result
Bounded[T, lo, hi].try_from(value) or an alias’s .try_from(value)Result[Bounded[T, lo, hi], RangeError]; RangeError.OutOfRange outside the interval
value.get()The stored T, copied without allocation

Expected literal contexts include annotated bindings, assignment, concrete function parameters and returns, defaults, constructor fields and typed collection elements. A suffixed literal must have exactly T. Plain computed integers require try_from, even when the programmer knows they fit. A try_from input is evaluated once and may be T or a reference to T. There is no implicit conversion between different bounded types or from a bounded value to its stored integer. The constructor and field are private.

def main() -> ():
    percent: Bounded[u8, 0, 100] = 101
outside `Bounded[u8, 0, 100]`'s inclusive bounds

Value behavior

A bounded integer is a copyable private single-field class. It stores only T; the bounds are type metadata and add no fields. It follows ordinary class borrowing, equality, hashing and cleanup behavior. Equality requires matching bounded types. Default formatting shows the class and its value; .get() gives the ordinary integer’s formatting and ordering operations.

Arithmetic requires .get(). The compiler does not derive new intervals for arithmetic or widen intervals automatically. Check the result again with try_from if it must re-enter a bounded domain. Arithmetic on the extracted integer retains ordinary overflow/error contracts.

Positive[T] and NonNegative[T] remain useful for their fixed lower bounds and domain-preserving operations. Use Bounded when a ceiling or another lower bound is part of the type. Existing standard-library limit signatures retain their documented types and dynamic checks; the availability of Bounded does not itself change them. Module constants may supply the bounds, but a module constant cannot yet have a Bounded result type.

Byte literals and bytes

A byte literal spells fixed binary data: protocol magic numbers, framing, test fixtures, search needles. Its type is the built-in immutable type bytes, the byte counterpart of str.

def is_png(header: bytes) -> bool:
    header == b"\x89PNG\r\n\x1a\n"

def main() -> Result[(), IoError]:
    request = b"GET / HTTP/1.1\r\n"
    print(len(request), request[0], request[-1])?
    print(is_png(b"\x89PNG\r\n\x1a\n"))?
    print(b'single "quotes"')?
    Ok(())
16 71 10
True
b"single \"quotes\""

Spelling

A byte literal is b directly followed by a quoted string: b"...", b'...', b"""...""", or b'''...'''. Triple quotes permit physical newlines, as for strings. Each rule has one form:

  • The prefix is a lowercase b. B"..." is rejected with a diagnostic. b followed by anything other than a quote is an ordinary name, so b and B remain usable identifiers.
  • bytes is a built-in type name like str: no function, type, or import binding may take it, so write import std.bytes, not import std.bytes as bytes. A local variable may still be named bytes; the type has no associated functions, so bytes.name always refers to that value.
  • The text holds only ASCII characters. Any other character is rejected at its position with a hint to use \xHH escapes or std.bytes.from_text. A source character never stands for its UTF-8 encoding.
  • Escapes are those of string literals (\0, \n, \r, \t, \\, \', \") plus \xHH, exactly two hexadecimal digits of either case, for any byte. \u{...} is rejected with a hint to write the scalar’s UTF-8 bytes as \xHH; \N and every other escape are errors.
  • There are no other prefixes: no raw form (rb, br) because there are no raw strings, and no formatted form (bf, fb). Build formatted text with a string and convert it with std.bytes.from_text.
  • Adjacent literals are not concatenated, as for strings.
  • A byte literal is not a match pattern; compare with ==.

Values

A bytes value refers to read-only data emitted into the executable. Creating, copying, passing, returning, storing, or comparing one never allocates, and it has no owner to release: assignment copies the reference, like an integer. The contents cannot change.

OperationResult
len(value), value.len()Number of bytes, i64
value[i]The byte at i, u8; list index rules: negative counts from the end, a missing index stops the program with index out of bounds
value[a:b:c]A slice: Result[std.bytes.Bytes, AllocError], a new owned buffer; no import needed
&value[a:b]A shared slice view, &[u8], of the selected bytes; no copy
for byte in value, reversed(value)Each byte as u8
x in valueWhether the u8 x occurs
==, !=Content equality with another bytes
print(value)The literal spelling, which reads back as the same bytes: printable ASCII, \\, \", \n, \r, \t, and \xhh for every other byte; see printing bytes

bytes values can be elements, set members, dictionary keys, class fields, and enum payloads, and can be captured and passed to scoped threads. len is the only method; it lets bytes satisfy a protocol that requires it. There are no ordering comparisons and no concatenation. A slice copies into an owned Bytes rather than producing another bytes. bytes and str are distinct: neither converts to the other implicitly.

Owned buffers

std.bytes.Bytes is the owned, growable buffer. std.bytes.from_bytes(source: &[u8]) copies a literal, or any other byte view, into a new Bytes and returns a Result, because the copy allocates.

Operations that only read bytes, such as find_bytes, matches, extend, and every std.io, std.net, and std.tls stream write, take a &[u8] view. A literal, a Bytes buffer, a list[u8], or part of any of them converts to that view and is read in place; nothing is copied, and there is no implicit conversion between bytes and Bytes. Changing or growing bytes needs a Bytes.

The literal type is a separate prelude type rather than a static &Bytes view so that the prelude does not depend on std.bytes: every program would otherwise know the library class. The &[u8] view is what lets one parameter accept every byte owner.

Type aliases

type Count = i32

def increment(value: Count) -> Count:
    value + Count(1)

type Name = Type declares a transparent, non-generic module-level alias to a supported type or another alias. Aliases are visible throughout the module and may refer forward. Unknown targets, cycles, duplicate aliases, and declaration or built-in name collisions are rejected even when the alias is unused. Local aliases are unsupported.

An alias adds no nominal identity, validation, or runtime wrapper. Two aliases for i32 are interchangeable with each other and i32. Existing storage and visibility restrictions apply through aliases.

Aliases work in signatures and local annotations. Numeric aliases support their target’s cast syntax and conversion semantics. Boolean, string, and unit aliases add no constructors or conversions.

Aliases do not change literal defaults or introduce suffixes: x: Count = 42 uses the annotation as context, while 42Count is invalid. A local expression name may shadow Count without changing its meaning in a type annotation.

Compile-time dimensions

A dimension identifies a physical dimension without introducing a runtime value. Declare a nominal base with its coherent-unit symbol, or a transparent derived dimension with an expression:

dimension Length("m")
dimension Time("s")
dimension Velocity = Length / Time
dimension Acceleration = Length / Pow[Time, 2]

@copy
class Sample[D: Dimension]:
    value: f64

def reading[D: Dimension](sample: Sample[D]) -> f64:
    sample.value

def main() -> Result[(), IoError]:
    speed: Sample[Velocity] = Sample[Length / Time](3.0)
    print(reading(speed))?
    Ok(())
3.0

Sample demonstrates a dimension argument. Its field is an ordinary f64; this class does not supply dimension-aware arithmetic or unit conversion. For physical values and arithmetic, use the built-in Quantity[T, D].

Declarations and identity

dimension is a keyword. Declarations belong at module scope and follow ordinary module visibility: pub dimension Length("m") is exported. Bases in different modules differ even when their names and symbols match. Import aliases preserve identity. A public derived dimension or signature cannot expose a private base dimension.

A derived declaration creates no new identity. Forward references work; cycles are rejected. One is the built-in dimensionless dimension. Coherent-unit symbols are retained as metadata; they do not select a conversion or change dimension equality.

Dimensions have no values. A dimension can be an argument to a generic parameter constrained by Dimension, but cannot be a parameter’s value type, a return value, a field, a collection element, or an ordinary type alias target. CopyType describes values and does not admit dimensions.

dimension Length("m")
def bad(length: Length) -> ():
    pass
def main() -> ():
    pass
dimension `Length` has no values

Algebra

ExpressionMeaning
OneEmpty product
Base, derived name, or dimension parameterThe named dimension
A * B, A / BProduct or quotient, left-associative; parentheses group
Pow[A, n]Integer power
Root[A, n]Exact integer root; n must be nonzero

n is an unsuffixed integer literal within i64 range, optionally negative. Decimal, hexadecimal, octal, binary, and ordinary integer separators are supported. Negative roots take the reciprocal of the corresponding positive root. Every base exponent must divide exactly by the root degree.

Pow and Root are fallback type-operator names. A local declaration or import with either name keeps its ordinary meaning; for example, importing std.fs.Root as FsRoot leaves the dimension operator Root available.

The normal form is a sorted list of nominal bases with nonzero i8 exponents. Products commute and associate when their intermediate exponents stay in range; division cancels exactly. Each subexpression must fit -128..127: a later cancellation does not rescue an overflowing intermediate result. Normal forms use base names and Pow terms; aliases do not change type identity.

dimension Length("m")
dimension Invalid = Root[Pow[Length, 3], 2]
def main() -> ():
    pass
not a whole dimension

Diagnostic alias notes

Type mismatches and quantity dimension errors show the normalized dimensions. When a matching derived alias is visible, a note connects that normal form to its source spelling:

dimension Length("m")
dimension Time("s")
dimension Velocity = Length / Time
def main() -> ():
    speed: Quantity[f64, Velocity] = Quantity[f64, Length](1.0)
note: dimension alias `Velocity` = `Length * Pow[Time, -1]`

The first matching alias in declaration/import order is used. A namespace import considers exported aliases in their declaration order. Notes use the written import binding, such as Fast for from units import Velocity as Fast or u.Velocity for import units as u. Private or unimported aliases from other modules are excluded, as are names shadowed by generic parameters. Aliases of One or of a single base work the same way.

The scope belongs to the expression or type that failed. A generic body error can therefore explain an alias in the defining module while the diagnostic also identifies the call that required that instance. Nested types and module constants receive the same notes. Errors encountered while resolving dimension declarations can explain aliases that have already resolved.

These notes do not change equality, specialization, coherent-unit output, or normalized quantity debug formatting.

Generic dimensions

D: Dimension accepts dimensions in functions, classes, and enums. A bare dimension parameter in an argument type can be inferred structurally, as in reading(sample). Explicit arguments use reading[Velocity](sample).

In a function parameter type, a dimension argument must be concrete or a bare parameter. An expression involving a parameter, such as Sample[D / Time], is rejected even for explicit calls. Return types and bodies may use dimension expressions, evaluated after substitution for each instance. An invalid root or overflowing exponent then reports the instantiation that required it.

Dimension arguments distinguish generic instances but add no fields or runtime metadata to their values. Specializations differing only in dimensions are not currently shared. See generic specialization.

Quantities

Quantity[T, D] stores a numeric value in the coherent unit of a dimension. T must be an integer, a float, or an integer domain such as Positive[i64]. Quantity values copy. Put domains inside the quantity: Quantity[Positive[i64], Length], rather than Positive[Quantity[i64, Length]]. std.units provides the SI dimensions, units and constants; the examples here declare their own dimensions to show the underlying rules.

dimension Length("m")
dimension Time("s")
dimension Velocity = Length / Time

def main() -> Result[(), IoError]:
    metre = Quantity[f64, Length](1.0)
    kilometre = 1000.0 * metre
    distance = 2500.0 * metre
    elapsed = Quantity[f64, Time](100.0)
    speed: Quantity[f64, Velocity] = distance / elapsed
    print(distance.to(kilometre), speed.get())?
    Ok(())
2.5 25.0

Construction and conversion

Quantity[T, D](value) accepts T, with ordinary literal inference and domain literal validation. .get() returns the stored T. The field is private. There is no implicit conversion to or from a plain number, including for Quantity[T, One].

For floating storage, q.to(unit) returns (q / unit).get(). Both quantities must have the same storage and dimension; the unit’s value determines the scale. For integer storage, / is unavailable, so use explicit // and .get() when integer floor conversion is intended.

Quantity[f32, D].from(q) and Quantity[f64, D].from(q) convert a quantity of the same dimension and plain numeric storage, with ordinary numeric cast semantics. They neither change dimensions nor infer a scale.

Quantity[Positive[T], D].try_from(n) returns Result[Quantity[Positive[T], D], RangeError]; the same operation supports NonNegative and NonZero. It takes an integer of the domain’s underlying width and checks the domain exactly as its ordinary try_from does.

Operators

The storage operation follows the existing numeric rules, including width checks, literal inference, domain restrictions, overflow and checked:. The table’s R is the storage operation’s result type.

OperationDimensions and result
a + b, a - bEqual dimensions; Quantity[R, D]
a * bQuantity[R, A * B]
a / b, a // bQuantity[R, A / B]; / requires floats, // integers
a % bEqual dimensions and integer storage; Quantity[R, D]
q * k, k * q, q / kNumeric scalar; Quantity[R, D]
k / qQuantity[R, One / D]
+q, -qSame dimension, storage result R
==, !=, <, <=, >, >=Equal dimensions; bool
+=, -=Equal dimensions and storage; result must fit the destination type
*=, /=Scalar only; result must fit the destination type

A scalar cannot be added to or compared with a quantity, even when it is zero. Bitwise operators and other compound assignments are unavailable. Operands are evaluated once in source order. Compound assignments preserve ordinary borrow checks and evaluate a collection index once.

For p: Quantity[Positive[i64], Length], p + p keeps positive storage, p * p has positive storage and dimension Pow[Length, 2], and p - p and -p have plain i64 storage while retaining Length. A plain integer quantity accepts a Positive or NonZero quantity divisor for // and %, with the same proof-based checks as the underlying integers. Domain dividends retain the underlying domain’s restrictions.

Inside checked:, failures remain OverflowError[T] and/or ZeroDivisionError, as for the underlying arithmetic. Integer domains use their underlying integer in OverflowError. Quantity construction and rewrapping do not change arithmetic policy.

Module constants

Declare units and derived values with const or pub const. Use a concrete Quantity[T, D] annotation; dimension aliases may appear in D. Initializers can use the Quantity constructor, other constants (including imports and forward references), numeric casts of extracted values, quantity operators, .get(), and floating .to(unit). The compiler evaluates them before generating code.

dimension Length("m")
dimension Time("s")

const metre: Quantity[f64, Length] = Quantity[f64, Length](1.0)
const kilometre: Quantity[f64, Length] = 1000.0 * metre
const minute: Quantity[f64, Time] = 60.0 * Quantity[f64, Time](1.0)
const speed: Quantity[f64, Length / Time] = kilometre / minute

assert kilometre.to(metre) == 1000.0

def main() -> Result[(), IoError | AllocError]:
    print(kilometre)?
    print(f"{speed:.2f}"?)?
    Ok(())
1000.0 m
16.67 m/s

Storage widths, dimensions and integer-domain results follow ordinary quantity arithmetic. A domain constructor accepts a valid literal or a value already in that domain. p - p still erases the storage domain, so a positive quantity cannot be declared from that subtraction. .get() on a domain-backed quantity returns the domain; call .get().get() to extract its plain integer.

Integer overflow and division by zero are compile errors. Conditional and short-circuit expressions check both branches but evaluate only the selected branch. A dimension mismatch in a skipped branch is still an error. Constant cycles are rejected. User function calls, fallible conversions and formatting are not constant expressions; for a float storage conversion, write Quantity[f32, D](f32(q.get())). See module constants for the common rules.

Formatting

print(q) and f"{q}" show the coherent value followed by a space and the coherent-unit symbol: 1.3 m/s. Numeric format specs apply to the number, including its padding; f"{q:08.2f}" gives 00001.30 m/s. Integer domains format as their underlying integers. Dimensionless quantities have no suffix.

Products use *, powers use ^, and multiple denominator factors are parenthesized: kg*m/s^2 or m/(kg*s). Symbols appear in the order their base dimensions are declared within a module, with modules ordered by name. std.units declares its bases in SI order, so a newton prints as kg*m/s^2. The order is presentation only: distinct dimensions with equal symbols remain distinct, and their symbols do not cancel.

dimension Mass("kg")
dimension Length("m")
dimension Time("s")

def main() -> Result[(), IoError | AllocError]:
    speed = Quantity[f64, Length / Time](1.3)
    print(speed)?
    print(f"{speed:.2f}"?)?
    print(f"{speed!r}"?)?
    momentum = Quantity[f64, Mass](2.0) * speed
    print(momentum)?
    print(f"{momentum!r}"?)?
    Ok(())
1.3 m/s
1.30 m/s
Quantity[f64, Length * Pow[Time, -1]](1.3)
2.6 kg*m/s
Quantity[f64, Length * Mass * Pow[Time, -1]](2.6)

str.repr(q), !r and quantities inside aggregates show Quantity[T, D](value), with the normalized dimension expression. That expression orders bases by qualified name, as dimension identity does, so it can differ from the symbol order. !a escapes non-ASCII characters in that debug spelling. !s converts the display spelling to text first, so f"{q!s:>12}" pads the whole number-and-unit text.

Quantities support display(out) and debug(out) with a mutable std.fmt.Formatter, including through std.fmt.display, debug, to_string and to_debug. Direct writer formatting and numeric f-string fields write the number and static symbols without allocating a quantity-specific string; ordinary formatter limits and writer errors apply. String-producing operations still allocate. Printing retains IoError, and str.repr retains AllocError; quantities do not add user formatting errors. Nested format specs and c retain their ordinary FormatError possibility.

Representation and current limits

Quantity[T, D] uses T’s native value and parameter/result representation. Dimensions add no runtime field, tag, or arithmetic operation. Construction and get retag the coherent numeric value without a constructor call; float to lowers to ordinary storage division. Integer-domain storage keeps the domain’s existing representation and arithmetic contracts. Ordinary classes keep their existing ABI.

Collections, aggregates, references, closures and generators use the same storage packing as their numeric payloads. Arithmetic and conversion do not allocate. The release instruction probe compares addition, multiplication and unit conversion with plain-number code. This does not promise that every surrounding function call or temporary is optimized away. Generic instances that differ only in dimension share one body unless they format a value or build a collection, whose type descriptor names the dimension.

Unit spellings live in static type metadata and compiled formatting calls, not in each value. A single quantity carries no type pointer: the compiler knows its dimension at every use and embeds the symbol in the formatting code. A list already holds one pointer to its type descriptor, whatever its element type, because the runtime’s shared printing, sorting, hashing and cleanup code reads the element type through it. A list[Quantity[f64, D]] stores plain 8-byte f64 rows; only its descriptor, and through it the unit symbol, differs from a list[f64]. Descriptors are static read-only data, so neither lists nor values allocate type information. std.units.wrap_list and strip_list therefore convert between the two list types by rewriting one header word, leaving every element untouched. See the remaining units work.

Type and dimension errors include scoped alias notes, while debug formatting keeps normalized dimensions.

Tuples

(a, b) creates a tuple; (value,) creates a one-component tuple. Parentheses without a comma group an expression, and empty () is unit. Annotations accept tuple[A, B] or (A, B). A one-component type requires (A,) or tuple[A]; (A) groups the type A, just as parentheses group an expression.

Tuples are structural products with fixed component types. Construction allocates nothing; component expressions retain their own allocation and error contracts. Tuples containing owned components move, while copyable tuples copy their components. Resolved owned closures can be stored; references and generators cannot.

A mut tuple binding permits mutable component borrows regardless of whether its components are copyable. A copied tuple has independent inline storage; changing the original does not change the copy. Components can be changed through &mut pair[0] or a mutable payload match. Direct indexed assignment (pair[0] = value) remains unsupported; assign through the reference instead.

pair[0] selects a component. Indices must be integer literals. A negative literal counts from the end, as in Python: for a tuple of length n, pair[-k] is pair[n - k], and pair[-0] is pair[0]. The same resolved component is borrowed by &pair[-1] and &mut pair[-1]. An index outside -n to n - 1 is a compile error that states the tuple’s length. Owned components may be observed or explicitly copied; moving them out requires whole-tuple unpacking:

number, text = (42, "cm")
number, text = 42, "cm"

Recursive unpacking transfers each selected component; _ destroys a discarded component. The parentheses are optional on the right of an unpacking statement, and targets may also be fields and collection elements: see assignment to several targets.

Unpacking a tuple reference borrows the components in place instead. With p: &(i64, str), n, s = p binds n = &p[0] and s = &p[1]; with a &mut reference, or n, s = &mut pair, each name is a &mut reference to its component, and _ skips one. The names are new immutable bindings: they cannot reuse existing names or be declared mut. for n, s in &pairs unpacks each element the same way, as described under iterable values.

def bump(p: &mut (i64, str)) -> Result[(), OverflowError[i64]]:
    n, _ = p
    *n += 1
    Ok(())

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    mut pair = (1, "one")
    bump(&mut pair)?
    n, s = &pair
    print(*n, s)?
    Ok(())
2 one

A slice with integer-literal bounds and step selects components at compile time: (1, "two", 3.0)[1:] has type (str, f64), and pair[::-1] reverses a pair. It allocates nothing, so it returns no Result and takes no ?. Slices of affine components are rejected for now.

Nested tuple targets mirror the value’s shape: ((x, y), label) = value. Parentheses without a comma group a target; (x,) requires one component. The same shapes work for owned and borrowed loop elements, comprehension bindings and generator-expression bindings. Borrowing a nested tuple keeps component-precise loans, so sibling components remain independent. Starred unpacking is unsupported. Unit components can be discarded or used as unit expressions without enabling standalone unit storage.

An owned list also supports unpacking with an exact runtime length check at each list position in a nested target; tuple positions use their statically known arity. See collection rules. Tuples of integers, bool, strings, and nested tuples of those types are copyable dictionary keys and set members. Hashing and equality use their contents, with no formatting or string-key encoding.

Tuples whose components are integers, floats, strings, or recursively orderable tuples support <, <=, >, and >=. Both operands must have exactly the same type. They compare lexicographically, using the first differing component, with the same allocation-free total order as list.sort(). This also works through references and in generic functions; comparison observes both operands. Booleans, unit, classes, ordinary enums, and collections are not orderable fields.

Floating-point tuple fields use the sorting order: negative zero precedes positive zero, and all NaNs compare equal in order and follow every number. Structural == and != still use IEEE equality. Consequently, (-0.0,) == (0.0,) and (-0.0,) < (0.0,) are both true; a NaN tuple is <= itself but not == itself. Scalar float operators keep their IEEE behavior.

The difference is deliberate. Tuple ordering exists to sort and rank records, so it must be total: every pair of tuples is ordered, and < on tuples agrees with sort(). IEEE ordering would make a tuple containing NaN unordered against every other tuple. Plenty has one total order rather than separate partial and total orders; the ordering proposal gives the rationale.

Distinct components of a named tuple can be borrowed independently. Reads of one component can continue while another is exclusively borrowed; whole-tuple access overlaps all components. See borrowing for projection and collection-element loan rules.

Anonymous sums

A | B holds one owned value of either whole type. It works in ordinary bindings, fields, arguments and returns, including Result[T, A | B] errors. The compiler stores a tag and the largest active payload inline; wrapping and widening allocate nothing. Payload construction retains its own costs.

Aliases, order, duplicates and nested sums normalize: A | B, B | A and (A | B) | A are identical; A | A is just A. Distinct nominal classes and enums remain distinct, and enum variants never become separate members. There is no stable tag numbering or foreign layout contract.

Structural formatting (print, str.repr, and std.fmt.represent) shows the active member directly, without an anonymous wrapper. For example, a Result[(), IoError | ThreadError] error can be represented as Result[(), IoError | ThreadError].Err(ThreadError.System(11)). Type names use the normalized A | B spelling. The structural fallback for an error returned by main follows the same rule; named enums keep their usual qualified variants.

A closure signature such as OnceClosure[[], i64] names no concrete environment, so it is a member only in a function’s declared return type, directly or inside a member such as SpawnError[OnceClosure[[], i64]]. The body’s returned or propagated closure resolves it; see closure return inference.

| binds less tightly than references and type applications. Use &(A | B) to borrow a sum. &A | B attempts to store a reference as a member and is rejected, as are other members forbidden by owned storage rules. Callable brackets delimit their types: Callable[[A | B], C | D]. Parentheses without a comma group a type; (A,) is a singleton tuple and () is unit.

An expected type permits lossless injection of a member or widening to a superset. This applies to annotated bindings, assignments, arguments, constructors and returns. Result[T, E] can widen to Result[T, F] when F contains every member of E; its success type must remain identical. Other generic containers and references do not gain covariance. Conversions evaluate their source once, move owned payloads and invoke no user conversion. Public return types and binding types remain declared, never silently widened. A constructor, tuple or collection literal takes its context from the one member it can build, so [1.0, 2]? passed as list[f64] | Unset checks its elements as f64, and []? or {}? needs no further annotation. Several viable members, such as list[f64] | list[i64], are ambiguous and rejected; bind the value with a member type first.

Match a whole member with case name: MemberType:. An ordinary wildcard matches any remaining value. A member annotation names a single normalized type; pattern | means alternatives, not a type union. Typed member patterns are supported at the case position, not inside other patterns.

An enum variant pattern also selects its member directly, including inside Err or another payload. The compiler tests the member before testing its variant, so equal variant names in different enums remain distinct:

def parse(text: &str) -> Result[i64, AllocError | ParseError]:
    Ok(i64.parse(text)?)

def main() -> Result[(), IoError]:
    match parse("x"):
        case Ok(value):
            print(value)?
        case Err(ParseError.Invalid):
            print("not a number")?
        case Err(ParseError.OutOfRange):
            print("outside i64 range")?
        case Err(_):
            print("out of memory")?
    Ok(())
not a number

Every member and each enum member’s variants must be covered, whether by variant patterns, a whole-member pattern, or a wildcard. Duplicate and unreachable cases are rejected; covering one variant does not cover its siblings. Payload literals still need a fallback for the other payload values.

The matched value supplies generic arguments, as in ordinary enum patterns. If a qualifier names several members, such as Choice[i64] | Choice[str], Choice.Value(value) is ambiguous even when only one payload type would fit its subpattern. Use a whole-member pattern with an explicit type and match that value’s variants inside the arm. Case guards remain unsupported.

An owned match moves noncopyable payloads. Matching &value or &mut value borrows the active member with the same permission and lifetime. Only the active payload is initialized or destroyed. Copyability and thread transfer require every alternative to qualify. If generic substitution collapses T | A to A, case value: A: binds that ordinary value; two equivalent member arms are still duplicates. Generic inference rejects ambiguous member substitutions; supply explicit type arguments in that case.

For reporting, a sum satisfies std.fmt.Formattable when every normalized member satisfies that protocol. Formatting dispatch borrows the active member without allocation or consuming the sum. A missing or mismatched method on any member rejects the whole bound, regardless of which member is active.

This version does not infer residual sums after recovery, introduce empty sums, forward arbitrary member methods or protocols, or erase errors. See propagation for ? and builtin Failure, the last-resort error that erases its cause.

The anonymous-sum proposal records the design rationale and migration boundary. The composition overview distinguishes sums from deliberate error mapping and typed stream APIs.

Collection keys

A sum satisfies KeyType when every member is a supported key type. Use set[i64 | str] or dict[i64 | str, V] directly. Hashing and equality include the active member type and its structural value: 1, 1i8, True and "1" stay distinct in a sum that admits them. Normalized aliases and member order do not change identity. Keys with an unsupported member are rejected even if that member is never inserted.

Insertion and lookup use the ordinary expected-type injection rules. Observed member values can be injected for lookup without moving them; their source remains borrowed through the operation. This does not make reference types covariant. Sums containing move-only classes keep their ownership rules, and stored keys are only exposed through shared borrows. Anonymous sums use the hash-table representation, including sums with finite members.

Classes and construction

A class declares typed fields and associated methods. It has no inheritance, dynamic attributes, class variables, properties, or runtime method lookup.

class Point:
    x: i64
    y: i64

    def squared_length(self) -> Result[i64, OverflowError[i64]]:
        Ok(self.x * self.x + self.y * self.y)

Without __init__, the generated constructor takes fields in declaration order. Point(3, 4) and Point.new(3, 4) return Point directly. Construction itself allocates nothing; arguments and initializer operations retain their own error contracts. new is a reserved class member.

A class name without arguments is a function value with the constructor’s positional signature: make = Point lets make(3, 4) construct a point. A generic class needs an explicit instance, such as Cell[i64], or a concrete expected Callable result naming the class instance. Constructor values require all positional arguments; parameter names and defaults belong to the statically resolved constructor call. Taking the value allocates nothing and preserves constructor visibility and fallibility.

An explicit __init__ overrides the generated constructor and returns either () or Result[(), E]. The fallible form makes both constructor spellings and constructor function values return precisely Result[Class, E]. The declared error type is independent of the caller’s arithmetic mode; successful fallible initializers write explicit Ok(()). Non-unit Result success types are rejected. Storage itself allocates nothing, so construction does not add AllocError. Initializer operations and parameter defaults must declare or explicitly map their actual errors into E, including allocation failures when applicable. __del__ and __exit__ still require (); cleanup cannot propagate failure. Class parameters in an initializer output, including a parameter naming the whole Result, are substituted before validating that output. Unit class type arguments remain unsupported; a concrete unit alias can still name an initializer output in an otherwise valid generic class. Fields may contain classes, enums, tuples, collections, and resolved owned closures, but not references, generators, or unit.

Every successful initializer exit must initialize every field. Branches merge their definitely initialized fields; a loop alone cannot establish initialization because it may run zero times. Field assignments performed inside an if or match block used as a binding or assignment’s value do not establish initialization after that value block. Assign the selected value directly to self.field instead; the outer assignment initializes that field. See conditional values. Initialized fields may be read. Borrowing or passing the whole partially initialized instance is rejected, and initialization cannot be delegated to another method. Constructors accept named arguments, using field names for generated constructors and parameter names for explicit initializers. Explicit initializer parameter defaults are inherited by both constructor spellings. See call rules. Field defaults, constructor overloading, and static methods are unsupported.

Failed initialization destroys initialized fields and remaining owned arguments and skips the class’s destructor. The destructor becomes active only after successful initialization; see cleanup.

Classes can define arithmetic operator hooks such as __add__ and __neg__, compiled as direct method calls. Any other __name__ method besides __init__, __del__, __enter__ and __exit__ is rejected, so Python special methods such as __str__ and __len__ cannot be silently ignored.

Class identity is nominal. Structural equality compares fields without calling user methods. Printing uses Point(x=3, y=4) unless the class defines display or debug. Comparison and hashing hooks such as __eq__, __lt__ and __hash__ are rejected; see equality, ordering and hashing. Automatic equality and formatting reject recursive data. Instance moves, field access, and explicit duplication follow ownership; receiver signatures follow methods, and constructor access follows visibility.

A class pattern such as case Point(x, y=0): takes a class apart in match, written like the generated constructor with every field named once.

One-field classes

A class with exactly one field and no __del__ is represented as that field. Wrapping a value in such a class, reading its field, or moving the field out does not copy or allocate, so a wrapper such as class Handle: _file: File costs the same as the bare File. The field’s type is unrestricted: a scalar, str, a collection, an enum, or another class. Nothing in the source changes: the class keeps its name, methods, equality, and printing (Handle(_file=...)), and the field keeps its visibility. A one-field class with __del__ keeps a separate record because its cleanup state needs storage. See value representations.

A field whose type is another class stores that class’s fields inside the owner, at a fixed position. Reading path.start.x costs the same as reading a field of path itself, and moving or returning path copies its bytes once, with no per-field address work. This is not visible in source: nested fields still move, borrow, and run __del__ as other fields do.

Copy classes

@copy on the line before class declares a copy class. Its instances copy on ordinary value use, as copyable tuples and enums do: assignment, owned argument passing, return, collection insertion, capture, and by-value receivers (self: Point) all leave the source usable. Copying duplicates the inline fields and never allocates; string fields share immutable storage. copy(value) still accepts the instance. Other classes remain affine.

@copy
class Point:
    x: i64
    y: i64

def main() -> Result[(), IoError]:
    p = Point(1, 2)
    mut q = p
    q.x = 5
    print(p, q)?
    Ok(())
Point(x=1, y=2) Point(x=5, y=2)

The declaration is checked as a whole:

  • Every field must copy: a scalar, str, another @copy class, or a tuple or enum whose components copy. The error names the field and its type.
  • The class cannot define __del__, because each copy would run it.

A generic @copy class copies when its type arguments copy, as a tuple does. Pair[i64] and Pair[str] copy; Pair[list[i64]] is accepted and moves. The field rule is checked at the definition, whether or not the class is used: a field must copy whenever every type argument copies. T, (T, U), Option[T] and Pair[T] qualify; a field such as items: list[T] or b: Box[T] moves for every argument and is reported at its declaration. A CopyType bound (or IntType or FloatType) restricts the class to instances that copy:

@copy
class Pair[T]:
    left: T
    right: T

def main() -> Result[(), AllocError | IoError]:
    numbers = Pair(1, 2)
    copied = numbers
    lists = Pair([1]?, [2]?)
    moved = lists
    print(numbers, copied, moved)?
    Ok(())
Pair[i64](left=1, right=2) Pair[i64](left=1, right=2) Pair[list[i64]](left=[1], right=[2])

Equality, printing, field access, and matching are unchanged. Classes are collection keys when every field is a supported key type. Custom __del__ cleanup is allowed, including in nested fields; such keys cannot be copied for set algebra or key-view to_set(). Hashing reads their fields structurally, just as equality does. Ordinary instances move into the collection; @copy instances copy. __hash__ and __eq__ hooks are unsupported.

Annotations

An annotation is @name alone on a line directly before a declaration. The set is closed and compiler-known; there are no user decorators. The annotations are @copy, @newtype, and @redacted. @copy and @newtype apply only to class declarations, including pub class; @redacted also applies to enums. An unknown name, arguments, a repeated annotation, or an annotation before any other declaration, member, or statement is an error.

@newtype makes a class forward its field’s operators; see newtype forwarding. The class must declare exactly one field and cannot define __init__. It can have type parameters (@newtype class Wrapper[T]: value: T) and __del__.

@copy and @newtype are independent. @newtype does not imply @copy: a @newtype class over i64 still moves on assignment unless it is also @copy. Forwarded operators borrow their operands, so a + a and reusing a after a + b work either way. A class can carry both annotations, in either order, with the same meaning:

@newtype
@copy
class Seconds:
    value: f64

def main() -> Result[(), IoError]:
    a = Seconds(1.5)
    b = a
    print(a + b)?
    Ok(())
Seconds(value=3.0)

Neither annotation changes layout or printing. Every class with one field and no __del__ uses the one-field representation, with or without @newtype, and a @newtype value prints as its class (Seconds(value=3.0)). Annotation names follow @ and do not reserve words: copy remains an ordinary name, and the builtin copy(value) is unaffected.

Redacted types

@redacted on a class or enum hides its contents from every automatic formatting path: print, str.repr, f-string fields, and the report main writes for a returned error. The value renders as its type name followed by (@redacted), at any nesting depth: in a list, dictionary, tuple, Option, Result, enum payload, or another class’s field.

@redacted
class ApiKey:
    secret: str

class Client:
    host: str
    key: ApiKey

def main() -> Result[(), AllocError | IoError]:
    client = Client("example.com", ApiKey("hunter2"))
    print(client)?
    print([ApiKey("other")]?)?
    print(client.key.secret.byte_len())?
    Ok(())
Client(host="example.com", key=ApiKey(@redacted))
[ApiKey(@redacted)]
7

The rules:

  • Redaction applies to the whole value. There is no per-field form; a class that mixes secret and public data either is redacted entirely or keeps its secret in a separate redacted field type, as Client does above.
  • The name includes type arguments and module qualification as ordinary printing would: a generic Secret[T] prints Secret[i64](@redacted).
  • A generic wrapper over a redacted type is not itself redacted; Holder[ApiKey] prints Holder[ApiKey](item=ApiKey(@redacted)). A @newtype over a redacted type likewise prints its own name with the redacted field.
  • In an f-string, !r, !s, !a, =, and every format spec give the same ApiKey(@redacted); there is no conversion that reveals the contents. Padding and alignment apply to the marker.
  • Explicit formatting is unaffected: a display or debug method the type defines is used by print, str.repr and f-strings as for any other type, and a std.report Formattable implementation still writes what it chooses.
  • Field access, equality, hashing, ordering, matching, moves, and layout are unchanged. @redacted combines with @copy and @newtype in any order.

Redaction is a guard against accidental logging, not a security boundary: code that can read a field can still print it.

Enums

enum Reading:
    Missing
    Value(i64)
    Invalid(str)

Enums are nominal module-level types. Variants have fixed positional payloads; nullary declarations omit parentheses. Reading.Value(7) and Reading.Value.new(7) return Reading directly. Nullary variants use Reading.Missing or Reading.Missing.new().

Construction allocates nothing. Payload expressions evaluate first and transfer their values into the enum; their own allocation and error contracts still apply. Payloads can include primitive values, classes, collections, tuples, other enums, unit, and resolved owned closures. An enum copies when every payload of every variant copies, and moves otherwise; @copy does not apply to enums. See the copy rule. References and generator-containing values cannot be user enum payloads. Standard Option/Result wrappers can contain generators, including through nested standard wrappers. A function result may carry a reference in the Some or Ok payload; such a value is held only in a local and never stored, as described in optional and fallible references.

Constructors can qualify variants with an enum name, transparent alias, or explicit generic instance; patterns use the enum name or an alias and never type arguments. Public enums expose every variant.

A payload variant named without arguments is a function value: Reading.Value has type Callable[[i64], Reading], and a variant with two payloads takes two arguments. Pass it where a callable is expected, for example i64.parse(text).map_error(ConfigError.InvalidCount) or iter(range(3)).map(Reading.Value). A generic enum needs an explicit instance, as in Wrapped[i64].One, or a concrete expected Callable whose result names that instance. This includes Option[i64].Some, Result[i64, str].Ok, and Result[i64, str].Err. The prelude Some, Ok, and Err specialize from an expected callable signature, for example wrap: Callable[[i64], Option[i64]] = Some. Nullary variants remain values of the enum, not functions. A constructor value cannot have unit or generator payload parameters; use a closure for those signatures. Enums with permitted element types can be list elements or dictionary values; they can also be dictionary keys or set elements when every payload is a supported key type. Hashing includes the variant tag and payloads. Recursive enums remain excluded. Anonymous sums qualify when every member is a supported key type.

Equality compares nominal identity, variant, and payload contents, including IEEE float comparisons; it never calls user methods, and comparison and hashing hooks are rejected. Printing uses qualified variant names unless the enum defines display or debug. Automatic equality and formatting reject recursive data types.

Matching selects variants and binds payloads. Standard Option and Result add prelude constructors and contextual inference. Generic enums follow generic data.

Methods

An enum can declare ordinary methods alongside its variants:

enum Reading:
    Missing
    Value(i64)

    def number(self) -> i64:
        match self:
            case Reading.Missing:
                0
            case Reading.Value(value):
                *value

Bare self borrows the enum; self: &mut Reading takes an exclusive borrow. Matching that receiver borrows the active payload under the ordinary matching rules. Methods may have their own generic parameters and use named arguments and defaults under the ordinary method rules. Generic enum parameters come from the receiver, as described in generic data.

Methods are private to their defining module unless declared pub def. Enum methods follow the class rule for reserved names: a __name__ method must be an operator hook, since enums have no lifecycle or context hooks. Variant and method names must be distinct. Class lifecycle hooks are not enum methods: variant construction and automatic payload destruction retain their existing behavior. Anonymous sums cannot declare methods. They forward only the std.fmt.Formattable contract when every member satisfies it; arbitrary member methods and protocols are not forwarded.

Enums can satisfy structural protocols with accessible matching methods. They can also supply display and debug methods for std.fmt; builtin printing and entrypoint reports continue to use structural rendering.

Recursive storage

Classes and user enums may refer to themselves or one another, including through Option, Result, tuples, and transparent aliases. Every storage cycle must pass through a heap boundary: Box[T], a list, a set, or a dictionary. A cycle without one would require an infinite-sized value and is rejected.

class Node[T]:
    value: T
    next: Option[Box[Node[T]]]

Finite concrete generic instances can recur, including mutual instances with different arguments. Alias-only cycles remain invalid. Endless type expansion, such as a Node[T] containing a Node[list[T]], is rejected when specialized; the diagnostic names the first three instances and points at the type application that grows. Finite nesting deeper than 64 levels reports that limit. These limits concern type descriptions rather than the number of nodes in a runtime chain or tree.

Recursive values and aggregates containing them move. Automatic copy, equality (including membership), and formatting reject types whose stored values can recur, even when a particular value is shallow. Callable signatures do not constitute stored recursive values. Consuming matching transfers payloads; borrowed matching preserves the owner. See ownership.

Moving a field through a Box, stored references, multiple mutable owners of one node, and source ownership cycles are unsupported. A terminating variant, Nothing, or an empty collection supplies a construction base case. A list can own all nodes while integer IDs represent parent links, shared targets, or cycles: such graph edges do not carry ownership or create loans. Index stability, stale IDs, and graph consistency remain the application’s responsibility. See representation choices. Shared[T] nodes may form immutable trees and DAGs, but not cycles: a Shared[T] whose content can reach it again through a Mutex or channel is rejected, so back-references use these IDs.

Automatic destruction of recursive owners uses bounded stack space and allocates nothing. Explicit recursion in user methods and destructor hooks still uses the native stack. Box construction, content conversion, projections, and borrowing are described under facilities.

The recursive representation proposal records the storage and destruction rationale. The owned-structure assessment compares linked lists, trees, and checked-index arenas using these facilities.

Expressions and evaluation

Arguments and ordinary binary operands evaluate once, from left to right. and and or short-circuit; conditions and Boolean operators require bool rather than truthiness. Conditional expressions evaluate only the selected branch. Assignment evaluates its value before destination indices.

Operator precedence, from highest to lowest:

GroupOperators or forms
PostfixCalls, member access, indexing, slicing, error propagation ?
Unary+, -, bitwise not ~, dereference *, borrowing & and &mut
Multiplication*, /, //, %
Addition+, -
Shift<<, >>
Bitwise and&
Bitwise exclusive or^
Bitwise or|
Comparison and membership==, !=, <, >, <=, >=, in, not in
Boolean negationnot
Boolean conjunctionand
Boolean disjunctionor
Conditionalvalue_if_true if condition else value_if_false

Arithmetic operators associate left to right. Parentheses override precedence. Chained comparisons are rejected. There is no exponentiation operator: two adjacent stars, as in a ** b, are rejected with a diagnostic that names base.pow(exponent), whatever the operand types. Multiplying by a dereference needs a space: a * *b. Numeric rules specify type matching, casts, overflow, and float behavior. Membership follows the container’s facilities contract.

Ordering operators accept matching integers, floats, strings, and orderable tuples. Tuple operators use the same lexicographic total order as sort(), including its signed-zero and NaN rules; scalar float comparisons retain IEEE behavior.

Format strings build text from fields with static specs. Suites and conditionals have value and exit rules. Enums support exhaustive matching; loops provide repeated execution. Postfix ? follows error propagation, and resource scopes follow cleanup.

Nesting

Blocks and expressions nest to at most 64 levels in total. A function body is level 1. A suite is one level below the statement that owns it, and so are that statement’s expressions. An operand, argument, index, or element is one level below the expression that holds it, and parentheses add a level around what they enclose.

Operators group to the left, so a + b + c is (a + b) + c: every binary operator, call, member access, index, or ? in a chain puts everything before it one level deeper. A statement in a function body can chain 62 of them. Each manager of a with statement is one level around the managers after it and the body, and each for or if clause of a comprehension is one level around the clauses after it and the element. The elif arms of an if are not nested.

Deeper source is rejected at the token that reaches level 65, with block and expression nesting exceeds the implementation limit of 64. Bind an intermediate value to a name, group a long chain with parentheses, or move inner blocks into a function. Types and patterns have their own limits of 64.

Binding and assignment

name = expression first declares an immutable local with inferred type. name: Type = expression adds an annotation. mut name = expression or mut name: Type = expression declares a mutable local.

The value of a binding or a single-target assignment is an expression, an if block, or a match block. The selected suite supplies a block’s value.

Later name = value assigns to an existing local and requires mut and the same type. Repeating mut or an annotation for that name is a duplicate declaration. Parameters are immutable, and declarations always have an initializer. Copies, moves, and exclusive access follow ownership.

Field and indexed assignment accept nested writable paths such as rows[0][-1] = 9 and records[0].values[1] = 7. The root must be mutable or an exclusive reference. Existing list slots change in place, without copying or allocating their containing owners. Dictionaries have no indexed assignment, so an assignment path cannot write through dictionary[key]; use insert(key, value)? to add or replace, or get_mut to change a value in place.

Assignment proceeds in this order:

  1. Evaluate the right-hand value.
  2. Evaluate destination indices once each, from root to leaf.
  3. Resolve the destination using the resulting contents, checking bounds or keys.
  4. Replace the destination while holding exclusive access.

Index expressions can read or resize the destination. Their values are saved before any address is resolved, and no user expression runs between address resolution and the write. Replacing an owned value destroys its previous value exactly once. If an index propagates an error, a pending right-hand owner is cleaned up and no write occurs.

Whole-instance assignment through *reference = instance is rejected for class references, including class-valued field references. Replace the owning binding or a named field instead.

Augmented assignment

target += value, -=, *=, /=, //=, %= and integer bitwise updates &=, |=, ^=, <<=, >>= update an existing numeric destination. The destination can be a mutable binding, field, collection element, or exclusive reference, including nested paths. The right side takes the destination’s numeric type. Integer division uses //=, float division uses /=, and %= currently accepts integers. Strings and collections do not support augmented assignment; string concatenation remains fallible and explicit. Inside a checked: block, an integer update that overflows or divides by zero exits the block with Err before writing back, so the destination keeps its previous value.

Unlike ordinary assignment, augmented assignment evaluates its target before the right side:

  1. Evaluate each target index once, from root to leaf, saving its value.
  2. Read the destination’s current scalar value.
  3. Evaluate the right side and apply the arithmetic operation.
  4. Write back to the same destination using the saved indices.

A shared loan protects the destination during right-side evaluation. The right side may observe it (count += count), but cannot move, replace, or resize its owner. Collection loans cover the whole collection unless distinct literal list indices separate them. Index expressions may mutate the owner before the destination is read. If an index or the right side propagates an error, the update performs no write. Invalid list indices trap as in ordinary indexed assignment; dictionary elements are not destinations. The update itself allocates nothing, introduces no binding, and cannot initialize an unread constructor field.

User-defined classes and enums can provide explicit mutable in-place hooks; see operator overloading. Those calls use the method’s implementation rather than the scalar read/compute/write sequence.

Several targets

target, target = value, value assigns to two or more targets at once:

def main() -> Result[(), AllocError | IoError]:
    mut items = [10, 20, 30]?
    items[0], items[2] = items[2], items[0]
    print(items)?
    mut low = 1
    mut high = 2
    low, high = high, low
    print((low, high))?
    Ok(())
[30, 20, 10]
(2, 1)

The right side is one tuple. With several targets its parentheses are optional, so a, b = b, a means a, b = (b, a); a single target still requires them. Its component count must equal the target count.

  1. Evaluate the whole right side.
  2. Assign each component to its target, from left to right, exactly as a single assignment to that target would.

Because the right side is complete before the first write, targets can exchange values. Each target evaluates its own indices when its turn comes, so an earlier target can change a later target’s index: i, items[i] = 2, 5 writes items[2]. A target is a name, a field, a collection element, *reference, or _, which destroys its component. A name follows the single-assignment rule: it assigns an existing mut local or declares a new immutable one. mut a, b = value declares mutable locals and accepts names or nested tuples of names only. Tuple targets nest recursively: (a, (b, c)) = value requires the same tuple shape. mut (a, b), c = ((1, 2), 3) and mut ((a, b), c) = ((1, 2), 3) declare the same nested components; the first tuple remains one target when an outer comma follows it. A singleton (a,) still requires one tuple component. Leaves are assigned left to right after the whole right side is evaluated; field, index and dereference leaves follow the same rules as flat targets. Naming the same local twice, starred targets, and a closure capture as a name target are rejected. In a constructor, a self.field target initializes its field, and later targets may read it.

The right side obeys ordinary ownership, so reading an owned element such as a nested list still cannot move it out of its collection. Exchange owned list elements with swap. When a target traps on a missing index or key, earlier targets have already been assigned. Nothing is allocated.

Replacing a field

replace(owner.field, replacement) follows the same evaluation order but returns the previous field value to the caller instead of destroying it. It requires the same type, visibility, mutability, and loan checks. It supports nested fields, boxed owners, and fields reached through collection elements. It allocates nothing and keeps the field initialized. It also accepts a list element and the target of a mutable reference, replace(*reference, replacement), but not a whole binding or temporary receiver. A user declaration named replace continues to resolve normally.

Use replace when the owner is reached through a reference or defines __del__, or when it must stay usable as a whole. A field of an owned binding can instead move out and be assigned later.

User-defined arithmetic operators

Classes and user-defined enums can implement arithmetic with Python-style special methods. The compiler selects the method statically and emits an ordinary direct call. There is no runtime lookup, boxing, or allocation for dispatch.

OperatorLeft methodReflected methodIn-place method
+__add____radd____iadd__
-__sub____rsub____isub__
*__mul____rmul____imul__
/__truediv____rtruediv____itruediv__
//__floordiv____rfloordiv____ifloordiv__
%__mod____rmod____imod__
&__and____rand____iand__
|__or____ror____ior__
^__xor____rxor____ixor__
<<__lshift____rlshift____ilshift__
>>__rshift____rrshift____irshift__

Unary +, -, and ~ call __pos__, __neg__, and __invert__. Precedence and associativity are unchanged.

Signatures and resolution

A value-producing hook has a shared self, exactly one other parameter for binary operators, and no other parameters for unary operators. The other operand is a shared reference or copyable value. Its type can differ from the receiver’s. The result is an owned value of any supported return type, including Result; unit and reference results are rejected. Hooks cannot declare method type parameters, parameter defaults, or yield. Generic classes and ordinary generic functions can use operators after specialization.

For a + b, the compiler selects a.__add__ if declared; otherwise it selects b.__radd__ if declared. A selected hook’s signature must match: a type error does not cause fallback. There is no dynamic NotImplemented protocol, subtype priority, or implicit conversion. A reflected method is allowed for operands of the same type when the left method is absent. Ordinary member visibility applies: exported APIs need pub hooks.

Both operands are evaluated exactly once, left to right, even for a reflected call. Shared operands stay borrowed through the call. Temporary owners survive the full expression and are destroyed normally; no implicit deep copy occurs. Methods can still be called explicitly.

An operation that allocates should return Result, just as a named method does. Write (a + b)? to propagate failure: ? binds more tightly than +. The operator syntax does not automatically propagate, unwrap, or discard errors.

In-place operations

target += value calls target.__iadd__(value); the other in-place names follow the table. Its signature requires self: &mut T, one shared or copyable operand, and -> (). It mutates the existing destination and does not replace it with a returned owner. The target must be a writable binding, field, collection element, or exclusive reference. Its indices are evaluated once before the right side. The ordinary mutable-method receiver reservation prevents invalidating that place while evaluating arguments.

There is no fallback from += to +. For an operation that creates a new value or can fail, use explicit assignment (a = (a + b)?) or a named fallible method. This keeps allocation and error handling visible. An in-place hook can perform explicitly handled allocation inside its body; dispatch itself allocates nothing.

Newtype forwarding

A class annotated @newtype wraps exactly one field and forwards every operator that the field’s type supports, so a wrapper keeps its field’s arithmetic without one hook per operator:

@newtype
class Metres:
    value: i64

def main() -> Result[(), IoError | OverflowError[i64]]:
    mut total = Metres(3) + Metres(4)
    total *= Metres(2)
    print(total, -total, total > Metres(10))?
    Ok(())
Metres(value=14) Metres(value=-14) True

a + b on two Metres values lowers to a.value + b.value through the ordinary operator rules for the field type, then wraps the result:

  • A result of the field’s type becomes the class: Metres + Metres is a Metres.
  • A Result[F, E] result, where F is the field type, becomes Result[Class, E]. @newtype class Name: text: str gives Name + Name -> Result[Name, AllocError], written (a + b)?.
  • Any other result is returned as is, such as the f64 from a field hook __mul__(self, other: &Vec2) -> f64.
  • <, <=, >, and >= forward and return bool. == and != stay structural, as for every class.
  • Unary +, -, and ~ forward in the same way.
  • target += value performs the field’s own in-place operation, target.value += value.value, so it follows the field type’s +=.

Forwarding happens only when no hook is selected and both operands have the same @newtype class. Typing stays nominal: Metres + i64, i64 + Metres, and Metres + Feet are errors unless a hook accepts them; there is no implicit conversion. An explicit hook replaces forwarding for its operator only: a @newtype class with __add__ uses it for + and still forwards -. Reflected hooks apply only to mixed operand types, so they never compete with forwarding.

Each operand is evaluated once, left to right, and borrowed as a hook operand would be; forwarding never moves or copies the wrapper. Overflow errors, division rules, allocation failure, and other errors are those of the wrapped operation. An operator the field type rejects is rejected with the field type’s diagnostic: a list[i64] wrapper has no +, and an f64 wrapper has no %. Forwarding allocates nothing beyond what the wrapped operation does.

Forwarded operations build the result through the class constructor, which is why a @newtype class cannot define __init__. The declaration rules are in annotations.

Equality, ordering and hashing

Equality, ordering and hashing never call user code:

  • == and != compare fields structurally, including inside collections.
  • <, <=, > and >= on a class come only from newtype forwarding.
  • Dictionary keys and set elements use built-in structural hashing. Classes are not keys yet; named enums with supported key payloads are accepted.

A class, enum or protocol that declares __eq__, __ne__, __lt__, __le__, __gt__, __ge__, __cmp__, __hash__ or __key__ is rejected, so a Python-style hook cannot compile and then be ignored. Rename a method that is not meant as a hook, or compare explicitly:

class Money:
    cents: i64

    def same_amount(self, other: &Money) -> bool:
        self.cents == other.cents

def main() -> Result[(), IoError]:
    print(Money(1) == Money(2), Money(1).same_amount(&Money(1)))?
    Ok(())
False True

User-defined ordering through one __cmp__ hook is planned under B37, and user-defined keys under B74; see the ordering and hashing proposals.

Reserved method names

A method named __name__, with two underscores on each side, must be a hook the compiler calls: an operator hook in the tables above, or, on a class, __init__, __del__, __enter__ or __exit__. A class, enum or protocol that declares any other such name is rejected, so a Python special method cannot compile and then be ignored. The names stay free for hooks the language may add. Where Plenty has its own form, the diagnostic names it:

Python methodPlenty form
__str__, __repr__, __format__display and debug methods
__pow__a pow method, called as base.pow(exponent)
__iter__, __next__a generator method
__len__, __contains__an ordinary shared len method for len(value); explicit search methods for membership
__bool__a predicate method such as is_empty
__getitem__, __setitem__named methods such as get and set

len(value) calls an ordinary shared len(self) -> i64 method on classes and enums. Membership through in and not in still uses only intrinsic operations; user-container membership remains under B49. A name with underscores on one side only, such as __size, is an ordinary method.

Boundaries

Custom truthiness, indexing, membership, assignment, dereference, **, and @ operators are not implemented; their Python hooks are reserved. and, or, and not keep their boolean and short-circuit rules. Numeric operators retain their overflow checks and division rules. Operators a newtype forwards to an integer field report OverflowError of the field type through the function’s declared Result by default, through a local checked: block, or as a violated precondition in unchecked:, just like the field’s own operators. A user-defined hook is an ordinary call with its own declared contract, independent of the caller’s mode; a fallible hook still needs explicit Result handling.

See the operator tutorial, assignment rules, and implementation and performance notes.

Operator lowering and performance

Operator hooks use the same typed call operations, loan checks, argument ABI, caller attribution, and reachability/cache machinery as named calls. Reflected operations evaluate arguments in source order and reorder their already computed values for the call. They never rerun operand expressions. Dispatch uses the specialized nominal types; it does not add vtables or runtime type tests.

Keep four costs distinct:

  • Dispatch: an ordinary direct call, eligible for the same bounded inlining as an explicitly named call.
  • Borrowing: shared parameters lend existing storage without copying matrix payloads. The compiler may need an addressable temporary for a computed owner.
  • Results: inline class result storage uses the normal return ABI. Returning a large matrix still requires whatever allocation its implementation specifies.
  • Expression chains: (a + b) + c invokes two operations and may build two results. This implementation does not fuse loops, build expression templates, or silently change evaluation order or floating-point association. Named destination-writing methods remain useful for large numerical workloads.

Bounded optimization and evidence

The release leaf inliner accepts shared class receivers that only project scalar fields, alongside its existing scalar inputs and arithmetic. It still requires a scalar output, no locals, no calls or writes, and at most 24 non-marker typed operations. Tail calls to eligible leaves also substitute their body when the caller has no managed locals to clean up before the call. This benefits ordinary named methods as well as operator hooks. Managed aggregates, matrix allocation, and result-producing ordinary class constructors remain ordinary calls. Quantity values use their storage type’s scalar eligibility.

The regression probe compares a + b, a.__add__(b), and a.x + b.x for a class containing one f64. Operator and named calls have identical native instructions in both build modes. In release mode all three produce identical 12-instruction x86-64 bodies, with no call or jump to the hook. Before the inliner extension, both call spellings added a five-instruction tail-call wrapper. This is code-generation evidence, not a claim about matrix-kernel throughput. Separate tests disable heap allocation across owned class addition and in-place updates, and check destructor and failure paths.

Quantity erasure

The builtin Quantity carries its storage type’s native ABI. Its constructor and numeric projection lower directly, with no call, allocation, dimension tag, or constructor state. Direct get and float to retain argument validation and receiver loans while avoiding addressable receiver temporaries for named numeric values. Module unit constants substitute their folded storage literal and type retag at each read.

tests/test_quantity_erasure.rs compares complete release instruction listings for Quantity and plain f32/f64 functions: a + b, a * b, q.to(unit), and q.to(km) with a constant coherent value of 1000.0. On x86-64 the paired listings are identical, including parameter/result registers, with no Quantity constructor, method, constant-getter or caller-context calls. The test uses the host backend; native ARM execution remains part of ARM qualification rather than evidence from this x86-64 run.

Other tests exercise every numeric width and representative integer domains through direct/indirect and tail calls, references, aggregates, collections, closures and generators in both build modes. Signed zero, NaN, infinity and worker transfers retain their numeric payloads. Existing allocation-denial, checked-arithmetic, formatting and diagnostic tests cover the language contracts. This evidence does not imply identical instructions for arbitrary programs or shared bodies for different dimension instances.

Design sources

Python supplies the familiar hook names and the separation between ordinary, reflected, and in-place operations. Plenty deliberately uses static resolution, shared value-producing receivers, and unit-returning mutable in-place methods instead of Python’s dynamic NotImplemented and replacement-result behavior. See the Python data model.

D’s operator specification rewrites operators into named member calls and has separate right-hand and assignment hooks. Its operator-string templates reduce declaration repetition; they do not themselves eliminate allocation or intermediate results. Plenty uses ordinary named methods without adding string-template machinery.

The C++ operator rules likewise transform operator notation to function calls while preserving the operator’s sequencing rules. That separation motivates keeping source evaluation order independent of which operand owns the selected hook. Inlining remains an optimization: even Clang’s always-inline attribute does not guarantee substitution. Plenty reuses its bounded leaf substitution rather than adding an operator-only optimizer or promising that every call vanishes.

Rust’s Add trait separates the right-hand operand and output types. Plenty’s signatures likewise need not return the receiver type; this permits scalar results and explicit Result values without introducing associated types for this feature.

Suites, conditionals, and returns

A suite’s final expression supplies its value. Other expressions evaluate and are discarded, except that implicitly discarding a Result is a compile error. Handle it with ?, matching, or a function; drop(result) explicitly discards it. This applies in non-final conditionals, matches, loops, resource scopes, and generator bodies. It does not prohibit unused bindings, Option, or aggregates containing Results.

Continuing branches of value-producing if/elif/else suites must agree on type. An if without else can only produce unit. The inline conditional uses value_if_true if condition else value_if_false; only its selected value is evaluated.

An if takes any number of elif arms. The arms are alternatives at one level, tested in order; they do not count toward the nesting limit. An if written inside an else suite does.

def clamp_low(value: i64, minimum: i64) -> i64:
    if value < minimum:
        return minimum
    value

return value exits the enclosing function from any suite. Bare return returns unit. return if condition: returns its selected suite’s value. return match value: returns the final expression of the selected case. Every explicit return must match the declared result, including returns in non-final or statically unchosen branches.

A returning branch contributes no value to paths that continue. Every continuing function path must still produce the declared result; an if without else cannot prove that all paths return. This analysis follows structure without constant-condition folding. Statements after an explicit return or an exhaustive conditional whose branches all return are unreachable errors.

break and continue likewise contribute no branch value when they exit a suite inside a loop. Ordinary exits perform cleanup.

An if as a statement’s value

A block if can be the whole right-hand side of a binding or single-place assignment, or follow return. Its selected suite’s final expression supplies the value:

def classify(flag: bool) -> Result[i64, OverflowError[i64]]:
    result = if flag:
        base = 10
        base + 1
    else:
        2
    Ok(result * 2)

def main() -> Result[(), IoError | OverflowError[i64]]:
    print(classify(True)?, classify(False)?)?
    Ok(())
22 4

mut and annotations work as for ordinary bindings. An annotation or the assigned destination’s type supplies the expected type to every continuing arm, including every elif and else; this allows contextual Nothing and empty collection literals. Continuing arms must agree. Arms ending in return, break, or continue contribute no value. If every arm exits, write an ordinary if statement instead of binding its nonexistent value. An absent else produces unit on its unselected path; unit cannot be stored in a binding.

Conditions are evaluated in order; only the selected suite runs. Its locals are cleaned up before the binding or assignment, including before evaluating destination indices. ?, return, break, and continue keep their enclosing function or loop. A binding or assignment cannot receive a reference or a closure with borrowed captures from the block; produce an owned value instead. These are the same lifetime and evaluation rules as a match used as a value.

return if condition: returns each selected suite’s final expression as though it ended with return expression. Returned references follow ordinary returned-reference rules, and calls in these final positions remain source-tail calls. A suite without a value returns unit. Every path leaves the enclosing suite, so a statement directly after the whole block is unreachable. An arm that executes break or continue still exits to its enclosing loop rather than returning from the function; code after that loop can run.

The block ends the statement. Block if and match remain unsupported inside call arguments, operands, augmented assignments, or assignments to several targets. Bind the selected value first, or use an inline conditional for two single expressions.

Pattern matching

match evaluates its scrutinee once and selects a case. The scrutinee is an enum, an anonymous sum, a class, a tuple, an integer, a str, or a bool, or a reference to one:

enum Reading:
    Missing
    Value(i64)

def amount(reading: Reading) -> i64:
    match reading:
        case Reading.Missing:
            0
        case Reading.Value(number):
            number

A case holds one pattern. A pattern is one of:

PatternMatches
_any value, binding nothing
nameany payload, bound to an immutable local
_ as namethe whole matched value, bound to an immutable local
name: MemberTypea whole member of an anonymous sum; see member patterns
Enum.Variant or Enum.Variant(p, ...)that variant, with one pattern per payload position
Some(p), Nothing, Ok(p), Err(p)that standard variant
(p, q) or (p,)a tuple with one pattern per component
Class(p, ..., field=p, ...)a class value, with one pattern per field; see class patterns
1, -1, 0u8an integer equal to the literal
"text"a str equal to the literal
True, Falsethat bool
p | qa value either alternative matches

A user-defined or library variant is always qualified by its enum’s name, a module path to it, or an alias of the matched type. Some, Nothing, Ok, and Err are never qualified. A pattern takes no type arguments, because the matched value supplies them: case Choice.Value(v) matches a Choice[i64] and a Choice[str] alike, and case Choice[i64].Value(v) is an error.

At an anonymous-sum position, an enum variant pattern selects that enum member before testing its variant. This also works in nested payloads, such as Err(ParseError.Invalid) for Result[T, AllocError | ParseError]. Coverage includes every variant of every member. A generic qualifier that names several members is ambiguous; select one with a typed member pattern before matching its variants. See anonymous sums.

A bare name binds only at a payload or field position; the case’s own pattern is a variant, a class or tuple pattern, a literal, _, alternatives of those, _ as name, or a typed member pattern. A bare name as the case’s own pattern is an error, and a payload name equal to a variant of that payload’s enum is rejected, so a forgotten qualifier cannot silently bind everything. A variant pattern at a Box[T] payload matches the box’s contents. Patterns nest to at most 64 levels.

enum Shape:
    Dot
    Circle(i64)
    Rect(i64, i64)

def size(shape: Option[Shape]) -> i64:
    match shape:
        case Some(Shape.Circle(side) | Shape.Rect(side, _)):
            side
        case Some(Shape.Dot) | Nothing:
            0

A literal pattern matches an integer, str, or bool position: the scrutinee itself, a payload, or the contents of a Box. An unsuffixed integer literal takes the position’s type and must fit it; a suffixed one must name that type exactly. Float literals are not patterns.

Tuple patterns

Tuple patterns are recursive fixed-arity products. (p,) matches a singleton; (p) groups a pattern. Empty () is unit, not an empty tuple pattern; use _ for a unit component. A tuple pattern can appear at the root or inside enum, class, tuple, or boxed payloads. Every component must have a pattern. At an anonymous-sum position, it selects a unique tuple member with that arity. If several members have the same arity, bind one with a typed member pattern before matching its components.

def combine(left: Option[i64], right: Option[i64]) -> Result[i64, OverflowError[i64]]:
    match (left, right):
        case (Some(a), Some(b)):
            Ok(a + b)
        case (Some(a), Nothing):
            Ok(a)
        case (Nothing, Some(b)):
            Ok(b)
        case (Nothing, Nothing):
            Ok(0)

def main() -> Result[(), IoError | OverflowError[i64]]:
    print(combine(Some(2), Some(3))?)?
    Ok(())
5

Coverage considers combinations of component patterns, so omitting (Nothing, Nothing) above is an error. Alternatives retain the existing 256-combination expansion limit and decision-tree work limit. Matching owns and moves each bound affine component once and destroys ignored components once. Matching a reference instead binds component references, preserving disjoint component loans.

Class patterns

A class pattern takes a class value apart, written as the class’s generated constructor takes its fields: positional patterns in field-declaration order, then field=pattern. It matches a class position, the contents of a Box, or that class’s member of an anonymous sum.

class Point:
    x: i64
    y: i64

def axis(p: Point) -> Result[i64, OverflowError[i64]]:
    match p:
        case Point(0, y):
            Ok(y)
        case Point(x=x, y=0):
            Ok(x)
        case Point(x, y):
            Ok(x + y)

The rules have no exceptions:

  • Every field appears exactly once; ignore one with _ or field=_. A class with no fields is written Name().
  • Positional patterns precede named ones. An unknown, repeated, or missing field, or one positional pattern too many, is an error.
  • The class is named as in an expression: by its name, a module path, or an import. As for enums, the matched value supplies type arguments, so case Wrap(item) matches a Wrap[i64] and case Wrap[i64](item) is an error.
  • Each field must be visible where the pattern is written, so a class with a private field cannot be taken apart outside its module.
  • A class pattern tests nothing itself: it is irrefutable when every subpattern is, and a nested literal or variant makes it refutable.
  • A bare name that spells the class at its position, such as case Err(OverflowError):, is rejected rather than bound, so the parentheses cannot be forgotten silently.
  • Owned matching moves bound fields into their bindings and destroys the rest, as for an enum payload. A class with __del__ expects every field initialized, so an owned match cannot bind its fields; match a reference such as &value to borrow them, or bind it whole. Borrowed matching binds field references, as for payloads.

A @newtype class or any one-field class is matched the same way; its pattern costs no projection at run time. The error of a checked: block is the motivating case:

def total(a: i64, b: i64) -> i64:
    result = checked:
        a + b
    match result:
        case Ok(sum):
            sum
        case Err(OverflowError(wrapped)):
            wrapped
def opcode(name: str) -> i64:
    match name:
        case "cpy":
            1
        case "inc" | "dec":
            2
        case _:
            0

def describe(reading: Option[i64]) -> str:
    match reading:
        case Some(0):
            "zero"
        case Some(_):
            "nonzero"
        case Nothing:
            "missing"

case _ as name: matches every value the earlier cases leave and binds the whole matched value, for any supported scrutinee type. It does not make floats, collections, or other unsupported scrutinees matchable. It is accepted only as a case’s entire pattern: not as a | alternative, and not at a payload position, where the name alone binds. Because it matches everything, a case after it is unreachable, and it is itself unreachable when earlier cases are exhaustive. name cannot be a variant of the matched enum, _, or mut.

def forward(shape: Shape) -> i64:
    match shape:
        case Shape.Dot:
            0
        case _ as other:
            size(Some(other))

Every alternative of p | q binds the same names with the same types, at any depth. Cases are tried in order and the first match runs. Bindings stay within the arm and may shadow outer names.

Coverage must be exhaustive; the error lists missing values as patterns, such as missing Ok(Some(Shape.Dot)), Ok(Nothing) or missing Point(_, _). A case, or a single alternative, that earlier patterns already cover is an error. Incorrect payload arity and wrong enum identities are errors. A pattern expands to at most 256 combinations of its alternatives.

Literals never cover an integer or str position: such a position also needs _ or a payload name, and a match on an integer or str ends with case _: or case _ as name:. True and False together cover a bool, so a _ after both is unreachable. A repeated literal is an unreachable case.

Matching preserves source-order behavior, including overlapping payload patterns. Native dispatch uses a jump table for at least 256 distinct integer values or enum tags forming one contiguous range, in any source order. Signed values and all integer widths are supported; out-of-range values still take the wildcard or defensive invalid-tag trap. Table storage grows with the case count. Smaller or sparse sets use comparison chains; a str case compares lengths and then bytes. These chains can be sensitive to which cases occur most often; the dispatch measurements explain the conservative table threshold. A match on at least 16 distinct str literals instead binary-searches them once, so its cost grows with the logarithm of the case count and does not depend on arm order; see the string match measurements. Keep using match for clear exhaustive branching; dispatch selection does not change its behavior.

Guards, float patterns, range patterns, and pattern as name for any pattern other than the case’s own _ are unsupported. A scrutinee of any other type, such as a float or a collection, is rejected with advice to use an if/elif chain.

Payload and whole-value bindings cannot be declared mut. To change an owned payload, move it into a mutable local inside the arm (mut local = payload); to change it in place, match &mut value.

Continuing arms must agree on their result type. Arms ending in return, break, or continue contribute no join value. A function-tail match supplies the final expression of its selected arm, just like a tail conditional. A match elsewhere in a suite is a statement, and its arms’ values are discarded, unless it is a statement’s value.

A match as a statement’s value

A match block can be the whole value of these statements, and of no other position:

StatementEffect
name = match value:, with mut or an annotation as for any bindingdeclares the local, or assigns an existing mut local
place = match value:, where place is a field, a collection element, or *referenceassigns the place
return match value:returns from the function
def area(shape: Shape) -> Result[i64, OverflowError[i64]]:
    side: i64 = match shape:
        case Shape.Dot:
            return Ok(0)
        case Shape.Circle(side):
            side
        case Shape.Rect(width, height):
            if width < height:
                width
            else:
                height
    Ok(side * side)

The statement ends with the last case; nothing follows the block on its line. Patterns, exhaustiveness, and ownership of the scrutinee are those of any other match.

In a binding or assignment, the final expression of the selected case is the value. A final if or match supplies the value of its selected suite.

  • Continuing cases must agree on one type. An annotation, or the type of the assigned local, is the expected type of every case’s final expression, as it is for an ordinary right-hand side; without one, each case is inferred alone, so Nothing or []? needs the annotation.
  • A case that ends in return, break, continue, or a propagating ? gives no value. At least one case must continue; a match whose cases all exit is rejected with advice to write it as a statement.
  • A case whose final statement has no value, such as pass or an assignment, makes the match unit, which cannot be bound.
  • The value cannot be a reference or a closure with borrowed captures, because the case’s bindings end before the statement uses it. Produce an owned value, or declare mut name and assign in each case.
  • Earlier expressions in a case are discarded as in any suite, so an unhandled Result there is still an error.

Evaluation order is: the scrutinee; the selected case’s suite; cleanup of that case’s bindings and locals, in reverse declaration order, and of scrutinee parts no pattern bound; then the binding or assignment. An assignment evaluates destination indices and destroys the replaced value after all of that. An exit from a case performs the ordinary cleanup for that exit and assigns nothing.

return match value: returns the final expression of the selected case, exactly as if that case ended with return expression. Every case therefore exits, a call there is a source-tail call, and a statement after the block is unreachable. A case that ends without a value returns unit, which is an error unless the function returns unit. A returned reference follows the rules of any other returned reference.

Augmented assignment, assignment to several targets, call arguments, operands, and every other expression position reject a match block; bind its value to a name first. A block if supports the same whole-statement value positions and retains these expression-position restrictions.

Ownership

Owned matching consumes the scrutinee: bound payloads move into their bindings and every part no pattern binds is destroyed. _ as name moves the whole value into name, with no copy or allocation. Testing a literal copies the integer, str, or bool it compares, so a later pattern can still bind or move the value; for a str the copy shares the text and allocates nothing. Matching a shared or exclusive reference binds shared or exclusive payload references instead; see borrowed matching. Unit payload bindings produce unit expressions and can be returned from a unit-returning function.

Loops and control transfers

while condition: repeats its suite while the Boolean condition is true. for name in source: evaluates its source and visits the supported sequence. Flat tuple unpacking is also accepted in the loop binding. Source kinds, iteration loans, generators, and comprehensions are described under iteration.

break exits the nearest loop. continue advances to its next iteration. Both must appear within a loop. Statements following either transfer are unreachable errors. Loops are unit-valued; loop else suites are unsupported.

Declarations inside a loop body do not establish values outside it. Assignments to existing mutable locals persist. A loop may run zero times and cannot alone prove that a function returns or that an initializer sets every field.

Control transfers destroy values whose ownership ends on that path. An explicit return exits the enclosing function, while ? follows the enclosing callable’s error propagation rule.

Format strings

An f-string is a string literal with the prefix f: f"...", f'...', f"""...""", or f'''...'''. Literal text uses the ordinary string escapes, including \u{H...}; {{ and }} are literal braces, and a single } is an error. Each {...} is a replacement field:

{field[=][!conversion][:spec]}

The expression f"..." has type Result[str, AllocError], like a collection literal: s = f"x={x}"? or print(f"x={x}"?)?. Its type is Result[str, AllocError | FormatError] when formatting can fail other than by allocation: a field runs a user display or debug method, directly or for an element of an aggregate, a spec has nested fields, or a spec has type c. The choice is made per site from the fields’ types; in a generic function it is made for each instantiation. out.format(f"...") writes one to an existing formatter instead. The compiler parses the template and checks every spec while compiling. It lowers the literal to calls on a std.fmt string formatter, with no format-string interpreter at run time. A module that uses an f-string loads std.fmt without an import. FormatError is a prelude name, so a signature names it without an import too (issue #632).

from std.fmt import Formatter

class Tag:
    name: str

    def display[W](self, out: &mut Formatter[W]) -> Result[(), FormatError]:
        out.format(f"#{self.name}")

def plain(n: i64) -> Result[str, AllocError]:
    pair = [n, n]?
    Ok(f"n={n:>4} {pair}"?)

def tagged(tag: &Tag, width: i64) -> Result[str, AllocError | FormatError]:
    Ok(f"{tag} {1:{width}}"?)

def main() -> Result[(), AllocError | IoError | FormatError]:
    print(plain(7)?)?
    print(tagged(&Tag("x"), 3)?)?
    Ok(())
n=   7 [7, 7]
#x   1

A function that declares only FormatError for an f-string with built-in spellings is rejected, and the diagnostic names AllocError as the replacement.

Fields

A field is any expression, for example {p.x}, {xs[i + 1]}, {total / count:.2f}, {a > b}, {'yes' if ok else 'no'}, or {parse(text)?}. As in Python, {a, b} is the tuple (a, b). A : or ! outside brackets ends the expression, so a lambda must be parenthesized. Errors inside a field are reported at their position in the source.

All fields, including the nested fields of specs, are evaluated once, left to right, before any text is built. Each value is lent to the formatting code, so formatting moves nothing; a temporary value lives until the f-string’s text is built. A ? in a field belongs to the enclosing function: on Err or Nothing it returns from that function before formatting starts, exactly as ? in a call argument does. The Result of the f-string itself reports only formatting failure, so f"{parse(s)?}"? has two independent ?s.

A field cannot contain a backslash, a comment, or the quote character that encloses the f-string. A field in a triple-quoted f-string may span lines; a line break in its spec is a character of the spec, such as a fill.

Only the f prefix starts an f-string. F"..." and the combinations bf/fb are errors: build text with an f-string, then convert it to bytes.

{expr=} writes the field’s source text, including the = and any spaces around it, then its value. Without a conversion or a spec the value is written as with !r, so {name=} gives name="Ada".

Quantity fields apply numeric specs to the stored number and append the coherent-unit symbol. Use !s to pad the whole display text, or !r for Quantity[T, D](value) debug output.

Values and conversions

Without a conversion, a field is written as print writes it, including its user formatting methods:

TypeSpelling
strThe text itself
boolTrue or False
IntegersDecimal
f32, f64Shortest round-trip, as str.from
A class or enum with a display methodThat method, with the depth accounting of std.fmt.display
A class or enum with only a debug methodThat method
Anything else, including bytesThe str.repr spelling, as print shows aggregates and b"..." values; elements with a debug method use it

A field calls a top-level display or debug from the module where the f-string is written, as a direct call there would. A private class, or a private method, formats in its own module. In another module, including through a generic function defined there, the class and the method must be public; otherwise the field is rejected with the ordinary private-access diagnostic. Methods of aggregate elements run as str.repr runs them, with their own module’s access.

!s is the same spelling. !r writes str quoted and escaped as Formatter.debug_text does, uses a type’s debug method when it has one, and otherwise the str.repr spelling, including element debug methods. !a, like Python’s ascii(), takes the !r text and spells each non-ASCII scalar as \u{hex} with lowercase digits, the escape debug_text uses for controls: f"{'café'!a}" is "caf\u{e9}", which is also a valid string literal. Python spells the same escapes \xe9, \u2603, or \U0001f600, with single quotes.

A @redacted type without its own display or debug method writes TypeName(@redacted) under every conversion and spec, including !r, !a, and =.

Format specs

A spec follows Python’s format specification mini-language:

[[fill]align][sign][z][#][0][width][grouping][.precision][type]
  • align is < (left), > (right), ^ (center; extra fill goes right), or = (fill between the sign and the digits, numbers only). Text defaults to left and numbers to right. fill is any single character.
  • sign is - (default), +, or a space. z (floats only) drops the sign of a result that rounds to zero, so {-0.0001:z.2f} is 0.00; -inf keeps it. # adds 0b, 0o, 0x, or 0X to integers, and keeps the point or trailing zeros of floats.
  • 0 pads numbers with zeros after the sign and prefix. With grouping, the zeros are grouped too: {-1234:010,} gives -0,001,234. After an explicit alignment, 0 is the fill character.
  • width is a minimum count of Unicode scalars; output is never truncated to it. grouping is , or _: every three integer digits, or every four with _ and types x, X, o, and b.
  • precision is the digits after the point for e, f, and %, the significant digits for g and for a float with no type, and the maximum number of scalars for text.

Width and precision are decimal numbers of at most 4096.

ValueTypesOther limits
Text (str, bool, values with a conversion or no built-in spelling)none, sNo sign, z, #, 0, grouping, or =
Integersnone, d, x, X, o, b, n, cNo precision or z; , only for decimal; c takes no sign, #, or grouping
f32, f64none, e, E, f, F, g, G, %, n

c writes the Unicode scalar whose code point is the integer, laid out like a number: right-aligned by default, and 0 pads with zeros. A value that is not a scalar (negative, a surrogate, or above U+10FFFF) makes the f-string return Err(FormatError.InvalidSpec), so a field with c adds FormatError to the f-string’s type. n is Python’s n in the C locale, which is the locale Python formats with unless a program calls locale.setlocale: d for integers and g for floats, with no grouping. Plenty has no locales, so grouping with n is rejected rather than ignored.

Float types follow Python, including exponents with a sign and at least two digits (1.23e+03), % multiplying by 100, and g choosing fixed or exponent notation. Typed floats spell infinity and NaN as inf and nan (INF, NAN for upper-case types). A float without a type or precision keeps the print spelling (1e16, NaN); with a precision it behaves as Python’s .Ng with at least one digit after the point. Because Plenty has a bool type, a spec on a bool formats its text rather than the number 1 or 0.

An unsupported option for the field’s type is a compile error at the spec, for example format spec `.2f` does not apply to i64: integers take no precision; convert with f64(...) to format a float. Convert explicitly to use another type’s options.

Nested fields in a spec

A spec may contain fields, as in {x:{width}} or {x:{fill}{align}{width}.{precision}f}. A nested field is an expression with no =, conversion, spec, or further nesting. Python pastes nested values into the spec text and parses the result at run time; Plenty instead parses the spec while compiling, and each nested field supplies exactly one option, chosen by its position and its type:

Nested fieldTypeSupplies
First, before an alignment character or a second str fieldstrThe fill
First otherwise, or second after the fillstrThe alignment
Where the width’s digits would beAny integerThe width
After .Any integerThe precision

Any other nested field, such as an integer before the alignment or two adjacent integer fields, is a compile error, as is 0 after a nested alignment (write the fill instead). Everything written in the spec, including its type, is checked against the field’s type as usual; {x:{w}.2f} with an integer x is rejected because of .2f whatever w is.

The nested values are checked when the f-string runs, and an invalid one makes it return Err(FormatError.InvalidSpec) before any text is built:

  • an alignment must be "", <, >, ^, or = (= only for numbers); "" means the default alignment;
  • a fill must be one scalar, or "" for the default space, and a non-empty fill needs a non-empty alignment;
  • a width or precision must be in 0..=4096.

For valid values the result is the one Python gives. Where Python would read a pasted value differently, such as a str field holding "<10" or a fill with an empty alignment, Plenty returns InvalidSpec rather than another layout.

Writing to a formatter

out.format(f"..."), where out is a std.fmt.Formatter[W] or a &mut reference to one, writes the f-string to out without building a str, and has type Result[(), FormatError]. It is the form to use in a display or debug method:

pub def display[W](self, out: &mut Formatter[W]) -> Result[(), FormatError]:
    out.format(f"({self.x}, {self.y})")

The argument must be an f-string literal; the compiler lowers the pair to calls on out, so out’s byte budget, depth limit, and partial-progress rules apply, and its scratch is reused. A field that needs a temporary string (a spec on a value without a built-in spelling, !r on str with a spec, !a) still allocates one. format is not an ordinary method of Formatter: it has no other form, and a user type’s own format method is unaffected.

Cost and failure

The result of f"..." is built in a new string formatter whose output and 32-byte scratch are allocated for each evaluation; a float spelling longer than the scratch grows it. A spec on a value without a built-in spelling first builds that value’s text as a temporary string. There is no byte budget: an f-string is bounded only by allocation failure, reported as AllocError whatever the f-string’s type; a user method’s own FormatError.Allocation(...) is reported as AllocError too. User display and debug methods may nest through std.fmt up to 32 levels, after which the result is FormatError.RecursionLimit. No partial string is returned on error.

Slices

A subscript whose brackets contain a top-level : is a slice. It copies the selected elements into a new value and returns Result[_, AllocError], so the copy is visible as a ? (issue #579). A tuple slice allocates nothing and returns the tuple itself (see tuple slices):

def main() -> Result[(), AllocError | IoError]:
    xs = [0, 1, 2, 3, 4, 5]?
    print(xs[1:3]?)?
    print(xs[-2:]?)?
    print(xs[::2]?)?
    print(xs[::-1]?)?
    print("Aé🙂Z"[1:3]?)?
    Ok(())
[1, 2]
[4, 5]
[0, 2, 4]
[5, 4, 3, 2, 1, 0]
é🙂

Syntax

slice = primary "[" [start] ":" [stop] [":" [step]] "]"

xs[:], xs[::], xs[a:], xs[:b], xs[::c] and xs[a:b:c] are all slices. A slice has the precedence of indexing and the other postfix forms: xs[1:3]?[0] indexes the copy and s[1:]?.upper()? calls a method on it. A comma inside slice brackets is rejected: there are no multi-dimensional subscripts. The receiver, then each written bound, evaluate once from left to right. The receiver is only read, so a named receiver stays usable.

Bounds and steps

start and stop are i64 expressions or references to i64; other integer types need a cast such as i64(n). Positions count elements, Unicode scalars for str, bytes for Bytes and bytes, and rows for clist.

Out-of-range bounds clamp exactly as in Python. A negative bound counts from the end, and positions outside the receiver select nothing rather than trapping. Omitted bounds depend on the sign of the step: a positive step runs from the start to the end, and a negative step runs from the last element back to the first.

The step has type NonZero[i64] and defaults to 1, so a zero step cannot be written or computed. A nonzero integer literal converts implicitly, and a literal 0 is a compile error. A computed step needs an explicit conversion, which reports a zero step as RangeError:

def main() -> Result[(), AllocError | IoError | RangeError | OverflowError[i64] | ZeroDivisionError]:
    xs = [0, 1, 2, 3, 4, 5]?
    step = NonZero.try_from(len(xs) // 2)?
    print(xs[::step]?)?
    print(xs[100:]?)?
    print(xs[-100:2]?)?
    Ok(())
[0, 3]
[]
[0, 1]

Receivers and results

ReceiverResultElements
list[T]Result[list[T], AllocError]Duplicated with copy(); Shared[T] handles with share()
strResult[str, AllocError]Unicode scalars
std.bytes.BytesResult[std.bytes.Bytes, AllocError]Bytes
bytesResult[std.bytes.Bytes, AllocError]Bytes; no import needed
clist[T]Result[clist[T], AllocError]Rows copied column by column
TupleThe selected tuple, or (); no ResultComponents selected at compile time

A list slice of a nested list copies each selected inner list, so later changes to the source do not reach the slice. A step-1 slice of a list whose elements own no storage, such as list[i64], is one block copy. Every slice reserves its result storage once, before copying; allocation failure returns Err(AllocError.OutOfMemory) and leaves the receiver unchanged.

A list element type that copy() rejects, such as std.bytes.Bytes, is a compile error. Transfer such elements out of an owned temporary with the slice() method instead. A Shared handle in an element, as in list[(str, Shared[T])], gains another owner, as copy() does.

A bytes literal slice gives an owned std.bytes.Bytes. Slice syntax makes the Bytes type available without an import std.bytes line; naming the type in a signature still needs the import. The compiler loads std.bytes for this only when some module of the program could produce a bytes value: one with a b"..." literal, include_bytes, or the word bytes. A program that slices only lists or text does not load it.

Tuple slices

Tuple slice bounds and step must be integer literals, because the result type depends on them: (1, "two", 3.0)[1:] has type (str, f64). A slice that selects no component is (). A tuple slice copies components inline and allocates nothing, so it returns no Result and takes no ? (issue #625):

def main() -> Result[(), IoError]:
    t = (1, "two", 3.0)
    tail: (str, f64) = t[1:]
    print(tail, t[::-1])?
    Ok(())
("two", 3.0) (3.0, "two", 1)

A tuple whose selected components are affine is rejected for now; unpack it and copy the components instead.

A clist receiver must be a named place, and the slice is subject to the same borrow checks as other clist reads.

Slice views

&xs[a:b] borrows the selected elements in place, and &mut xs[a:b] borrows them exclusively (issue #581). A view copies nothing and allocates nothing, so it needs no ?. Use one to pass part of a list or buffer to a function:

def total(values: &[i64]) -> Result[i64, OverflowError[i64]]:
    mut sum = 0
    for x in values:
        sum += *x
    Ok(sum)

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    mut xs = [1, 2, 3, 4, 5]?
    print(total(&xs[1:4])?)?
    tail = &mut xs[-2:]
    tail[0] = 40
    for x in &mut xs[:2]:
        *x *= 10
    print(xs)?
    Ok(())
9
[10, 20, 3, 40, 5]

The bounds clamp as for a copying slice, so a view is empty rather than out of range. A view selects adjacent elements, so it takes no step; copy every k-th element with xs[a:b:k]?.

Viewed valueShared viewExclusive view
list[T]&[T]&mut [T]
std.bytes.Bytes&[u8]&mut [u8]
bytes&[u8]rejected: a literal is read-only
A viewA narrower viewA narrower view of an exclusive view
Shared[list[T]]&[T] of the contentrejected: the content is shared

&[T] and &mut [T] are types: parameters and local annotations may name them. A list of u8, a Bytes buffer, and a bytes literal all give &[u8], so one &[u8] parameter reads any of them. The viewed value must be a named place, such as xs, self.items, or rows[0], or a bytes or str literal, as in &b"GET /"[:3]. A shared view argument may also view a call result, as in w.write(&encode(x)?[2:]): the result lives until the end of the statement. Bind a call result to a name before binding a view of it to a name, and before passing an exclusive view of it.

OperationMeaning
len(v)Number of elements in the view
v[i]Element i of the view, with list index rules: a negative index counts from the end, and an index outside the view traps
v[i] = xReplace element i; needs an exclusive view in a mut binding
v.swap(i, j)Exchange elements i and j in place, with list.swap’s contract; needs an exclusive view
&v[i], &mut v[i]Reference to element i
&v[a:b], &mut v[a:b]Narrower view; bounds count from the start of v
for x in v&T per element, or &mut T for an exclusive view; reversed(v) and enumerate(v) also work
iter(v)A pipeline over the elements, with the items of for x in v
v == w, v != wEqual lengths and equal elements; both sides are views of one element type
print(v), str.repr(v), f"{v}"The elements as a list, [2, 3]; a &[u8] as a bytes literal, b"bc"

In --unchecked assume builds, view indexing inside a plain unchecked: suite assumes a nonnegative index below the view’s own length, with no bounds check or negative-index normalization. The view’s construction and swap method keep their normal contracts.

A view’s length never changes, so v[i] in for i in range(len(v)) or for i, x in enumerate(v) compiles without a bounds check, as described for lists. An in-place reversal, while lo < hi: starting with v.swap(lo, hi) or v[hi], checks its first pass’s indices once, before the loop, as described there.

def even(x: &i64) -> bool:
    two: NonZero[i64] = 2
    *x % two == 0

def main() -> Result[(), AllocError | IoError]:
    xs = [1, 2, 3, 4, 5]?
    middle = &xs[1:4]
    print(middle, iter(middle).filter(even).count())?
    data = b"abc"
    print(&data[1:])?
    Ok(())
[2, 3, 4] 2
b"bc"

A view prints as a reference does, through to its elements; a Bytes buffer itself keeps its own spelling. Other list operations, such as methods other than swap, in, list(v), and copying slices of a view, are rejected. Copy from the owner, xs[a:b]?, for an owned value.

v.swap(i, j) counts both indices within the view, so a negative index counts from the view’s end, and an index outside the view traps before either element moves. Both indices are evaluated before the view is borrowed, so they may read it, as len(v) - 1 does. Equal indices leave the view unchanged. It accepts every element type and allocates, copies, retains, and drops nothing. For numbers and bool it compiles to the inline bounds checks, loads, and stores of list.swap; other element types make one runtime call, which repairs inline payload addresses. A shared view is read-only and has no swap:

def reverse(v: &mut [i64]) -> Result[(), OverflowError[i64]]:
    mut lo = 0
    mut hi = len(v) - 1
    while lo < hi:
        v.swap(lo, hi)
        lo += 1
        hi -= 1
    Ok(())

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    mut xs = [1, 2, 3, 4, 5]?
    reverse(&mut xs[1:])?
    print(xs)?
    Ok(())
[1, 5, 4, 3, 2]

Views borrow their owner

A view holds a loan of its run of elements in its owner: a shared view a shared loan, and an exclusive view an exclusive one, under the borrowing rules. While a view is live, its owner cannot grow, shrink, move, or be reassigned, and len(xs) and &xs conflict with an exclusive view. Two views are disjoint when one stops where the other starts, at one literal or non-mut i64 name, so &mut xs[:m] and &mut xs[m:] can both be live. Overlapping or unprovable ranges, such as &mut xs[:3] with &xs[2:], conflict. Disjoint slice views states the full rule and why it is sound.

Where a view type is expected, a borrow of its owner converts to the whole view, as [:] would: &xs or &mut xs for a list, &buffer or a b"..." literal for &[u8], and &handle for a Shared list. A reference binding to an owner, such as an xs: &list[i64] parameter, converts too. The conversion copies nothing and allocates nothing. Expected view types are parameters, including a generic &[T], and annotated locals:

def total(values: &[i64]) -> Result[i64, OverflowError[i64]]:
    mut sum = 0
    for x in values:
        sum += *x
    Ok(sum)

def width(data: &[u8]) -> i64:
    len(data)

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    xs = [1, 2, 3]?
    whole: &[i64] = &xs
    print(total(&xs)?, total(&xs[1:])?, total(whole)?, width(b"abc"))?
    Ok(())
6 5 6 3

&xs[:] keeps working. An owned list, total(xs), is not a view and is rejected. A &T type parameter takes a reference to a whole value, not a view. Standard-library operations that read bytes take &[u8], so buffer.extend(&other[2:]) and writer.write(b"ok") pass a view. A function with a view parameter may return a reference into it: the first reference or view parameter is the origin of a returned reference. Views may be passed to scoped workers.

Views are local until stored references land (backlog B11). A view cannot be returned, stored in a class field, tuple, collection, or enum payload, or captured by an anonymous function. Return or store the bounds, or an owned copy.

Substring views

&s[a:b] on a str borrows the selected text in place as a &substr (issue #595). Like other views it copies nothing, allocates nothing, and takes no ?. Its bounds are those of the copying slice s[a:b]?: they count Unicode scalars, a negative bound counts from the end, and bounds outside the text clamp. A view therefore always starts and ends on a scalar boundary, and s[a:b]? == &s[a:b] for any a and b.

def vowels(text: &substr) -> Result[i64, OverflowError[i64]]:
    mut n = 0
    for c in text:
        if c in "aeiouéö":
            n += 1
    Ok(n)

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    s = "héllo, wörld"
    word = &s[7:]
    print(word, len(word), word.byte_len(), vowels(&s[:5])?)?
    print(word == "wörld", &s[:1] < word, "ör" in word, word.find("l"))?
    for c in reversed(&word[1:3]):
        print(c)?
    owned = word[:2]?
    print(owned)?
    Ok(())
wörld 5 6 2
True True True Option[i64].Some(3)
r
ö
wö

&substr is a separate type from &str. A &str refers to a whole str, whose representation stores its byte length and scalar count ahead of the text, and every str operation reads them. A run inside a string has no such prefix, so a view holds the run’s address and byte length instead. substr is only written as &substr; there is no owned substr and no &mut substr, since a str is immutable. A &substr parameter or annotated local also takes a whole string as &s, a &str binding, or a str literal, converted as &s[:] would be.

OperationMeaning
len(v)Number of scalars; scans the view’s text, as positions do
v.byte_len()Number of UTF-8 bytes; constant time
v[i]Scalar i as a one-scalar str; a negative index counts from the end, and an index outside the view traps
&v[a:b]Narrower view; bounds count scalars from the start of v
v[a:b]?An owned copy of part of the view, Result[str, AllocError]
for c in vEach scalar as a one-scalar str; reversed(v), enumerate(v), and iter(v) also work
v == w, v != w, <, <=, >, >=Compare by Unicode scalar value, as str does; either side may be a str or a &substr
needle in v, needle not in vSubstring membership; the needle is a str or a &substr
v.startswith(x), v.endswith(x)Exact prefix or suffix; x is a str or a &substr
v.find(x)First matching scalar position within the view, or Nothing
print(v), f"{v}"The text itself; str.repr(v) and f"{v!r}" quote it as a str

Each scalar a loop or index yields fits in a string’s inline form, so these operations allocate nothing. Finding a bound, len(v), v[i], and v.find(x) scan the text before the position, as they do on a str. str methods other than those listed are not available on a view; copy it with v[:]? to use one. A str method does not take a &substr argument, and v in s for a str s is written v in &s[:].

A substring view borrows its string as other views borrow their owner: while it is live, the string cannot be reassigned or moved, and the view stays local until stored references land (backlog B11).

Limits

SourceDiagnostic
ys: list[i64] = xs[1:]slice returns Result[list[i64], AllocError]; use `?` to propagate allocation failure
xs[::0]slice step cannot be zero
xs[::k] with k: i64slice step must be `NonZero[i64]`, not `i64`
&xs[::2]a slice view selects adjacent elements, so it takes no step
&mut s[1:3] on a stra `str` is immutable, so a substring view is shared only; write `&s[a:b]`
def f(xs: &list[T]) -> &[T]a function cannot return a slice view yet (backlog B11)
total(xs) for total(v: &[i64])this parameter takes a slice view: pass `&xs` for a whole list or buffer, or `&xs[a:b]` for part of it
x in v for a view vslice views do not support `in` yet
xs[1:3] = ysslice assignment is not supported; assign elements by index, or build a new list
m[1:2, 0]a slice cannot have several dimensions
t[n:] on a tupletuple slice bounds must be integer literals; the result type depends on them
t[1:]? on a tuplea tuple slice `t[a:b]` does not allocate, so it takes no `?`; write `t[a:b]`
d[1:] on a dictdict does not support slicing
class Window:
    rows: &[i64]

def main() -> ():
    pass
slice views cannot be stored in class fields yet (backlog B11)

First-class slice(...) values are not implemented. The slice syntax proposal records the design and alternatives.

Ownership, borrowing, and lifetimes

An owner controls a value’s lifetime. Copyable values can have independent owners; affine values transfer their ownership when moved. References lend access without taking ownership. Access permissions end at a loan’s last use; owned resources follow the specified destruction scopes.

Types define which values can be stored together. Iteration and resource scopes apply these same ownership rules to their hidden owners.

Bindings, moves, and copies

Bindings and mutable reassignment follow the declaration and assignment rules. Ordinary value use (assignment, owned argument passing, and return) either copies or transfers ownership according to the value’s type:

Numbers, bool, str, bytes, ranges, and named function values copy. A tuple, Option, Result, enum, or anonymous sum (A | B) copies when all of its parts copy. A class copies only when it is declared @copy, and a generic one only when its type arguments also copy. Every other value moves: collections, boxes, closures that capture, generators, and other class instances.

A class of integers is still affine unless declared @copy. Copied strings may share immutable storage. Inserting an affine value into a collection or capturing it transfers ownership. A moved-from binding cannot be read until a permitted reassignment initializes it.

copy(value) observes its argument and returns Result[T, AllocError] containing an independent value. It recursively duplicates owned mutable contents; immutable strings can share storage. A nested Shared handle is not duplicated: the copy holds another owner of the same content, as share() would. copy(handle) on a handle itself is rejected in favour of handle.share(). Copying does not supply mutable aliases. For a value that already copies, copy(value)? equals value; checking reports it as a redundant-copy warning. Generators, captured closures, recursive data, and types containing custom destruction cannot be copied. Allocation and destruction describe failure and cleanup.

Only paths that continue contribute to availability at a branch join. Moving on some continuing paths makes a binding possibly moved; moving on all makes it moved. Reinitializing a mutable binding restores availability. Every loop backedge, including continue, must preserve outer owners available at loop entry. Moving and then reinitializing is accepted; carrying the move to a backedge is rejected. Break paths join the zero-iteration path.

Moving fields

An affine field can move out of a class instance held by an owned binding, including an owned self receiver. The path from the binding to the field may pass through further class fields, as in outer.inner.resource. Availability is then tracked per field, with the same rules as bindings:

  • The moved field, and any place inside it, cannot be used until it is assigned.
  • The other fields remain usable and can move out too.
  • The binding cannot be used as a whole while a field is moved: it cannot be moved, passed, returned, borrowed, or used as a method receiver. Assigning the field (value.field = replacement) or the whole binding restores it.
  • At a branch join, a field moved on some paths is possibly moved. A field moved in a loop must be assigned again before the backedge.
  • When the binding goes out of scope, or a path returns early, only the fields still initialized are destroyed, each exactly once.
class Resource:
    name: str

    def __del__(self) -> ():
        print("close", self.name).unwrap()

class Pair:
    left: Resource
    right: Resource

def consume(resource: Resource) -> ():
    print("consume", resource.name).unwrap()

def main() -> ():
    mut pair = Pair(Resource("a"), Resource("b"))
    consume(pair.left)
    print(pair.right.name).unwrap()
    pair.left = Resource("c")
    whole = pair
    print("end").unwrap()
consume a
close a
b
end
close c
close b

Using the binding as a whole before the field is assigned is rejected:

class Resource:
    name: str

class Pair:
    left: Resource
    right: Resource

def main() -> ():
    pair = Pair(Resource("a"), Resource("b"))
    left = pair.left
    whole = pair
use of partly moved binding `pair`: `pair.left` is moved; assign `pair.left` before using `pair` as a whole

Moving out needs no mut binding, as moving the whole value does not, but assigning the field back does. Fields cannot move out:

  • through a reference, including bare self and &mut receivers; use replace, which transfers the old value while leaving the field initialized;
  • of a class that defines __del__, which receives the whole object and so needs every field initialized; replace works there too;
  • through a Box, a Shared handle, a collection element, a closure capture, or a temporary.

Tuple elements move by unpacking the whole tuple, and enum payloads by consuming match. Use an explicit copy or a collection removal operation for collection elements. Two list elements can be exchanged in place with swap.

Borrowing and reborrowing

&place creates a shared reference &T; &mut place creates an exclusive reference &mut T. Exclusive access requires a mutable owner or an existing exclusive reference. A shared loan permits other shared reads; an exclusive loan prohibits competing reads, writes, and borrows. Neither acquires ownership. &mut str permits replacing a string binding, not mutating string bytes.

A reference binding can be initialized from an explicit borrow, a named reference, or a reference-returning call. Naming or passing an existing reference reborrows it. An exclusive parent can lend shared or exclusive access; conflicting parent access is prohibited while the child is live. A shared reference cannot be upgraded to exclusive access. Reference bindings are immutable unless declared mut; that declaration controls reassignment of the reference, independently of the referent’s shared or exclusive access mode.

Loans are checked through branches, loops, and early exits and normally end at their last use. Moving, replacing, or destroying an owner conflicts with any live loan into it. References cannot be stored in aggregates or retained across generator suspension. General lifetime annotations are unsupported. Fields can move out of an owned binding; see moving fields.

Class fields and fixed tuple indices are projected places. Disjoint fields can be borrowed independently; access to an ancestor or whole owner overlaps its descendants. &items[index] and &mut items[index] borrow list elements or dictionary values in named storage, including projected collections. Indices and keys follow ordinary indexing, including negative list indices and traps for absent entries. Element loans at distinct nonnegative integer literal list indices, such as xs[0] and xs[1], are disjoint. Every other element loan protects the whole collection: a variable, computed, or negative index, and every dictionary key. Growth, removal, replacement, movement, and whole-list access such as len(xs) or &xs overlap every element loan. &xs[a:b] and &mut xs[a:b] borrow a run of list or byte-buffer elements as a slice view, &[T] or &mut [T]. A view’s loan covers its run of elements; see disjoint slice views for when two runs are disjoint. Nested fields and collection elements can be projected through these references. Borrowing itself allocates nothing.

Disjoint slice views

Two views of one owner are disjoint when the checker can prove their runs do not overlap for every owner length. It accepts two forms:

  • One view stops where the other starts, at the same integer literal or the same non-mut i64 binding: &mut xs[:m] and &mut xs[m:], or &mut xs[:-2] and &mut xs[-2:].
  • Both bounds compared are literals of the same sign, and one view stops at or before the other starts: &mut xs[0:2] and &mut xs[4:6]. An empty view such as &xs[3:1] overlaps nothing.

A list element at a literal index k is disjoint from a view that starts at a nonnegative literal above k or stops at a nonnegative literal at or below k. Nested views and rows of one view follow the same rules relative to that view. Every other pair overlaps, including bounds of different signs, two different names, a mut binding, and a computed expression such as m + 1.

The rule is sound because a view clamps each bound to the owner’s length when it is created, and the length cannot change while a view is live. A nonnegative bound v becomes min(v, len) and a negative one max(len + v, 0); both are monotone, so equal or ordered bounds of one sign stay equal or ordered after clamping. A shared bound name reads one value for both views: assigning it while a view bounded by it is live is rejected.

def main() -> Result[(), AllocError | IoError | OverflowError[i64] | ZeroDivisionError]:
    mut xs = [1, 2, 3, 4, 5]?
    m = len(xs) // 2
    left = &mut xs[:m]
    right = &mut xs[m:]
    left[0] = right[0] + right[1]
    print(xs)?
    Ok(())
[7, 2, 3, 4, 5]
def main() -> Result[(), AllocError | IoError]:
    mut xs = [1, 2, 3, 4, 5]?
    left = &mut xs[:3]
    right = &mut xs[2:]
    left[0] = right[0]
    Ok(())
conflicting borrow

The diagnostic names both views and states the rule. Rewrite the bounds to meet at one name, or end the first view before taking the second.

Distinct fields of one borrowed loop element are disjoint, as they are for any other reference. Under for p in &mut pairs, swap(&mut p[0], &mut p[1]) borrows two fields of the current element; under for n, s in &mut pairs, n and s are such fields. Borrowing the list or another element while they are live still conflicts, because the loop element’s index is unknown.

Integer node IDs do not borrow their target collection. They permit sequential updates to different nodes, but do not make simultaneous indexed loans disjoint. Use the arena lesson for owned storage with parent links and a traversal worklist.

An ordinary &T parameter implicitly borrows an owned argument: inspect(value) and inspect(&value) lend the same owner. An &mut T parameter requires &mut value, except when forwarding an existing exclusive reference. Methods borrow their receivers according to their declared self parameter. Generic calls and callbacks follow the same signature-based rules.

A class method with an exclusive receiver reserves its place before evaluating arguments, then activates exclusive access just before entering the method. The receiver and arguments still evaluate once, left to right. During argument evaluation the reserved place permits shared reads, but cannot be moved, replaced, or mutably borrowed. Thus buffer.fill(0, buffer.len(), 0u8) works even when buffer is an &mut Bytes: the length is a completed value before the exclusive call starts. Generic methods use the same rule. A shared reference into the receiver that remains live as an argument still conflicts with activation; a nested mutation of that receiver also conflicts. Bind a nested mutation’s result to a local before the outer call when sequential mutations are intended. Explicit function arguments such as f(&mut x, &x) do not receive this implicit receiver reservation.

Shared call arguments may be literals or owned temporaries. They evaluate once in argument order and stay alive through later arguments and the containing full expression, including cleanup on ?. A reference returned into such a temporary can be used within that expression, but cannot be bound, returned, or retained by a scoped worker. Mutable temporary arguments and general temporary lifetime extension are unsupported.

Reference binding reassignment

A mutable reference binding covers the place it first borrowed, its first place. A list element’s first place is the whole list. The binding may be assigned, with an unchanged type:

  • a reference borrowed through itself: a field, element, box content, borrowed-match payload, or the result of a reference-returning call that received it. The binding’s loan continues.
  • another reference into its first place that does not go through the binding, such as cur = head or best = &items[i]. This retargets the binding: the loan it held ends at the assignment, and later uses hold the new loan.

A reference to a different owner, or to a place of the same owner outside the first place (&p.right after &mut p.left), is rejected. The first place is borrowed while the binding is live, so the owner cannot change between a retarget and a later use of the binding. On a path that skips a retarget, the old loan continues; a reborrow of the old target keeps its own loan.

class Node:
    value: i64
    next: Option[Box[Node]]

def total(head: &Node) -> Result[i64, OverflowError[i64]]:
    mut sum = 0
    mut cur = head
    while True:
        sum = sum + cur.value
        match &cur.next:
            case Some(next):
                cur = next
            case Nothing:
                break
    Ok(sum)

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    chain = Node(1, Some(Box(Node(2, Nothing))?))
    print(total(&chain)?)?
    Ok(())
3

This traversal uses constant stack space and allocates nothing after constructing the chain. Because a reassignable reference can point anywhere within its first target, loans through it protect that whole target. Two such field loans are not considered disjoint. An exclusive binding cannot be accessed while a live child reference conflicts with it. Immutable reference bindings keep precise field loans.

Retargeting lets a search remember its best element without an index:

def longest(words: &list[str]) -> &str:
    mut best = &words[0]
    for i in range(1, len(words)):
        if words[i].byte_len() > best.byte_len():
            best = &words[i]
    best

def main() -> Result[(), AllocError | IoError]:
    words = ["fig", "banana", "kiwi"]?
    print(longest(&words))?
    Ok(())
banana

Because the old loan ends at a retarget, a collection can grow after the old target’s last use and before the binding is pointed at a new element:

def main() -> Result[(), AllocError | IoError]:
    mut values = [1, 2]?
    mut slot = &mut values[0]
    *slot = 10
    values.append(3)?
    slot = &mut values[2]
    *slot = 30
    print(values)?
    Ok(())
[10, 2, 30]

Returned references

A function or method returning &T or &mut T borrows its result from its first reference parameter, the origin. A slice view parameter, &[T] or &mut [T], counts as a reference parameter. A function without a reference parameter cannot return a reference, and a mutable result requires an &mut origin. Every return path must originate in the origin; later reference parameters, local owners, and owned parameters cannot escape. Direct parameter references, reborrows, field and element projections, borrowed-match payloads, and forwarding calls are supported. Branches may use explicit returns; conditional reference expressions are unsupported.

class Pair:
    first: i64
    second: i64

def first(pair: &Pair) -> &i64:
    &pair.first

def main() -> Result[(), IoError]:
    pair = Pair(3, 4)
    value = first(&pair)
    print(*value)?
    Ok(())
3

Later reference parameters are ordinary inputs. Their loans end when the call returns, so a temporary key or a borrow of another owner does not restrict the result. For a method, the origin is its reference receiver:

class Scores:
    names: list[str]
    points: list[i64]

    def points_of(self: &mut Scores, name: &str) -> &mut i64:
        for i, candidate in enumerate(&self.names):
            if candidate == name:
                return &mut self.points[i]
        &mut self.points[-1]

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    mut scores = Scores(["ada", "bob"]?, [3, 5]?)
    mut name = "bob"
    points = scores.points_of(&name)
    name = "eve"
    *points = *points + 10
    print(scores.points)?
    print(name)?
    Ok(())
[3, 15]
eve

This small lookup assumes equally sized, nonempty lists and uses the last score as its fallback. Enumeration avoids manual index arithmetic while keeping the reference-returning method infallible. A general lookup can return an optional reference instead.

To return a borrow of a different argument, make it the first reference parameter. A function cannot choose between two arguments; that relationship needs annotations Plenty does not have. Signatures alone fix the origin, so Callable types and generic functions follow the same rule.

The result extends its origin loan until its last use. A body that directly returns its parameter or a fixed field projection permits a precise returned field loan; the original call still requires access to its whole argument. Indexed origins, control-dependent returns, and other bodies conservatively protect the whole argument, even after further projection.

A &mut T result can be assigned through directly, without a binding: *f(&mut xs) = value and *f(&mut xs) += value call f once and write its referent. The call runs first, as in r = f(&mut xs); *r = value, so its loan is live through the right-hand side: *f(&mut xs) = xs[1] is a conflicting borrow. Read the value into a local first.

def head(xs: &mut [i64]) -> &mut i64:
    &mut xs[0]

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    mut xs = [1, 2, 3]?
    *head(&mut xs[:]) = 9
    *head(&mut xs[1:]) += 10
    print(xs)?
    Ok(())
[9, 12, 3]

A caller may bind a result from named storage and reborrow it. A temporary origin survives only its full expression, the statement or condition that evaluates it. A shared temporary passed to an ordinary function follows this rule: len(view("text")) can use a returned &str, but r = view("text") is rejected even when r is unused. A temporary method receiver, such as make().label(), follows the same rule: the result may be used within the statement, and the receiver drops at its end. Binding the result, or returning it, is rejected:

class Point:
    x: i64
    name: str

    def label(self) -> &str:
        &self.name

def make(name: str) -> Point:
    Point(0, name)

def main() -> Result[(), IoError]:
    print(make("origin").label())?
    print(len(make("east").label()))?
    Ok(())
origin
4

A temporary receiver that itself borrows other storage, such as a returned &Point in pick(&a).view(), cannot lend a returned reference: its loan of that storage would end before the result is used. Bind the receiver first, p = pick(&a), then call p.view().

A method on an element or field of a temporary, as in [make("a")]?[0].label() or holder.items()[0].view(), borrows that element in place. An owned temporary lives to the end of the statement, as a temporary receiver does; a returned reference lends as a named reference does.

Returned references cannot enter aggregates, scoped workers with temporary origins, or generator suspension. No lifetime annotation or callee-body inference changes the signature’s first-reference origin.

A borrow that may be absent or may fail is returned as Option[&T] or Result[&T, E] under the same origin rule; see optional and fallible references.

Optional and fallible references

A function may return Option[&T], Option[&mut T], Result[&T, E], or Result[&mut T, E]: a borrow that may be absent or may fail. The success payload may also be another such enum, as in Result[Option[&T], E]. These reference-carrying results follow the returned-reference rule: the payload borrows the first reference parameter, a function without one is rejected, and Option[&mut T] requires an &mut origin.

class Item:
    name: str
    count: i64

def find(items: &mut list[Item], name: &str) -> Option[&mut Item]:
    for item in items:
        if item.name == name:
            return Some(item)
    Nothing

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    mut items = [Item("ada", 1)]?
    match find(&mut items, "bob"):
        case Some(item):
            item.count += 1
        case Nothing:
            items.append(Item("bob", 1))?
    match find(&mut items, "ada"):
        case Some(item):
            item.count += 1
        case Nothing:
            pass
    print(items[0].count)?
    print(len(items))?
    Ok(())
2
2

Nothing, Err(e), and Some or Ok around such a value hold no loan and are always accepted. Every other return path must borrow the origin, as for &T results. Err payloads cannot hold references: copy the part of the input that the error reports.

Using the result

The caller’s origin loan lasts until the last use of the result or of any reference taken from it. A Nothing or Err arm holds no payload, so it may modify the origin. These forms take the reference out or inspect the result:

  • match result binds the payload reference directly; match &result is rejected because the result already holds a reference.
  • ? propagates Nothing or Err and produces the reference.
  • unwrap(), ok_or(e), is_some(), is_none(), is_ok(), and is_err().
  • Returning the result from a function with the same origin, directly or from a local.
  • Printing and str.repr, which show the referenced value: Option[&str].Some("ada").

unwrap_or(default) is rejected: it would choose between two borrows. Use match.

Holding the result

A result can be bound to an immutable local. The local is a holder, not storage: like a reference binding, it extends the origin loan to its last use. Copying it shares that loan. A reference-carrying binding must hold a borrow, so o: Option[&str] = Nothing is rejected.

A reference-carrying value is never stored. Tuples, lists and other collections, class fields, enum payloads, generic type arguments, closure captures, scoped workers, and values live across yield all reject it. Match it first and store an owned copy or an index instead.

Current limits

  • mut locals and parameters of a reference-carrying type are rejected. Pass the reference after matching.
  • A match used as a value cannot produce one.
  • Patterns cannot match nested variants through the carried reference: case Some(Some(v)) on Option[&Option[i64]] needs a second match.
  • A returned reference from a temporary argument follows the returned-reference rule: the result cannot be bound to a local.
  • Standard library lookups such as dict.get still return copies or trap; borrowed optional lookup methods are planned work.

Borrowed matching

match &value, or matching an existing &E, inspects an enum without consuming it. Named payload bindings have type &T, even for scalar or unit payloads. match &mut value, or matching an existing &mut E, binds &mut T payloads. _ creates no binding and moves no borrowed payload. _ as name binds &E or &mut E: a reborrow of the whole scrutinee, whose loan protects all of it. The scrutinee evaluates once; patterns, coverage, and arm scope follow ordinary matching. A binding inside a nested pattern is a reference to that inner payload, through boxes as well, and a name bound by p | q refers to the payload of whichever alternative matched.

Mutable matching supports standard sums and user enums, both copyable and affine. Mutating a copyable enum’s inline payload leaves independent copies unchanged. Matching and its payload borrows allocate nothing.

Payload loans descend from the scrutinee loan and protect the active variant against conflicting access, replacement, movement, or destruction. A nested binding’s loan also protects each enclosing variant. Disjoint payload fields can be borrowed mutably together; precise class-field origins remain precise, while collection and imprecise returned origins protect their whole target. Reading a discriminant conflicts with an existing exclusive loan into the enum.

Bindings exist only in their arm. Last-use checking permits replacement after the final payload use, even within that arm. No live payload loan may cross yield. Payload references can be returned under the single reference-parameter rule; the returned payload conservatively protects the caller’s whole argument. Borrowing a temporary, returning a local owner’s payload, and storing payload references in aggregates are rejected. A mut reference binding can be reassigned to a payload borrowed from itself; the arm’s payload bindings are immutable.

A returned Option[&T] or Result[&T, E] already holds a reference: match it by value, not with match &result. Its payload bindings are the carried reference and extend the call’s origin loan; see optional and fallible references.

Deterministic destruction

Live owned locals drop at lexical scope exit in reverse binding declaration order, with inner scopes before outer scopes. return, ?, break, and continue clean up the scopes they leave. Moving transfers the cleanup obligation; moved or already dropped values are not dropped again. Replacement destroys the previous initialized value before installing the new one; see assignment for evaluation order.

drop(value) -> () consumes an owner early. It cannot destroy a borrowed referent or consume an owner during a conflicting loan. Dropping a shared immutable string releases that owner’s share, without freeing other owners’ storage. Ending a borrow at last use does not move observable resource destruction away from its specified scope exit or explicit drop.

Owned expression temporaries drop at the end of their full expression unless transferred. A block transfers its result before destroying its other locals. Loop conditions and each comprehension iteration finish their own temporaries. An early propagation cleans pending evaluated operands before enclosing scope cleanup; later operands do not run.

A class may declare def __del__(self) -> () (an exclusive receiver) for additional cleanup. The hook runs before automatic field cleanup. It cannot return a recoverable error, yield, allow references to the dying value to escape, or be called as an ordinary method. Because the hook receives every field, fields of such a class cannot move out. Fields clean up automatically, skipping any field that has moved out:

OwnerCleanup order
Class fields and active enum payload fieldsDeclaration order
Tuple componentsComponent order
List elementsIndex order
Dictionary entriesInsertion order, each key before its value
Closure capturesReverse capture order

A child finishes destruction before its next sibling begins. Recursive heap cleanup requires no allocation or recursive native stack growth; source-level recursive calls inside user hooks can still use the native stack. User cleanup retains its own effects and allocation costs. Types with custom destruction, including nested fields, cannot be copied. Explicit fallible shutdown should return Result; a destructor provides fallback cleanup rather than reporting shutdown failure.

Bounded-stack destruction does not make a recursive traversal bounded-stack. An explicit descendant cursor or worklist is still needed for unbounded-depth input. See linked-data examples.

Suspended or never-started generators drop their remaining captures and locals without resuming the body. Statements following yield are not cleanup hooks. Resource scopes insert exit calls into ordinary cleanup. Tail transfers finish caller cleanup before entering the callee. Fatal traps and process aborts do not unwind.

Functions and callable values

Functions declare complete signatures. Named functions and capture-free anonymous functions have Callable types; explicit captures create concrete closure environments. Reusable calls borrow an environment, while consuming calls transfer it into the body.

Generic constraints allow one callback API to accept different concrete callable representations without erasing their types.

Signatures, methods, and calls

Named functions are declared at module scope or as class or enum methods. Every parameter and result has an explicit type, except the receiver shorthand self. All signatures are available before bodies are checked, permitting forward calls and mutual recursion. Nested named functions, overloads, and redefinitions are unsupported.

def clamp_low(value: i64, minimum: i64) -> i64:
    if value < minimum:
        return minimum
    value

def main() -> Result[(), IoError]:
    print(clamp_low(2, 5))?
    Ok(())
5

The final expression supplies the function’s result. return value exits from any suite; bare return supplies unit. Every return must match the declared type, and every continuing path must produce that result. The branch and return rules define how continuing paths join and unreachable statements are rejected.

A string literal that begins a body is the function’s documentation, not its final expression. A str function whose body is only that string is rejected with this explanation; write return "text" to return the string.

Calls evaluate the callee once, then arguments once from left to right. Parameter types supply expected types, including numeric literal context. Owned arguments copy or move; reference arguments follow borrowing. A later argument’s ? cleans up already evaluated arguments without entering the callee.

Named functions, class and enum methods, and class constructors accept name=value arguments. Positional arguments must come first; named arguments can follow in any order. Unknown names, duplicate bindings (including a positional argument plus its name), and missing required arguments are errors. Names bind statically; no runtime dictionary is constructed. Supplied expressions still run in written order, with the expected type of the parameter they name. self is supplied by the method receiver and cannot also be named. Parameter names are part of the source API.

Parameters may have defaults: def f(x: i64, scale: i64 = 2) -> i64:. Required parameters must precede defaulted parameters. Each omitted expression is evaluated afresh, after all supplied arguments, in declaration order. An explicit value, including Nothing, skips that default. Names in defaults resolve in the defining module; defaults cannot depend on parameters or self. Concrete declarations are checked even when their defaults are never omitted; generic defaults are checked on specialization.

Owned defaults produce fresh owners. For a borrowed parameter, the default produces an owner of the referenced type and lends it for the call. For example, values: &mut list[i64] = []? creates a new list for each omission. It requires a compatible Result return type: allocation failure is explicit. ? inside a default produces an error from the called function, not its caller. Evaluated owners are cleaned up and the body is skipped on failure. Supplying every argument does not change the return type. A returned reference cannot borrow an omitted temporary; pass caller-owned storage instead.

Calls through callable values retain full positional arity. Builtins and enum payloads remain positional. Protocol requirements cannot declare defaults; named method calls in protocol-constrained generic bodies are currently rejected conservatively, including calls whose receivers happen to have a concrete type. Those bodies must also supply every method argument: an implementation’s defaults are not part of its protocol contract. Generic parameters must be inferable from supplied arguments or passed explicitly; omitted defaults do not drive type inference. Anonymous functions and C import/export declarations do not accept parameter defaults. No splat unpacking or variadic parameters are supported. Field defaults are separate from parameter defaults.

A method’s first parameter is self. Bare self borrows the enclosing class or enum immutably in ordinary methods. Class lifecycle methods __init__ and __del__ instead borrow &mut Class; enums have no lifecycle methods. Mutating ordinary methods declare self: &mut Class (or the enum type); their receiver must be a mutable owner or exclusive reference. Methods resolve statically. Temporary owned receivers are supported when no reference escapes. Attempting mutation through an ordinary method’s bare self reports the explicit self: &mut Class annotation needed for an exclusive receiver. For a generic class, use its declared type parameters in that annotation, such as self: &mut Cell[T]; the diagnostic points to these parameters rather than a particular specialization. See classes for initialization and field rules and returned references for receiver lifetime restrictions.

An explicit self: Class (or self: Enum) is a by-value receiver. Calling value.finish(args) transfers an affine receiver just like passing it as the first owned argument to a function; the old binding is unusable even when the method returns an error. Copyable enums retain ordinary copy semantics. Generic owners can use self: Cell[T] or the current-owner shorthand self: Cell. There is no dynamic dispatch or allocation introduced by this call syntax.

The receiver is evaluated before the remaining arguments, including named arguments. Once moved, it cannot be used in a later argument; save needed values before the call. If a later argument propagates an error, the pending owned receiver is cleaned up. Two-phase mutable-receiver reservation does not apply. The callee owns cleanup unless it returns or otherwise transfers the receiver. To support retry, return the owner in the error payload explicitly.

Owned self is an immutable binding, like any value parameter; use mut owned = self to modify it. Consuming methods do not permit moving individual class fields, consuming through an affine reference or indexed element, or returning a borrow of the consumed owner’s storage. Constructor/destructor receivers and operator-hook contracts are unchanged. See the consuming-method lesson.

Named and anonymous function values

A named function can be assigned, stored, passed, and returned. To accept a callback, take a type parameter bound by Callable and a parameter of type &F, or &mut F when the callback changes a capture: def apply[F: Callable[[i64], i64]](f: &F, value: i64) -> i64. This form accepts named functions, capture-free anonymous functions, and closures; see generic functions. Parameter names do not affect a signature. Callable[[], T] takes no arguments, and Callable[[T], ()] returns unit. Aliases may name these signatures.

Callable[[parameter types], result] is also a value type: a code address with no environment, the type of a named function or capture-free anonymous function. A capturing closure also needs an inline environment, so a concrete Callable parameter or field cannot hold one; use the bound form above, or a boxed callable for storage. Function values are copyable, display as <function>, and compare native code addresses rather than behavior. They cannot be dictionary keys or set elements. A payload enum variant written without arguments, such as AppError.Io, and a class constructor, such as Point, are also function values; see enums and classes. Generic constructor values specialize from an explicit data instance or an expected Callable, including prelude Some, Ok, and Err. Like named functions, constructor values can be borrowed as &Point or &Reading.Value; &mut borrows a fresh copy of the code address.

An arbitrary callee expression is allowed: choose()(n), (f)(n), handlers[index](n), and record.callback(n). The callee evaluates before its arguments. Brackets following a local binding or a module constant index it; brackets following a function declaration or module path provide type arguments. Following a field or method on a value, brackets index when their contents start at a local binding and otherwise provide method type arguments.

Take a generic function value with explicit specialization, such as identity[u8], or let a concrete expected Callable from an annotation, parameter, or return specialize the bare name. Both input and result positions provide evidence; unconstrained type parameters need explicit arguments. Signatures do not implicitly convert, and this does not introduce higher-rank polymorphism. Generic inference also works inside callable inputs and results.

An anonymous function is def(parameters) -> T: followed by a body. Its position fixes the form of the body: outside brackets the body is an indented block; inside (), [], or {} it is one expression, whose value is returned.

def apply[F: Callable[[i64], Result[i64, OverflowError[i64]]]](f: &F, value: i64) -> Result[i64, OverflowError[i64]]:
    f(value)

def main() -> Result[(), IoError | OverflowError[i64]]:
    twice = def(value: i64) -> Result[i64, OverflowError[i64]]:
        Ok(value * 2)
    print(twice(3)?)?
    print(apply(&(def(value: i64) -> Result[i64, OverflowError[i64]]: Ok(value + 1)), 3)?)?
    Ok(())
6
4

The one-expression body is the final expression of a block, so both forms create the same value and follow the same type, capture, and ownership rules. Only the layout differs:

  • The expression extends as far as it can and ends at the , or closing bracket of the enclosing element. Line breaks inside brackets do not end it.
  • Statements have no one-expression form: return, a binding, an assignment, or a second expression is rejected. Give such a function an indented body and bind it to a name first.
  • A string body is the returned value, not documentation.
  • Outside brackets a body on the def line is rejected. Parentheses are brackets: twice = (def(value: i64) -> Result[i64, OverflowError[i64]]: Ok(value * 2)).

In both forms an anonymous function must be a complete expression: a bound or returned value, an argument, an element, or the contents of parentheses. As the operand of an operator, including & and &mut, it needs parentheses: &(def [offset](value: i64) -> Result[i64, OverflowError[i64]]: Ok(value + offset)). Parameter and result types are always written. lambda is reserved and rejected with the def spelling.

Capture-free anonymous functions have ordinary Callable types, can nest, and can be returned. Their parameters and locals form a separate scope; enclosing generic type parameters are substituted. Implicit capture of an enclosing local, including one shadowing a module function, is rejected. Use an explicit capture list for environments.

Callable signatures exclude generator frames and closure environment values. A named function can be borrowed directly: apply(&double, value) or callback = &double. Each borrowed declaration or explicit specialization, such as &identity[u8], has one immutable code-address slot in the borrowing function, initialized at entry without allocation. Its references follow the ordinary local borrow and escape checks. A local binding shadowing a function name takes precedence. A generator can borrow one for a call, but a reference to that local slot cannot remain live across yield. Generic reference bindings require explicit specialization; a concrete callable parameter can instead supply the expected signature. &mut double borrows a fresh copy of the code address, so a named function also passes to a &mut F parameter; distinct &mut borrows of one declaration do not conflict because a function has no state. Callable constraints, as distinct from storage types, accept reusable named functions and concrete captured closures.

Closure environments

def [captures](parameters) -> T: creates a reusable concrete environment. Captures are explicit; no implicit deep copy occurs. The body takes either anonymous function form: an indented block outside brackets, one expression inside them.

CaptureAccess inside the body
valueOwn the value; read or borrow it
mut valueOwn private mutable state
&valueShare a surrounding binding
&mut valueExclusively borrow a mutable surrounding binding

Scalars copy into captures, immutable strings keep their sharing semantics, and affine owners move. An owned capture’s source need not be mutable; a borrowed mutable source must be. Captures cannot be redeclared or shadowed by loop, comprehension, unpacking, or match bindings inside the body.

def counter(start: i64) -> Closure[[], Result[i64, OverflowError[i64]]]:
    def [mut start]() -> Result[i64, OverflowError[i64]]:
        start = start + 1
        Ok(start)

def main() -> Result[(), IoError | OverflowError[i64]]:
    mut count = counter(2)
    print(count()?)?
    print(count()?)?
    Ok(())
3
4

Every closure has its own declared arithmetic contract, independent of where it is defined or called. A checked: block or unchecked: suite around its definition does not change its body. There is no private callback error channel or checked-block lifetime restriction; normal capture ownership and borrow rules still apply. Callers handle a closure’s Result explicitly, just as they handle a named function’s result.

Calls borrow the environment and may repeat. Mutable captures require an exclusive environment borrow and a mutable closure binding or owner. Captured owners can be borrowed and mutated as permitted but cannot be moved out; use a consuming closure when ownership must leave the body. Assigning a closure moves it; copy and equality are unavailable. Display is <closure>.

Closure[[parameter types], result] describes a concrete environment’s signature. Its producer identity is retained: different closure expressions do not unify merely because their signatures match. Factories and generic consumers preserve that concrete layout. Prefer F: Callable[...] with &F or &mut F for reusable callback APIs accepting both named functions and closures; the generic constraint is separate from a code-only Callable value. Diagnostics identify a concrete closure by its definition’s source file, line, and column (line and column for source supplied without a filename). This location describes its origin; different generic instantiations of the same expression can still have distinct environment types.

Creating, moving, passing, returning, calling, and dropping inline environments adds no heap allocation. Captured owners drop in reverse capture order. Owned environments can capture other owned environments, ranges, and standard sums, and can travel through Option/Result or suspended generator frames. Reusable closures cannot capture generator frames. Body operations and capture expressions retain their own allocation and error contracts.

Capture loans follow moves and remain live through environment borrowing and later argument evaluation. Shared captures prohibit owner mutation or destruction; exclusive captures also prohibit competing reads. Borrowing environments can be passed by reference, including through generic callbacks, but cannot cross a function boundary by value, escape their creating function, own-nest, enter sum/container storage, or suspend in generators. Borrowing another environment retains its transitive loans.

Calls can target temporaries, such as make(3)(4) or wrapped.unwrap()(4); the callee survives the full expression and cleans up if an argument propagates. Capturing and one-shot closures cannot return references or yield. Environment nesting is limited to 64 levels and must fit the native stack layout limit. Closure calls do not perform native tail transfers.

Consuming and stored closures

def once [captures](parameters) -> T: creates an affine one-shot environment. Calling consumes it and transfers owned captures into the body, where they can be moved into the result. mut value permits changing a capture during that call; the closure binding itself need not be mutable. Even a capture-free def once (...) is affine. Calling twice, through a reference, or through a reusable Callable constraint is rejected.

OnceClosure[[inputs], result] describes its concrete signature and producer identity. It does not unify with reusable Closure. The generic constraint F: OnceCallable[[inputs], result] accepts named functions, reusable closures, and one-shot closures. A consuming callback API takes F by value and may bind a mutable local to accommodate mutable reusable closures. Specialization still checks concrete move rules: calling a one-shot argument twice is rejected. OnceCallable is a constraint, not a storage type.

A function returning OnceClosure or Closure, alone or inside a sum such as Result, takes the concrete environment from a closure value it returns. The signature alone does not specify the captured environment, so a body that returns no closure value on any path, such as one that only returns Err, is rejected with cannot infer the concrete one-shot closure returned by this function (or closure for a reusable one).

? returns its error, so it also resolves a closure signature in the declared error, including inside a retained-job error such as SpawnError or SubmitError and as an anonymous sum member. The first closure returned or propagated fixes that position’s producer; another producer is rejected. Several closures from one factory function share its producer. To propagate distinct jobs, drop_job() converts each error to the job-free SpawnError[()] or SubmitError[()]. A member pattern with the same signature, such as case failed: SpawnError[OnceClosure[[], i64]]:, names the resolved member. Only declared return types infer this way: a closure signature in a parameter’s or local binding’s sum is rejected.

Uncalled environments drop captures in reverse order. After entry, the body owns cleanup: moved result captures survive and the rest drop on exit. Argument failure drops the pending environment without entering the body. One-shot environments can own and resume concrete generator frames; mut source permits advancing a captured frame. Creation, movement, invocation, and cleanup of the inline environment allocate nothing.

One-shot borrowed captures permit a local consuming call. Their loans remain live through argument evaluation and invocation and may end afterward. Such environments cannot escape, pass by value across functions, own-nest, enter sums/containers, or suspend in a generator. Passing one by reference does not grant permission to consume it. Reference returns, yielding, and capture shadowing have the same restrictions as reusable closures.

Wholly owned concrete environments can be stored in generic class fields, tuple components, list elements, dictionary values, and generic enum payloads, including through standard sum wrappers. A signature annotation alone does not resolve a storage layout. Different tuple slots or enum variants may hold different producers; a list’s elements share one concrete producer type. Borrowed captures and environments containing generators cannot enter heap storage. A rejection names its cause: an unresolved signature, a borrowed capture, a captured generator, a reference, or a generator. Classes and enums themselves are inline; their construction adds no environment allocation. Collection construction and growth remain fallible. For different shared-call producers behind one signature, use an explicitly allocated BoxedCallable.

Calling a stored reusable callback borrows its field or element, exclusively for mutable captures. A live indexed callback loan protects the collection against invalidating mutation. Moving, replacing, and dropping storage retain ordinary ownership and cleanup rules. Values containing environments cannot be copied or compared; environments cannot be dictionary keys or set elements.

A stored one-shot callback must first be extracted as an owner, using pop, consuming iteration or matching, or tuple unpacking. Calls through references or indexed places cannot consume it. A class field can own such a callback; move the field out of an owned binding to call it. Invocation, extraction, and cleanup add no allocation; growing its collection can allocate.

Boxed callable environments

BoxedCallable[[inputs], result] owns an explicitly allocated callback with a structural signature. Unlike concrete Closure, its type does not include the producer or capture layout. Different owned environments with the same signature can share list elements, dictionary values, class fields, enum payloads, and factory return types. Named Callable values can use the same owner type.

BoxedCallable(callback) consumes the callback and returns Result[BoxedCallable[[inputs], result], AllocError]. Construction always attempts one allocation, even for a named function or capture-free closure. There is no implicit conversion or allocation when assigning or returning a callback. On allocation failure the consumed environment and its captures are dropped.

The initial contract accepts reusable shared-call callbacks with wholly owned captures. Borrowed captures, generator-containing environments, mutable closures, one-shot closures, and callbacks returning references are rejected. Signatures otherwise use the same rules as Callable, including reference parameters and ordinary owned arguments, with at most 255 parameters plus the hidden environment slot. A BoxedCallable satisfies a matching generic Callable constraint; its affine ownership remains visible to specialization.

Invocation borrows the owner immutably, including through shared or mutable references, fields, and collection indexes. The binding need not be mutable. Temporary owners survive argument evaluation and the call, then drop normally. Live borrowed callbacks prevent moves, replacement, removal, and invalidating collection changes. Moving a box transfers its sole owner; copying, equality, membership equality, hashing, and use as a dictionary key or set element are unsupported, as for concrete closure environments.

Calls, moves, and cleanup allocate nothing beyond allocations explicitly performed by the callback itself. The heap box retains its concrete environment descriptor for relocation and destruction, together with a compiler-generated adapter address. Each capture is dropped exactly once when its owner is replaced or destroyed. Calls use that adapter and the public signature, so the caller does not need the concrete capture layout. Formatting shows <boxed callable>.

Erasure does not certify captures or indirect effects for concurrency. Boxed callbacks, including nested fields and container elements, cannot cross worker or channel boundaries. Retain a concrete environment when worker eligibility must be checked. Mutable and consuming erased modes require a separate contract.

Recursion and tail calls

A source-tail call supplies the function’s result without further work. This includes final expressions, explicit returns, selected conditional-expression branches, tail if/match arms, the arms of return if, the cases of return match, and result branches of and/or. An if or match block bound to a local or used as an assignment’s value is not a tail position. Arithmetic, wrapping in Ok or Some, conversion, or ? after a call makes it non-tail. Evaluating arguments before the call does not make it non-tail.

For a direct source-tail call within its caller’s direct recursion cycle, compilation requires an approved native tail transfer. Otherwise it reports the callee and the lifetime, resource-scope exit, or representation restriction at that call. Cycles are computed after concrete generic specialization and include both tail and non-tail direct edges. Different specializations are different nodes. A cycle that also has non-tail edges has no constant-stack guarantee.

A transfer evaluates arguments left to right, then drops pending temporaries and the caller’s remaining owned parameters and locals in ordinary exit order, then enters the callee. Moved arguments belong to the callee. Destructors on unmoved locals run before the callee body and do not by themselves prevent a transfer. An argument’s ? takes an ordinary error exit instead.

These restrictions retain the caller for an ordinary call:

  • Any reference argument into the caller’s locals, owned parameters, or temporaries, including a receiver in that storage.
  • A pending with exit action after the call.
  • An owned inline argument, such as a range, record, closure, or generator frame.
  • A result requiring conversion, wrapping, propagation, or incompatible result storage.

Nonrecursive ordinary fallbacks are legal. Reference arguments originating in the caller’s reference parameters may be forwarded, including reborrows, projections, borrowed-match payloads, borrowed iteration items, and forwarded reference results. Their owners belong to an older caller. Exact-compatible inline results can also transfer, including ranges, records, nested sums, and concrete generator frames. No hidden heap allocation extends argument lifetime.

Indirect Callable calls follow the same transfer restrictions but are outside the direct recursion guarantee. Concrete closure invocation, worker starts, generator construction, and generator bodies are outside that graph. Ordinary recursive calls during generator resumption may consume stack. Explicit direct-call edges in generated adapters participate; an indirect call into an adapter does not create a direct edge. See the tail-call ABI for implementation.

Generics and constraints

Functions, classes, enums, and methods can declare type parameters. They also accept value parameters, such as const N: i64, for compile-time integer and boolean specialization.

Type parameters are written as follows:

def identity[T](value: T) -> T:
    value

def add[T: IntType](a: T, b: T) -> T:
    a + b

Concrete uses preserve ordinary type and ownership rules. Generic bodies are checked for each used concrete instance. Calls remain statically dispatched; generic syntax adds no runtime type lookup or allocation. The toolchain documents specialization limits and caching.

Functions and methods accept unconstrained parameters, IntType, FloatType, CopyType, KeyType, Dimension, structural protocol constraints, and callable constraints. Dimension accepts compile-time dimensions, which have no values. IntType accepts the eight fixed-width integer types and FloatType accepts f32 and f64, including aliases; each is a constraint rather than a value type. No constraint admits both integers and floats; an unconstrained parameter does, because a body is checked for each instance. Callable capabilities follow functions.

KeyType

KeyType accepts exactly the supported dictionary keys and set members: integers and their supported domains, bool, str, bytes, and tuples, classes, enums or anonymous sums whose every field, payload or member is a supported key type. Floats, collections, boxes and recursive types remain excluded. Custom cleanup is allowed, but copying operations still reject such keys. It is a built-in constraint, not a protocol or a stored value type; no __hash__ method is needed or accepted.

Functions, methods, classes and enums can declare K: KeyType. Explicit and inferred arguments are checked against the same admission rule as dict[K, V] and set[K]. This makes the requirement visible and reports unsuitable arguments at the generic boundary, even when its body does not hash anything. Generic bodies remain checked per concrete instance: an unconstrained K can still be used with collections when its concrete type is valid.

def distinct[K: KeyType](values: list[K]) -> Result[set[K], AllocError]:
    mut result: set[K] = set()?
    for value in values:
        result.add(value)?
    Ok(result)

def main() -> Result[(), AllocError | IoError]:
    print(len(distinct(["red", "blue", "red"]?)?))?
    Ok(())
2

KeyType implies neither CopyType nor eligibility for explicit copy(). The loop consumes its input and transfers each key, so it also works for keys with custom cleanup. A generic body that copies keys or calls non-mutating set algebra is rejected when specialized with such a key type.

def accept[K: KeyType](value: K) -> ():
    pass

def main() -> ():
    accept(1.5)
f64 does not satisfy KeyType

Finding an unsupported key field

When a collection or KeyType constraint rejects a structured key, the diagnostic follows the first unsupported component in declaration order. It names class fields, enum variants and payload positions, and tuple element positions. Positions start at one. For example, a class field sample whose second tuple element is f64 reports field sample -> tuple element 2: f64 is not a supported key type. Unsupported sum members are named in the diagnostic path. This applies to type annotations, inferred collection literals and conversions, and explicit or inferred generic arguments.

CopyType

CopyType accepts every type argument that copies under the copy rule: numbers, bool, str, bytes, ranges, NonZero, unit (), named function values, tuples, enums, and anonymous sums whose parts copy, and @copy classes whose type arguments copy. Closures, collections, boxes, and other classes do not satisfy it. Type arguments are never references, so &T never satisfies it; a &T parameter borrows a T that does.

def twice[T: CopyType](value: T) -> (T, T):
    (value, value)

def main() -> Result[(), IoError]:
    print(twice(3), twice("a"), twice((1, Some(2.5))))?
    Ok(())
(3, 3) ("a", "a") ((1, Option[f64].Some(2.5)), (1, Option[f64].Some(2.5)))
def twice[T: CopyType](value: T) -> (T, T):
    (value, value)

def main() -> Result[(), AllocError]:
    pair = twice([1]?)
    Ok(())
list[i64] does not satisfy CopyType: it moves on ordinary use

A body is checked per instance, so twice with an unconstrained T also compiles for these calls. The bound states the requirement in the signature and reports a moving argument at the call. Functions, methods, classes, and enums accept it.

Argument inference determines parameters from inputs or a complete explicit argument list. Generic data and methods retain nominal identity. Protocols describe exact structural interfaces without inheritance or registration.

Generic function type arguments can be unit (), including when inferred from Result or Option payloads. Substituted signatures and bodies retain ordinary unit restrictions: unit can be a return type or built-in enum payload, but not a direct function parameter or stored collection element. A &T or &mut T parameter instantiated with () borrows unit, so a helper such as std.test.is_err accepts a Result[(), E]. Explicit calls and specialized function values also accept ().

Generic function type arguments cannot be references. A signature can borrow T with &T or &mut T. A callable-constrained parameter can borrow a concrete closure while keeping its capture loans live across the call. Data and protocol arguments have their additional restrictions on their own pages.

Diagnostics from instances

A generic body is not checked where it is declared. Each used instance is checked once, and an error found in it is reported at the use that first required the instance, in code that is not itself a generic instance. A note gives the position in the definition, the instance being checked, and the reason:

main.plenty:7:9: cannot instantiate `outer[i64]`: its definition is rejected
  pairs.plenty:5:13: note: in instantiation `pairs.scaled[i64]`: unknown binding `pairs.factor`

The first line names the instance the use asked for. The note names the instance whose definition failed; the two differ when generic code requires further instances, and the instances between them are not listed. Further notes belong to the reason. Names longer than 96 characters are cut short.

When the body is rejected only because of a type argument, the first line names that argument instead. Three causes are recognized: an implicit copy of an affine argument (moving it out of a field, an element, or a reference, or using a moved binding of it), an operator the argument does not support, and a method the argument does not have:

main.plenty:8:9: `outer[list[i64]]` requires `pairs.twice[list[i64]]`, which copies `T`, but `list[i64]` cannot be copied
  pairs.plenty:2:13: note: in instantiation `pairs.twice[list[i64]]`: use of moved binding `value`
  pairs.plenty:2:6: note: `value` is moved here

The note then describes the rejected operation without advice for changing the body, which the caller may not own. A cause is attributed when the rejected value comes from a type parameter and its type is exactly one parameter’s argument. Equal concrete types alone do not establish that origin: an inferred row = [1]? is not a T value merely because T = list[i64]. Conversely, an unrelated concrete annotation does not hide a failure on an actual T.

The diagnostic tracks parameter origins through local aliases, borrows, collection displays, and field or element projections whose declared shape identifies the parameter. Projecting inside an opaque T does not establish the origin of its contents. Attribution is conservative: mixed assignments, untracked transformations such as call results, or two parameters with the same concrete argument retain the first form. These rules affect diagnostic wording only, not which programs are accepted.

This covers generic functions, generic methods, functions written inside them, and the methods of a generic class or enum instance. A method of an instance is checked when it is first used: called, including by a constructor call for __init__, or required by a protocol bound that the instance satisfies. A __del__ method is checked when the instance is first required, because constructing a value arms it. An instance can therefore exist although one of its methods would be rejected for it, as long as that method is not used:

main.plenty:9:13: `Cell[list[i64]].get` copies `T`, but `list[i64]` cannot be copied
  main.plenty:5:9: note: in instantiation `Cell[list[i64]].get`: cannot move out of a field

A method required through a protocol bound is reported at the bounded use, naming the bounded instance, such as accept[Holder[Guard]]. Unused methods are not compiled for the instance either.

Since bodies are checked only per instance, a body error that no type argument causes, such as an unknown name, is reported in the first form, and not at all while the declaration is unused. Errors in the declaration itself (type parameters, constraints, signature types) and uses that do not satisfy a constraint are reported at one location. Instance field types, constraints on data arguments, and substituted method signatures are checked when the instance is first required.

Checking every method

Checking per use keeps compilation fast while code changes, but a library’s tests then check a method for a type argument only if they use it. The check modes accept --check-all-methods (plenty --check, --check-module, and plenty check): after the used methods, every remaining method of every class and enum instance the program creates is checked as if it were called. Such an error has no use, so it is reported where the program first required the method’s type, such as a constructor call or a type annotation, with a note naming the flag:

main.plenty:8:9: `Cell[list[i64]].get` copies `T`, but `list[i64]` cannot be copied
  main.plenty:4:9: note: in instantiation `Cell[list[i64]].get`: cannot move out of a field
  note: `Cell[list[i64]].get` is checked because of --check-all-methods; the program does not use it

The flag checks only types the program’s own package declares: the entry file’s directory (or --root) for plenty --check, and the package’s src/ for plenty check. Generic types from std and from dependencies keep per-use checking, because their authors chose which methods each instance supports. LruCache[K, V].get copies V, for example, so an LruCache[str, list[i64]] is accepted as long as the program does not call get. An own type is still checked when its type arguments come from std.

The flag checks only instances: a generic class or enum with no instance stays unchecked, since its bodies are checked with concrete type arguments and there are none. Compiling does not accept the flag, so it never changes generated code or the compile cache.

Type argument inference

Calls supply all generic arguments, as in identity[list[i64]](values), or infer all from inputs, as in identity(values). Partial explicit lists and inference from an expected result type are unsupported. Output-only or unused parameters need an explicit list.

Inference structurally matches concrete arguments: &T against &Message learns T = Message; &dict[K, V] against &dict[str, u8] learns both parameters. Collections, ranges, generators, tuples, sums, callable signatures, and nominal generic class/enum arguments participate. Repeated parameters must agree after alias resolution. Nominal matching requires the same declaration as well as matching arguments. Value parameters are inferred from nominal arguments by the same structural match: Bounded[T, lo, hi] against Bounded[u8, 0, 100] learns all three parameters.

A shared-reference parameter supplies ordinary implicit borrowing, so a parameter value: &T can accept an owner directly. Mutable borrowing still needs &mut or an existing exclusive reference. Arguments evaluate once in source order.

A generic parameter position supplies no expected type to its argument. Unsuffixed integers default to i64 and floats to f64, independently of other arguments or the result context. add(1u8, 2u8) works; add(1u8, 2) conflicts. add[u8](1, 2) guides both literals. Nongeneric parameter positions retain their ordinary contextual typing. Empty literals and incomplete sum constructors may need an annotated binding or explicit arguments.

Callable constraints contribute evidence from a known callback signature: F: Callable[[], T] can determine T from the callback result. Parameterized protocol bounds likewise contribute evidence from accessible methods on the known class. Evidence from all inputs and constraints must agree. Inference does not widen numbers or search for an implementation class.

A generic function becomes a concrete function value using identity[u8] or an expected concrete Callable signature. A fully concrete callable constraint can also select a generic named function passed directly by value. This contextual selection does not change ordinary generic-call inference. OnceCallable constraints accept consuming callbacks with the same inference rules. Constraint signature shapes are checked at declaration; concrete ownership capabilities are checked when the body is specialized. Polymorphic function values are unsupported.

Value generic parameters

Functions, classes, enums and methods can specialize on compile-time integers and booleans as well as types. Write const before a value parameter:

def selected[const Enabled: bool](value: i64) -> i64:
    if Enabled:
        return value
    0

def main() -> Result[(), IoError]:
    print(selected[True](7), selected[False](7))?
    Ok(())
7 0

A value parameter’s declared type is one of i8, i16, i32, i64, u8, u16, u32, u64 or bool. It may instead name a preceding type parameter constrained by IntType, as in [T: IntType, const lo: T, const hi: T]. This dependent form keeps a bound’s width and signedness identical to T. Aliases in the const parameter’s own type annotation are not supported. Protocols and their requirement parameters do not accept const parameters.

Arguments and identity

Each value argument is an integer or boolean literal, an optionally negated integer literal, a module constant (including an imported public constant), or a bare value parameter being forwarded. Unsuffixed literals take the parameter’s declared type; suffixed literals and constants must match it exactly. There is no widening. Floating-point values, strings, calls, numeric type constants such as i64.MAX, and expressions such as N + 1 are not value arguments. A module constant may name a value computed by a supported constant initializer.

const maximum: i64 = 4 * 4
class Tag[const N: i64]:
    def number(self) -> i64:
        N

def number[const N: i64](value: &Tag[N]) -> i64:
    value.number()

def main() -> Result[(), IoError]:
    tag: Tag[16] = Tag[maximum]()
    print(number(&tag))?
    Ok(())
16

Instance identity compares resolved values, so Tag[maximum] and Tag[16] are the same type. Different values remain different nominal types even when no field uses the parameter. Aliases can name complete instances. Values add no runtime fields and do not change copying, borrowing or cleanup rules. A value argument cannot appear where a type argument is required.

Bodies and inference

Within a specialized body, a const parameter is an immutable value of its declared type. Reads become typed literals. Ordinary arithmetic on it uses the usual arithmetic contracts; const parameters do not make a function a compile-time evaluator. A local binding cannot reuse a generic parameter’s name.

Structural inference learns values from nominal arguments: Tag[N] matched against Tag[16] supplies N = 16. Repeated evidence must agree. An ordinary integer argument’s runtime value supplies no evidence, and an output-only parameter needs an explicit argument. As with type parameters, supply all arguments explicitly or infer all of them. Methods supply only their own parameters; the receiver already fixes its class or enum arguments.

Each distinct argument combination uses the existing specialization cache and instance limits. Value argument expressions cannot grow recursively because bracket arithmetic is rejected.

Bounded[T, lo, hi] is the first built-in consumer. Inline arrays, derived sizes in signatures, and general per-instance static assertions are not implemented; B83 tracks that scope.

Generic data and methods

class Cell[T]:
    value: T

    def read(self) -> T:
        self.value

enum Choice[T]:
    Value(T)

Cell[u8](7) constructs a Cell[u8] directly using the ordinary class rules. Class constructor parameters can be inferred from generated field order or an explicit initializer’s signature: Cell(7u8) selects u8, while Cell(7) selects i64. Output-only or phantom parameters need explicit arguments; a result annotation does not influence argument inference.

A constructor’s enum qualifier requires explicit arguments: Choice[u8].Value(7) constructs a concrete instance. A pattern takes none, because the matched value supplies them: case Choice.Value(value): matches it. Aliases can name complete instances. Different arguments produce different nominal identities even when a parameter is absent from every payload. This includes value parameters: Tag[3] and Tag[4] are distinct instances.

Bare self uses the current concrete class or enum; an explicit mutable receiver is self: &mut Cell[T] or self: &mut Choice[T]. A by-value receiver is self: Cell[T] or self: Choice[T] (the current-owner spellings Cell and Choice also infer their declared parameters). Ordinary methods may declare their own parameters and integer, protocol, or callable constraints. Data parameters are already fixed by the receiver; method parameters cannot shadow them. Lifecycle methods cannot add parameters.

Explicit method calls use receiver.method[Types](arguments). Supply every method parameter without repeating data parameters. Inferred method arguments follow ordinary inference. Fields and temporary owned receivers support generic method calls; returned references retain their ordinary named-receiver and lifetime restrictions. Brackets beginning with a local binding index a callable field instead, as in table.callbacks[index](argument).

Data parameters accept IntType, FloatType, CopyType, KeyType, Callable, and OnceCallable constraints; protocol bounds on data are unsupported. Callable signatures may mention other data parameters and contribute constructor inference evidence. Explicit instances and aliases are checked too; unused declaration bounds still require valid signature shapes.

Data type arguments cannot be unit, references, or generators. Resolved owned closures can be arguments; see stored callables. Generic fields, construction, moves, and destruction keep their ordinary data semantics, including recursive storage.

Public instances must expose public concrete arguments, including phantom parameters and types named only in callback constraints. Declaration and member visibility apply normally. A generic method does not currently satisfy a protocol’s monomorphic method requirement.

Structural protocols

protocol Readable[T]:
    def read(self) -> T:
        pass

def read[T, R: Readable[T]](source: &R) -> T:
    source.read()

A protocol declares required ordinary method signatures with pass bodies. A bound accepts a class or enum whose accessible inherent methods match every requirement, even requirements unused by the generic body. Receiver borrowing, arity, parameter types, and return type must match exactly; parameter names may differ. There is no inheritance, registration, or import-driven method activation.

Bare self means a shared receiver. An owned requirement uses self: Readable[T] or self: Readable; its implementation must also take its receiver by value. Owned, shared and mutable receivers are distinct requirements. Mutable requirements use self: &mut Readable[T] or self: &mut Readable. Within requirements the protocol’s own name denotes the implementing type, including in returned references.

Protocol arguments substitute into all requirements. Once an input determines the implementing type, its accessible concrete signatures supply inference evidence for other parameters, including nested types and returns. Evidence must agree with ordinary arguments and callable constraints. A method type that no arguments can fit, such as -> () for -> Result[(), E], does not satisfy the protocol. For a concrete implementation, a protocol parameter absent from every requirement still needs explicit arguments. A Box[P[Args...]] supplies its fixed arguments directly. Inference follows known signatures without searching for a type or implementation.

Visibility is checked from the bound’s defining module before method signatures supply evidence. Public contracts cannot expose private protocols, requirement types, or concrete arguments. Importing a protocol adds no methods to a type.

A requirement may declare its own type parameters, each unconstrained or constrained by one protocol. An implementation then declares a generic method with as many parameters, the same constraints, and the same signature once its parameter names are replaced by the requirement’s; the names may differ. Calls through the bound instantiate the implementation for each argument type:

protocol Sized:
    def size(self) -> i64:
        pass

protocol Sink:
    def put[S: Sized](self: &mut Sink, item: &S) -> Result[i64, OverflowError[i64]]:
        pass

class Crate:
    width: i64

    def size(self) -> i64:
        self.width

enum Parcel:
    Letter
    Packet(i64)

    def size(self) -> i64:
        match self:
            case Parcel.Letter:
                1
            case Parcel.Packet(width):
                *width

class Total:
    sum: i64

    def put[T: Sized](self: &mut Total, item: &T) -> Result[i64, OverflowError[i64]]:
        self.sum += item.size()
        Ok(self.sum)

def feed[K: Sink, S: Sized](sink: &mut K, item: &S) -> Result[i64, OverflowError[i64]]:
    sink.put(item)

def main() -> Result[(), IoError | OverflowError[i64]]:
    mut total = Total(0)
    feed(&mut total, &Crate(3))?
    feed(&mut total, &Parcel.Letter)?
    print(feed(&mut total, &Parcel.Packet(4))?)?
    Ok(())
8

Crate and Parcel both satisfy Sized, so feed accepts either. Use an enum to model a sum type, a value that is one of several variants, and a class to model a product type, a record that holds all of its fields. Either can meet the same protocol.

A generic method cannot satisfy a requirement without type parameters, and a non-generic method cannot satisfy one with them.

Protocols constrain class and enum arguments, the built-in bytes, and an explicit protocol box exposing the identical protocol. A protocol alone is not a runtime value type. bytes has one method, len(self) -> i64; other built-in types have none to offer. Protocol parameters cannot have their own bounds; protocol arguments exclude unit, references, generators, and closure environments. Fields, inheritance, multiple bounds, defaults, associated types, and lifecycle requirements are unsupported.

Generic bodies are checked per concrete instance. Requirements express the minimum interface rather than a separately checked abstract body. IntType, FloatType, CopyType, and KeyType remain separate built-in constraints.

std.fmt.Formattable additionally accepts an anonymous sum when every normalized member satisfies its contract. The compiler dispatches to the active member through a shared borrow without allocation. This is specific to that standard-library protocol: a user-defined protocol with the same signature does not gain sum forwarding.

Design history

The original protocol design records the shipped foundation. Protocol method bodies propose defaults, final methods, and explicit private-hook grants; those extensions are not implemented. For an existing-language solution, see library-owned operations. The boxed protocol design records the research and rationale for implemented explicit erased owners.

Boxed protocol objects

Box[P](value) -> Result[Box[P], AllocError] consumes a concrete class, enum, or bytes value satisfying protocol P. Different implementations have the same owner type, so a list[Box[P]] or dictionary with Box[P] values can call protocol methods without matching. Protocol arguments must be fixed, for example Box[Readable[i64]].

Construction makes one allocation, including for an empty payload. Failure destroys the consumed input. A static method table is shared by objects with the same concrete implementation and protocol. Calls, moves, and compiler cleanup add no allocation; user methods and destructors retain their own costs. Collection insertion retains its ordinary consumed-input failure contract.

Every method operates on the same concrete payload. Shared methods borrow it; mutable methods require a mutable binding or exclusive reference. Loans remain rooted in the owner, including through fields and collection elements. An active borrow prevents conflicting replacement, removal, mutation, or movement. Read-only calls on temporary owners keep them alive through the expression; mutable calls require a place.

Only protocol methods are visible. Box[P] satisfies the identical P bound, including its fixed arguments. Generic inference can use those arguments without knowing the hidden concrete implementation. Structural matching against another protocol, even one with identical methods, is unsupported. Conformance checks required methods from the protocol’s defining module; adapters grant no private access.

Boxing eligibility

Every requirement must have a shared or mutable receiver and fixed argument and result types. Ordinary owned arguments, borrowed arguments and slice views, unit results, aggregate results and fallible results are supported. Calls take positional arguments. The complete protocol is checked when Box[P] is used.

These requirements prevent boxing, while remaining valid for static generic use:

  • Method-local type parameters.
  • A consuming receiver.
  • The implementing type outside the receiver, including nested occurrences.
  • Returned references or views, including references inside Option or Result.

Generator-frame and concrete-closure signature types remain unsupported by the indirect-call ABI. Mutually recursive boxed interface signatures are diagnosed rather than expanded indefinitely.

std.fmt.Formattable has a method-local writer parameter and cannot be boxed. The constructor accepts specialized generic classes and enums, but does not introduce new builtin conformances or forward arbitrary anonymous sums.

Ownership and limits

Bare P, list[P], and &P remain unsupported value types. Use &Box[P] to borrow an erased owner. Box(value) still makes a concrete Box[T]; annotations and arguments never implicitly erase or allocate. Box[P](existing_box) is rejected: construct from the concrete owner directly, or move an existing Box[P] without another constructor.

Protocol boxes can be stored in ordinary aggregates and returned from factories. They cannot be copied, compared for equality, hashed, used as keys/set elements, or extracted with *box. Downcasts and protocol conversions are unsupported. Automatic formatting writes <boxed protocol> without inspecting the payload. These restrictions apply through containing aggregates too.

Native worker/channel transfer is rejected, including through nested containers. A protocol box cannot conceal executors, futures, channel endpoints, or erased callback captures; these would hide ownership obligations from static checks. Shared owners cannot contain protocol boxes, because their erased payloads could conceal ownership cycles or owners whose destruction waits. Indirect calls retain the compiler’s conservative effect checks.

The channel restriction includes Sender[T] and Receiver[T] nested in classes, enums or containers. Hiding an endpoint would prevent the compiler from checking whether it can block a worker during a join, result wait or pool shutdown. Keep such payloads in concrete owners such as Box[T], and drop the relevant endpoint holder before waiting.

See the tutorial, concrete boxes, and design and research.

Errors, allocation, and resource scopes

Recoverable failure is a value: Option represents absence and Result represents success or an error. ? propagates a failure through the enclosing function with ordinary cleanup. Allocating operations expose checked failures; fatal traps terminate without unwinding.

There are no exceptions. A try, except, raise, or finally statement is rejected with a hint toward Result:

def main() -> ():
    try:
        pass
`try` is not supported; Plenty has no exceptions: return `Result[T, E]`, propagate an error with `?`, and handle it with `match`

Destruction defines cleanup order. The entrypoint defines process exit status and reporting of a main error. std.error adds operation and path context around a typed error and reports it on standard error.

Standard sums and propagation

Option[T].unwrap_or(default: T) -> T selects a present payload or the default. Option[T].ok_or(error: E) -> Result[T, E] converts Some(value) to Ok(value) and Nothing to Err(error). Both consume owned receivers, evaluate receiver then argument exactly once, and drop the unused argument. They add no allocation and support affine payloads and arguments; their construction expressions retain their own allocation contracts. Unit payloads and unit errors are supported. Bare absence needs enough type context to determine T. Conversion is explicit: use option.ok_or(error)? in a Result-returning function; ? alone does not invent an error for absence.

Defaults and recovery

Result[T, E].unwrap_or(default: T) -> T returns the Ok payload or the supplied default, discarding the error. Like the Option form, it evaluates the receiver and default eagerly, in that order. In option.unwrap_or(compute()?), compute runs and can propagate an error even when the option is Some. Choose an eager default only when that is intended.

The lazy alternatives consume an owned callback and invoke it at most once:

MethodCallbackWhen calledResult
Option[T].unwrap_or_else(f)() -> TNothingT
Result[T, E].unwrap_or_else(f)E -> TErr(error), passing the errorT
Option[T].ok_or_else(f)() -> ENothingResult[T, E]

Named functions, reusable closures, and owned one-shot closures use OnceCallable. Constructing a callback and evaluating its captures still happens eagerly; only its body is conditional. Unused callbacks and their captures are dropped. These operations add no allocation, deep copying, or implicit error propagation. The existing restrictions on passing borrowed environments by value apply. Use match for reference-carrying payloads and callbacks that cannot be passed as owners.

Both sums also support map_or(default, mapper) and map_or_else(fallback, mapper). The mapper consumes T and returns U. map_or evaluates its default U eagerly. map_or_else invokes only the selected callback: Option’s fallback takes no arguments, Result’s fallback consumes E, and both return the same U as the mapper. Receiver and callback expressions evaluate from left to right, once. No automatic flattening occurs.

For a fallible default, choose U = Result[T, E], wrap a present value with Ok, and propagate after selection:

def default_number() -> Result[i64, ParseError]:
    i64.parse("23")

def choose(value: Option[i64]) -> Result[i64, ParseError]:
    Ok(value.map_or_else(default_number, Ok)?)

def main() -> Result[(), ParseError | IoError]:
    print(choose(Some(7))?, choose(Nothing)?)?
    Ok(())
7 23

These fallback semantics follow Rust’s Option and Result operations, with Plenty’s explicit allocation and callback contracts.

Deliberate assertions

Option[T].expect(message: str) -> T and Result[T, E].expect(message: str) -> T return the success payload or trap with the supplied message and the call site’s source location. The message is evaluated eagerly, including on success. Reporting adds no allocation; constructing a dynamic message retains its ordinary allocation contract. The trap reports the message, without formatting the error payload, and uses Plenty’s existing trap behavior rather than Rust panic unwinding. Prefer ? and explicit recovery for expected failures.

Inspection and propagation

Result[T, E].is_ok() -> bool and is_err() report which variant a result holds; Option[T].is_some() -> bool and is_none() do the same for an option. They observe the receiver, so a named value remains usable afterwards and a temporary is dropped at the end of the full expression. They take no arguments, ignore the payload, and allocate nothing. Asking a Result for is_some or an Option for is_ok is a compile-time error that names the available pair. Classes may define methods with the same names.

Option[T] has Some(value) and Nothing; Result[T, E] has Ok(value) and Err(error). These prelude variants are ordinary typed values. Some can infer its type from its payload; Ok, Err, and Nothing need expected context for their missing type arguments. Qualified constructors and aliases are supported. Payload storage follows enum rules.

Constructing, passing, returning, matching, and propagating standard sum wrappers allocates nothing. Payload operations retain their own costs and ownership rules. Wrapping a generator or closure does not make it copyable. References cannot be sum payloads. Nested sums preserve all scalar values; there is no source-visible null or numeric sentinel reserved for absence.

Postfix value? evaluates once. Some(value) or Ok(value) produces its payload. Nothing returns Nothing from the enclosing function; Err(error) returns Err(error). The function must return the same sum family; Result errors must be identical or widen losslessly into a declared anonymous sum, except for explicit Failure erasure below. Success types may differ. There are no implicit Result/Option conversions. A unit success produces unit.

For explicit error conversion, result.map_error(mapper) consumes Result[T, E] and returns Result[T, G], invoking the mapper only for Err. It forwards Ok without copying and adds no allocation. A payload variant is a mapper, as in print(line).map_error(AppError.Io)?. The std.error.map_error(result, mapper) function has the same behavior. Neither changes the propagation rule.

def first(values: &list[i64]) -> Option[i64]:
    Some(values.get(0)?)

def checked_first(values: &list[i64]) -> Result[Option[i64], OverflowError[i64]]:
    match first(values):
        case Some(value):
            Ok(Some(value + 1))
        case Nothing:
            Ok(Nothing)

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    values = [3]?
    print(checked_first(&values)?)?
    Ok(())
Option[i64].Some(4)

Here first is a focused demonstration of Option propagation; an ordinary first-element lookup can simply return values.get(0). Absence and arithmetic failure are distinct: checked_first returns Ok(Nothing) for an empty list and propagates overflow as Err, never as absence. ? does not implicitly convert Option to Result.

Propagation may appear in arguments, conditions, loops, and comprehensions. It composes with other postfix operations: values?[0] indexes the unwrapped value. Failure releases already evaluated pending operands and initialized locals, including hidden iterators/builders and entered resource scopes, in ordinary cleanup order. Later operands and statements do not run. Error payload ownership transfers to the caller. Generator bodies reject ? because their declared return family is Generator[T]; explicitly match instead.

Prefer a concrete error or a declared sum when combining fallible operations, so callers can inspect or report the cause. Builtin Failure is the last resort, for when no precise error can be written. It is a copyable allocation-free builtin enum containing only Failure.Unspecified. In a function returning Result[T, Failure], ? accepts any Result error type. Failure consumes and drops the original error payload, then returns Err(Failure.Unspecified) after normal cleanup. Success keeps its original payload. Erasure keeps no details, allocates no wrapper, does not catch traps, and does not convert Nothing. User cleanup retains its own effects and allocations.

Erasure applies only at ?. Assignments, arguments, direct returns, and explicit Err(payload) still require the declared error type or lossless anonymous-sum membership/widening. Use Err(Failure.Unspecified) to construct that error and Ok(()) for unit success. An allocating expression propagated this way still has its own AllocError type before propagation.

An expression statement cannot implicitly discard a Result, including in non-final branches, loops, context blocks, or generator bodies. Handle it, return it, or use explicit drop(result). Unused bindings, Option, and aggregates containing Results are not rejected by this rule. Builtin print also rejects a direct or borrowed Result; wrapper inspection uses str.repr(result) with its formatting Result handled separately.

Allocation and traps

Source-level heap construction and allocating operations use checked paths. AllocError has two allocation-free nullary variants: AllocError.OutOfMemory for allocator rejection and AllocError.CapacityOverflow for impossible sizes or layouts. Constructing, copying, comparing, matching, or propagating it inside Result allocates nothing. AllocError itself is not a family on which ? operates.

Class instances, tuples, and enum values are inline and construct directly without allocation. Allocations in their component expressions keep their own Results. A fallible class initializer can make construction return Result[Class, AllocError]; class initialization defines cleanup of a partially initialized instance. String literals, scalar values, and ranges require no heap allocation.

Collections, boxes, and newly allocated dynamic text expose allocation errors. Collection displays, comprehensions, constructors, growing updates, and copy(value) return Result. Their APIs have no separate try_ naming family. Successful reservation permits subsequent insertions within that capacity without storage allocation. Failed growth preserves logical contents and order, although capacity may have increased. Failed deep copy preserves the source and destroys partial output. Exact APIs and input-transfer behavior are in collections and text.

Failure does not generally roll back argument effects, consumed generator steps, external input, or partial writes. Operations specify what is preserved and what has already been consumed. Allocation-free wrapper transport, borrowing, inline environments, generator frames, and cleanup bookkeeping do not make allocations performed by user expressions or destructors disappear. Cleanup on an allocation-error path must still handle its own fallible work. No public allocator switching or custom-memory-manager API is exposed.

value.unwrap() extracts a success payload without allocation, moving it when owned. Err or Nothing deliberately traps. Invalid indexing, absent dictionary keys in indexed access, a numeric cast to an integer type that cannot represent the value, invalid range parameters, and an empty split separator are also fatal runtime errors. Optional lookups provide recoverable absence where supported. Integer overflow and division by zero return Err from the function’s declared Result by default, or from a local checked: block. Only an explicit unchecked: suite reaches arithmetic traps from source operators, where they currently enforce its no-failure precondition. Explicitly trapping primitives such as atomic fetch operations retain their contracts. Traps terminate the process without unwinding language scopes. They are not caught by Result propagation or resource scopes.

Trap reports

A trap flushes standard output, writes its report to standard error, and exits with status one. The line gives the position of the failing operation in the form of a compile-time diagnostic, error: file:line:column: message:

error: main.plenty:7:11: index out of bounds: index 5, length 3

The position is the start of the expression that trapped: the left operand of an arithmetic operator, the type name of a cast, the indexed collection, the receiver of unwrap or split, or the range call. A trap in an imported function or a generic instance reports the position in the function’s body, in the file that defines it, not the call or instantiation. The file is relative to the source root, tools/geometry.plenty for the module tools.geometry, so an executable does not depend on the directory it was built in. Compile-time diagnostics print the full path. Reporting a trap allocates nothing.

When the failing body belongs to the embedded standard library, an additional line names the nearest non-std caller:

error: std/collections.plenty:20:10: index out of bounds: index 0, length 0
note: called from main.plenty:7:12

This caller is the actual invocation, including generic methods, indirect calls, tail calls, and nested standard-library helpers. A user callback that calls std becomes the nearest caller; a trap in the callback itself keeps its own ordinary report. A suspended std generator uses its current resume site, not its creation site. The note is omitted when there is no source caller on the current thread, such as a std function started directly as a worker. Caller context does not cross thread creation or executor submission. This is one caller note, not a stack trace; it does not change the trap message into an API-specific error.

Caller locations

The same nearest non-std caller is available to standard-library code that reports a failure without trapping. std.test assertions use it to begin their report with file:line:column: . The compiler records it as described above, so a std function needs no extra parameter and user code no annotation. Plenty code outside the standard library cannot read it: the hook, __caller_location, is rejected with `__caller_location` is reserved for the standard library. Without a source caller on the current thread the report has no position.

TrapMessage
Integer overflow in +, -, *, //, pow, or negation, inside unchecked:integer overflow
// or % by zero, inside unchecked:division by zero
Integer pow with a negative exponentnegative integer exponent
Shift count negative or not below the integer widthshift count outside integer width
Numeric cast to an integer type that cannot represent the valueinteger cast out of range
unwrap() on Nothing or Errunwrap on Nothing, unwrap on Err
List, text, range, or Bytes index out of rangeindex out of bounds: index I, length N
Absent dictionary key in d[k]dictionary key not found
A range longer than i64range length exceeds i64
Empty split separator held in a variable; a literal "" is a compile errorstring split requires a nonempty separator
A standard-library argument outside its domain, such as a Bytes range past the lengthThe operation and argument, such as Bytes.get: index is outside 0..len()

A bounds report prints the index as written, before a negative index counts from the end. A missing dictionary key is not printed. Slices never trap on their bounds: out-of-range bounds clamp as in Python.

Standard-library code raises the last kind with the statement __trap("message"), whose argument is a string literal. It reports the position of the statement in the std file, with the caller note, and never returns, so it can end a branch of a function that returns a value. Code outside the standard library is rejected with `__trap` is reserved for the standard library.

Stack overflow is reported separately and has no position. Trap sites describes how positions reach the runtime.

Resource scopes

with expression as name: evaluates an owned manager once and moves it into a hidden mutable owner. A concrete class manager exposes visible methods __enter__(self: &mut Class) -> T and __exit__(self: &mut Class) -> (), with no other parameters. The entry result becomes a body-local binding. Without as, an unused owned result drops before the body. Entry is an ordinary call: it does not implicitly unwrap a Result. Perform fallible acquisition with ? in the manager expression.

Comma-separated managers nest left to right, each one a level of nesting. Each acquisition and entry completes before evaluating the next expression, which can use earlier entry bindings. If a later acquisition propagates failure, only already entered managers exit.

On leaving the body, its locals drop in reverse order, then __exit__ runs once, then the manager drops normally. Nested managers exit in reverse order. Return values are evaluated and preserved before cleanup. return, ?, break, and continue use this order for the scopes they leave; control inside an inner loop does not exit an outer manager. __del__ is a separate fallback hook, so exit and later destruction must be safe together. Exit is not an implicit flush or durability guarantee; fallible operations expose their own Results.

with &mut existing: instead reborrows a mutable named manager or class field. The exclusive loan lasts through __exit__, preventing competing access, replacement, and movement. After exit the caller retains its mutated manager; its destructor remains at the owner’s normal scope boundary. An existing exclusive reference must also be explicitly reborrowed with &mut.

__enter__ may return &T or &mut T under the returned-reference contract. The entry binding carries that loan and must finish before exit: it cannot escape an owned context or stay live across a borrowed manager’s exit call. Owned entry results may move out normally.

Resource-scope bookkeeping adds no runtime allocation; manager methods retain their own costs. yield inside with is unsupported. Traps and process aborts do not unwind managers. Intrinsic thread, channel, and executor managers use the same scope-exit obligations; their shutdown contracts are described under concurrency. mutex.lock() is an intrinsic manager too: it borrows the mutex shared rather than moving it, binds an exclusive reference to the value, and unlocks on every exit; see mutexes.

Iteration and generators

Iteration either consumes an owned iterable or lends access to a collection. Comprehensions build fallible collection results. Generators own concrete frames and resume only when advanced; construction and frame management add no heap allocation.

Loop control defines binding scope, break, continue, and return checking. Collections and text define their operations and ordering.

Iterable values and borrowing

for name in iterable: evaluates the iterable once. Owned collections and generators transfer into the loop’s hidden owner; it drops on exhaustion or early exit. There is no implicit snapshot. Ranges, strings, and copyable elements retain their ordinary copy semantics.

Iterating a range, as the source of for, reversed, enumerate, zip, a comprehension, or a generator expression, compiles to a counted loop. The element count is computed once, before the first iteration, and no step calls the runtime, so it costs no more than the equivalent while loop. This holds for a range(...) call written directly as the source and for a range value stored in a variable, field, or generator frame or returned by a call. The loop copies the range when it starts: changing a bound variable, or assigning a new range to the variable, inside the body does not change the count. A generator expression’s outer source is stored in its frame, so each element also costs one frame resume.

IterableYielded value
ListElement
DictionaryKey, in insertion order
SetElement, with no promised order
RangeInteger of the range’s element type
StringUnicode scalar value as str
bytesByte as u8
std.bytes.BytesByte as u8
GeneratorDeclared yielded value

Consuming list iteration transfers affine elements. for item in &list preserves the list and always yields &T, including scalars and strings. for item in &mut list yields &mut T, including scalar elements. Existing collection references follow the same access mode. This includes direct reference-returning calls: for item in owner.column() and iter(owner.column()) need no intermediate reference binding. The call runs once when the source is prepared, and its returned loan protects the owner through the loop, including backedges, against invalidating mutation, movement, or destruction. Element loans follow the borrowing rules: only distinct literal list indices are disjoint. Iteration references cannot escape into stored references.

Borrowed temporary iterable owners remain alive through the loop and drop when its scope exits, including on early exit.

Borrowed dictionary and set iteration yields shared references to keys or elements. Borrowed range, string, and bytes iteration also yields shared references; each refers to a computed value in an iteration-local slot and cannot escape that iteration. A Bytes buffer has addressable storage: for b in &buffer and for b in &mut buffer loop over a slice view of the whole buffer and yield &u8 or &mut u8 into it. Comparisons and printing observe references uniformly; dereference explicitly when passing an element to a function that takes a value or storing it in a new collection. Other arithmetic observes numeric references as well. Borrowed generators cannot be loop sources; advance one with next(&mut g) instead. Use for item in copy(values)? for an explicit independent snapshot when the original must be mutated during iteration.

for key, value in dictionary.items(): borrows a named dictionary or field and preserves insertion order. Keys are &K and values are &V, regardless of whether their types are copyable. (&mut dictionary).items() or &mut dictionary.items() instead yields &mut V values with shared &K keys. These loops allocate no item tuples or snapshot buffers. items() is a loop/comprehension intrinsic, not a storable iterator, and requires two distinct binding names (or _). Its source loan prohibits invalidating dictionary mutation.

dictionary.keys() creates a bindable local KeyView[K]; iteration yields &K in the same insertion order, without allocation. dictionary.values() is a loop/comprehension source yielding &V, including affine values; (&mut dictionary).values() yields &mut V; see updating every value. Both protect the whole dictionary under their loans. Collect explicitly ([*key for key in d.keys()]?), or use keys_snapshot() / values_snapshot() for owned lists and into_keys() / into_values() to consume the owner. See dictionary views for the key view’s set operations and storage restrictions.

Loop targets permit recursive tuple unpacking, such as for (x, y), label in rows. Comprehensions and generator expressions accept the same targets. Each name is a new immutable binding local to that iteration scope, shadowing any outer local without changing it. A clause evaluates its source before introducing its own bindings; filters and subsequent sources see the new bindings. Enclosing tuple parentheses do not change the shape: for ((x, y), label) in rows is equivalent, including over borrowed zip and dictionary item sources. Over borrowed elements, each name borrows its component in place: for n, s in &pairs binds n as &item[0] and s as &item[1], and for n, _ in &mut pairs makes n a &mut reference, so *n += 1 changes the list. The names borrow distinct fields of the element, so swap(n, s) under for n, s in &mut pairs passes both as &mut arguments, as swap(&mut p[0], &mut p[1]) does for an element p of a &mut loop. Borrowing the list or another element while they are live conflicts. Iteration itself, including string scalar production and generator next, adds no allocation; iterable creation and body expressions keep their own contracts. There is no arbitrary user iterator protocol, special-method dispatch, or async iteration. Generator expressions construct lazy owned frames and follow the generator iteration rules.

Comprehensions and collection construction

List, set, and dictionary comprehensions return Result[collection, AllocError]. Multiple for and if clauses nest from left to right, each one a level of nesting. Each iterable evaluates when its enclosing iteration reaches it; filters run before the result expression. Dictionary keys evaluate before their values. Iteration uses the same consuming or borrowed contracts as loops.

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    values = [n * n for n in range(5) if n > 1]?
    print(values)?
    Ok(())
[4, 9, 16]

A comprehension with one for clause and any number of if clauses lowers to the same loop as the equivalent pipeline: [f(x) for x in s if p(x)] runs as iter(s).filter(p).map(f).collect() would, with no hidden function. Its iterable is evaluated before the output is allocated. Over a known-length range or list without filters, the output reserves its full length once and never grows. Comprehensions with several for clauses keep the nested loop.

Comprehension bindings have their own scope; the first iterable sees the enclosing scope. Flat tuple targets are supported. Conditional expressions in iterables or filters must be parenthesized. Parenthesized generator expressions produce lazy owned frames instead of collection Results. Async comprehensions are unsupported.

Output ownership and growth are fallible. On growth failure, the partial output, current yielded owners, and active hidden iterators clean up. No iterator advances and no later filter or element expression runs. Nested expressions keep their own Result types and failure behavior. A nested ? propagates from the enclosing function rather than becoming an error inside the surrounding comprehension Result. Already performed side effects are not rolled back.

Collection displays [elements], {elements}, and {key: value} follow the same recoverable output-allocation policy. Their owner allocates before entries evaluate left to right; allocation failure prevents later entry evaluation. Expected Result types supply element types for empty displays. See collection construction for literal types and duplicate-entry behavior.

list[T].from(source) and set[T].from(source) consume an owned collection, range, or generator and return Result with AllocError. Failure destroys partial output, the current element, and remaining source. Generator effects before failure persist. Sets require hashable elements and keep the first occurrence of each value. Borrowed sources and strings are unsupported by these .from(...) methods. The call-style constructors additionally accept strings and bytes; see collection conversion.

A directly written range in a comprehension can receive numeric element context from the output annotation through direct arithmetic, for example squares: list[u8] = [n * n for n in range(8)]?. This inference does not run backward through arbitrary calls, filters, stored ranges, or explicit range type arguments. Use range[u8](...) at those boundaries. Fixed-width arithmetic follows its normal overflow rules.

Generators

A function containing yield declares Generator[T]. Ordinary call syntax producer(arguments) constructs its owned frame; .new(arguments) is an equivalent alias. Arguments evaluate first and copy or move into the frame. Construction runs no body code and requires no heap allocation. Argument expressions keep their own allocation and error contracts.

def numbers(stop: i64) -> Generator[i64]:
    mut value = 0
    while value < stop:
        yield value
        # value starts at zero and value < stop <= i64.MAX.
        unchecked:
            value = value + 1

def main() -> Result[(), IoError]:
    for value in numbers(3):
        print(value)?
    Ok(())
0
1
2

Resumption executes until a statement-only yield value, bare return, or fallthrough. Yield types are exact and owned payloads transfer to the caller. Reference, generator, closure, and unit yields are rejected; a generator body cannot return a value. An ordinary factory without yield may return another frame by moving it. ? is unsupported in a generator body; match fallible results explicitly. Recoverable arithmetic failures cannot propagate from a generator: handle a local checked: result, call a helper that returns a Result and yield or match it, or use a small unchecked: suite with a documented bound.

next(&mut g) -> Option[T] borrows a mutable generator owner or reborrows an exclusive generator reference. next(g) is shorthand for named bindings and exclusive references. Shared references, immutable owners, and mutable temporaries cannot be advanced. The borrow lasts only through the call; exhaustion is stable and produces Nothing on every later call. Resumption and its Option wrapper add no allocation beyond body operations.

Loops, comprehensions, and iterable collection constructors consume frames. break destroys a loop’s hidden frame without executing subsequent generator statements. Assignment, arguments, and returns transfer ownership. Standard Option/Result wrappers may own frames, including nested wrappers, and remain affine. Frames cannot enter collections, class fields, tuples, user enum payloads, or heap-stored closures. Printing, comparison, length, and membership are undefined for frames.

Each producer and concrete specialization has a distinct frame identity. Generator[T] describes the yield contract without erasing that identity. Local annotations preserve the initializer’s producer, and parameters, references, aliases, and standard sum wrappers specialize to actual frames. A factory must resolve to one producer per specialization; equal yield types do not allow different producers to join. Nothing or Err alone cannot infer an absent producer. No hidden boxing makes frames interchangeable.

A suspended frame retains its live owned locals. Completion releases them; dropping a suspended or never-started frame releases remaining locals and captures without resuming the body or allocating. Short local borrows completed within one resume are permitted, but generator parameters cannot be references and no live loan may cross yield. yield inside with is unsupported. There is no yield-from, send/throw, or async/await.

Fluent transformations

Generators expose the pipeline adapters and terminals: numbers(100).filter(even).map(square).fold(0, add) consumes the generator through a fused adapter loop. Source resumption retains its ordinary cost; the fluent adapters do not each construct another suspended frame.

An owned pipeline can become a returnable resumable stream through explicit .generator(). It transfers already prepared source and capture state into one frame without advancing any item. Borrowed sources or callback captures are ineligible, including when mapped items are owned. There is no implicit conversion at a Generator[T] parameter or return annotation. The resulting frame follows the specialization, suspension and cleanup rules above.

Generator expressions

(expression for name in source if condition) creates a concrete owned Generator[T], with its yielded type inferred from expression. Multiple for and if clauses have the same scopes and nesting limits as comprehensions. The outer source evaluates once at creation; inner sources, filters, and the element expression run only as the frame advances. A range outer source is stored in the frame and advances as a counted loop. A sole call argument can omit parentheses: list(n for n in range(4)). Arithmetic in an element or filter follows the same generator restriction; a Result-returning helper may produce Result items. With other arguments, parenthesize the generator expression.

Inputs and referenced enclosing local values copy or move into the frame at creation. Copyable captures are snapshots; later assignments do not change them. Affine captures become unavailable to the caller. Captures are taken in order of first use in the remaining clauses, then the yielded expression, after the outer source. References cannot be captured, including a borrowed outer source; use an owned source or copy(source)?. Adapter sources enumerate, zip, and reversed evaluate their inputs at creation without allocating an adapter.

Construction and advancement add no heap allocation beyond the expressions they run. The same yield restrictions, propagation restrictions, destruction, and concrete-frame specialization rules apply as for generator functions. Dropping an unstarted frame releases its captures without evaluating filters or elements; short-circuit consumers destroy the remaining frame immediately.

Recursive inline layouts are rejected. Nested frame layouts are limited to 64 levels, native frame offsets to signed 32-bit sizes, a function’s native stack frame (which holds the inline frames it owns) to Cranelift’s 1 GiB, a compile-time error naming the function, and concrete consumer specializations to 16,384 per consumer and 65,536 per compilation, with the dependency depth limit of generic functions guarding expanding recursion (see specialization). These are implementation limits, not heap-backed fallback behavior. See generator implementation.

Enumeration and paired iteration

enumerate(iterable[, start: i64]) and zip(left, right) are intrinsic sources for for loops and comprehensions, and can start local pipelines. Pipelines also have .enumerate(start=0) and .zip(right) methods. They compose the existing loop plans and allocate no iterator object, snapshot, or item buffer. Inputs evaluate once in source order; both zip inputs evaluate even if one is empty. Owned items use inline tuples, allowing one tuple binding or two unpacked bindings. Nested adapters work with owned items.

Enumeration yields (i64, element), starting at zero by default. Its counter increments with checked arithmetic only after another element is available. A last index of i64.MAX is valid; a further item traps at the adapter’s source location. continue advances the source and keeps the count aligned. Enumerating a forward list or view, as enumerate(xs), enumerate(&xs), enumerate(v) or iter(&xs).enumerate(), without a start counts with the loop’s own row position. That count cannot overflow and needs no separate counter. Indexing the same &list[T] parameter or view with it then needs no bounds check.

Zip yields (left element, right element) and stops when either input ends. It tests the left condition before the right, including generator advancement. When the right generator ends first, the left can have one unmatched pending item; cleanup drops it. Generator bodies therefore may have one extra left side effect. break, return, and propagation drop all remaining owned inputs.

Each input follows the ordinary iteration ownership rules. Borrowed elements retain their reference types and source loans. Such adapters require two unpacked bindings and bind the references directly, without constructing a tuple containing references. Aliasing exclusive inputs and invalidating mutation remain errors. An inner borrowed adapter cannot be used as an outer adapter’s tuple-valued input. dictionary.items() yields two references; it can start a pipeline but cannot become a nested tuple containing references.

The calls do not construct first-class iterators, support arbitrary iterator protocols, or implement zip strictness. Pipeline enumeration also accepts the start keyword. Reverse traversal can supply either input. A user function or callable binding named enumerate or zip retains its normal call behavior.

The free forms and methods yield the same items: enumerate(&names, 1) and iter(&names).enumerate(1) are interchangeable as a loop source or pipeline start. Use the method form to enumerate or zip partway through a chain, after a filter or skip, where it counts or pairs only the items that reach it. reversed is not a pipeline adapter: reverse the source instead, as in iter(reversed(&values)) or zip(reversed(&values), &values).

Reverse traversal

reversed(iterable) is a source for loops and comprehensions over a list, string, bytes, std.bytes.Bytes, or range. It evaluates the source once and visits its elements from last to first, without changing the source order or allocating a reversed copy. Text is traversed by Unicode scalar value, with linear total byte traversal; combining marks remain separate scalars. Ranges retain their element type and original step, including descending and unsigned ranges.

The ordinary iteration ownership rules apply. An owned list transfers its elements and drops its unvisited elements on early exit. reversed(&values) yields shared element references; reversed(&mut values) yields exclusive references for a list. A borrowed temporary source stays alive until the loop scope exits. continue advances backward once, and break, return, and propagation clean up the remaining source and current item. Flat tuple unpacking and comprehensions work as in forward iteration.

Use reversed as an input to enumerate or zip, including borrowed inputs, or as a pipeline source: iter(reversed(&values)). Pipelines have no reversed method. Its own input must be a concrete list, string, byte literal, byte buffer, or range; it cannot reverse enumerate, zip, a generator, or a hash collection. It does not create a stored iterator or dispatch to a user iterator protocol.

There are no free any and all functions. Test boolean items with the pipeline terminals iter(flags).any() and iter(flags).all(), or pass a predicate: iter(values).any(is_negative). reversed may supply their input: iter(reversed(&flags)).all(). A call to any(...) or all(...) that names no user function is an error naming these replacements.

Functions and callable bindings named reversed, any, or all retain their normal call behavior. Imported generic functions and generator bodies can use reversed.

Local iterator pipelines

iter(source) starts a lazy, single-use pipeline. It needs no import. Source expressions and callback captures are evaluated immediately, from left to right; items are visited only when a loop, comprehension, or terminal consumes it. The compiler fuses the adapters into the source’s ordinary iteration loop. Generators expose the same adapters and terminals directly: numbers(100).filter(even).map(square).collect() is equivalent to starting with iter(numbers(100)). Each adapter extends the fused plan.

p = iter(&payments).filter(def(p: &Payment) -> bool: p.name == "bob")
total = p.map(def(p: &Payment) -> i64: p.cents).fold(0, add)

Use a plain loop when the body has several statements and a comprehension when building a collection is clearest. Pipelines are useful for borrowed transformations and reductions that would otherwise need nested generator adapters or an intermediate collection.

Sources and lifetime

Sources follow the ordinary iteration rules: owned collections and generators move into the pipeline; borrowed collections retain their loan. Integer ranges use counted loops. Text, bytes and collection views keep their ordinary item types. A slice view keeps its owner’s loan and yields &T, or &mut T for an exclusive view, as in iter(&xs[1:4]).filter(even).count().

A source may be a reference-returning call, such as iter(owner.column()). The call runs once during construction; its owner stays borrowed until the pipeline is consumed or dropped. An intermediate reference binding is optional.

A local may have an Iter[T] annotation. Its concrete source and callback identity remain part of its type. Consuming it, adapting it, or drop(p) moves it; a second use is rejected. Dropping it releases the owned source and captures without advancing a generator or calling a callback. Early exit from an owning consumer releases the unconsumed source and captures.

Pipelines cannot be returned, passed by value, shared, captured, or put in stored data. In generator functions, a pipeline must be consumed by a terminal in the expression that creates it, such as yield iter(&flags).any(); a pipeline local or a loop over a pipeline is rejected. A pipeline can be lent to a &mut Iter[T] parameter, and an eligible owned pipeline can leave its scope after explicit .generator() conversion.

Advancing one item

next(p) takes one item from a pipeline local in its creating scope and returns Option[T]. The binding must be mut; next(&mut p) is equivalent. The pipeline keeps its progress, so a later next, loop, adapter or terminal continues after the items already taken:

mut rows = iter(lines).filter(def(l: &str) -> bool: l != "")
header = next(rows)
records: list[Record] = rows.map(parse_row).try_collect()?

next expands the fused step inline: there is no frame and no dispatch. It returns Nothing once the pipeline is exhausted, and every later next, loop or terminal sees an empty pipeline: take_while stays stopped after its first rejected item, and zip stays stopped after either side ends. Items must be storable in an Option, so a pipeline of references is rejected; map to an owned value first. Loans held by the pipeline stay live until it is consumed or dropped, so mutating a borrowed source between two next calls is a borrow error. next on a consumed pipeline is a use of a moved binding.

Lending a pipeline

A function can take a pipeline as &mut Iter[T], consume part of it, and return. The lender then continues after the items the borrower took:

def skip_header(rows: &mut Iter[str]) -> Result[(), CsvError]:
    match next(rows):
        case Some(header):
            check_header(&header)
        case Nothing:
            Err(CsvError.Empty)

def main() -> Result[(), CsvError | AllocError | IoError]:
    mut rows = iter(lines).filter(def(l: &str) -> bool: l != "")
    skip_header(&mut rows)?
    records: list[Record] = rows.map(parse_row).try_collect()?
    ...

The lender writes &mut p and the binding must be mut. Inside the borrower, p supports next(p), for x in p, terminals, and adapters that build a sub-pipeline consumed in the borrower. A break, return or ? inside the borrower’s loop leaves the remaining items in the pipeline. The borrower may lend p on, as f(p) or f(&mut p), including recursively. It cannot move, return, store or capture p, and &Iter[T], by-value Iter[T] parameters and Iter results are rejected, as are Iter parameters of exported and foreign functions.

&mut Iter[T] names only the item type. Like a &mut Generator[T] parameter, the borrower is compiled once per lent pipeline, as a frame specialization; recursive and repeated lends of the same pipeline share it. A method can take a lent pipeline the same way.

A generic borrower such as def first[T](p: &mut Iter[T]) -> Option[T] infers type parameters only from the lent pipelines’ item types. A call that needs any other type parameter, even one an ordinary argument would determine, supplies all type arguments explicitly:

def tagged[T, K](p: &mut Iter[T], tag: K) -> Option[(K, T)]:
    ...

tagged(&mut p, "first")             # error: cannot infer type parameter `K`
tagged[i64, str](&mut p, "first")   # Some(("first", 0))

A borrower is checked only through the calls that lend it a pipeline. One that no call lends a pipeline to is not checked, like an unused generic function, so its errors appear when the first lending call is added.

The lender keeps the pipeline’s loans live across the call, so mutating a borrowed source while the pipeline is unfinished stays a borrow error. Lending a pipeline whose binding a consumer already moved is a use of a moved binding.

Costs and limits:

  • The borrower receives one pointer per owned state slot (source cursor, owned captures, counters, chain phase, flat_map inner source, zip and take_while flags) and the value of each reference the pipeline holds, as hidden arguments. It copies owned state into its own locals on entry and writes it back on every exit, including ?, return and an early loop exit. No pipeline state is heap-allocated, and a pipeline that is never lent pays nothing. next(p) in the borrower expands one step inline.
  • References the pipeline holds, such as a borrowed source or a borrowed capture of a closure literal, are read-only in the borrower; the lender’s loans cover them.
  • A pipeline of references, such as iter(&names), can be lent as &mut Iter[&T]. Each item borrows the lender’s source, which stays borrowed for the whole call, so the borrower can use items in loops, adapters and local closures, and lend the pipeline on. An item cannot leave the call: returning it or a reference derived from it is an error, and next(p) on reference items is rejected as in the creating function. A generic borrower cannot take one, since type arguments cannot be references; write &mut Iter[&T] with a concrete T.
  • A pipeline cannot be lent when it holds a reference inside other state, such as a closure with borrowed captures stored in a variable and then passed to an adapter, or a reference into its own state. The diagnostic points at the adapter argument; write the closure literal inside the adapter call instead, as in .filter(def [&limit](x: &i64) -> bool: *x > limit). Its captures then become references the lender’s loans cover.
  • A lent pipeline may need at most 64 state slots.

Returning a resumable stream

.generator() consumes a pipeline and transfers its prepared source, callback environments, counters and other iteration state into one concrete generator frame. It takes no arguments and adds no heap allocation. It does not revisit source expressions, reconstruct callbacks, or advance the source. Adapters remain fused inside the frame; next resumes that frame until its next accepted item or exhaustion.

def squares(stop: i64) -> Generator[i64]:
    iter(range(stop)).map(def(n: i64) -> i64: n * n).generator()

def total(values: Generator[i64]) -> i64:
    values.fold(0, def(acc: i64, n: i64) -> i64: acc + n)

The result can be returned, passed to a generator-consuming helper, moved, advanced with next, or used as another fluent source. Owned sources and owned captures are required. Borrowed sources and borrowed captures are rejected even when a later map produces owned items; conversion never copies borrowed state. Items must satisfy the ordinary generator yield restrictions. The existing concrete-producer and finite-frame rules apply, including the restriction against joining unrelated generator producer types.

Dropping an unstarted or partly consumed frame releases all its owned state without invoking more callbacks. Exhaustion is stable. A zip frame preserves left-first pulls and drops an unmatched left item when the right input ends. Mutable owned captures keep their state between resumptions.

Conversion is always explicit. An expected Generator[T] type does not convert an Iter[T], and a generator expression over an unconverted pipeline is not an alternative conversion spelling. Prefer local consumption when a resumable stream does not need to cross a function boundary.

Adapters

MethodBehavior
map(f)Call (Item) -> U once per item; emit the returned value
filter(f)Retain items for which the shared-reference predicate is true
take(n)Emit at most n items; zero or negative n advances nothing
skip(n)Discard the first n items; zero or negative n skips nothing
take_while(f)Stop at the first false shared-reference predicate, dropping that item; stay stopped
filter_value(value=True)Retain items equal to value, compared with ==; no callback
enumerate(start=0)Emit the checked i64 index and item; count accepted items at this point in the chain
zip(right)Pull one accepted item from each input, left first; stop when either ends and stay stopped
chain(other)Emit every item of this pipeline, then every item of other
flat_map(f)Call (Item) -> source once per item; emit each item of the returned source

Predicates receive &T for owned T, borrowed &T, and mutable &mut T items. Mapping receives the item itself. Callback parameters and results require explicit types. In serial pipelines, every reusable callable is accepted by value: a named function, an anonymous function, a capturing closure with owned or borrowed captures, a stored Callable value or parameter, a BoxedCallable, or a class or payload-variant constructor. Generic constructors need explicit instances here, for example map(Option[i64].Some), because a pipeline does not supply an expected callback result type. Parallel pipelines accept a narrower set of callback forms. One-shot closures are rejected because the pipeline calls the callback once per item, and a reference such as &f is rejected: pass the function or closure itself. map cannot currently return references.

A closure literal written directly as a callback is lowered into the consumer’s loop instead of being called. It behaves as the call would: captures are evaluated once, when the chain is built; borrowed captures stay borrowed until the pipeline is consumed or dropped; owned captures are dropped with the pipeline; and a mut capture keeps its state across items. A closure stored in a local, and a literal whose body is more than one expression or contains ?, match, or another closure literal, is called as before. This optimization preserves each callback’s declared arithmetic contract. Callbacks never inherit the consumer’s checked or unchecked mode. Arithmetic callbacks keep their function boundary; use try_fold for a callback returning Result[A, E]. An infallible callback must handle arithmetic locally or contain an explicit, justified unchecked: suite.

Integer sum() returns Result[T, OverflowError[T]], including inside unchecked:, and stops at the first overflow. Floating sum() returns an ordinary float using sequential IEEE addition, including infinity on overflow.

filter_value(value) evaluates value once, at construction, and compares each item with item == value in the loop; the item passes through unchanged. The value takes the item type T as its expected type, so filter_value(2) over u8 items needs no suffix, but it is not otherwise converted; a different type is rejected at the call. For a borrowed &T or &mut T item, value may be a T or a shared &T, and the comparison reads through both references (*item == *value). A borrowed value keeps its loan until the pipeline is consumed or dropped, like a borrowed source; moving or assigning its owner while the pipeline is live is a borrow error. Owned items require a T.

Any type that == accepts is allowed, with the same semantics. In particular, floats use IEEE equality: a NaN value keeps nothing and -0.0 keeps 0.0. The argument defaults to True, so filter_value() keeps true bool items and is a type error for any other item type. Pair items are rejected; use filter with a two-parameter predicate. An owned value is dropped with the pipeline, and a .generator() conversion moves it into the frame.

Free enumerate(source[, start]) and zip(left, right) also start pipelines. dictionary.items() can be used directly in a chain or through iter. A pair item spreads into two callback parameters: map and for_each receive the components themselves; predicates receive shared references to both. fold and try_fold take (accumulator, left, right), and reduce takes (accumulator_pair, left, right). The accumulator is always one value, even when it is a tuple. Only the item’s top level spreads. Owned nested pairs are ordinary tuples; a nested pair containing references cannot be formed.

Zip pulls through each arm’s filters and skips independently. If the right arm ends, a left item may already have been consumed; that unmatched item is dropped. Ending a consumer drops both arms and their pending items. Enumeration permits a final index of i64.MAX, trapping only if another item is available. There is a 64-adapter/type-depth limit, alongside expression nesting limits.

chain(other) accepts any pipeline source, including another pipeline. Both sides must yield exactly the same item type, pair shape included. The second source is evaluated at construction but not advanced until the first ends. A take_while that stops the first side moves on to the second; it does not end the chain. An item borrowed from either side keeps both sources borrowed.

flat_map(f) calls f for an outer item only when the previous inner source has ended. One rule applies: the callback returns any owned iterable (a list, set, dictionary, str, bytes, range, or generator), and flat_map emits its items as a for loop over that value would. A reference, a Result, a non-iterable value, or a nested Iter is rejected; borrowed outer items are allowed, but the returned source may not borrow them. Each inner source is released as soon as it ends, and an unfinished one is released when the consumer exits or the pipeline is dropped. Inner items are owned, so flat_map adds no loan.

Terminals

MethodResult and behavior
fold(initial, f)Left fold with callback (A, Item) -> A
reduce(f)Left fold without an initial value; Option[Item], empty is Nothing
count()Checked i64 item count
sum()Integer Result[T, OverflowError[T]] (Ok(0) when empty); floating T (0.0 when empty)
min() / max()Least / greatest numeric or str item in Option[Item]; empty is Nothing; ties keep the first
any(f) / all(f)Short-circuit shared-reference predicates; empty is false / true
any() / all()Short-circuit over bool or &bool items; empty is false / true
find(f)First matching item in Option[Item], then release the rest
for_each(f)Call (Item) -> () for each item
try_fold(initial, f)Left fold with (A, Item) -> Result[A, E], stopping at the callback’s first error
try_for_each(f)Call (Item) -> Result[(), E], stopping at the callback’s first error
collect()Build an expected list[T], set[T], or dict[K, V]; return Result[Collection, AllocError]
try_collect()Collect Result[T, E] items, returning `Result[Collection, E
generator()Transfer an eligible owned plan into a lazy Generator[Item]

any() and all() test each item itself, as filter_value() does; other item types need a predicate. There are no free any and all functions.

find and reduce copy a borrowed &T or &mut T item when T is copyable, such as a number or str, as sum, min, and max do: iter(&values).find(f) returns Option[i64] for a list[i64], and a reduce callback takes (i64, i64) -> i64. Other borrowed items cannot be stored in an Option; map them to an owned value first.

sum, min, and max read a borrowed &T or &mut T item as its value (*item), so iter(&values).max() returns Option[i64] for a list[i64] and copies the selected str. They take no arguments and add no allocation. sum accepts numeric items; min and max accept numeric and str items. Pairs, tuples, and other types are rejected: map them to a supported item first, or use fold or reduce. Float min and max follow IEEE 754-2019 minimum and maximum: -0.0 orders below 0.0, and any NaN item makes the result Some(NaN). The NaN returned is the first NaN item. They still consume every item. std.math.min_all and max_all share this rule and give the same result for the same items. A float sum adds in source order with IEEE 754 addition, with no compensated summation: any NaN item makes the sum NaN, and an empty float sum is 0.0. Integer sums return the first overflow’s wrapped payload without allocation. Handle their ordinary result explicitly; an outer checked: suite does not catch errors returned by a terminal.

Collecting needs an expected collection type, from a binding annotation, argument, or return type. There is no default collection:

values: list[i64] = iter(range(10)).map(square).collect()?
result: Result[list[i64], AllocError] = iter(range(10)).map(square).collect()

Borrowed items must be mapped to owned values before collection. Dictionary collection consumes owned (K, V) pairs; the last value for a duplicate key wins. try_collect into a dictionary needs Result[(K, V), E] items. A failed collection drops its partial output and the remaining owned source. try_collect stops at the first item error. try_fold and try_for_each instead stop on callback errors: their callbacks must explicitly handle any Result items. They add no allocation error of their own.

Parallel pipelines

.par(&pool) runs the rest of a pipeline on a ThreadPoolExecutor’s workers. The pool is borrowed: write &pool, or pass a &ThreadPoolExecutor value.

with ThreadPoolExecutor(8, 64)? as pool:
    lengths = iter(paths).par(&pool).map(file_size).collect()?
    best = iter(range(1, n)).par(&pool, chunks=16).map(score).reduce(better)?
PartAccepted
Sourcean owned list[T], a borrowed &list[T], or a range[i64], through iter(source), iter(source).enumerate(start) or enumerate(source, start)
Steps after parmap, filter, and filter_value
Callbacksnamed functions, or reusable immutable closures that own their captures; constructor values (class or variant), stored capture-free closures and Callable values are not yet accepted
Terminalscollect, try_collect, count, any, all, sum, min, max, reduce

enumerate must come before par. Other steps (take, skip, take_while, zip, chain, flat_map) are rejected after par, as are steps before it other than enumerate. Generators, other range types, and lists reached through &mut are rejected. A parallel pipeline must end in a terminal: it cannot be stored, looped over with for, converted to a generator, or advanced with next. fold, try_fold, for_each, and try_for_each are rejected because their callbacks have no associativity guarantee or would run in no fixed order. Mutable, one-shot, and borrowing closures are rejected; callbacks also pass the worker eligibility checks.

Borrowed sources. As in a serial pipeline, iter(&xs).par(&pool) yields &T items, and the list stays usable afterward. Every chunk reads its rows of xs in place. collect, try_collect and reduce cannot store &T items; map them to owned values first. sum, min, max and filter_value read through the reference, as in the serial form.

An owned list is consumed: each chunk moves its items out of the list’s rows, and the list is dropped after every chunk has finished, which drops any items a chunk did not consume (after an early any/all exit or a callback error, or in chunks never submitted). Each item is dropped exactly once.

filter_value(value=True) follows the serial rules: value is evaluated once, on the calling thread, takes the item type, and is compared with ==. Every chunk compares against that one value, so the items (or, for &T items, the referenced values) must be copyable (scalars, str, and tuples of them); use filter for other item types. The value must also pass the worker checks.

Chunks. The source is split into contiguous chunks, and each chunk runs the fused steps as one pool job. By default the chunk count is the pool’s worker count, read when the pipeline runs. .par(&pool, chunks=N) sets it explicitly; N has type Positive[i64]. A positive literal converts at compile time; zero or negative literals are compile errors. Convert a computed count once with Positive.try_from(count)?, which returns RangeError.OutOfRange for zero or negative values. An input with fewer items than the count gets one chunk per item, so no chunk is empty. Chunk boundaries depend only on the input length and the count, never on scheduling.

Terminals.

TerminalResult
collect()Result[list[U], PoolMapError], in input order
try_collect() on Result[U, E] itemsResult[list[U], ParallelError[E | AllocError]]
count()Result[i64, PoolMapError]
any(keep), all(keep); any(), all() on bool itemsResult[bool, PoolMapError]
sum() on integer itemsResult[T, PoolMapError | OverflowError[T]]
sum() on float itemsResult[T, PoolMapError]
min(), max()Result[Option[T], PoolMapError]
reduce(f)Result[Option[T], PoolMapError] for (T, T) -> T; Result[Option[T], PoolMapError | E] for (T, T) -> Result[T, E]

try_collect reports the error at the lowest input index, across chunks, as map_result does. Chunk allocation failure inside a worker is reported as ParallelError.Worker(AllocError). As in the serial form, any() and all() without a predicate test bool items. any and all evaluate every chunk; there is no early stop across chunks.

Integer sum retains each chunk’s mapped numeric values, waits for the jobs to finish, then adds the values serially in input order. It returns the first OverflowError[T], with the exact wrapped result of the failing addition, whenever the serial left-to-right running total would leave the type’s range. Later cancellation does not erase an overflow; an incoming negative total can rescue a prefix that would overflow if its chunk started at zero. This ordinary Result contract is independent of the caller’s default, checked: or unchecked: mode. Map and filter callbacks are not replayed to find the error; on a successful mapping pass each runs exactly once per item reaching it. Allocation or submission failure can prevent some items from being visited.

Floating-point sum adds each chunk serially, then adds the chunk totals in input order, preserving IEEE behavior (including infinity and NaN). It can differ from the serial sum, since chunk totals regroup the additions; it is deterministic for a given input length and chunk count. With the default count, a different pool size can change the result; pass chunks=N when the same answer is needed from any pool.

min and max take each chunk’s serial result and combine them in input order, so they equal the serial results: the first NaN in input order wins, and -0.0 is less than 0.0. Empty input gives Nothing.

reduce(f) takes a named function (T, T) -> T or (T, T) -> Result[T, E]. The latter returns Result[Option[T], PoolMapError | E], preserving E without wrapping it in ParallelError. Like float sum, it reduces each chunk serially, left to right, on its worker, then reduces the chunk results in input order, skipping chunks with no items; empty input gives Nothing. No list of all items is built. The result is deterministic for a given input length and chunk count, but it can differ from the serial left fold when f is not associative, and with the default count a different pool size can change it. Code that needs one grouping for any pool or chunk count can collect the items and use pool.reduce_tree, whose fixed adjacent-pair tree does not depend on the worker count. Neither reduction uses integer sum’s serial-prefix grouping.

For a fallible reducer, each chunk stops at its first callback error. All accepted chunk jobs drain before inspecting their Results. The first failed chunk in input order wins, before any cross-chunk combining calls. If every chunk succeeds, chunk results combine in input order until the first error. Infrastructure failure from the mapping pass returns its existing PoolMapError before inspecting chunk Results. Sources, captures, unused values and errors are released exactly once, and borrowed sources can be reused after return. Empty input and singleton chunks invoke no local combining callback; a single surviving item invokes no callback at all. Inline closures and indirect Callable reducers remain unsupported; a named generic function is accepted.

Errors and order. Every terminal reports PoolMapError.Shutdown for a closed pool and PoolMapError.Allocation when chunk storage or job submission cannot allocate. Callbacks may run in any order and on any worker. Every submitted chunk finishes before the terminal returns, on success, on a callback error and after a failed submission; this is what lets chunks read the source’s rows without owning them. Integer sum overflow is checked only after that mapping pass has drained. Partial outputs and captures are dropped once.

Enumeration index overflow remains a deliberate integer overflow process trap, as in serial enumerate, even in a checked: caller. It is not a recoverable integer-sum overflow: an arbitrary enumeration start can overflow while splitting chunks or advancing their indices.

Arguments are evaluated in written order, as in a serial pipeline: the source, the enumerate start, the pool and chunks arguments of par, then each later step’s value or callback captures and the terminal’s arguments, left to right. The pool stays borrowed until the terminal returns, so a later argument cannot move it.

Cost. Splitting a list or range allocates one small list of chunk bounds without copying the source’s rows. Each chunk is one pool job, and its worker reads the items from the source’s rows. Collecting terminals allocate one output list per chunk, then append each chunk’s storage to one reserved result list. Integer sum also allocates a numeric output list per chunk and retains all surviving mapped values until the ordered addition pass; this uses O(n) temporary numeric storage, not just O(chunks) totals. It does not build a flattened list. The scalar additions themselves allocate nothing, but the parallel terminal is allocating and fallible. Float sum and reduce allocate nothing beyond the chunk lists and the jobs.

Each chunk is still a pool job, so with fewer than a few thousand cheap items the job overhead exceeds the work. See the parallel pipeline measurements.

Cost

A local (non-parallel) pipeline itself adds no heap allocation or per-adapter generator frame. Source expressions, callbacks, and destructors retain their own costs. A generator source still pays its normal resume cost. Collecting reserves once for a known-length range or list pipeline containing only map, take, skip, enumerate, or chain of two known-length sides; a filter, filter_value, zip, flat_map, or unknown-length source grows as it goes. A terminal over a chain runs one loop per side, so it tests no flag per item; when zip or flat_map follows the chain, and in for loops, collect, comprehensions, and generator conversion, one loop tests a flag per item. flat_map adds the callback and the inner source’s own setup per outer item: a list-returning callback allocates one list per outer item, while a range-returning callback allocates nothing. filter_value adds one == comparison per item and no call; for scalar items the optimized loop matches a hand-written if *item == value loop. Closure literals in a chain add no call. Optimized native code also substitutes small, straight-line scalar leaf functions and stored closures at their call sites (at most 24 meaningful operations). Immutable scalar captures can stay in registers. Arithmetic checks and source locations are preserved. Larger named callbacks and stored closures retain direct calls; fusion alone does not guarantee inline-loop performance for every body. The reproducible benchmark covers named callbacks, closure literals, scalar captures and a branching callback.

Explicit conversion adds one suspension boundary for the whole pipeline, rather than one frame per adapter. A generator source inside that frame still has its own suspension boundary. Local fluent consumption keeps its fused loop without relying on generator-frame fusion as an optimization.

The proposal describes the broader design; B75 tracks follow-up work. Pipelines replace the removed std.iter generator functions.

Built-in facilities and interoperability

Operation contracts state what they consume or borrow, whether they allocate or block, and what remains after failure.

These APIs follow the ownership and error/allocation rules. They need no import; modules that do are in the standard library.

Built-in types and operations

These operations are available through the prelude or compiler-known type and method names. Their contracts complement the language rules for types, ownership, and iteration.

An operation’s allocation guarantee excludes constructing its arguments and running user cleanup. Result and Option wrappers add no allocation. Operations that observe inputs leave them available; owned arguments follow ordinary move or copy rules. References are accepted where an operation observes a value, unless its contract says otherwise.

Numbers

The integer types are i8, i16, i32, i64, u8, u16, u32, and u64; the floating-point types are f32 and f64. Literal typing, arithmetic, comparisons, and casts follow the numeric language rules. There is no builtin int type or implicit numeric widening.

Parsing

T.parse(text: str) -> Result[T, ParseError] observes its input and allocates nothing. Parsing trims Unicode White_Space. ParseError.Invalid means malformed input; OutOfRange means a valid number outside the target’s range. Both errors are allocation-free.

Integer parsing accepts an optional ASCII sign and decimal ASCII digits, and checks the requested width. Empty input, prefixes, underscores, and non-ASCII digits are invalid.

Float parsing accepts ASCII decimal/exponent forms and case-insensitive inf, infinity, and nan, with an optional sign. It rounds directly to the requested width. Finite overflow returns OutOfRange; underflow may round to zero. Signed zero survives. Separators and hexadecimal forms are invalid.

Checked conversion

T.try_from(value) -> Result[T, RangeError] is defined for every integer type T and takes one value of any integer type. It returns Ok with the same mathematical value when T can represent it and Err(RangeError.OutOfRange) otherwise. RangeError has just that one variant, so an exhaustive error match needs one arm. Conversion allocates nothing and does not call the runtime. Float and non-numeric arguments are compile-time errors, as is try_from on a float type; float conversions use casts. NonZero.try_from(value) fails with the same error for zero.

Parsing still returns ParseError, which distinguishes malformed text from an out-of-range number. A function that parses and then narrows can return Result[T, ParseError | RangeError] and propagate both operations with ?, preserving each cause. It can instead explicitly match the conversion error and return Err(ParseError.OutOfRange) when that mapping is its intended contract; there is no implicit conversion between these distinct error types.

Wrapping conversion

T.wrapping_from(value) -> T is defined for every integer type T and takes one value of any integer type. It keeps the low bits of the value’s two’s-complement representation, sign-extending a narrower signed value first, so the result is the value modulo 2 to the power of T’s width: u8.wrapping_from(300) is 44, u64.wrapping_from(-1) is 18446744073709551615, and i8.wrapping_from(200u8) is -56. It never fails, allocates nothing, and does not call the runtime. Float and non-numeric arguments are compile-time errors, as is wrapping_from on a float type.

The bare cast T(value) is the form that requires the value to fit.

Limits and constants

T.MIN and T.MAX are the smallest and largest values of an integer type T. Float types provide T.PI, T.E, T.TAU (nearest to π, e, and 2π), T.INF (positive infinity), and T.NAN (a quiet NaN). Each constant has type T, so u8.MAX - small needs no cast. Float types have no MIN or MAX, and integers have no float constants. Names are upper-case; unknown names are compile-time errors listing the supported constants.

Powers

base.pow(exponent) is defined on every numeric type and takes one exponent of the receiver’s type. There is no ** operator.

For integers the result is exact. A result outside the type returns Err(OverflowError[T]) from the enclosing function by default; its declared Result must include that error. Inside a checked: block, it exits that block instead. Inside an unchecked: suite, it traps with integer overflow. A negative exponent always traps with negative integer exponent, since it has no integer result. x.pow(0) is 1 for every x, including zero. Convert to a float for a fractional or negative power.

For floats pow is the platform math library’s pow: (2.0).pow(0.5) is the square root of two, and a negative base with a fractional exponent gives NaN.

An integer literal receiver needs parentheses, as in (2).pow(10), because 2. begins a float literal. A suffixed literal does not: 2u8.pow(7).

Float functions

These methods are defined on f32 and f64 and return the receiver’s type, except div_checked, which returns a Result. None traps or allocates; results outside a function’s domain are NaN or an infinity, following IEEE 754 and the platform math library.

MethodResult
x.sqrt()Square root; NaN for a negative x
x.exp()e to the power x
x.exp2(), x.expm1()2 to the power x; e to the power x minus 1, accurate near zero
x.log()Natural logarithm; -inf for zero, NaN for a negative x
x.log2(), x.log10(), x.log1p()Base-2, base-10, and natural log of 1 + x (accurate near zero)
x.sin(), x.cos(), x.tan()Trigonometric functions of x radians
x.asin(), x.acos(), x.atan(), y.atan2(x)Inverse trigonometry in radians; atan2 preserves quadrant and signed zeros
x.sinh(), x.cosh(), x.tanh()Hyperbolic functions
x.asinh(), x.acosh(), x.atanh()Inverse hyperbolic functions
x.cbrt()Real cube root, including negative inputs
x.floor(), x.ceil(), x.trunc()Whole value toward negative infinity, positive infinity, or zero; still a float
x.round()Nearest whole value, choosing even at halfway values; preserves signed zero
x.fabs(), x.copysign(y)Absolute magnitude; magnitude with the sign bit of y
x.pow(y)x to the power y
x.fmod(y)Remainder of x / y with the sign of x; NaN when y is zero
x.div_checked(y)Result[F, ZeroDivisionError]: Ok(x / y), or Err when y is 0.0 or -0.0; an infinite or NaN y divides normally

An integer receiver is a compile-time error naming the float cast. % remains integer-only; fmod is the float remainder. floor and ceil return floats, and the cast i64(x.floor()) produces the integer.

Each binary method takes one argument of the receiver’s type. Unary methods take no arguments. Generic FloatType functions can use every method. The imported std.math functions wrap these primitives with recoverable domain and overflow errors and checked integer rounding.

Formatting

str.from(number_or_bool) -> Result[str, AllocError] observes one scalar. Integers use decimal; floats use shortest round-trip debug formatting, including a decimal point for whole finite values, -0.0, inf, -inf, and NaN; booleans use True and False. str(value) is not a cast; its diagnostic names str.from.

Formatting uses bounded temporary storage and fallible output construction; small text results require no heap allocation. Aggregate formatting and format specifications are unsupported here; an f-string applies a compile-time spec.

Tuple operations

Tuples provide fixed component access with an integer literal index; negative literals count from the end, so t[-1] is the last component. Construction, indexing, unpacking, and component-loan rules are defined in tuple types.

Supported tuples compare structurally without allocation. copy duplicates copyable components and mutable contents fallibly; formatting displays supported component values. Types containing stored recursive data are excluded from these automatic traversals.

A tuple can store concrete owned callbacks of different types. Reusable calls borrow their indexed component; consuming calls require extraction by unpacking. See stored callable values.

Boxes

This page describes concrete Box[T]. Explicit Box[P](value)? for a protocol creates a protocol object with indirect method dispatch and no payload extraction or copying.

Box(value) -> Result[Box[T], AllocError] moves a value into one heap allocation. Failure drops the input. Boxes are affine. Moving, borrowing, extracting content, and destroying the box add no allocation; explicit copying and operations on its content retain their own costs. Boxes provide a heap boundary for recursive data. A box has one owner; Shared[T] has several.

For insertion into a borrowed structure, allocate a detached node before moving existing contents into it. Otherwise a failed allocation can destroy the old contents as part of its input. The insertion lesson demonstrates preservation without adding a successful-path allocation.

Where a T is required, Box[T] converts to T by moving out the content and freeing the box, without failure. This applies to arguments, returns, annotated bindings, assignments, constructor fields, and collection elements. Borrowed boxes convert to references to their contents. Nested boxes convert through every level.

Without an expected content type, the box remains a box: other = box moves its owner. Operators and conditions do not convert it. *box explicitly extracts the content; &*box and &mut *box borrow it. Matching an owned box consumes it; matching a borrowed box lends its content.

Class fields and methods are reached directly through boxes. Builtin list, dictionary, set, and string methods also reach through nested boxes. Calling a method on a boxed binding, field, or element borrows its content and preserves the box. Mutation requires a mutable owner or exclusive reference.

Read-only methods accept owned temporary boxes, keeping content alive through the expression. Mutation needs a binding, field, or element rather than a temporary receiver. Read-only builtin methods also accept boxed projections from temporary collections or classes: the temporary owner stays alive until the end of the full expression, without copying its content or requiring a named binding:

def main() -> Result[(), AllocError | IoError]:
    print([Box([1, 2]?)?]?[0].get(1))?
    Ok(())
Option[i64].Some(2)

A returned reference to a box still needs a named binding before calling a builtin method directly on it. See returned references for temporary origins and projected receivers.

Automatic copy, equality (including membership), print, and str.repr reject types whose stored values can recur, including through containers or standard sums. Use explicit traversal to inspect such values. Automatic box cleanup allocates nothing and does not recurse on the native stack; user methods and destructor hooks retain ordinary recursion and effect rules.

Shared owners

Shared[T] is one heap value with several owners. Each owner is an affine handle; the value is released when the last handle drops, on whichever thread drops it. Use it to give pool jobs, or any owners that cannot borrow from one scope, the same atomic, mutex, or immutable value, or a value that synchronizes itself, such as an HTTP client and its connection pool. It is a prelude name, like Box.

OperationResultEffect
Shared[T](value), Shared(value)Result[Shared[T], AllocError]Move value into one new allocation
handle.share()Shared[T]Another owner of the same value
handle.field, handle.method(...)as for TUse the content through a shared borrow
handle[i]elementCopy a copyable element; borrow an affine one with &handle[i]
&*handle&TBorrow the content
*handleTCopy copyable content
with handle.lock() as value:binds value: &mut VLock a Shared[Mutex[V]]; see mutexes

Construction makes one allocation, a counted box; failure drops the input and returns AllocError. share() increments the count and allocates nothing. Dropping a handle decrements it; the last drop runs T’s destructor once and frees the allocation. The count is atomic, so handles may be moved into, shared with, and dropped on any worker.

Access

A handle gives only shared access to its content. Every access borrows the content through *handle, and that loan is rooted at the handle’s binding: while a field reference, element reference, or &*handle is live, the handle cannot be moved, dropped, or reassigned. The handle may be a &Shared[T] reference; share() through it makes a new owner.

The content changes only through types that synchronize their own mutation: an Atomic, a Mutex, a channel endpoint, or a cancellation token. &mut *handle, assignment to a field or element, &mut self methods, and mutating collection methods are rejected. *handle copies copyable content; moving affine content out is rejected, since other owners may still use it.

Where a &T is expected, &handle converts to &T as a borrowed Box does: a &T parameter or an annotated r: &T = &handle borrows the content, and the loan is rooted at the handle as &*handle’s is. The same holds where a view is expected: &handle of a Shared[list[T]] is a &[T]. The conversion is shared only, so &mut handle for a &mut T parameter is rejected. There is no conversion from Shared[T] to T.

A temporary handle, such as Shared(5)?.share(), has no binding to root a borrow at and must be named first.

Rules

  • Handles have no equality, formatting, or print, including when stored inside other values. Compare or print values read through the handle.
  • copy() of a value that contains handles, such as a tuple, class, enum payload, list, or dictionary, copies its owned parts and gives each nested handle another owner, as share() would. The content is not copied and the count is shared. Sharing a handle allocates nothing; the copy allocates only for its owned parts. copy(handle) on a handle itself is rejected: write handle.share(). Slices copy their elements the same way.
  • T follows the storage rules of a box: no references, generators, or borrowed environments.
  • T cannot reach a ThreadPoolExecutor or Future. The last owner could drop on one of the pool’s own workers, which would then wait for its own pool.
  • T cannot reach Shared[T] again through a Mutex or a channel endpoint. Their contents change after construction, so a value could come to hold a share of itself and never be released. Immutable sharing, as in a tree or DAG of Shared nodes, is accepted: each node exists before any share of it. Link back to a parent or a sibling with an index into an owning list instead; see recursive data. There is no weak handle. For the same reason a BoxedCallable cannot be stored behind such a mutex or channel: its captures are not visible to the check.

Rules are checked at each construction, with the concrete T.

Threads

A handle crossing a thread boundary is checked like any other value: its content must be worker-eligible. Because the last owner can drop on any holder’s thread, the content’s destructors are checked as if every holder could run them. Mutexes reachable through a handle count for the compiler’s lock checks: a join, result wait, or pool shutdown while holding a lock on a Mutex[V] that a worker can reach through a share is rejected, and so is nesting a lock through a handle inside a lock on another Mutex[V] of the same V, since two handles can name one mutex.

Lowering

A handle is a pointer to the same counted record as a box: a header with an atomic reference count and destructor, the content’s type, and the content slot. share() is a relaxed atomic increment; a release decrements with release ordering, and the last owner acquires before destroying. Borrowing the content is an address computation. See the memory architecture.

Formatting values

str.repr(value) -> Result[str, AllocError] observes a printable value, including references, and produces its representation. Strings are quoted and escaped; aggregates display their structure. Direct Result wrappers can be inspected this way. Formatting uses checked buffer growth and preserves the input on failure.

print(values...) -> Result[(), IoError] observes its arguments, writes a trailing newline, and uses raw top-level strings. Printing an existing string needs no temporary formatting allocation; other values use checked growth. Formatting failure emits no bytes; output errors can leave a written prefix.

Zero arguments print a newline. Multiple arguments evaluate once from left to right before output, with one space between values and raw top-level strings. Their source loans remain live through argument evaluation and printing; later arguments cannot mutate an earlier observed owner. Printing keeps every owner intact and does not build an intermediate tuple or list. Multiple values use a checked formatting buffer, even when all arguments are strings.

Print rejects a direct or borrowed Result in any argument: handle it first, or intentionally format the wrapper with str.repr(result) and handle that formatting Result. Option and printable aggregates containing Results are allowed. Their Results must be handled or explicitly dropped, following the error rules.

Quantities display their coherent number and unit at the top level. Debug output, including in aggregates, uses Quantity[T, D](value). These builtin spellings keep the ordinary print and str.repr error types.

User formatting methods

print, str.repr and f-strings share one formatting path. A class or enum controls its spelling with two methods that write to a std.fmt.Formatter:

  • display writes the text print(x) and f"{x}" produce at the top level;
  • debug writes the text str.repr(x) and f"{x!r}" produce, and the spelling of x as an element of any formatted aggregate.

A type without display uses its debug; a type without debug uses its structure. Strings are raw at the top level and quoted inside aggregates, so print(["a"]) writes ["a"].

from std.fmt import Formatter

class P:
    n: i64

    def display[W](self, out: &mut Formatter[W]) -> Result[(), FormatError]:
        out.format(f"<{self.n}>")

    def debug[W](self, out: &mut Formatter[W]) -> Result[(), FormatError]:
        out.format(f"P#{self.n}")

def main() -> Result[(), AllocError | IoError | FormatError]:
    p = P(3)
    print(p)?
    print(str.repr(p)?)?
    print([P(3)]?)?
    print(f"{p} {p!r}"?)?
    Ok(())
<3>
P#3
[P#3]
<3> P#3

When a value’s type, or any type it holds, has one of these methods, the method can fail, so the operation’s error widens with std.fmt.FormatError: print returns Result[(), IoError | FormatError], str.repr returns Result[str, AllocError | FormatError], and an f-string returns Result[str, AllocError | FormatError]. Other values keep the narrower types: Result[(), IoError], Result[str, AllocError] and Result[str, AllocError]. A method’s error is returned unchanged and nothing is printed. The method runs with no byte budget beyond the operation’s own and the nesting depth of an f-string field (32); a private type’s method runs with its own module’s access, so an aggregate holding another module’s private type still formats.

display and debug are reserved for this form. Formatting a type whose method of either name has another signature is a compile error that names the type, the method and the required signature; rename the method if it serves another purpose. std.log.Logger (a debug that logs) and std.cli.Progress (a display that draws) are such types, so they cannot be printed.

main’s error report and runtime trap messages do not call these methods; they use structure (issue #634).

f-strings build a str from fields, with Python format specs checked at compile time. A field without a spec uses the same spelling as print.

A @redacted class or enum without the method in use formats as its type name followed by (@redacted) in str.repr, print, f-string fields with any conversion or spec, and main’s error report, at any nesting depth. The marker uses the same checked buffer as ordinary formatting.

Automatic formatting rejects stored recursive types even when an individual value is shallow. Print selected scalar fields or write an explicit traversal. A top-level closure, generator or scoped task, and sums that hold one (for example SpawnError[OnceClosure[[], i64]]), cannot be formatted by str.repr. The diagnostic names the unformattable kind and, when it is nested, the containing type: match the value and format its parts, or call a closure and format its result. print renders a closure as an opaque marker. Numeric spelling follows scalar formatting; stream behavior follows standard output.

Text

str is immutable valid UTF-8 representing Unicode scalar values. Embedded U+0000 is ordinary content (\0 is a literal escape). Equality and hashing compare all bytes without normalization; strings have no public terminator. Ordering (<, <=, >, >=) is lexicographic by Unicode scalar value, matching the order used by list[str].sort(). Prefixes precede longer strings; comparison observes both inputs and allocates nothing. len(text) counts scalars, including combining marks individually, rather than bytes or grapheme clusters. text.byte_len() -> i64 is the length of the UTF-8 encoding in bytes: len("héllo") is 5 and "héllo".byte_len() is 6. It observes the receiver, takes constant time, and allocates nothing; use it to enforce a size limit before copying or encoding text. str has no len method, so the two counts cannot be confused at a call site.

s[a:b]? copies the scalars from a to b into a new str; &s[a:b] borrows them in place as a &substr, which loops, compares, and searches without allocating. Both count bounds in scalars; see substring views.

String literals require no allocation. Results of at most seven UTF-8 bytes, including every single character and the empty string, also require no heap allocation. Longer constructed results use fallible storage. This threshold does not change the public type or the result type of an operation.

Escapes

String literals and f-string text accept the escapes \0, \n, \r, \t, \\, \', \", and \u{H...}. The last names one Unicode scalar by code point with 1 to 6 hexadecimal digits of either case, so invisible and combining scalars stay visible in source:

def main() -> Result[(), IoError]:
    print(len("e\u{301}"), "e\u{301}" == "é", "\u{1F600}".byte_len())?
    Ok(())
2 False 4

It is the spelling the !a conversion writes, so !a output pastes back into source as the same text. A surrogate (\u{d800} to \u{dfff}) or a value above \u{10ffff} is not a scalar and is rejected, as are \u without braces, an empty or longer digit run, and every other escape, including Python’s \x, \uXXXX, \U, and \N{...}. A byte literal rejects \u{...}: it holds bytes, so write the UTF-8 encoding with \xHH.

SourceDiagnostic
"\u200d"`\u` needs 1 to 6 hexadecimal digits in braces, as in `\u{200d}`
"\u{d800}"`\u{d800}` is a surrogate code point, not a Unicode scalar value
"\u{110000}"`\u{110000}` is beyond the largest Unicode scalar value, `\u{10ffff}`

Character lookup

text[index] -> str accepts an i64 scalar index, with negative indices counted from the end. A missing index traps. text.get(index) -> Option[str] instead returns Nothing, including for either extreme i64 index.

Both observe the text and index once in source order, accept references, and produce an independent single-scalar string without allocation. ASCII lookup is constant-time; other text requires scanning to the scalar boundary. String iteration likewise yields one-scalar strings without allocation.

ord(text) -> Result[u32, ParseError] converts a one-scalar string to its Unicode code point. Empty text or multiple scalars return ParseError.Invalid. chr(code: u32) -> Result[str, ParseError] performs the reverse conversion; surrogate code points and values above U+10FFFF return ParseError.OutOfRange. Both allocate nothing, including for non-ASCII scalars. Unsuffixed chr arguments infer u32; other integer widths require explicit conversion. Indexing and iteration retain the existing single-scalar str representation; there is no separate char type.

Classification

text.isascii() -> bool is true for empty text and for every ASCII character, including controls and NUL. text.isspace() -> bool requires nonempty text whose characters all have Unicode’s White_Space property. NUL and U+200B are not whitespace. The bundled Unicode tables also govern trimming; Python’s additional whitespace classifications are not promised.

text.isdigit() and text.isalpha() require nonempty text consisting entirely of ASCII digits (0–9) or ASCII letters (A–Z, a–z), respectively. They deliberately do not implement Python’s Unicode digit/letter predicates.

All classification methods observe their receiver, take no arguments, and allocate nothing.

text.lower() and text.upper() return Result[str, AllocError], applying Unicode’s per-scalar case mappings without locale-specific or contextual rules. Mappings may expand a scalar (ß becomes SS in uppercase). They observe the receiver and allocate only the final output, fallibly; short results stay inline. They do not normalize or implement case folding. Other operations are grouped under search, building, and splitting.

Text search

All these methods observe their strings once in source order, accept references, and allocate nothing. Matches are literal UTF-8, with no normalization, case folding, or regex interpretation. Optional bounds are unsupported.

OperationResultBehavior
text.startswith(prefix)boolExact prefix; empty prefix always matches
text.endswith(suffix)boolExact suffix; empty suffix always matches
text.find(needle)Option[i64]First matching scalar position, or Nothing
text.rfind(needle)Option[i64]Last matching scalar position, or Nothing
text.count(needle)i64Number of non-overlapping matches, left to right

Each method takes exactly one string argument. Prefix and suffix checks do not accept tuples of alternatives. An empty needle matches at zero for find, at len(text) for rfind, and at all len(text) + 1 scalar boundaries for count. The last rfind match can overlap an earlier match. Converting a found byte position to a scalar position scans the preceding prefix.

needle in text and needle not in text provide substring membership. A substring view &s[a:b] supports in, startswith, endswith, and find, with a str or &substr argument.

Building text

These methods observe their inputs once, receiver first, and return Result[str, AllocError]. Inputs remain unchanged on success and failure; the result outlives them. References are accepted. Length/layout overflow returns CapacityOverflow; storage exhaustion returns OutOfMemory. Only the final text needs storage, with no intermediate strings; small outputs require no heap allocation. The one exception is join over a generator, described below.

OperationBehavior
text.concat(other) or text + otherConcatenate two strings
separator.join(parts)Insert the separator between consecutive elements of an iterable of str
text.repeat(count)Repeat exact contents; count is i64; zero or negative yields empty
text[start:stop:step]Slice by scalar position, with Python’s clamping, omitted bounds, and steps
text.slice(start, stop)Forward scalar slice with required i64 bounds
text.replace(old, new)Replace all non-overlapping literal matches, left to right
text.removeprefix(prefix)Remove at most one exact prefix
text.removesuffix(suffix)Remove at most one exact suffix
text.strip()Remove Unicode White_Space at both ends
text.lstrip() / text.rstrip()Remove White_Space at the left/right end

Empty input to repeat produces empty output for any count.

Joining

join accepts every iterable whose for element is str, and joins exactly the pieces that loop would yield:

ArgumentPiecesAllocations
list[str]Elements in orderOne, for the result
set[str]Elements, in no promised orderOne, for the result
dict[str, V]Keys in insertion orderOne, for the result
strUnicode scalars, so "-".join("abc") is "a-b-c"One, for the result
Generator[str], including a generator expressionYielded valuesScratch growth plus one for the result

An empty iterable produces empty text; a single piece adds no separator. Empty pieces are retained. Other element types are rejected: write an f-string or convert with str.from to join numbers.

Collections and strings are observed, with or without &. join measures them before allocating, then copies once.

A generator is one-shot, so join consumes it as a for loop does: a named generator moves (sep.join(g)), and a reference to one is rejected. Its total size is unknown in advance. Pieces are appended to a scratch buffer whose capacity starts at the first piece’s size (at least 8 bytes) and at least doubles each time it fills. The result is then copied into exact-size text and the scratch is freed before join returns. Producing n > 8 bytes takes at most ceil(log2(n / 8)) + 2 allocations and holds less than 3n bytes at the peak. For example, four 10-byte pieces with ", " use scratch capacities of 10, 20, 40, and 80 bytes, then one 46-byte result. Each yielded piece is released once it is copied. On failure the generator stops where it is; its remaining state drops with it, and no partial text is returned. To bound the cost, collect the pieces into a list first or use std.text.join with a byte limit.

Choose by the shape of the text:

  • an f-string for a fixed number of pieces of mixed types, such as f"{host}:{port}"?. It reports AllocError, plus FormatError when a field runs a user formatting method or a spec is computed;
  • concat (or +) for two strings in an AllocError function;
  • join for any number of strings from one iterable.

slice includes start and excludes stop. Negative bounds count from the end; bounds clamp to [0, len(text)], without overflow for extreme i64 values. Reversed bounds yield empty text. ASCII bounds map directly to bytes; other text requires scanning scalar boundaries. Slice syntax text[a:b:c]? follows the same scalar positions and also accepts omitted bounds and a NonZero[i64] step; text[::-1]? reverses the scalars.

replace observes receiver, old, and new in that order. Inserted text is not searched again. An empty old string matches every scalar boundary, including both ends; on empty input it inserts new once. Empty new text removes matches. There is no count limit, regex interpretation, or normalization.

Prefix/suffix removal uses exact strings, including NUL, without repeated removal. Empty or missing patterns preserve the contents. Trimming takes no character-set argument, leaves interior text unchanged, and uses the same White_Space classification as isspace.

Splitting text

Both operations observe their text and arguments without consuming them. They return independently owned lines or pieces in Result[list[str], AllocError]. All list storage and long output strings are constructed fallibly. Failure reclaims the initialized result prefix and preserves the inputs.

Explicit separators

text.split(separator) requires one nonempty string, including a reference. A literal "" separator is a compile error, split requires a nonempty separator. A computed empty separator traps; this invalid-argument error is not an AllocError. Maximum-split arguments are unsupported.

Matches are literal, non-overlapping, and scanned left to right. Leading, trailing, and adjacent separators produce empty pieces. Empty input yields [""]; no match yields one piece containing the input’s contents. Multibyte separators and NUL work without normalization.

Whitespace

text.split() with no separator splits on runs of Unicode White_Space, using the same bundled property as isspace() and trimming. It omits leading, trailing, and repeated empty pieces; empty or whitespace-only text yields an empty list. NUL and U+200B are ordinary content. The output remains fallible and independently owned, like explicit-separator splitting.

Lines

text.splitlines() discards terminators. text.splitlines(True) retains each original terminator without translation; the optional argument is bool and positional.

Recognized boundaries are LF, CR, CRLF, vertical tab, form feed, U+001C–U+001E, U+0085, U+2028, and U+2029. Empty text yields no lines; interior blank lines are retained; a final terminator adds no extra empty line. This is broader than file universal-newline reading, which recognizes CR/LF only.

Collections

For primitive-field class records scanned by field, see columnar lists. Their packed columns and borrowed scalar pipelines have a deliberately smaller interface than ordinary lists.

list[T], dict[K, V], and set[T] have concrete element types, including nested collections. Dictionary keys and set members are integers, bool, str, bytes, or tuples, classes, enums and anonymous sums recursively containing those types. Every class field and every payload of every variant must be a supported key type. Hashing includes class fields, variant tags and payload contents, independently of where the value is stored. Ordinary classes move into a collection; @copy classes keep their usual copy behavior. Stored keys are exposed only through shared borrows, so their identity cannot change in place. Floats, collections, boxes and recursive types are not supported keys. Keys with custom __del__ cleanup are supported, including nested cleanup, but cannot be duplicated: copying collections, non-mutating set algebra and key-view to_set() reject these keys. Borrowing and ownership transfers work normally. User-defined hash/equality hooks remain unsupported. See the heterogeneous collections tutorial. Unit elements are rejected. Sets of supported finite enums or tuples with at most 256 possible values use bitsets instead; they do not hash or request entropy.

Dictionary and hash-backed set bucket selection uses SipHash-1-3 over the complete key, the schedule Rust’s HashMap, CPython and Swift use. A str, bytes, integer or bool key is hashed as its contents alone; keys nested in tuples, classes and enums are framed by field count, variant tag and string length. One secret 128-bit key is obtained from OS entropy on the process’s first hash operation and shared across worker threads. Initialization and hashing allocate no runtime storage. Constructing or reserving an empty collection does not need entropy. The first hash may wait for the OS entropy source to initialize. If the OS cannot provide the key, the operation traps with collection hashing requires OS entropy; there is no predictable fallback. Interrupted reads retry.

Randomized bucket placement makes precomputed hash-flooding inputs harder to construct; it does not guarantee worst-case constant-time lookup. Hash values are private implementation details, not persistent identifiers or cryptographic digests. No test-mode fixed seed is needed: dictionary insertion order is independent of buckets. Set iteration order remains unspecified.

Lists preserve order and duplicates. Dictionaries preserve first insertion order: replacing a value keeps its key’s position. Sets remove duplicates and promise no iteration order. Equality is structural: list order matters; dictionary insertion order and set order do not. Structural comparisons allocate nothing. Collections have no identity test.

len(collection) -> i64, in, and not in observe collections. Dictionary membership tests keys. Indexed reads use i64 list indices (including negative indices) or the dictionary’s key type; missing keys and out-of-bounds indices trap with exit status one, reporting the source position. An index is one expression evaluated once; multiple comma-separated indices are rejected. std.bytes.Bytes uses the list index rules for buffer[i] reads and writes.

len(value) also calls an accessible ordinary def len(self) -> i64 method on a class or enum, including standard-library containers such as Deque and Bytes. It borrows the receiver, including a temporary, and evaluates it once. A mutable or owned receiver, extra parameters, method type parameters, or another return type does not meet this contract; call such methods directly instead. The method body determines its cost and whether it allocates. Built-in collections retain their intrinsic length operations. Shared arguments follow the ordinary call borrowing rules: len(&"ada"), len(&b"abc"), and len(&make_collection()) are valid. Borrowing itself does not allocate; producing the argument may allocate.

in and not in do not dispatch to methods on classes or enums, even if they have an ordinary contains method. Use the container’s explicit API; for example, Bytes.find searches a byte range. A type being iterable or indexable does not imply that it supports membership.

Collections move on assignment, owned calls, and returns. Independent duplication uses copy. Indexed reads do not extract affine owners: use an explicit element reference, an owned removal operation, or copying. Element loans protect the whole collection unless they use distinct nonnegative integer literal list indices. See borrowing and iteration.

Flat unpacking also consumes a list: a, b = values requires exactly two elements and transfers their owners; _ drops an ignored element. A length mismatch traps at the unpacking expression before any target is written. Unpacking itself allocates nothing. Borrowed lists, nested targets, and starred targets are unsupported; explicitly copy a list first to preserve it.

replace(collection[index], value) transfers the previous element out while writing its replacement. It requires exclusive access, evaluates the replacement before the target index, and adds no allocation. List indices must exist. Dictionaries reject it, as they reject all indexed writes; use replace(*value, replacement) on a reference from get_mut.

Shared mutation rules

Mutation requires a named mut binding, mutable class field, or exclusive reference. Arguments and indices evaluate before the target is exclusively borrowed, so items.append(len(items)) is valid. Active conflicting loans still reject mutation; temporary mutation receivers are unsupported.

Copying and borrowed lookup

Optional lookups named get, and peek for containers with an order such as caches and heaps, return a copy: dictionary.get(key) -> Option[V] and list.get(index) -> Option[T]. Indexing traps on a missing key or index instead.

A copying get has no hidden cost. It accepts only copyable values: scalars, immutable strings, ranges, function values, instances of @copy classes, and tuples or enums of those. Copying one is a fixed-size copy; a string shares its immutable storage. Neither allocates nor duplicates a buffer. Values that own storage, including every collection and every class instance not declared @copy, are affine, and get rejects them at compile time (“get cannot return owned dictionary values”), so owned data is never copied by accident.

Copying is the default because the common case reads a small value and uses it directly, as in counts.get(key, 0) + 1. A borrowing get would need a dereference at each such use, and its get(key, default) form could not exist: returning either the stored value or the default would choose between two borrows, which returned references do not allow.

Borrowed forms take a suffix: get_ref and peek_ref for shared references, get_mut and peek_mut for exclusive ones. Builtin and standard library containers follow the same rule, so the method name tells the reader whether they hold a copy or a borrow. User functions can return Option[&V]. Of the builtin containers, only dictionaries provide an optional borrowed lookup, get_mut; get_ref is not provided yet.

collection.clear() -> () removes all entries while retaining capacity and allocating no runtime storage. Removed owners drop before return: lists in element order, dictionaries in insertion order with keys before values, sets in unspecified order. Independent immutable owners survive. Empty clear is harmless.

See construction and copying for constructors, growth, reservation, and failures, then lists, dictionaries, and sets for their operations.

Columnar lists

clist[T] stores each field of T in a contiguous column of native-width values. Choose it for repeated scans of a few fields in many rows. Ordinary list[T] remains useful for general objects and whole-record access.

T must be a fully defined, nonempty class containing only bool, signed or unsigned 8/16/32/64-bit integers, f32, or f64. Custom destructors and fallible initializers are rejected. Strings, nested classes, enums, nullable fields, collections and references are not supported. Creating a clist or copying or removing a row requires access to every field; field selection respects module privacy.

Construction and mutation

OperationResult and behavior
clist[T]() or clist[T].new()Empty owner, without allocating
clist[T].with_capacity(n)Result[clist[T], AllocError]
len(items) or items.len()Live row count as i64
items.capacity()Row capacity as i64
items.reserve(additional)Result[(), AllocError]; capacity for len(items) + additional rows
items.append(row)Result[(), AppendError[T]]; moves the row
items.remove(i)Allocation-free Option[T]; takes the row and preserves remaining order
items.clear()(); empties the owner without releasing its capacity
items[i].fieldChecked scalar read; negative indices count from the end
items[i].field = valueChecked scalar replacement
items[i].field += valueNumeric augmented assignment; evaluates the index once
items.copy_row(i)Allocation-free Option[T], independent of subsequent writes
items[a:b:c]Result[clist[T], AllocError]; a slice copying the selected rows; items[:]? duplicates the owner
items.column(T.field)Local, one-shot Iter[FieldType] borrowing the owner

Capacity growth allocates replacement storage, copies each column, and publishes the new storage only after success. Failure preserves length, capacity and all existing values. Negative or overflowing capacity requests return AllocError.CapacityOverflow. Capacity may exceed the requested size.

AppendError.OutOfMemory(row) and AppendError.CapacityOverflow(row) preserve the complete submitted row for retry. Successful append moves its scalar fields into the columns without rerunning an initializer. An append within reserved capacity allocates nothing. The empty owner and all live rows are cleaned up on every ordinary exit; moving a clist transfers its storage.

Field replacement evaluates the RHS before its indices, following ordinary assignment. Enclosing owner indices precede the row index, and each is evaluated once. Augmented assignment evaluates these indices, reads its current value, then evaluates the RHS under a shared owner loan before writing. Out-of-bounds indices trap at the source position; copy_row and remove return Nothing instead.

remove accepts an i64 index; negative indices count from the end. An invalid index leaves the owner unchanged. The returned row is an independent ordinary T, assembled without rerunning its initializer. Removal shifts later rows left in every column. clear makes all columns empty. Both require mutable access, allocate nothing, and retain capacity for later appends. Neither changes the owner pointer or column starts. Removing a row requires access to every field, just like copy_row; clearing does not expose any field.

A slice reserves exactly the selected row count, then copies each column; a step-1 slice copies each column as one block. It reads every field, so it needs the same access as copy_row, and the receiver must be a named clist.

Column iteration and ownership

T.field is a compile-time selector accepted only by column. It is not a string lookup or a general reflection value. Each column plan borrows the entire clist until consumed. While that loan is live, reserve, append, remove, clear, field writes, movement and destruction of the owner are rejected. Several shared column plans can be zipped together. Their lengths and row order agree.

Columns yield scalar values, not &Field references. The source remains borrowed, but each element is loaded directly from its native-width buffer. Ordinary Plenty references address tagged value slots; treating packed bytes as those slots would be invalid. Use scalar callback parameters with existing map, filter, fold, zip and other local pipeline operations. Recreate a column plan for a second traversal. Plans cannot escape into stored values or suspended generators. Column iteration is currently rejected inside generator functions, even if the scan would finish before a yield.

items[i], &items[i], packed field references and direct row-method calls are unsupported: there is no contiguous T at that row position. copy_row creates an ordinary owned T on which methods can run; writes to that copy do not write back. Whole-clist copy, mutable column views and general row iteration are not part of this initial interface. Equality compares row values; formatting reports clist(len=N) without materializing rows.

Layout and cost

The current implementation uses one allocation for an owner header and all columns. Every column contains tightly packed elements (bool uses one byte). Capacity rounding aligns column starts; this is an internal layout, not an Arrow, Parquet, C ABI or serialization contract. Growth copies all columns, and can temporarily hold both old and new allocations.

Ordered removal moves (len - normalized_index - 1) values in each column, so its cost grows with the number of following rows and the total native row width. Removing the last row only gathers that row. Repeated removal near the front can be quadratic; this is not a bulk filtering operation. clear takes constant time because all fields are primitives with no destructors. It does not erase old bytes, shrink storage, or allocate; subsequent appends overwrite the newly live rows. Neither operation uses tombstones.

Column loops load the pointer and length before iteration. Checked arithmetic and floating-point rules are unchanged. Indexed field reads and writes use checked native-width loads and stores. copy_row checks its index once, then gathers the known fields directly into fresh ordinary row storage without runtime schema dispatch. Each access observes the current owner, so growth and mutation between accesses are safe. Whole-row workloads can still cost more than ordinary lists: gathering separated fields and moving the reconstructed object have costs of their own. The access measurements compare these paths with the initial implementation.

Automatic SIMD counts

With the default optimized profile, a column of i32 or u32 followed by one filter and count() uses four-lane SIMD when the predicate compares its borrowed argument against a constant or immutable by-value capture of the same type (==, !=, <, <=, >, >=). Either operand order is recognized. The predicate may also join two such comparisons with and, for example *x >= low and *x < high. Captured bounds may be runtime values; the closure retains its own snapshot. This bounded pattern accepts at most two captures, both of the column’s scalar type. Named functions and closures are recognized. For example: items.column(T.score).filter(def(x: &i32) -> bool: *x > 5i32).count(). The ordinary scalar program remains valid with --fast-compile.

The compiler proves this small pattern from the checked operations; it requires no user annotation, temporary collection, allocation, aligned column start or lane-count bookkeeping. Full groups use unaligned-safe vector loads; the scalar loop handles the remaining zero to three rows. Counts use 64-bit accumulators, including for columns larger than a 32-bit lane counter could represent. The generated instructions use the existing x86-64 SSE2 baseline, so this slice needs neither a newer CPU nor runtime feature dispatch.

Borrowed or mutable captures, or, more than two comparisons, checked arithmetic in predicates, additional adapters, other scalar types and other terminals retain scalar execution. Only total comparisons qualify: eagerly combining their SIMD masks cannot introduce effects or traps from a skipped and operand. Side effects, traps and floating-point order are unchanged. This is a bounded optimization, not general auto-vectorization; parallel scans remain unimplemented. See the validation and measurements.

See the tutorial and the design rationale and measurements.

Collection construction and copying

Constructors and displays

list[T].new(), dict[K, V].new(), and set[T].new() return Result[collection, AllocError]. with_capacity(capacity: i64) has the same result and reserves at least that many entries, including hash storage. Aliases expose these constructors too. Empty construction allocates its owner; negative counts and impossible layouts return CapacityOverflow; exhaustion returns OutOfMemory. Partial storage is reclaimed on failure. Small enum bitsets need only the owner allocation; valid capacity hints require no additional storage because all possible members already fit in the membership mask.

[elements], {key: value}, {elements}, and typed empty constructors such as list[i64]() also return Result[collection, AllocError]. {} is a dictionary. Entries must have identical types. Empty displays require contextual types, such as the annotation in names: list[str] = []?; a typed constructor such as list[str]()? needs none. Nonempty displays infer from their first entry unless context supplies the type. Already typed entries never implicitly narrow.

List displays reserve space for all entries before evaluating any of them; their element buffer does not grow during insertion. Other displays and comprehensions start with an empty owner and grow as needed. Entries evaluate left to right after initial storage succeeds. A failed insertion stops evaluation and destroys the initialized prefix; unevaluated entries have no effects. Nested fallible expressions require their own ?. A display’s error is always AllocError, so ? on it needs a function returning Result[_, AllocError] or an error sum containing AllocError, such as Result[_, AllocError | IoError] when it also prints. Otherwise match the result and convert the error explicitly. Propagation leaves the enclosing function, not merely the display. Comprehension evaluation follows the iteration rules.

A directly iterated nonempty i64 list display with constant integer entries at a constant step, such as for n in [2, 4, 6]?:, is lowered to a counted loop without collection storage. Repeated entries, descending sequences, singletons, .unwrap(), and reversed(...) are supported too, provided the step fits i64. Borrowed sources, other element types, nonuniform sequences, and expressions with effects retain ordinary materialization. This is an allocation optimization: the source still needs valid Result handling, including a compatible enclosing error type for ?. Use range when a sequence is naturally described by bounds and a step.

Reservation and insertion

These exclusive methods return Result[(), AllocError]:

ReceiverOperationBehavior
Any collectionreserve(additional: i64)Room for at least this many entries beyond current length
Listappend(value: T)Append one element
Setadd(value: T)Insert if absent
Dictionaryinsert(key: K, value: V)Insert or replace without changing existing key order

dictionary.get_or_insert(key: K, default: V) follows the same growth and failure rules but returns Result[&mut V, AllocError]: a present key keeps its entry and drops both arguments; see dictionary lookup.

Growth reserves all required storage before mutation. Failure preserves contents, order, and lookup behavior; capacity may change. Negative or impossible counts return CapacityOverflow. A successful reserve permits the specified number of subsequent insertions without storage allocation. Existing-key replacement and duplicate set insertion do not grow storage. An equal incoming key is destroyed; the existing key owner is retained. This can run custom cleanup even when the collection length does not change.

Insertion consumes its arguments on both outcomes; failure drops owned inputs. Reserve before moving an input that must survive a storage failure. The Result remains fallible even when no growth is needed.

Conversion

list[T].from(source) and set[T].from(source) consume an owned collection, range, or generator and return Result[collection, AllocError]. list(source) and set(source) infer the element type; list[T](source) and set[T](source) specify it explicitly. These call-style constructors also accept str and bytes: list("aé")? produces ["a", "é"], and list(b"AB")? produces [65u8, 66u8]. The .from(...) methods do not accept those two source types. dict(d) transfers a dictionary. Set conversion retains the first occurrence of each permitted member.

Failure destroys partial output, the current element, and the remaining source; generator effects already performed are not rolled back. Borrowed sources and pair-iterable dictionary construction are unsupported.

Explicit copying

copy(value) -> Result[T, AllocError] observes its operand once, including references to T. Mutable contents are recursively duplicated; immutable strings can share storage. Scalars and wholly immutable values need no allocation. Classes with custom cleanup, aggregates containing them, and generators are uncopyable. A nested Shared handle gains another owner, as share() would, and its content is not copied, so content behind a handle may be of any type. copy(handle) of a handle itself is rejected; write handle.share().

A failed copy preserves the source and releases newly initialized contents. Partial records do not run whole-record destructors. Copying follows the ownership rules; it introduces no mutable aliases.

Lists

Lists use the common collection rules. Growth through append, reserve, and extend is recoverable; indexed replacement allocates nothing and traps on a missing index. Lists have no * repetition; a comprehension such as [0.0 for _ in range(count)]? builds repeated elements.

With --unchecked assume, list indexing lexically inside a plain unchecked: suite requires a nonnegative index below the list length. Reads, writes and borrows omit both the bounds check and negative-index normalization; violating this precondition is undefined behavior. unchecked trap: and default builds keep normal indexing behavior. The optimizations below describe checked accesses.

Inside for and while loops, scalar indexing of a named list already in scope reads the list’s length and row address once, before the loop, and then uses checked direct loads and stores. An immutable shared reference always qualifies. An owned list, exclusive reference, or retargetable binding qualifies only when the loop cannot resize, replace, or move the list: the loop makes no call and performs no list operation other than scalar element reads, writes, and length queries. Each list keeps its own bounds check, and a negative index is still normalized; unrelated lists need not have equal lengths. Generator bodies and other loops keep ordinary indexed access.

An access that cannot fail has no bounds check. In for i in range(start, len(xs)) with a nonnegative constant start and step 1, xs[i] cannot fail when xs is a &list[T] parameter or a slice view: its length cannot change while the function runs. The stop may also be a local set from len(xs) earlier and not changed since, as in n = len(xs) followed by for i in range(n). With the stop written as len(xs) for the same list xs, an owned list, an exclusive reference, or a shared reference made inside the function also qualifies when the loop uses the row cache described above: the loop cannot change the list, so the cache takes its length from the range’s stop; a length saved in a local before the loop does not qualify for these lists. Indexing any other list with i, a range with any other stop, or a loop that can resize the list keeps the check, so a length read before a clear() still traps. The counter of for i, x in enumerate(xs) over such a list or view qualifies the same way, when the call has no start; indexing another list with it still checks the bound but skips negative-index normalization.

for x in xs over an owned list reads the list’s length and rows once, before the loop, like iteration over a borrowed list: the loop owns the list, and no other code can reach it. Each element read has no bounds check, and the position advances without an overflow check.

A while lo < hi loop over a slice view checks once, before the loop, when its body begins by indexing the view with hi, and possibly lo, as v.swap(lo, hi) or t += v[hi] does. The view must be one the loop does not reassign, lo must start nonnegative and only grow, and hi must change only by checked subtraction of constants. Each pass’s hi is then at most its first value, and lo is below hi. The check runs only when the loop does, and fails with the first pass’s message and site. A body that starts with any other operation that can fail, such as t * 10 + v[hi], keeps its checks.

Any other for i in range(start, stop) loop with step 1 and a nonnegative start is compiled twice when its body is small, makes no call, and has no nested range loop, as in for i in range(n): dst[i] = src[i]. start is nonnegative when it is a constant, a range counter, or a checked sum of such values, as in for j in range(i + 1, n). One test before the loop compares stop with the length of each list or view the body indexes by i or i + k, for a constant k from 0 to 64, and checks each index that no pass changes, such as bodies[i] in that inner loop. When every index is in bounds, the copy without those checks runs; otherwise the checked copy runs, so a body that breaks, returns, or fails before reaching a bad index behaves as before, and the bad index traps with its usual message and site. A negative unchanged index also selects the checked copy, which counts it from the end. The length must be read once before the loop: a borrowed list or view, or a list that uses the row cache described above. A division keeps its own checks in both copies. The body limit, 256 operations, bounds the code size the second copy adds. An overflow check on i + k with k > 1 remains.

A while loop without the lo < hi form above is compiled twice under the same conditions. Either kind of loop may hold a nested while loop. Its test covers each unchanged index and each nonnegative constant index, such as perm[0] in while perm[0] != 0:; a negative constant index keeps its check in both copies. A nested loop also sees the lengths its enclosing loops never change: with n = len(u), a for j in range(n) loop inside for i in range(n) reads u[j] and u[i] without checks.

Storage costs

Scalar lists store elements at their declared widths: one byte for bool, i8 and u8; two for i16 and u16; four for i32, u32 and f32; eight for i64, u64 and f64. Thus 128 live u8 elements occupy 128 bytes of row storage, and 128 i64 elements occupy 1,024 bytes. The allocation also includes a fixed owner, spare capacity from growth, and rounding to 16-byte allocation units. reserve counts elements, not bytes.

Other elements use one 16-byte value slot plus any inline payload storage; lists reserve no dictionary value slot. Scalar-only Option and Result elements still take 16 bytes. Records and multi-field enum variants retain 16-byte field slots, so an optional variant with three bool fields takes 64 bytes per row. These are private implementation layouts, not an FFI ABI. Element references remain valid under the existing borrowing rules, including for packed scalar elements. See value representations.

Lookup and removal

OperationResultBehavior
items.get(index: i64)Option[T]Observe an element; Nothing outside the list
items.pop()Option[T]Remove and own the last element
items.pop(index: i64)Option[T]Remove and own the selected element
items.count(value)i64Number of equal elements
items.find(value) / items.rfind(value)Option[i64]First/last matching position

Indices can be references; negatives count from the end. get and pop handle extreme i64 indices safely. Neither allocates; a missing pop leaves the list unchanged. pop requires exclusive access, preserves remaining order and capacity, and supports all list element types. Last-element removal touches no surviving rows and is constant-time in the list length, including for inline elements. Other positions shift and repair only subsequent rows. Extracted inline payloads are copied into the caller’s own storage before another list mutation or destruction; the freed capacity can be reused by a later append.

get observes the list and index once in source order. It supports only non-affine values, including copyable enums; retained results survive replacement or destruction of the list. Use pop or explicit references for affine elements. There is no default argument.

Search/count accept exactly the element type, including references, and support integers, floats, booleans, and strings. They observe both operands once, allocate nothing, and use IEEE float equality (NaN never matches; signed zeros do). Aggregate searches, optional bounds, and throwing index are unsupported.

Reordering and extending

items.reverse() -> () requires exclusive access and reorders all element types without allocation, copying, retaining, or dropping them. Empty and singleton lists are unchanged; later destruction follows the new order.

items.swap(i: i64, j: i64) -> () exchanges two elements under the same contract: exclusive access, every element type, and no allocation, copy, retain, or drop. Negative indices count from the end, and a missing index traps. Both indices are evaluated before the list is borrowed, so they may read it. Equal indices leave the list unchanged. swap is the way to exchange owned elements, which cannot be moved out of indexed storage. On a list of numbers or bool, swap compiles to inline bounds checks, loads, and stores with no runtime call; other element types call the runtime once. An exclusive slice view, &mut [T], has the same swap, with indices counted within the view.

items.sort() -> () orders the elements ascending, in place, without allocating. It requires exclusive access and one of these element types:

Element typeOrder
IntegersNumeric
f32, f64Numeric, with -0.0 before 0.0 and every NaN after all numbers
strBy code point; a prefix comes before the longer string
Tuples of these typesBy the first component, then the next where those are equal

Every other element type, including bool, classes, enums, and collections, is a compile-time error. Elements that compare equal are indistinguishable, so the result is fully determined. items.sort(True) orders descending; an omitted or False argument orders ascending. Sorting takes O(n log n) comparisons. Wide inline rows use three-way introsort with insertion sort for small partitions and a heapsort fallback; rows without inline payloads use Rust’s unstable slice sort. Row swaps repair inline addresses immediately, without retaining or dropping elements. The wide-row algorithm uses O(log n) stack space and no heap workspace. A tuple of numbers and strings has its field offsets resolved once per sort.

items.sort_by_key(key) -> () accepts any element type. The key is a named function, a reusable shared-call closure, or an anonymous function written in place, such as items.sort_by_key(def(x: &i64) -> i64: *x); it takes &T for the element type T and returns an integer, float, string, or tuple of those. items.sort_by_key(key, True) orders descending. The list is exclusively borrowed for the operation; callbacks cannot also borrow it through a capture. Keys use the same total order as sort, including signed zero and NaN placement. The runtime sorts the rows with the wide-row introsort above, for every element width, and calls a generated comparator that computes both keys. The sort is unstable: equal-key elements may change relative order. It allocates no buffer and recomputes keys during O(n log n) comparisons, so the callback should give consistent keys and remains responsible for its own explicit allocations. An inconsistent key yields an unspecified order but keeps every element. A key returning Result[K, E] is also accepted: the sort then returns Result[(), E] instead of (). The first error stops further key calls; the runtime finishes its safe row moves and returns that error unchanged. The list may already be reordered, but every element remains owned exactly once. Both keys and errors may own strings. Callers handle this public result normally, even inside unchecked:; no private callback channel is involved. For example, a negating key declares def(x: &i64) -> Result[i64, OverflowError[i64]]: Ok(-*x). One-shot and mutable-call closures and reference-valued keys are unsupported. Elements are moved by row exchange, with no copying or early destruction.

sorted(items) -> Result[list[T], AllocError] observes a list and explicitly copies it before sorting by natural order. sorted(items, True) gives a descending copy. The source survives either result; the element type must support copying and natural ordering. For key ordering of a copy, use mut result = copy(items)? followed by result.sort_by_key(key). User-defined comparison functions and class ordering protocols are unsupported.

items.extend(other) -> Result[(), AllocError] consumes a same-typed owned list on both outcomes. It reserves the entire additional count before transfer. Success moves elements in order without cloning; failure preserves destination contents and drops the source. Sufficient capacity, including empty sources, needs no allocation. Source references, arbitrary iterables, and self-extension by moving the destination are rejected. Use extend(copy(source)?) to preserve it.

Slices

Slice syntax items[start:stop:step]? copies the selected elements into a new list with Python’s clamping, omitted bounds, and negative steps. It duplicates elements with copy() and Shared handles with share(), and leaves the source unchanged; element types that copy() rejects are a compile error.

items.slice(start: i64, stop: i64) -> Result[list[T], AllocError] is the transfer form for an owned temporary. It takes both bounds, observes receiver and bounds once in order, and accepts references. Start is inclusive; stop is exclusive. Negative bounds count from the end and clamp to [0, len(items)]; reversed or extreme bounds safely yield the corresponding clamped slice. It has no step.

Non-affine elements are retained into an independent list. Affine elements require an owned temporary receiver: selected owners transfer, and the remainder drops in source order. Failure drops the whole temporary; observed sources remain unchanged. copy(items)?.slice(start, stop) explicitly preserves an affine source. All output storage is reserved before transfer: two allocations for a nonempty slice, one for an empty owner.

Sorting design alternatives

User-defined ordering and Timsort are unimplemented proposals for comparison semantics and stable sorting respectively. They do not change the current sorting contract above.

Dictionaries

Dictionaries use the common collection rules. insert(key, value) adds a key or replaces its value. Counting uses *counts.get_or_insert(key, 0)? += 1, which looks the key up once; see lookup and removal. The ? exposes allocation failure.

Dictionaries have no indexed assignment. dictionary[key] = value, dictionary[key] += n, and &mut dictionary[key] are compile errors whose messages name insert or get_mut: key presence is a run-time fact, so a write through an absent key would trap where --check cannot see it. Reading dictionary[key] remains, and traps on an absent key. Writing through dictionary[key].field or calling a mutating method on dictionary[key] is rejected the same way; use get_mut for one key, or a mutable value loop for every value.

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    mut stock = {"apples": 3}?
    stock.insert("pears", 5)?
    match stock.get_mut("apples"):
        case Some(count):
            *count -= 1
        case Nothing:
            pass
    print(stock)?
    Ok(())
{"apples": 2, "pears": 5}

Updating every value

for value in (&mut dictionary).values(): lends each value as &mut V in insertion order; for key, value in (&mut dictionary).items(): adds a shared &K key. &mut dictionary.items() is the same loop. Each step reaches the next entry directly, with no key lookup, item tuple, or copy, so it works for affine values and allocates nothing. Inside a function taking d: &mut dict[K, V], write (&mut d).values(); d.values() stays a shared loop.

class Player:
    name: str
    hp: i64
    items: list[str]

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    mut players = {"ada": Player("Ada", 3, []?), "lin": Player("Lin", 9, []?)}?
    for player in (&mut players).values():
        player.hp += 10
        player.items.append("potion")?
    for key, player in (&mut players).items():
        if *key == "lin":
            player.hp = 0
            break
    for key, player in players.items():
        print(*key, player.hp, player.items)?
    Ok(())
ada 13 ["potion"]
lin 0 ["potion"]

The loop holds an exclusive loan of the whole dictionary until it ends, so the body cannot insert, remove, clear, read, or borrow the dictionary again, including through get_mut or a nested loop; these are conflicting borrow errors. break, return, and ? end the loan without dropping or moving any value. The references are loop-local: storing one in a collection, assigning it to an outer reference binding, or returning it from a function that does not take the dictionary as its first reference parameter is rejected. A mutable projection is an immediate loop source; (&mut dictionary).values() cannot be bound to a name. There is no values_mut() or items_mut(): the exclusive receiver selects the mutable loop, as for x in &mut xs does for a list, and those names are compile errors that point here. To change values while also adding or removing keys, collect the keys first with keys_snapshot() and use get_mut per key.

Lookup and removal

dictionary.get(key) -> Option[V] observes receiver and key once in source order, including references. It copies or retains only non-affine values; collections, classes, and enums containing affine payloads are rejected. Lookup allocates nothing. Results outlive source replacement or destruction. Copying and borrowed lookup explains why get copies and how borrowed lookups are named.

dictionary.get(key, default) -> V accepts a second positional argument of the value type and returns it only when the key is absent. Receiver, key, and default evaluate once in that order; the default is eager even for a present key. Lookup still requires a copyable value and adds no allocation. Zero, False, and an empty string are present values. This does not insert the key. A stored Nothing produces Some(Nothing), distinct from a missing key.

dictionary.get_mut(key) -> Option[&mut V] lends the value for key exclusively, or returns Nothing. It works for every value type, including affine ones, and allocates nothing. The key is observed before the receiver is borrowed. The result holds an exclusive loan of the whole dictionary until its last use, as a reference-returning method’s result does, so the dictionary cannot be read, mutated, or borrowed again meanwhile. Match on the result to update a value in place: *count += 1 for a scalar, items.append(x)? for a list, or point.x = 0 for a class. replace(*value, new) returns the previous value and keeps the entry’s position.

dictionary.get_or_insert(key, default) -> Result[&mut V, AllocError] lends the value for key exclusively, first inserting key and default when the key is absent. Key and default evaluate in that order before the receiver is borrowed, and both move into the call. A present key keeps its stored entry and value; the call drops the incoming key and the unused default before returning. Either way the key is hashed once, so counting or grouping needs one lookup per item rather than the two of get_mut followed by insert. A missing key may grow the dictionary as insert does; on AllocError the key and default are dropped and the dictionary is unchanged. The result holds the same whole-dictionary loan as get_mut’s. The default is eager: it is built even when the key is present, so prefer get_mut when building it costs an allocation and the key is usually present.

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    mut counts: dict[str, i64] = {}?
    for word in "to be or not to be".split()?:
        *counts.get_or_insert(word, 0)? += 1
    mut lengths: dict[i64, list[str]] = {}?
    for word in counts.keys():
        group = lengths.get_or_insert(len(*word), []?)?
        group.append(*word)?
    print(counts, lengths)?
    Ok(())
{"to": 2, "be": 2, "or": 1, "not": 1} {2: ["to", "be", "or"], 3: ["not"]}

dictionary.pop(key) -> Option[V] requires exclusive access and supports all permitted value types. A hit removes the entry, drops its stored key, and transfers the value’s owner into Some; a miss returns Nothing without mutation. The key is observed once before exclusive receiver access. Removal takes exactly one key, with no default argument.

Removal allocates nothing, preserves capacity and remaining insertion order, and appends a reinserted key at the end. Expected lookup/removal cost is constant with well-distributed keys, plus hashing/comparison and cleanup; adversarial collisions can make it linear.

Snapshots and views

dictionary.keys() -> KeyView[K] creates a shared, local-only borrowed view without allocating. A view can be bound to a local, iterated, and used with builtin set operations. Iteration yields &K in insertion order; use [copy(key)? for key in dictionary.keys()]? to collect owned keys explicitly, including move-only class keys without custom cleanup. Keys containing __del__ can be borrowed here but cannot be copied; use into_keys() to transfer them. len(view) is O(1), and key in view / not in use dictionary lookup. The value holds a borrowed owner handle with a compiler-known key projection; the owner retains its row descriptor and each loop keeps its own cursor.

== and != compare keys as sets, ignoring insertion order, against KeyView[K] or set[K] with the same key type. Dictionaries may have different value types. issubset, issuperset, and isdisjoint take &set[K] or &KeyView[K] and return bool. These observations allocate nothing and have no AllocError; relationships perform O(n) expected hash lookups (equality first compares lengths). Empty views are subsets of every set-like operand and disjoint from every operand.

union, intersection, difference, and symmetric_difference take those same operands and return Result[set[K], AllocError]. to_set() materialises the keys as that result type. Algebra reserves its result before inserting keys and leaves both sources unchanged on failure. Move-only keys are copied into independent result storage. All four algebra methods and to_set() reject keys containing custom cleanup, because those keys cannot be copied. Relationship checks remain available. Result order is unspecified. Operators such as |, &, and <= are not part of the view API. Set relationship and algebra methods accept mixed set/view operands in either receiver order. set.intersection_update(view) and set.difference_update(view) also accept key views. Equality accepts either operand order. These builtin methods observe their operands, so an explicit & may be omitted without moving a named set or view.

set(view) and list(view) are not supported. Use view.to_set(), dictionary.keys_snapshot(), or [*key for key in view]? to choose the owned result explicitly.

A bound view holds a shared loan on the whole dictionary until its last use. Insertion, get_mut, get_or_insert, update, removal, clearing, and moving the owner conflict with that loan, including through aliases. Views cannot be stored in fields, collections, tuples, enum payloads, generators or worker/closure captures, or passed to or returned from user functions. Binding a view of a temporary dictionary is rejected; immediate iteration may borrow a temporary. drop(view) and copy(view) are rejected like ordinary references; end the loan by finishing the view’s last use. Conditional view bindings are not supported. General stored/returned views remain separate work.

dictionary.values() creates a shared local view yielding &V, without allocating or copying affine payloads. Both values = dictionary.values() and items = dictionary.items() can be bound and reused in loops; len(view) is O(1). iter(values) and iter(items) start pipelines, with item callbacks taking separate key and value parameters. These views have the same local-only borrow restrictions as key views and do not provide set operations. (&mut dictionary).values() yields &mut V in immediate loops; mutable views cannot be bound. See updating every value. Automatic box dereferencing applies to keys(), values() and items(), including nested boxes and borrowed boxed dictionaries. Their loans protect the boxed owner throughout the observation or loop.

dictionary.keys_snapshot() -> Result[list[K], AllocError] and dictionary.values_snapshot() -> Result[list[V], AllocError] take no arguments and preserve insertion order. Each returns a new list, retaining immutable element storage. Observed dictionaries remain unchanged on failure, and snapshots survive later source mutation or destruction.

A snapshot with affine selected elements requires an owned temporary: this applies to keys_snapshot() for move-only keys and values_snapshot() for move-only values. Those elements transfer and the temporary is consumed on success or failure. Use copy(d)?.keys_snapshot() or copy(d)?.values_snapshot() to preserve such a source explicitly. The unselected side may be affine even when observing a named dictionary. Both snapshots reserve their complete output before transfer: two allocations when nonempty, one for an empty owner.

dictionary.into_keys() -> Result[list[K], AllocError] and dictionary.into_values() -> Result[list[V], AllocError] consume a named owner or owned expression directly, including dictionaries with affine payloads. They preserve insertion order and transfer the selected elements without copying. The dictionary is consumed on success and failure. Output reservation happens before any transfer; allocation failure drops the remaining source exactly once. Use for value in dictionary.into_values()?: when the dictionary is no longer needed. This is an owned snapshot, not a stored borrowed view.

dictionary.items() creates a shared local borrowed view, with an independent cursor for each loop, and creates no item tuples or snapshot buffers. Shared views load shared references to both keys and values, including scalars and strings; mutable views lend mutable values while keys remain shared references. Its loop loans follow the iteration rules.

Transferring an update

dictionary.update(other) -> Result[(), AllocError] requires exclusive access and consumes a same-typed owned dictionary on every outcome. Existing keys keep their position and key owner; incoming values replace old values. New keys append in source insertion order.

All storage for absent keys is reserved before replacement or insertion. Failure preserves destination contents and lookup behavior, though capacity may change, and drops the source. Success transfers entries without copying; old values drop in source traversal order after replacement. Empty and replacement-only updates, or updates fitting reserved capacity, allocate no runtime storage.

Use update(copy(source)?) to preserve the source. References, pair iterables, and keyword updates are unsupported.

Sets

Dictionary KeyView[K] values borrow their dictionary’s keys without allocating and provide the same relationship and algebra methods. Their operations accept &set[K] or &KeyView[K]; equality also compares a key view and a set in either operand order.

Sets use the common collection rules; members are integers, booleans, strings, bytes, or tuples, classes, enums and anonymous sums recursively containing those types. Iteration order is unspecified. add and reserve follow the recoverable growth contract.

Removal and update

set.discard(value) -> bool requires exclusive access. It observes one same-typed value or reference before borrowing the receiver, removes and releases the stored member if present, and returns whether removal occurred. A miss changes nothing. It allocates no storage and retains capacity. Set pop is unsupported.

set.update(other) -> Result[(), AllocError] consumes one same-typed owned set on both outcomes. It reserves for absent members before changing membership. Failure preserves destination contents and lookup behavior and drops the source; success transfers new owners and releases duplicate incoming owners. Empty and duplicate-only sources, or sufficient reserved capacity, need no allocation. Use copy(source)? explicitly to retain a source. References and general or multiple iterables are unsupported.

set.intersection_update(other) -> () keeps common members; set.difference_update(other) -> () removes members found in other. Both accept a same-key-type dictionary KeyView as well as a set. They require exclusive destination access and observe the source before borrowing the destination. The source remains borrowed during mutation; aliases into the destination are rejected. Removed members drop, capacity remains, and the operations allocate nothing. Use clear() to empty a set directly.

Set algebra

Each method takes exactly one same-key-type set or dictionary KeyView (with an optional explicit borrow), observes both inputs, and returns Result[set[T], AllocError]:

OperationMembers selected
set.union(other)Present in either input
set.intersection(other)Present in both
set.difference(other)Present in the receiver, absent from other
set.symmetric_difference(other)Present in exactly one input

Inputs may be the same set and remain unchanged on failure. Output is independent, though immutable strings can share storage. All storage for the actual result is reserved before copying members, including move-only class records. Hash-backed results use four allocations when nonempty, one for an empty owner. Small enum bitsets use one owner allocation regardless of membership. Arithmetic operator overloads and iterable conversions are not implied.

Keys containing __del__ cannot be duplicated, so all four non-mutating algebra methods reject them, even for empty sets or a currently cleanup-free variant. copy(set) is also rejected. Use consuming update to combine owners, and intersection_update or difference_update to filter in place. These operations preserve each retained key’s ownership and destroy removed or incoming duplicate owners. Relationship checks only borrow and remain available.

Relationships

issubset(other), issuperset(other), and isdisjoint(other) return bool. They take one same-key-type set or dictionary KeyView, accept references, observe both operands once left to right, and allocate nothing. Empty sets are subsets of every set and disjoint from every set. Heterogeneous coercion and comparison operators are unsupported.

Small enum bitsets

A set of a named enum or tuple with a supported finite domain of at most 256 distinct values uses a size-selected membership mask instead of rows and hash buckets: one 64-bit word for 1–64 values, two words for 65–128, and four words for 129–256. Selection uses the complete type domain, not the number of current members; clearing or draining a set preserves its selected width. This includes payload-free named enums and named enums or tuples composed recursively of bool and other such enums or tuples. Count each variant’s payload combinations, then add the variants: Empty plus Flag(bool) has three values; a variant with six boolean fields has 64. Option[bool] also has three values. This is automatic: write set[Permission], with the same methods and Result handling as any set. The owner still allocates once; construction, copying and algebra can therefore return AllocError. Adding members, valid reservation, removal and in-place updates need no further allocations. Invalid capacity requests still return CapacityOverflow.

Membership, insertion and removal encode the payload into a dense index and operate on its bit. Encoding visits the active payload’s fields; length and relationships between two such sets use bit operations. Their union, intersection, difference and symmetric difference use word operations plus the result-owner allocation. Iteration visits present values; borrowed iteration still yields shared references, and the borrow checker still protects the source owner. Set iteration order remains unspecified; do not rely on the order seen with a particular representation.

Dictionary key views remain borrowed hash-table views. Their mixed operations with bitsets work normally, and to_set() can produce a bitset. Dictionary storage itself is unchanged. Domains larger than 256, integer or string payloads, and classes keep the ordinary hash-set representation. Anonymous sums also use hash sets, even when their members have finite domains. Recursive keys remain unsupported. The compiler emits immutable canonical values for payload domains, including inline record storage, so borrowed iteration needs no allocation or temporary value cache. This adds static data proportional to the domain size and value layout. All three mask alternatives fit inside the existing owner storage; small-domain operations process only their selected words. No target-specific SIMD support is required. See the measurements and storage tradeoffs for the per-owner savings and shared metadata costs.

Ranges

range(stop), range(start, stop), and range(start, stop, step) return range[T] directly. Stop is exclusive; omitted start is zero and omitted step is one. A range is a small copyable value: construction, passing, returning, standard-sum wrapping, and repeated iteration allocate nothing. Iterating a stored range is a counted loop, like a direct range(...) source. Collecting its elements into a collection is fallible.

range[T](...) selects any integer type. Start and stop have type T; step is always NonZero[i64], permitting descending unsigned ranges. Without explicit type arguments, a typed bound or range annotation selects T; otherwise it defaults to i64. Bounds must fit T, so an exclusive u8 stop of 256 is rejected.

A step literal such as -2 converts at compile time, and 0 is a compile-time error. A computed step is converted first, which handles zero as an error value:

def evens(stop: i64, stride: i64) -> Result[list[i64], AllocError | RangeError]:
    step = NonZero.try_from(stride)?
    Ok(list(range(0, stop, step))?)

def main() -> Result[(), AllocError | IoError | RangeError]:
    print(evens(9, 2)?)?
    print(list(range(5, 0, -2))?)?
    Ok(())
[0, 2, 4, 6, 8]
[5, 3, 1]

Passing an i64 step is a compile-time error: range step must be `NonZero[i64]`, not `i64`; write a nonzero literal, or convert a value with `NonZero.try_from(step)?` . A range therefore never has a zero step. A length exceeding i64 traps. Bounds and length handle the full u64 domain without intermediate overflow. len, in, not in, and i64 indexing are supported; negative indices count from the end and missing indices trap. Membership is constant-time.

In a directly written comprehension range, a numeric output annotation can guide the loop variable through direct arithmetic in the output expression:

squares: list[u8] = [n * n for n in range(8)]?

This context does not flow backward through arbitrary calls, filters, stored ranges, or explicit range arguments. Use range[u8](...) at those boundaries. Arithmetic retains its fixed-width rules.

Input, output, and files

Text I/O uses strict UTF-8; binary I/O preserves raw bytes. Both use explicit IoError results. Input consumption and external writes may have partial effects before failure; errors do not promise rollback. Error values and their Result/Option wrappers allocate nothing. Successful writes and ordinary flushing do not promise durable storage.

These APIs are prelude builtins. File I/O currently supports Linux; unsupported hosts report IoError.Unsupported. Standard-stream input uses the process descriptor on Unix. Configurable encodings, public OS descriptors, and lazy file iteration are unsupported. The imported std.fs/std.path modules add native paths, lazy directory streams, metadata and a Plenty walker. The finder example searches directory roots through these APIs.

I/O errors

IoError is an allocation-free enum:

VariantMeaning
System(i32)Native OS error code, only when the OS supplied one
Data(DataError.InvalidUtf8)Invalid or truncated UTF-8
Data(DataError.Allocation(AllocError))Recoverable buffer/result allocation failure
InvalidModeUnsupported computed open mode
ClosedFile is closed
NotReadableFile lacks read capability
NotWritableFile lacks write capability
InvalidInputInvalid argument, such as NUL in a path or negative truncation size
UnsupportedUnsupported host operation
OtherFailure without a native code or more specific variant

Constructing, matching, and propagating these errors adds no allocation. The ordinary error rules determine whether details are retained or erased by builtin Failure.

File operations report Closed before mode errors. Reads and writes validate capabilities even for zero-length work. Negative read limits mean unbounded reading; they are valid, unlike negative truncation sizes.

An error can follow consumed input, a written prefix, or other external effects. The operation’s page states its cleanup and preservation guarantees.

Standard streams and arguments

Output

write_stdout(text: str) -> Result[i64, IoError] and write_stderr(text) observe text and write its exact UTF-8 bytes without adding a newline. Success returns the Unicode-scalar count. Errors can follow partial output. Stderr writes go straight to descriptor 2, so a closed or read-only stderr is IoError.System(9) (EBADF). Stdout writes are buffered by the Rust standard library, which reports EBADF as success; a closed or read-only stdout discards output without an error.

print(values...) -> Result[(), IoError] formats supported values separated by spaces and adds a newline; no arguments produce an empty line; see formatting for allocation and Result restrictions. flush_stdout() and flush_stderr() return Result[(), IoError], exposing deferred buffered-write errors. These operations retain the process-owned stream handles. Success and flushing do not imply durable storage.

Input

input() -> Result[Option[str], IoError] reads one stdin line, removing LF or CRLF and preserving other characters. Some("") is an empty line; Nothing is EOF. Decoding is strict UTF-8.

Buffer growth and result construction are recoverable. Failure may consume a prefix or complete line; the reader does not read ahead into the next line. The current Unix backend reads the process descriptor directly. Prompt arguments, bare-CR newline translation and buffered stream objects are unsupported. Use input_bytes for binary records without newline conversion.

Arguments

args() -> Result[list[str], IoError] snapshots all executable arguments, including the invocation name at index zero. Each call produces an independent list with fallible text construction. Invalid UTF-8 is rejected rather than replaced or escaped; failed construction releases partial output. OS argument storage is only borrowed during construction. Use args_bytes for lossless Unix arguments and paths.

Bytes and lossless Unix paths

Raw transport uses owned list[u8] values. These are ordinary mutable lists, with checked indexing, borrowing, and fallible growth; the current representation uses general list slots rather than a compact byte buffer. A Unix path is a byte list used without decoding. Filesystem calls reject embedded NUL and otherwise preserve every byte, including CR, LF, tabs, and invalid UTF-8.

All operations below are prelude builtins. Arguments are observed in source order and remain available. Results use IoError; allocation failures are IoError.Data(DataError.Allocation(...)).

OperationResultContract
encode_utf8(text)Result[list[u8], IoError]Independent owned copy of the exact UTF-8 bytes
decode_utf8(bytes)Result[str, IoError]Strict validation and independent text; invalid input returns DataError.InvalidUtf8
args_bytes()Result[list[list[u8]], IoError]Independent snapshot including invocation name at index zero; no decoding
input_bytes(delimiter: u8)Result[Option[list[u8]], IoError]Read one stdin record, removing only its delimiter
write_stdout_bytes(source: &[u8])Result[i64, IoError]Exact bytes with no added framing; returns byte count
write_stderr_bytes(source: &[u8])Result[i64, IoError]Same contract on stderr
read_bytes(path)Result[list[u8], IoError]Read whole file without decoding or newline translation
write_bytes(path, source: &[u8])Result[i64, IoError]Create or truncate, write exact bytes, check close, return byte count

The writers read source as a slice view: pass &bytes for a list[u8] or std.bytes.Bytes, &buffer[a:b] for part of one, or a b"..." literal. Nothing is copied or converted.

input_bytes(0u8) provides NUL framing for Unix paths, whose names cannot contain NUL. Some(empty) represents an empty record; Nothing means EOF before any record byte. A final unterminated nonempty record is accepted. CR and LF have no special meaning. Input reads no bytes beyond the delimiter and retries interrupted reads. Allocation or I/O failure can consume part of a record; no rollback or resynchronization is promised. Construction may fail even at EOF.

Writers read the view in place without allocating an output buffer. They retry interrupted writes and complete all bytes on success, but errors can follow a written prefix. Use the existing flush_stdout() or flush_stderr() to report deferred output errors. Calls and NUL records are not atomic against concurrent writers.

Binary file helpers use the same Linux open/creation/symlink rules as text helpers. Path conversion is fallible and precedes opening, so allocation failure cannot truncate a destination. No output allocation follows opening. Reads release partial buffers and descriptors on error; successful file operations check close errors. Success does not imply atomic replacement or durable storage. File helpers on other hosts return IoError.Unsupported; raw stdin uses the Unix process descriptor.

For display, decode explicitly or format the byte list with str.repr; keep the original bytes for filesystem operations. Replacement decoding is never used. The imported std.path module provides Linux Path owners over this transport, and std.fs provides directory enumeration and metadata. std.bytes provides compact buffers that convert to and from list[u8], and std.io reads and writes them through files and the process streams without that conversion. First-class byte views and portable path representations remain within issue #49’s broader scope. The finder example exercises raw arguments, matching, and NUL input/output against real Unix filenames.

Whole-file helpers

Paths are UTF-8 strings, absolute or relative to the process working directory. NUL in a path returns IoError.InvalidInput; ordinary symlink resolution applies. Arguments are observed once in source order and remain available. All temporary storage is fallible, and descriptors are closed on success and failure.

OperationResultBehavior
read_text(path)Result[str, IoError]Read a whole file, translating CRLF and bare CR to LF
write_text(path, text)Result[i64, IoError]Create or truncate, then write exact UTF-8
append_text(path, text)Result[i64, IoError]Create or append exact UTF-8 without truncating

read_text preserves content NUL, validates UTF-8 strictly, and does not remove a BOM or use a locale. Path conversion, buffer growth, and output construction are recoverable; errors reclaim partial buffers. It does not promise a snapshot against concurrent writers.

write_text and append_text return the Unicode-scalar count. Path allocation precedes opening, so allocation failure cannot truncate or append to the file. No output allocation follows opening. OS errors can leave a truncated destination or written prefix. Success checks close errors but does not imply synchronization or atomic replacement.

Creation uses 0666 restricted by the process umask. Append creates a missing file even for empty text. Each native write appends at the current end, but one logical call may use several writes and is not an atomic concurrent record. For explicit lifetime and cursor control, use File. For raw path bytes and untranslated contents, use read_bytes and write_bytes.

Owned files and modes

The builtin accepts two positional call forms:

  • open(path: str) -> Result[File, IoError]
  • open(path: str, mode: str) -> Result[File, IoError]

Both create an opaque, affine file owner. open(path) opens an existing file for reading, equivalent to open(path, "r"). Like other builtins, open takes positional arguments only: open(path, mode="w") is rejected. Named functions and methods accept named arguments and defaults; see signatures and calls. Pass an explicit mode positionally. Literal modes are checked at compile time; computed invalid modes return IoError.InvalidMode.

ModeCapabilitiesOpening behavior
r / r+Read / read and writeExisting file required
w / w+Write / read and writeCreate or truncate
a / a+Write / read and writeCreate or append
x / x+Write / read and writeAtomic exclusive creation

Exclusive creation fails for an existing path, including a symlink, without truncation. a+ starts at EOF; other modes start at byte zero. Append writes use OS append semantics even after seeking. Paths and creation permissions follow the whole-file rules. Owner and path storage are allocated fallibly before opening, so allocation failure cannot truncate the destination.

Files can move, return, live in aggregates, and be borrowed, but cannot be copied. file.closed -> bool observes state. Printing shows File(open) or File(closed); equality is owner identity rather than path equality. file.readable() and file.writable() return Result[bool, IoError] through shared access, without allocation. They report the open mode, not a guarantee that a subsequent OS operation will succeed; closed files return Closed.

Closing and resource scopes

file.close() -> Result[(), IoError] requires exclusive access and is idempotent. It marks the owner closed before closing its descriptor, reports native errors, and allocates nothing. Automatic destruction closes any live descriptor without reporting errors or allocating. Closing does not promise durability.

File is an intrinsic context manager: entry lends &mut File, exit closes it, and destruction releases the owner. with &mut file as stream: leaves the original owner closed afterward. Entry permits an already closed file; its operations report Closed.

Exit discards close errors to preserve the body’s control flow. Call close()? inside the body when those errors matter. All ordinary resource-scope cleanup rules apply.

Reading files

All reads require exclusive access to a live readable File, even for zero limits. They use strict UTF-8, preserve NUL, and translate LF, CRLF, and bare CR to LF. Output is independent of the file. Buffers, long strings, and list growth are fallible; failures may consume input but leave the file valid for another operation or close. Invalid UTF-8 is reported after consuming input. No rollback or concurrent-file snapshot is promised.

OperationResultBehavior
file.read()Result[str, IoError]Read to EOF
file.read(count: i64)SameRead at most count translated Unicode scalars
file.readline()Result[str, IoError]Read one line, retaining its translated terminator
file.readline(count: i64)SameStop at count scalars or the first translated newline
file.readlines()Result[list[str], IoError]Eagerly collect remaining lines with translated terminators

Negative read counts mean read to EOF; negative readline counts mean a complete line. Zero returns empty text without consuming input or a pending LF. Positive limits never read past a scalar and can leave a partial line for the next call.

EOF returns "" from read/readline. An empty line is "\n"; a final unterminated line receives no added terminator. readlines returns an empty list for empty input and adds no synthetic line at EOF. On failure it destroys the collected prefix. Bounded readline controls memory better than eager readlines.

At CR, line reading returns immediately. The next read consumes an optional LF; it does not read beyond a line merely to decide whether CR was followed by LF. Saved positions preserve this translation state. File iteration, size hints, and configurable encodings are unsupported. To split text already in memory using broader Unicode line boundaries, use splitlines.

Writing and synchronization

file.write(text) -> Result[i64, IoError] observes text once before exclusive receiver access. It writes exact UTF-8 without newline insertion or translation, returning the Unicode-scalar count. The file must be live and writable even for empty text. No runtime write buffer is allocated; errors can leave partial output.

file.writelines(lines: list[str]) -> Result[(), IoError] observes a list or shared list reference once before exclusive access. It writes each string’s exact bytes without separators, preserves the list, and allocates nothing after argument construction. Empty lists still validate the file and capability. Errors may leave a written prefix. General iterable sources are unsupported.

Update streams are unbuffered, so switching reads and writes requires no flush. Writes use physical byte positions and can overwrite part of a UTF-8 sequence; they do not insert text. Nonempty writes discard pending CRLF translation state. Append modes use OS append semantics, but a logical write need not be one atomic record against concurrent writers.

Flush and sync

file.flush() -> Result[(), IoError] flushes runtime buffers. Files are currently unbuffered, so it validates the open state and normally does no extra work. file.sync() -> Result[(), IoError] requests OS synchronization of data and metadata. Both require exclusive access and report errors without allocating.

Neither close nor flush substitutes for sync. Filesystem, device, and directory entry durability rules still apply; sync is not an atomic-save operation.

Positions and truncation

file.tell() -> Result[u64, IoError] returns an opaque text-position cookie. file.seek(cookie: u64) -> Result[(), IoError] restores it. Both require exclusive access, allocate nothing, and report native seek errors, including nonseekable streams.

Zero rewinds. Other cookies must come from tell() on the same stream with unchanged contents. They preserve pending CRLF translation state, rather than representing a plain byte offset. Arithmetic and persistence of cookies are unsupported. A failed seek leaves decoder state unchanged; arbitrary cookies remain memory-safe but can yield invalid text or surprising positions. Relative and end-relative seeking are unsupported.

file.truncate(size: i64) -> Result[i64, IoError] changes a writable file’s length in bytes, returns that length, and leaves the cursor unchanged. Omitting size uses the current physical byte position, not a text cookie. Negative sizes fail before mutation. The operation requires exclusive access, allocates nothing, can fail at the OS, and does not synchronize storage.

Extending regular Linux files adds zero bytes. Shrinking can split UTF-8, which later reading reports as invalid data.

Concurrency and parallelism

There is no async/await or event loop in Plenty. The separate async/await exploration includes a standalone spike, not an implemented language feature or an adoption decision.

Concurrency is explicit. Native workers can overlap without an interpreter lock; ordinary loops and comprehensions remain serial. Ownership transfers and checked loans still apply across threads. Shared immutable storage does not permit unsynchronized mutation; std.atomic values are updated through shared references with atomic instructions, and a mutex grants one lock holder at a time exclusive access to its value. A Shared[T] handle gives several owners, such as pool jobs, shared access to one value; the last owner, on any thread, drops it.

The Go-style scoped-task exploration assesses a possible lightweight alternative, including a fixed-stack native experiment. It introduces no implemented task API or change to these contracts.

Worker eligibility is a closed-world compiler check, not a user-defined marker protocol. It examines concrete stored types, reachable helper and callback bodies, captures, results, and destructors, including finite recursive type graphs. Scalars, strings, ranges, collections, classes, enums, and Shared handles are eligible when their contents and effects are eligible. A shared value’s destructor is checked as if any holder’s thread could run it. Stored concrete callbacks retain their eligibility evidence. Thin Callable values are certified against every matching function address in the program.

Each native operation has a declared effect. Files, directory streams and child pipes cross threads under the ordinary ownership and loan rules; changing the working directory is rejected. Foreign pointers/calls, uncertified indirect callable effects, and generator frames are conservatively excluded. Imports do not grant thread permission. Ordinary checked standard-stream I/O is usable, with scheduler-dependent ordering. Owned transferred values cannot contain references or borrowed environments.

Scoped threads can borrow parent storage for the task’s lexical scope. Channels and executor jobs require wholly owned storable values. Executors and futures cannot enter worker code, including through containers or messages: worker-side submission and future waiting are unsupported.

Allocation guarantees exclude argument expressions, worker bodies, and user cleanup. Error wrappers add no allocation. Blocking operations offer no strict waiter fairness, and application communication can deadlock. Normal scope exits join workers; fatal traps terminate without unwinding. A nonterminating worker prevents its scope or executor from finishing. Worker stack exhaustion terminates the process rather than becoming a job error.

Scoped threads

with spawn(worker, arguments...)? as task: starts a native thread and joins it before leaving the block. Named functions, including inferred generic instances, use their ordinary typed parameters. spawn(&job) or spawn(&mut job) runs an explicitly borrowed reusable zero-argument closure. A binding that already holds a &Closure or &mut Closure, such as a parameter or a get_mut match binding, is reborrowed for the task’s scope: spawn(callback) means spawn(&*callback), or spawn(&mut *callback) when the closure has mutable captures, which requires a &mut reference. Wholly owned jobs use consuming spawn.

task.join() waits and transfers the result exactly once, consuming the task binding. A worker returning Result[T, E] yields that same result. Join itself is infallible under the compiler/runtime contract; unexpected native join failure terminates the process to avoid reclaiming storage still used by a worker.

Omitting as, or leaving a task unjoined, waits and drops its result at exit. Application errors disappear with an unclaimed result; join explicitly to inspect or propagate them. Owned results, including collections, recursive records, and owned concrete callbacks, can transfer back. References, borrowed environments, and generator frames cannot be returned from a worker.

Scope and loans

Task bindings are lexical join tokens: they cannot be copied, borrowed, returned, stored, captured, or passed to a function. spawn must be the direct manager expression, using ? or .unwrap(). Comma-separated managers form nested scopes. Each successful task joins on normal completion, return, ?, break, and continue. Failure to start a later task first joins earlier tasks. Detachment and forced cancellation are unsupported. A join that could wait for a task holding a token before the function’s later cancel is rejected; put stop.cancel_on_exit() after the task to cancel it on every exit. See cancel on every exit.

A join, explicit or automatic, is rejected while the function keeps a channel endpoint that the worker could be waiting on. See joins and live endpoints for the rule and its recommended patterns.

Argument loans and borrowed capture dependencies last until the end of the with block, including after explicit join. Shared readers can coexist; exclusive loans prohibit conflicting access, moves, or drops. Ordinary field disjointness and enum-payload rules apply. yield inside a task scope is rejected.

Scoped work may borrow or own its inputs. Named arguments use ordinary call semantics: &T and &mut T borrow, copyable values copy, and affine values move into the worker. Pool jobs must own their inputs instead; only .par chunks read a borrowed list internally, because the terminal waits for every chunk before it returns. Borrowing callbacks from a collection protects the collection through scope exit; separate tuple slots can have disjoint loans. All workers satisfy the shared eligibility rules.

Start failures

A named start whose arguments are all owned or copied, or that has none, binds them into a one-shot job, as pool.submit(function, arguments...) does, and starts that job. Its start expression has result Result[(), SpawnError[F]], where F is the bound OnceClosure[[], R]; a failed start returns the job with every argument, as an owned job start does. spawn(worker, sender) with an owned Sender therefore returns the sender on failure. Call drop_job() on the start to drop the arguments there instead.

A start that borrows, a named start with a &T or &mut T argument or spawn(&job), returns the job-free SpawnError[()].Unavailable((), code), so every start fails with SpawnError. A borrowed argument cannot be returned in an error, so moved arguments of a borrowed start drop exactly once on the parent thread, in reverse argument order; on success they belong to the worker. The worker has not run and cannot have changed borrowed inputs; already evaluated argument expressions keep their effects.

spawn(...)? follows ordinary ? propagation: the enclosing Result’s error may be the start error or an anonymous sum containing it.

Every nonzero pthread_create status is a start failure; none terminates the process. The error carries it: SpawnError[F].Unavailable(job, code), with code: i32. The usual code is EAGAIN (11), for a thread or memory limit. The runtime passes no thread attributes, so other codes come only from the environment: a seccomp filter that denies clone surfaces its own code, such as EPERM (1).

Task bookkeeping and error handling allocate no Plenty heap storage; native thread stacks and OS bookkeeping still require resources. The current runtime supports joinable native threads on x86_64 Linux GNU.

Joining synchronizes worker completion and its cleanup, but does not synchronize Linux’s later removal of the task from /proc/self/task. Diagnostic tools can briefly observe an exiting task there after a successful join; this does not mean the worker is still running Plenty code or still owns its dropped handles.

Consuming thread jobs

with spawn(job)? as task: moves a concrete wholly owned zero-argument closure into a worker. Reusable and one-shot closures are accepted; def once permits moving a capture into the result. Borrowed captures and generator frames are excluded. A named start with owned arguments binds them into such a job. The scoped-thread lifetime and join rules still apply.

The start expression has result Result[(), SpawnError[F]], where F is the concrete job type. Its one variant, SpawnError[F].Unavailable(job, code), returns the complete unstarted job and the i32 native code after any native start failure. Returning, matching, dropping, and retrying the error and job allocate nothing. Capture expressions have already run, but the worker has not. ? widens the error into an enclosing sum containing SpawnError[F], which keeps the job. A kept job’s endpoints stay alive through the joins that the ? exit performs, so the live-endpoint join rule counts them. A generic signature names the job as F. A nongeneric one writes its signature, such as SpawnError[OnceClosure[[], list[i64]]], alone or in a sum; ? resolves that member to the job it propagates, as for a returned closure. Each such member names one producer, so start several jobs from one factory function or a generic helper.

Job-free errors

with spawn(job).drop_job()? as task: gives up the job instead. drop_job() converts Result[(), SpawnError[F]] to Result[(), SpawnError[()]]: on failure it drops the job and its captures at the start, and the error keeps its variant and code, SpawnError[()].Unavailable((), code). When applied directly to spawn, the compiler constructs this job-free error in the failed-start block, without materializing an intermediate job-carrying result in the caller frame. On success it changes nothing. One SpawnError[()] member covers every job, so a function can start distinct closures and propagate one precise error:

def main() -> Result[(), IoError | SpawnError[()]]:
    with spawn(def once () -> i64: 1).drop_job()? as first:
        print(first.join())?
    with spawn(def once () -> str: "two").drop_job()? as second:
        print(second.join())?
    Ok(())

Because the job is gone before ? runs its exit’s joins, its endpoints cannot block them. The conversion copies a tag and the code and allocates nothing. drop_job() also applies to a SpawnError[F] value, or a Result carrying SubmitError[F], as for pool submissions. Erasing the error into Failure drops the job too, but loses the error type; use it only when no precise error fits.

After success, only the worker owns the captures. One-shot invocation moves them into the body; reusable invocation borrows them once and then drops the environment. Join transfers the result exactly once; automatic join drops an unclaimed result. Job storage and the ownership handoff require no heap allocation.

Bounded channels

channel[T](capacity: Positive[i64]) -> Result[tuple[Sender[T], Receiver[T]], AllocError] constructs a bounded multi-producer, multi-consumer FIFO queue. Capacity must be a Positive[i64]: positive literals convert at compile time; computed counts use Positive.try_from(capacity)?. Zero and negative literals are compile errors. Size overflow or exhaustion returns AllocError. Failed construction exposes no partial queue.

OperationResultBehavior
sender.send(value)Result[(), SendError[T]]Wait for space or receiver disconnection
sender.send_nowait(value)SameReturn Full(value) if no space
receiver.recv()Result[T, RecvError]Wait for a message or sender disconnection
receiver.recv_nowait()SameReturn Empty if no message is ready
endpoint.share()Same endpoint typeCreate another owner of the queue

Methods observe endpoints; mut is unnecessary. Endpoint assignment moves; share() explicitly retains the queue. Copy and equality are unsupported. Sending moves affine messages. SendError[T].Disconnected(value) and Full(value) own the complete unsent value; blocking send never returns Full. Receiving transfers one message to one receiver. Shared receivers compete rather than broadcast. Accepted messages follow insertion order, but concurrent senders and waiters have no predetermined or strictly fair order.

Construction uses one allocation. Sharing, send, receive, waits, disconnection, and error handling allocate nothing. nowait avoids waiting for queue space/data but can contend on synchronization; it does not guarantee immediate execution. Timed operations and receive selection are described separately.

Disconnection and cleanup

Dropping the last sender wakes receivers, which drain buffered messages before returning RecvError.Disconnected. Dropping the last receiver wakes senders, returns their pending messages as errors, and drops queued messages.

Endpoints are owned context managers. with receiver as inbox: moves the handle into the scope, lends &Receiver[T], and drops that owner on every normal exit. Other shared handles remain independent owners.

Workers must handle disconnection and terminate.

Joins and live endpoints

A worker waiting for queue space is released only by a receive or by the drop of the last receiver. A join that waits for such a worker while the joining function keeps a receiver alive, and does not receive, never returns. The compiler therefore rejects a join that could wait this way:

A scoped task whose worker can hold Sender[T] cannot be joined while a live binding of the joining function can hold Receiver[T].

Likewise, a worker that can hold Receiver[T] cannot be joined while a live binding of the joining function can hold Sender[T]: an empty queue could leave the worker waiting forever for a message or the last sender’s drop.

The rule applies to task.join() and to the automatic join on every exit from the task’s with block: its end, return, ?, break, continue, and the failed start of a later task in the same statement. A task that task.join() has consumed on every path is not joined again at exit. A join is likewise rejected while the joining function holds a lock on a Mutex[T] that the worker can reach; see mutexes.

These patterns satisfy the rule on every path, and keep satisfying it when the block gains early exits. They are the canonical form; the tutorial teaches them in avoid waits that hang.

  1. Move each endpoint into the worker that uses it, as a named argument: spawn(produce, sender).drop_job()?. A failed start returns its owned arguments inside SpawnError; drop_job() drops them before ? leaves.

  2. Guard each endpoint the function keeps as the last manager of the task’s with. Contexts exit in reverse order, so every exit drops it before the join:

    with spawn(produce, sender).drop_job()?, receiver as inbox:
        ...
    with spawn(consume, receiver).drop_job()?, sender as outbox:
        ...
    

    Use a new name for the guard binding: with outbox as outbox: is a duplicate binding. The blocked-join diagnostic suggests an unused name, adding a numeric suffix when inbox or outbox is already bound. Use that guard binding for endpoint operations inside the block.

  3. To join explicitly, end the endpoint’s guard first. Open the guard before any statement that can leave the block:

    with spawn(produce, sender).drop_job()? as task:
        with receiver as inbox:
            ...
        result = task.join()
    

    drop(receiver) before the join also satisfies the rule, but only on paths that reach it: an earlier ? still leaves with the receiver alive. Moving the receiver into another worker or function likewise ends the binding.

  4. For several producers, give the receiver to a consuming worker started first, and start the producers inside a sender guard:

    with spawn(consume, receiver).drop_job()? as consumer:
        with sender as outbox, spawn(produce, outbox.share()).drop_job()?, spawn(produce, outbox.share()).drop_job()?:
            pass
        total = consumer.join()
    

    One receiver guard after several ? starts is rejected: a failed later start joins the earlier tasks before the guard exists. Alternatively, start the later producers in a function that takes the receiver by value.

  5. Call drop_job() on a start the function will not retry: spawn(job).drop_job()? or spawn(produce, sender).drop_job()?.

  6. Wait in a statement of its own, result = task.join(), and move an endpoint into a value or return it only after the block.

  7. Start tasks and pools outside lock blocks; see mutexes.

  8. Cancel a token that a worker waits for on every exit. Like an endpoint guard, stop.cancel_on_exit() goes directly after the task or pool:

    with spawn(serve, stop.share()).drop_job()?, stop.cancel_on_exit():
        ...
    

    See cancel on every exit.

Pools follow the same patterns, with pool.submit(...) in place of the start; see shutdown and result waits.

What can hold an endpoint

A worker can hold a sender that is an argument or capture, a field or element of one, behind a reference, or a message of a channel it holds. A binding can hold a receiver the same way, except that a local reference is ignored because its owner is checked instead. A &Receiver[T] parameter, or a parameter that refers to a value holding one, always counts: its owner outlives the call. The same holder rules apply to receiving workers and surviving senders, including borrowed sender parameters, aggregates, and concrete callable captures. Other shared handles count separately, and a drop on only one branch does not release a handle on the other paths.

Besides live bindings, these values hold the endpoints moved into them:

HolderHeld throughRejected exampleFix
The error that ? propagatesThe function’s return, after the exit’s joins and pool shutdownswith spawn(consumer)? as task, spawn(producer)?: where the producer closure captures the senderspawn(producer).drop_job()?
The value a return or final expression hands backThe joins and pool shutdowns of every block it leavesreturn Err(sender) inside the consumer’s blockDrop the endpoint in the block, or return it after the block
A statement’s expression and the value it buildsUntil the statement stores, drops, or tests the valuepair = (sender, task.join()) inside the consumer’s blockJoin in an earlier statement
The final expression of a match case used as a valueThat case’s cleanupA case whose value is an endpoint, ending a block that the endpoint blocksProduce the endpoint after the block

A failed closure start’s SpawnError[F] retains the unstarted job and every endpoint it captured, so the first row also applies to an enclosing task block. Erasing the error into Failure drops the job too, as a last resort.

The returned-value and expression checks follow the bindings that the expression moves, not its type, so return Ok(()) of Result[(), Sender[i64]] is accepted. A tuple, class, enum payload, Some, or call argument that receives a moved endpoint counts. The check is conservative: an endpoint moved into a call inside the expression counts even if the call consumes it, so return Ok(consume(sender)) is rejected; call consume(sender) first. A wait nested in an expression, such as task.join() or future.result()?, is rejected while an operand evaluated before it holds a matching endpoint.

The expression check covers bindings, assignments, unpacking, discarded expressions, yield, if, elif, and while conditions, for iterables, match subjects, and with managers. A wait inside a called function is seen only for a future or pool passed to the call; a scoped task cannot be passed to a function.

The diagnostic reports the join or the with statement, each binding, propagated error, returned value, or unfinished expression that holds a matching endpoint, the early exit that reaches the join, and, for an explicit join, where the task starts.

Pool shutdown checks sending jobs against surviving receivers. See shutdown and live receivers.

What the rule does not decide

Endpoints are matched by message type, not by queue, and the compiler does not count messages. These safe programs are rejected:

  • the parent receives everything the worker sends, or drains to disconnection, through a receiver that is still alive at the join;
  • the live receiver belongs to a different queue of the same message type;
  • the worker sends only with send_nowait or send_timeout, or its sends always fit the queue’s capacity;
  • another thread also receives from the queue;
  • the receiver’s caller-side owner would have received.

The receiving-worker rule likewise rejects a sender for an unrelated queue of the same message type, a worker that only receives with recv_nowait or recv_timeout, and a worker whose receives are already satisfied by buffered messages or another thread. It does not count sends or receives.

The rule is not a deadlock-freedom guarantee. It does not cover:

  • an endpoint captured by a Callable or boxed callable held by the joining function, whose captures have no type;
  • endpoints held by the function’s callers through bindings it was not passed;
  • a worker blocked for any other reason.

Dropping or guarding the surviving endpoints removes the rejection. Workers must still handle disconnection and terminate.

Messages

Messages must be concrete wholly owned storable values satisfying the worker checks, even for a channel used on one thread. Recursive data and sender-based reply paths are supported, but recursive message-type paths back to a receiver are rejected: a queue retaining its last receiver could prevent destruction. This conservative check also rejects noncyclic instances of such a type. Zero-capacity rendezvous is unsupported.

Executors and shutdown

ThreadPoolExecutor(max_workers: Positive[i64], queue_capacity: Positive[i64]) returns Result[ThreadPoolExecutor, PoolError]. Both counts are Positive[i64]; positive literals convert at compile time, and computed counts use Positive.try_from(count)?. Construction starts all fixed native workers before success and provides a bounded FIFO queue. Partial startup failure stops and joins workers already started. PoolError has Allocation(AllocError) and Thread(ThreadError).

OperationResultBehavior
pool.submit(job)Result[Future[T], SubmitError[F]]Move an owned zero-argument job; wait for space
pool.submit(function, arguments...)Same, with a bound job typeBind a statically named function and owned arguments
pool.submit_nowait(...)SameReturn Full(job) if the queue is full
pool.cancel_pending()()Cancel every queued job; running jobs continue and the pool stays open
pool.shutdown()()Stop accepting, drain accepted work, join workers

cancel_pending() does not wait for running jobs. It removes the queued jobs, so their queue slots are free for later submissions; their futures report FutureError.Cancelled, and each job’s captures are dropped with its future. To stop without running queued work, call pool.cancel_pending() and then pool.shutdown(). shutdown() takes no argument; shutdown(True) is rejected with a diagnostic naming cancel_pending().

Methods observe the pool; mut is unnecessary. Executors are affine and do not support copying or equality. Jobs may differ in concrete argument/result types on one pool. Reusable closures run once then release their environment; one-shot closures transfer captures into their body. Named submissions infer ordinary argument types and return a callable bound job on failure.

Inputs and results must be wholly owned and satisfy the worker checks. Jobs share state through Shared[T] handles: pass each job its own handle.share(). Borrowed captures/arguments, generator frames, and uncertified indirect effects are rejected. Executors and futures cannot enter workers.

User jobs must own their inputs because a future can outlive the borrow that would feed it. Parallel pipelines are the one internal exception: .par chunk jobs read a list’s rows in place, which is sound because every .par terminal waits for all submitted jobs before returning, including after a failed submission. User-facing scoped pool jobs, a deferred B22 extension, would build on the same guarantee. Channel protocols can still deadlock.

SubmitError[F] has Full(job), Shutdown(job), OutOfMemory(job), and CapacityOverflow(job). Each owns the complete unstarted job for retry, synchronous invocation, or drop. A named submission’s job is a one-shot closure binding the arguments, so pool.submit(stopped, token)? can propagate as SubmitError[OnceClosure[[], bool]]; a signature names jobs as for owned starts. pool.submit(job).drop_job()? instead drops the job of a rejected submission at the call and gives SubmitError[()] with the same variant, such as Shutdown(()); one such member covers every job, as for job-free start errors. Blocking submit never returns Full. Shutdown is reported before job allocation; waiting for space precedes allocation and transfer. submit_nowait can contend on synchronization.

Construction uses one allocation; each accepted job needs one fallible cell. Status, cancellation, result transfer, waiting, and shutdown allocate nothing.

Lifetime

with ThreadPoolExecutor(workers, capacity)? as pool: owns the executor and lends &ThreadPoolExecutor. Every normal exit drains accepted work and joins workers; ordinary drop does the same. Explicit shutdown is idempotent and always waits. Detachment and wait=False are unsupported; running jobs are not forcibly killed. Tokens permit cooperative stopping; a shutdown that could wait for a job holding a token before the function’s later cancel is rejected, and with ThreadPoolExecutor(...)? as pool, stop.cancel_on_exit(): cancels it on every exit. See cancel on every exit.

Workers must finish their communication before the scope can close. See futures for result ownership and parallel operations for batched work.

Shutdown and live receivers

Shutdown waits for every accepted job, and a job waiting for queue space is released only by a receive or by the drop of the last receiver. The compiler therefore applies the task join rule to pools:

A pool cannot be shut down while a live binding of the function can hold Receiver[T], once the function has submitted a job that can hold Sender[T].

A pool shuts down at pool.shutdown(), including through a &ThreadPoolExecutor, at drop(pool), when its binding is reassigned, and wherever its owner leaves scope: the end of its with block or of the function, return, ?, break, and continue. A call that receives a pool, by value or by reference, counts as a shutdown when a function it reaches, directly or through further calls, calls shutdown() or owns a pool. A call through a Callable, closure, or boxed callable value, or to a function that makes one, is assumed to shut down a pool passed to it. Locals are released last declared first, so a receiver bound after the pool is dropped before the pool shuts down.

Jobs are not matched to pools and are never assumed to have finished. A job submitted with submit or submit_nowait counts at every later shutdown in the function, and at every shutdown in a loop that contains the submission. A submit(...)? or submit(...).drop_job()? that fails has submitted nothing on the path that leaves. map, map_result, and reduce_tree finish their jobs before returning and do not count. What a job and a binding can hold is defined as for tasks.

The recommended patterns for tasks apply. When the function does not wait for the job’s result inside the block, guard the receiver as the last manager of the pool’s with, so every exit drops it before the pool shuts down:

with ThreadPoolExecutor(2, 8)? as pool, receiver as inbox:
    pool.submit(produce, sender).drop_job()?
    ...

That guard is still alive at a future.result() inside the block, which its own check rejects. To wait for the result, end the receiver’s guard first:

with ThreadPoolExecutor(2, 8)? as pool:
    future = pool.submit(produce, sender).drop_job()?
    with receiver as inbox:
        ...
    result = future.result()

A channel created inside the pool’s block, and a drop(receiver) on every path to the shutdown, also satisfy the rule. The holders of a task join apply too: a returned Ok(receiver), an unfinished pair = (receiver, future.result()?), or a match case whose value is receiver holds it through the shutdown. The diagnostic reports the shutdown or the pool’s binding, each binding, returned value, or unfinished expression that holds a matching receiver, where the job is submitted, and the early exit that reaches the shutdown.

The rule has the task rule’s false positives and gaps. It also rejects these safe programs:

  • the job has finished, for example after future.result() returned;
  • the job was submitted to a different pool than the one shutting down;
  • a submit failure handled without ? returned the job to the caller.

It does not cover:

  • a job submitted by another function, through a &ThreadPoolExecutor or a pool it returns;
  • a pool passed to a call inside a class, tuple, or collection;
  • a map, map_result, or reduce_tree callback that waits to send.

Blocking future retrieval has its own check at result(), before any later receiver cleanup.

Shutdown and held locks

A job waiting for a mutex this function holds would keep the shutdown waiting too. Submission therefore also records the guarded types of the Mutex[T] values a job can reach, including through Shared handles:

A pool cannot be shut down while the function holds a lock on a Mutex[T], once it has submitted a job that can reach a Mutex[T] of the same T.

Shutdown points, job counting, and the false positives are those of the receiver rule; mutexes are matched by type, as for joins under a lock. End the lock’s with block before the pool shuts down.

Futures

Executor submission returns an affine Future[T] with one result consumer. Futures can move and live in collections, classes, and enums, but cannot be copied, compared, or passed into worker code. There is no shared-result handle, result cloning, async/await, or event loop.

OperationResultOwnership and waiting
future.result()Result[T, FutureError]Consume the future and wait for its result
future.done()boolObserve completion or cancellation
future.cancel()boolPrevent pending work from starting
future.wait_timeout_ms(timeout_ms: u64)boolObserve completion/cancellation within a millisecond budget

Methods other than result observe their receiver without needing mut. Cancellation succeeds only before execution begins. Repeated cancellation of a cancelled job returns True; running or completed jobs return False. Successful cancellation immediately makes done() true and result() return Err(FutureError.Cancelled) without waiting. It does not immediately reclaim queue space: captures drop after both the queue and public future release them.

Dropping a future neither cancels nor waits. Its job still finishes; the worker drops an unclaimed result. A future surviving executor shutdown retains the completed result until retrieval or drop. Futures do not retain the executor and cannot prevent shutdown. Mutable results transfer exactly once.

An application Result remains nested: a worker returning Result[T, E] yields Future[Result[T, E]]. The first ? on result() handles cancellation; the next handles application failure. Retrieval, cancellation, status, and error transport allocate nothing. Timed observation does not cancel or consume work; see timeout rules.

Scoped Task.join() returns T directly because scoped tasks cannot be cancelled. Future.result() needs its outer Result to report cancellation; for an application Result, (future.result()?)? deliberately handles two different failure layers.

Result waits and live receivers

result() cannot wait for a locally submitted job holding Sender[T] while the calling function still holds Receiver[T]. A full queue could keep the job and the result wait blocked forever. End the receiver’s own guard before retrieving the result:

with ThreadPoolExecutor(1, 1)? as pool:
    future = pool.submit(produce, sender).drop_job()?
    with receiver as inbox:
        ...
    result = future.result()

A guard alongside the pool, with ThreadPoolExecutor(1, 1)? as pool, receiver as inbox:, disconnects only at pool shutdown, which is too late for a result wait inside the block. A drop(receiver) on every path to the wait also satisfies the rule. See the recommended patterns.

The diagnostic points to the wait, receiver holders, and submission. Tracking follows local future moves, ?, unwrap(), drop_job(), extraction from submission results, and calls that return a future. Separate futures with the same result type keep their own job origins. Receivers dropped on every continuing path are accepted. done(), cancel(), and wait_timeout_ms() are unaffected, and result retrieval still consumes the future and preserves both cancellation and application errors.

The same wait is rejected while the function holds a lock on a Mutex[T] and the job can reach a Mutex[T] of the same T, for example through a Shared[Mutex[T]] share: the job could be waiting for that lock. End the lock’s with block before waiting. Mutexes are matched by type.

This is a conservative check, not a proof of deadlock freedom: endpoints are matched by message type, not channel identity. Completion, cancellation, queue capacity, and earlier receives do not discharge the check. Assignments to the same local are conservatively combined.

A call that receives the future, by value or by reference, is checked as a wait at the call when a function it reaches, directly or through further calls, calls result() on a Future[T] of the same T. The diagnostic names the call, and finished(future) below is rejected:

def finished(future: Future[()]) -> bool:
    future.result().is_ok()

def main() -> Result[(), AllocError | PoolError | SubmitError[OnceClosure[[], ()]]]:
    sender, receiver = channel[i64](1)?
    with ThreadPoolExecutor(1, 1)? as pool:
        future = pool.submit(produce, sender)?
        ok = finished(future)   # rejected: `receiver` is still live
        drop(receiver)
    Ok(())

Matching by type means a callee that waits for a different Future[T] also counts. A call through a Callable, closure, or boxed callable value, or to a function that makes one, is assumed to wait.

A future returned from a call keeps its jobs. When the call’s result can hold a Future[T], directly or in a Result, Option, or other aggregate, it is assumed to hold every future of the same T that its arguments carry, and every job with a T result that the callee, or a function it reaches, submits. Both waits below are rejected like waits for a local submission:

def submitted[T, F](result: Result[Future[T], SubmitError[F]]) -> Result[Future[T], str]:
    match result:
        case Ok(future):
            Ok(future)
        case Err(_):
            Err("submission failed")

def start(pool: &ThreadPoolExecutor, sender: Sender[i64]) -> Result[Future[()], str]:
    submitted(pool.submit(produce, sender))

def main() -> Result[(), AllocError | PoolError | str]:
    with ThreadPoolExecutor(1, 1)? as pool:
        sender, receiver = channel[i64](1)?
        future = submitted(pool.submit(produce, sender))?
        ok = future.result().is_ok()   # rejected: `receiver` is still live
        other_sender, other = channel[i64](1)?
        later = start(&pool, other_sender)?
        done = later.result().is_ok()  # rejected: `other` is still live
        drop(receiver)
        drop(other)
    Ok(())

The result is assumed to hold those futures even when the callee drops or waits for one of them. A call through a value keeps only its arguments’ jobs: jobs submitted by a callable, closure, or boxed callable value, or by a function compiled elsewhere, are not seen. Provenance is not tracked through collections, class fields, or multi-field aggregate storage; indirect submissions and blocking mapping callbacks retain the executor limits.

Ordered parallel mapping

pool.map(worker, inputs) -> Result[list[U], PoolMapError] accepts a statically named function or a concrete reusable immutable closure (T) -> U, where U is non-unit and owned storable data. It consumes an owned list[T] or copies an integer range[T] normally. Generic worker types are inferred. Ordinary worker restrictions apply.

A closure moves into the operation and must wholly own its captures. One environment is shared by all jobs and remains pinned until every accepted job finishes, including on empty input, allocation failure, worker error, and shutdown. It drops once on the owning thread before the map call returns. Captures are neither copied per input nor given separate heap environments. Mutable and one-shot closures cannot be shared across jobs; borrowed closures belong in scoped spawn. Concrete closures passed through an owned generic F: Callable[[T], U] parameter remain eligible. Indirect code-address Callable values lack worker-effect evidence and are rejected, even if initialized from a named function; pass that function directly instead.

The result preserves input order regardless of completion order. Mapping is eager: output capacity is reserved before execution, and at most max_workers + queue_capacity jobs remain outstanding. Each job cell can fail allocation. Memory is the output list plus that bounded window, rather than one future per input.

PoolMapError has Allocation(AllocError) and Shutdown. Infrastructure failure stops submissions, waits for accepted work, and drops partial outputs and remaining inputs. The input list is consumed on every outcome; previous effects remain visible. Allocation failure leaves the pool usable. A closed pool returns Shutdown, including for empty input. Worker Result values remain ordinary list elements.

Borrowed inputs, generators, multiple iterables, and streaming output are unsupported.

Fallible workers

pool.map_result(function, inputs) instead accepts (T) -> Result[U, E] and returns Result[list[U], ParallelError[E]]. U must be non-unit; inputs, outputs, and errors meet owned job eligibility. ParallelError[E] has Allocation(AllocError), Shutdown, and Worker(E), with allocation-free error transport.

Success follows input order. If several workers fail, the lowest input index’s error wins. A failed submission stops further submission; an earlier accepted input’s worker error takes precedence over that submission failure. Allocation failure before any submission returns Allocation; a closed pool returns Shutdown, including for empty input.

On observing a worker error, submission stops and all accepted jobs are drained. Partial successes, later errors, and unsubmitted inputs drop exactly once. Some later inputs may already have run, and effects are not undone. Workers must not depend on every input being scheduled. Output capacity and the bounded job window are reserved before execution; submitted cells remain fallible.

A parallel pipeline builds on map and map_result: iter(items).par(&pool) maps and filters contiguous chunks on the pool. Unlike pool.map, it also accepts a borrowed list, iter(&items), whose chunks read the caller’s rows in place.

Fixed-tree reduction

pool.reduce_tree(function, inputs) -> Result[Option[T], PoolMapError] accepts a statically named (T, T) -> T worker and an owned list or integer range. A named (T, T) -> Result[T, E] worker instead gives Result[Option[T], PoolMapError | E], preserving the ordinary public callback error (including managed, affine payloads). Generic types are inferred and worker eligibility applies. Lists are consumed on every outcome; ranges retain copyable semantics. Empty input returns Nothing; otherwise the result is Some. No identity is inserted or copied.

Each round combines adjacent pairs in left/right order; an odd final element advances unchanged. Five inputs use f(f(f(a, b), f(c, d)), e). Worker count, queue capacity, and completion order do not change the tree. A successful nonempty input of length n invokes the worker exactly n - 1 times.

The rounds define the grouping, not the schedule. For infallible callbacks, one job reduces an aligned run of adjacent inputs, a power-of-two count chosen so there is about one run per worker, through as many rounds as the run spans; jobs then combine the run results pairwise. A job’s calls are serial, and runs proceed independently, so calls from different rounds can overlap in time.

This reassociates operations rather than performing a left fold. Floating-point results can differ from serial accumulation; pure workers produce the same result for the same input and target across pool sizes. Effect order is unspecified. Workers must not depend on another pair being scheduled. Workers retain their own arithmetic contracts. General integer addition uses a callback returning Result[T, OverflowError[T]] with explicit Ok(left + right). A caller’s checked: or unchecked: mode never changes the callback or API type.

Fallible callbacks use separate adjacent-pair rounds, not multi-level run jobs. All accepted jobs drain before returning. Callback errors are observed in level order, then left-to-right pair order, never completion order. No later level runs after an error. An accepted callback error takes precedence over a later submission failure; without such an error the precise allocation or shutdown failure is returned. Unused errors and intermediate values drop once. This adds per-level synchronization and potentially more jobs than the infallible optimized path, without changing successful grouping or replaying callbacks. Empty and singleton input never invoke the callback. A parallel pipeline’s reduce groups by chunk instead, so its result can depend on the chunk count; use reduce_tree when one grouping is needed for every pool. Unlike reduce_tree, a pipeline can read a borrowed list: iter(&xs).par(&pool) borrows xs and maps its items to owned values before reduce.

Empty and singleton reductions allocate nothing. Larger inputs need one fallible scratch buffer proportional to input length, including for ranges, a bounded job window, fallible job cells, and, when a job reduces more than one input, one small buffer of per-job run descriptors and pair scratch. Pair arguments transfer without tuple allocation or implicit copying.

Infrastructure failure stops submission, drains accepted jobs, and destroys intermediate and unsubmitted values. A closed pool returns Shutdown even for empty input. When the callback returns exactly T and T is itself a Result, those Results remain ordinary elements: only a distinct Result[T, E] callback is fallible.

Mutexes

std.sync.Mutex[T] holds a value that several workers change through shared references or shared owners, one lock holder at a time. Use it when several values must change together to keep an invariant; a single independent counter or flag is an atomic. The operations are listed in std.sync.

Locking

with place.lock() as value: borrows the mutex at place shared, waits until no other thread holds the lock, and binds value: &mut T for the block. as may be omitted. The place is a name, a field, an element, or a dereference, including a &Mutex[T] parameter, or a Shared[Mutex[T]] handle, which locks the mutex through *handle; a temporary is rejected. lock() anywhere other than a with manager is a compile-time error naming that form.

The lock is released on every exit from the block: its end, return, ?, break, and continue. Comma-separated managers nest, so with a.lock() as x, b.lock() as y: takes a then b and releases b first.

value is an ordinary exclusive reference. Its loan belongs to the lock, not to the mutex’s owner, which stays shared-borrowed until the block ends: the mutex cannot be moved, replaced, or consumed while locked. Like any local reference, value cannot be stored, returned, captured by an anonymous function, or used after the block.

into_inner() consumes an owned mutex and returns its value without locking. Ownership proves that no thread holds or waits for the lock.

Waits the compiler rejects

A lock waits as long as another holder keeps it. The compiler rejects these patterns that would wait forever:

  • Relocking in the block. Locking a place that an enclosing lock block in the same function already holds is an error. Places are compared as written: m and *m match.
  • Nesting locks that could alias. Locking a Mutex[T] inside a lock block that holds another Mutex[T] of the same T is an error when either place is reached through a reference, a Shared handle, or a computed index (cells[i]). Such places can name one mutex (r = &a, b = a.share(), i == j), and only types are compared. Two owned places built from local names, fields, and constant indices, such as a and b or cells[0] and cells[1], are distinct mutexes and may nest. Otherwise wrap one value in a distinct class. A relock inside a called function is not detected and deadlocks at run time.
  • Joining under a lock. A scoped task cannot be joined, by task.join() or by leaving its with block, while the joining function holds a lock on a Mutex[T] and the worker can reach a Mutex[T] of the same T: as an argument or capture, a field or element of one, or behind a reference. The worker could be waiting for that lock. Mutexes are matched by type, like the channel rule. Start the task outside the lock: with spawn(...)?: around with m.lock(): unlocks before joining. A join inside a function called while the lock is held is not detected and deadlocks at run time.
  • Waiting for a pool job under a lock. A job submitted to a pool records the Mutex[T] types it can reach, including through Shared handles. future.result() on its future, and a shutdown of any pool after it was submitted (shutdown(), drop, reassignment, or the end of the pool’s scope), is an error while the function holds a lock on a Mutex[T] of a recorded T. A future is traced like the channel rule traces it, through local bindings and calls that return it.
  • Yielding under a lock. yield inside a lock block is an error: a suspended generator would keep the lock indefinitely.

Locking two mutexes in different orders on different threads can still deadlock; take nested locks in one fixed order.

No poisoning

A trap terminates the process without unwinding, so no program continues after a holder fails mid-update. A Plenty mutex therefore has no poisoned state and lock() has no error result.

Not provided

There is no try_lock, timed lock, reentrant lock, condition variable, read-write lock, or fairness guarantee, and no unlocked access through &mut Mutex[T] (Rust’s get_mut); lock it instead. Executor and pool jobs own their inputs and cannot borrow a mutex; give each job a share of a Shared[Mutex[T]] instead.

Lowering

The state word is 0 when unlocked, 1 when locked, and 2 when locked with possible waiters. Lock is an inline compare-exchange from 0 to 1; on failure the runtime exchanges in 2 and sleeps on the word with a Linux futex until an exchange finds 0. Unlock is an inline exchange to 0 that calls the runtime to wake one waiter only if it replaced 2. See the runtime mechanisms.

Cancellation tokens

CancellationToken() -> Result[CancellationToken, AllocError] creates an affine owner of a shared cancellation state using one fallible allocation. Tokens may cross threads and live in owned containers; assignment moves them. Copy and equality are unsupported.

OperationResultBehavior
token.share()CancellationTokenAnother owner of the same state
token.cancel()()Permanently request cancellation and wake all token waiters
token.is_cancelled()boolObserve the request
token.wait()()Wait until requested
token.wait_timeout_ms(timeout_ms: u64)boolTrue if requested within the millisecond budget
with token.cancel_on_exit():guardCancel on every exit from the block; see below

All methods observe their receiver without requiring mut and allocate nothing. Cancellation is idempotent. Dropping a handle does not request cancellation; the last owner releases the state. Timeout does not consume or reset it. wait_timeout is also the cancellable sleep; std.time supplies its budget from a Duration.

Workers must check the token at safe boundaries, finish required cleanup, and return normally. A token imposes no return type or error variant. It does not change future state by itself: Future.cancel only prevents pending work from starting.

Ordinary blocking channel operations and foreign calls do not observe tokens. Use timed waits or an explicit shutdown message when work must periodically regain control. Scope exit and executor shutdown still join running workers, so cancellation-aware code must actually return, and the code that cancels must reach its cancel on every exit; see cancel on every exit.

Cancel on every exit

A task or pool job that waits for a token keeps its join, or its pool’s shutdown, waiting until someone cancels the token. token.cancel_on_exit() is a with guard that cancels the token on every exit from its block: its end, return, ?, break, continue, and the failed start of a later manager in the same statement. It holds a share of the token, allocates nothing, has no value to bind with as, and is valid only as a with manager.

Contexts exit in reverse order, so put the guard directly after each task or pool that holds the token:

with spawn(serve, listener, stop.share()).drop_job()?, stop.cancel_on_exit():
    ...
with ThreadPoolExecutor(2, 8)? as pool, stop.cancel_on_exit():
    ...

To join inside the block, or to wait for a job’s result(), cancel first: either call stop.cancel() before the wait, or end a nested guard before it:

with spawn(serve, stop.share()).drop_job()? as task:
    with stop.cancel_on_exit():
        ...
    task.join()

A worker that cancels a token when it finishes, so that its caller can stop waiting, puts its body in with stop.cancel_on_exit():.

Waits the compiler rejects

The compiler rejects a wait that could skip the function’s own later cancel:

When a function cancels a token binding, a join of a task, a result() of a job, or a pool shutdown that waits for work holding that token must come after a cancel on every path that reaches it.

A task or job holds the token when its arguments move, share or borrow that binding, including a share() of it bound to another name. The rule covers task.join(), future.result(), drop(pool), and the automatic join or shutdown on every exit listed above. A guard placed before the task in the header is rejected too, since it cancels after the join. The diagnostic names the exit, the job’s start where it helps, and the later cancel, and suggests the guard.

A worker is held to its own final cancel: when a function’s normal end always cancels a token parameter, an earlier return or ? that skips the cancel is rejected. A conditional cancel is not checked.

What the check does not decide

The check is local to one function and linear in its size. It does not see:

  • tokens reached through a field, a Shared[T], a class value built before the start, or a closure capture;
  • the tokens that map, map_result or reduce_tree jobs hold;
  • a cancel made by a called function;
  • a worker’s deadline or timed wait, so a worker that stops by itself is still checked;
  • which pool runs which job: a function-exit shutdown counts every owned pool;
  • every path through a loop: a loop body is walked at most twice.

A program outside these forms can still wait forever; the guard is the habit that avoids it.

Timed waits and selection

Prefer the typed deadline waits in std.sync for cancellable application code. They take &Deadline and &CancellationToken. The low-level primitives below use u64 milliseconds using a monotonic total budget, including repeated wakeups. Zero checks immediately. Readiness or disconnection at the final check wins over timeout. Lock contention, cleanup, and scheduling may delay return beyond the budget; timeouts are not hard execution deadlines. std.time converts a Duration or the remainder of a shared Deadline to this budget.

OperationResult
sender.send_timeout_ms(value, timeout_ms)Result[(), SendTimeoutError[T]]
receiver.recv_timeout_ms(timeout_ms)Result[T, RecvTimeoutError]
future.wait_timeout_ms(timeout_ms)bool
token.wait_timeout_ms(timeout_ms)bool

The original _timeout spellings remain compatibility aliases.

SendTimeoutError[T] has Disconnected(value) and TimedOut(value), both returning ownership of the unsent message without copying or allocating. RecvTimeoutError has Disconnected and TimedOut; buffered messages remain receivable after the last sender drops. All channel waits borrow endpoints.

A future wait observes completion or cancellation without consuming its result or cancelling work. A token wait observes the permanent cancellation request without consuming or resetting it. Neither timed observation allocates.

Receive from either of two channels

OperationResult
select_recv(&first, &second)Result[Selected[A, B], SelectError]
select_recv_nowait(&first, &second)Same
select_recv_timeout_ms(&first, &second, timeout_ms)Same

Selection borrows both receivers and transfers exactly one message. Selected[A, B] has First(A) and Second(B) variants. Shared & markers are optional, and two handles sharing one queue are accepted. The unselected queue keeps its messages. Ordinary message eligibility applies.

SelectError.Empty means neither had a message during a nonblocking check; TimedOut means the budget expired. Disconnected means both queues are drained and have no senders. One disconnected queue does not stop waiting for the other. Blocking selection returns only Disconnected.

The first receiver has priority whenever both are ready; sustained traffic can starve the second. Selection receives a message before returning, rather than reporting readiness that another receiver could invalidate. It allocates nothing.

Only two-channel receive selection is supported: no send cases, arbitrary channel sets, or cancellation-token case. Selected and ordinary receives compete for messages. Application protocols can still deadlock.

Native effects on worker threads

Worker certification asks every native operation a body can reach for its effect. The compiler classifies each primitive in one exhaustive table (src/effects.rs), so a new primitive cannot be added without stating its effect. Standard modules have no exemption: a std helper is certified by the same walk as user code, and a foreign call is still rejected.

EffectOperationsOn a worker
ComputeText, collections, bytes, hashing, compression, UnicodeAllowed
ClockMonotonic and wall clocks, sleepAllowed
Entropystd.secrets readsAllowed
Process streamStandard input, output and error, print, argsAllowed; no ordering between threads
Pathopen, read_text, metadata, read_dir, std.fs.root, std.fs.symlink, std.fs.canonicalizeAllowed; a relative path resolves against the working directory at the call
HandleMethods of File, ReadDir and std.fs.Root, rooted mutation, std.ready waits and selectors, operations on an open socketAllowed under the loan rules below
Environmentstd.env.get, vars, current_dirAllowed; reads only
Childstd.process spawn, wait, poll and signalAllowed; the runtime’s child table is synchronized
NetworkOpening a std.net socket, name lookupAllowed; the kernel and C library synchronize sockets and the resolver
Process globalstd.env.set_current_dirRejected
Signal dispositionstd.signal.listenRejected

Changing the working directory would change how every other thread resolves a relative path, so a worker body, helper, callback or destructor that reaches set_current_dir is rejected:

error: worker `worker` reaches `std.env.set_current_dir`: changing the working directory affects every thread's relative paths; change it on the main thread before starting workers

Catching termination signals changes how every thread’s process ends, so std.signal.listen is rejected on a worker the same way. A worker may wait on the Signals handle the main thread opened. Nothing in the language writes the environment.

When the rejected function is reached through others, the diagnostic lists them from the worker outward. A rejected type in a called function’s signature or locals names that function and its path the same way. A generic function is named by the location of its definition, and a method as Class.method:

error: worker `work` reaches `std.env.set_current_dir` through `outer` -> `Parser.parse` -> the generic function at std/error.plenty:7:5 -> `wrap`: changing the working directory affects every thread's relative paths; change it on the main thread before starting workers

Thin callable targets

A Callable stores a code address with a concrete signature. The compiler collects every address of that signature in the closed program and certifies all of their bodies, signature types, stored values and destructors. This allows named and capture-free function values, including payload variant constructors, in worker helpers such as result.map_error(Domain.Allocation). Moving a callback through parameters, returned values, classes, lists or channels preserves this check. Recursive signatures and bodies are inspected once per identity.

This is conservative certification by signature. Taking an unsafe address on the main thread can reject a worker’s use of a different function with the same signature, even when that unsafe value never enters the worker. Standard library callbacks share this candidate set: an unsafe user callback with the same signature as a std error mapper can prevent a std worker from certifying. A conditional with one safe and one unsafe candidate is rejected too. An absent address set supplies no evidence and is rejected; a function signature alone is never an effect contract. A concrete captured closure has its own body and capture evidence and can avoid unrelated signatures, while retaining all checks on its actual effects.

Foreign interfaces cannot produce or reinterpret a Callable: their scalar and opaque-pointer ABI is separate from Plenty function values. Erased BoxedCallable environments remain ineligible. The direct worker-entry APIs keep their existing requirements for statically named functions or concrete closures; certification of thin values used inside workers does not add a new spawn or parallel-worker entry form.

Native handles

File, directory streams (ReadDir), entries and metadata cross thread boundaries like any other owned value:

  • An owned handle can move into a worker, a channel or an executor job. The receiving thread closes it, explicitly or when its owner drops.
  • A scoped worker can borrow a handle. Reading, writing, seeking and closing need an exclusive loan, so the parent cannot use a handle a worker holds as &mut File, and two workers cannot hold it mutably at once.
  • A shared loan (&File) permits only operations that do not move the cursor or change the handle: the closed property, readable, writable, rooted openat operations, and std.ready waits. Several workers may wait on one shared handle.
  • A failed thread start drops each moved handle once, which closes it.

Foreign pointers (opaque types) remain ineligible: the compiler cannot know whether a foreign resource tolerates use from another thread.

Blocking and cancellation

Blocking calls stay ordinary function calls; there is no async syntax. A worker that must not block indefinitely waits with a deadline through std.ready, which makes one bounded ppoll per step on the handle and the cancellation token’s wake descriptor, so cancelling the token wakes the wait at once. A deadline that passes leaves the handle and any borrowed buffer intact; the caller decides whether to retry, close or return. A token still does not interrupt an unbounded read, write, recv or child wait.

Resource use

Each worker is one native thread, started explicitly by spawn, an executor or a parallel operation, so a program’s thread count is what it asks for. No facility starts a thread per handle or per connection; a std.ready selector lets one worker wait on many. An idle std.ready waiter does not wake until its handle is ready, its deadline passes or its token is cancelled. std.net.resolve is the exception: each lookup runs on a short-lived helper thread, at most 32 at once. Descriptors are released when their owner closes or drops them, on whichever thread that is.

Supported target: x86_64 Linux GNU. Elsewhere std.ready waits report IoError.Unsupported.

C interoperability and libraries

Trusted .plentyi modules describe foreign ABI and lifetime promises. The compiler checks declared types, ownership, and loans; binding authors and native callers must supply matching implementations, valid storage, and obey the stated aliasing rules. Foreign code can violate memory safety. No foreign unwinding or nonlocal jump may cross Plenty frames.

The native ABI targets are x86_64-unknown-linux-gnu and aarch64-unknown-linux-gnu; the compiler, runtime and library object must match. See ARM qualification for deployment limits. Calls and owned exported objects remain on their creating caller thread, with serialized access as required by the generated C contract. Ordinary pub controls Plenty imports and does not export a native symbol. Building and linking is explicit; source imports alone neither link nor load a library.

Trusted declarations and ABI types

Imports resolve name.plenty or name.plentyi; having both is ambiguous. .plentyi permits foreign declarations alongside ordinary Plenty types and wrappers:

pub opaque Image
extern def load_raw(code: i32) -> Image = "image_load"
extern def free_raw(image: Image) -> () = "image_free"
pub extern def version() -> u32 = "image_version"

extern def requires fully typed parameters, a return type, and an explicit C symbol or function-address parameter. Bodies, generics, varargs, defaults, and implicit header translation are unsupported. pub only controls source imports. Checking does not invoke native functions or inspect C headers.

The raw ABI supports i8–i64, u8–u64, f32, f64, nominal opaque pointers, and unit for void results, plus the explicit borrowing/text adapters. On x86-64 and AArch64 GNU/Linux, these match C intN_t, uintN_t, float, double, and pointers. Binding aliases can name C types: int is i32, long and ptrdiff_t are i64, and size_t is u64. Bool, aggregate values, and private Plenty layouts have no raw ABI.

opaque Name declares a nominal pointer without a pointee layout. Distinct names do not interchange. Values can be copied, compared for address equality, passed back to C, or tested with .is_null(); Name.null() constructs null. They cannot be dereferenced, converted to integers, or used in pointer arithmetic. They do not own or free their pointees. Public signatures cannot expose private opaque types. Prefer private pointers behind owning wrappers.

C symbols must be identifiers outside the main, plenty_, and __plenty_ namespaces, except the checked loading primitives. Conflicting declarations are rejected. The binding author must supply the exact ABI, lifetimes, permissions, and error/ownership rules. Callbacks and C records are unsupported.

Calling a native address

opaque Function
extern def invoke(function: Function, value: i32) -> i32 = function

The identifier after = must name an opaque pointer parameter. It selects the callee and is omitted from C arguments; remaining arguments use ordinary adapters. The binding must provide a non-null address of the exact declared signature and keep the originating code resident. Arbitrary addresses are not validated. Keep this primitive behind a checked private wrapper; it is not a source-language callable value.

Borrowed arguments and text adapters

Foreign borrowing is call-scoped. An extern def parameter &T or &mut T, where T is a supported scalar or opaque pointer, becomes C const T * or T * to initialized storage. Borrowing an opaque pointer passes a pointer to the pointer, suitable for an output parameter.

Native code must not retain these addresses or write through a shared borrow. Mutable arguments obey exclusive borrowing; native code may access exactly the declared scalar’s bytes, with no assumptions about Plenty slot padding.

Text representations

extern def inspect(text: &str as utf8) -> u64 = "inspect_bytes"
extern def length(text: &str as c_string) -> Result[u64, CStrError] = "strlen"
AdapterActual C argumentsConstruction
&str as utf8const uint8_t *data, size_t byte_lenNo allocation; existing bytes including NUL
&str as c_stringconst char *dataOne temporary terminated copy; rejects embedded NUL

Both buffers are read-only and call-scoped: C cannot mutate, retain, or return an address into them. Empty UTF-8 views have length zero and must not be dereferenced. Ordinary strings retain their counted UTF-8 representation.

Any declaration with c_string must return Result[T, CStrError], with T a supported C scalar/pointer or unit. The actual C return type is T; the adapter wraps it in Ok. Conversion runs left to right before native entry. Embedded NUL or allocation failure skips the call, releases earlier buffers, and returns Err; normal return also releases all temporary buffers.

CStrError has allocation-free EmbeddedNul and Allocation(AllocError) variants. It reports conversion failure rather than native status: wrappers must interpret native errors separately.

Returned native strings are not implicitly decoded. Returned pointers need an independent lifetime. Retained/returned buffer loans and copying arbitrary C bytes directly into str are unsupported.

Strided numeric lists and byte views

&list[T] as strided and &mut list[T] as strided, for primitive numeric T, pass three C arguments: a data pointer, size_t len, and size_t stride_bytes. Element i lives at data + i * stride_bytes. The pointer may be null for an empty list and must then not be dereferenced. The stride describes Plenty storage; it is not necessarily sizeof(T).

The adapter does not allocate or pack a temporary collection. Native code may read only each declared scalar’s bytes and may write those bytes only through a mutable loan. It must preserve padding and runtime metadata, cannot resize the list, and cannot retain or return the pointer. Use memcpy or unaligned scalar accesses if the native type requires alignment. No booleans, strings, objects, nested lists or nullable elements are admitted. Exclusive borrowing rules still apply. Current numeric lists store naturally sized elements; generated and runtime reads sign-extend narrow signed values directly, so mutable views need no post-call normalization pass. The optional dataframe engine uses this adapter for bulk transfer.

&[u8] as strided uses the same three C arguments with a byte stride of 1. It borrows a literal, a Bytes buffer, a list[u8], or a subview without allocation or copying. The native call may read exactly len bytes and must not retain the pointer. Empty views must not be dereferenced. Mutable byte views and views of other element types are not admitted by this adapter; mutable numeric lists retain the contract above.

Owning foreign resources

A public class can own a private opaque pointer and invoke its matching foreign destroy function in __del__. It moves, cannot be copied, and cleans up on ordinary scope exits and error propagation. Borrowing the class retains native ownership. The binding chooses shared or exclusive methods from the native library’s actual mutation and aliasing guarantees, not merely its use of const.

For fallible acquisition, construct an owner containing null first, then acquire into its pointer inside a Result-returning factory. Its destructor is active before any native resource exists, so partial acquisition or subsequent failure still cleans up. Class storage is inline and requires no allocation; allocation explicitly requested by fields or native acquisition retains its own contract.

The destructor guards null unless the native destroy operation accepts it, and uses the matching native release function rather than Plenty’s allocator. Native allocation failure is recoverable only if the library exposes such an error.

Transfer policies

  • If C consumes unconditionally, move the owner into the wrapper and clear its pointer before calling C, on both success and failure paths.
  • If C consumes only on success, clear the pointer on success and return the intact owner on failure, for example as Err(owner) in Result[(), Handle].
  • Borrowed calls retain ownership and expose &Handle or &mut Handle according to the permitted access. C must not retain borrowed addresses.

Raw declarations cannot prove these policies. The wrapper and native implementation must agree, including partial effects, null handling, and thread restrictions. For Plenty objects exported to C, use the generated handle contract.

Exports and generated C contracts

export def gives a top-level function an explicit C entry point in a library build. It remains an ordinary Plenty callable; pub independently controls imports. In application builds, export functions retain private native symbols.

export def add(a: i32, b: i32) -> i32 = "calc_add":
    "Add two checked integers."
    a + b

Parameters can be numeric scalars, owned exported classes, or shared/exclusive references to either. Returns can be numeric, unit, or Result[T, E], where T is numeric/unit and E is numeric, AllocError, ParseError, RangeError, Failure, or a nongeneric user enum whose variants have no payloads. Result[Class, AllocError] factories use owned handles. Bool, other managed objects, raw pointers, returned references, generic exports, and other aggregate results are unsupported; bodies can use ordinary features. Enums with payload variants and error sums such as AllocError | ParseError are rejected; sums are planned with the broader C representations (B27).

C symbols begin with the library name plus _. NAME_plenty_ and _plenty_ source names are reserved for generated metadata and compatibility adapters. Duplicate symbols, import/export collisions, and generated-name collisions fail checking or artifact validation. The same namespace applies to imported source modules’ exports. Library builds require at least one export but no main.

Scalar borrows

&i32 becomes const int32_t *; &mut i32 becomes int32_t *. Pointers must be non-null, aligned, initialized, and live for the call. Shared arguments exclude mutation through every alias; exclusive arguments cannot overlap another loan or any concurrent/reentrant access. The callee retains no storage.

Adapters read/write exactly the C type’s size, allocate nothing, and copy mutable values back on every normal return. A trap does not return and promises neither copy-back nor rollback. Generated Plenty wrappers preserve the reference types so ordinary loan checking applies.

Result outputs

Result exports return uint32_t status: 0 for Ok, 1 for Err. After input arguments come T *out_ok (omitted for unit) and E *out_error. Both must be non-null, aligned, writable, mutually disjoint, and disjoint from input loans; previous contents need not be initialized. Only the selected output is written. Err does not undo input mutations. Adaptation and Result reconstruction allocate nothing and do not expose private sum layouts.

Enum errors use uint32_t *out_error. A variant’s code is its declaration index. The builtin errors have these codes:

ErrorCodes
AllocErrorOutOfMemory = 0; CapacityOverflow = 1
ParseErrorInvalid = 0; OutOfRange = 1
RangeErrorOutOfRange = 0
FailureUnspecified = 0

Prefer a precise enum; Failure is a last resort when no precise error exists. For a user enum, the generated interface re-declares the enum, so it stays self-contained, and its wrappers return that interface type:

enum CalcError:
    Empty
    Negative
export def check(value: i64) -> Result[f64, CalcError] = "calc_check":
    ...

The header names each code once per enum as LIBRARY_ENUM_VARIANT, before the functions that use it:

/* CalcError codes, written to *out_error when a call returns 1. */
#define calc_CalcError_Empty UINT32_C(0)
#define calc_CalcError_Negative UINT32_C(1)

/* On Err, *out_error is one of the calc_CalcError_* codes. */
uint32_t calc_check(int64_t p0, double *out_ok, uint32_t *out_error);

Interfaces use the enum’s unqualified name, so two exported enums with the same name, an enum named like an export or exported class, or a constant that collides with another generated C identifier fails the build. Runtime loaders also reserve Library and load.

Status and error code are distinct: code zero still means failure when status is one. Generated headers list these codes and caller requirements. Author docstrings appear as separate escaped documentation.

Headers require serialized calls on one caller thread and prohibit unwinding or nonlocal jumps across Plenty frames. Traps and explicit unwrap failures can terminate the process; adapters invent no recovery mechanism. Keep the generated header, interface, and binary together; see contracts and library artifacts.

Owned library objects

An export returning Result[Class, AllocError] publishes an opaque owned handle. Its adapter reserves a box before invoking the Plenty body. Allocation failure skips the body, releases consumed handle arguments, and reports AllocError. A body Err frees the empty box. Class fields and layout remain private.

For library calc and class Resource, the C declarations include:

typedef struct calc_Resource calc_Resource;
uint32_t calc_create(int64_t p0, calc_Resource **out_ok, uint32_t *out_error);
void calc_Resource_destroy(calc_Resource *value);

Status zero returns one non-null owned handle; status one writes the error code without changing out_ok. Destroy each successful handle exactly once with its originating destroy function; never use free or another library instance’s function. Null destruction is a no-op. Destruction runs ordinary class and field cleanup and invalidates all aliases. No borrow or concurrent/reentrant access may remain; calls stay on the creating thread and code stays resident.

Generated .plentyi wrappers own private pointers and destroy them automatically. Wrapper construction is inline and allocates nothing. Factory failure drops an empty wrapper; ordinary moves, drop, scope exit, and ? apply. Owners cannot be copied or expose the native object’s fields. Only allocation-error class factories are supported; methods require explicit free-function exports.

Class names need distinct ASCII basenames across source modules. Handle types, destroy functions, and generated namespaces cannot collide with user exports or foreign imports. Names beginning plenty_ are reserved. For example, Thing and Thing_destroy conflict as exported classes.

Borrowing

&Resource becomes a read-only call-scoped const calc_Resource *. Shared arguments may alias, but no alias may mutate or destroy the live handle. &mut Resource becomes an exclusive calc_Resource *: no other input or alias may access it or its fields. Handles must belong to the same library instance. Borrow adaptation allocates nothing.

Ownership remains with the caller. Whole-class replacement through a reference is rejected, but field changes survive Err; errors do not roll back mutations. Inputs and output storage must not overlap. Plenty wrappers preserve these loans.

Consumption

A by-value class argument consumes its C handle on entry, including when the call returns Err. It cannot alias another input or have outstanding borrows. The caller cannot later access or destroy it. A supported class result explicitly returns a new usable handle, even when the body returns the received instance.

Generated wrappers clear their private pointers immediately before native entry, preventing double destruction. The source call consumes arguments on every path; the factory’s box-allocation failure also cleans up consumed inputs.

Embedded contracts and compatibility

Library artifacts contain the exact generated UTF-8 .plentyi contract, including its format and target. The versioned discovery function NAME_plenty_interface_v1(size_t *length) returns immutable bytes and writes their length. The bytes have no NUL terminator; callers cannot modify or free them and must keep the code resident. Discovery allocates nothing.

Contracts survive supported ELF stripping and section garbage collection once their archive member is linked. Inspection reads dedicated contract sections, not arbitrary matching strings. It never loads or executes the binary. ELF objects, shared objects, executables, and regular static archives are supported; thin/compressed archives and malformed contracts are rejected without following thin-archive paths.

Inspection validates version, namespace/section agreement, C ABI, UTF-8, and target. Limits are 512 MiB per binary, 4 MiB per contract, 16 MiB total contract data, and 256 contracts. Duplicate namespaces are rejected even if bytes agree. Extraction preserves a previous output on failure and cannot overwrite the input. See build commands for extraction and verification.

Compatibility fingerprints

Generated interfaces carry a SHA-256 fingerprint and require NAME_plenty_contract_v1_HASH before each native call. An incompatible binary lacking that symbol fails linking or symbol resolution. The guard allocates nothing; C headers declare it for optional C-side checking.

Signatures, borrowing, cleanup/adaptation rules, namespace, and target affect the fingerprint. Parameter spelling, docstrings, private layout, and bodies do not. Matching is conservative: adding an export changes the fingerprint. This is not a general stable Plenty binary ABI or a promise of compatibility between compiler/runtime builds when combining static runtimes.

Verification validates the exact contract and checks its guard symbol is defined; shared libraries must expose the guard dynamically. Metadata alone cannot pass. Older format-1 contracts without fingerprints can be extracted but must be regenerated for verification. Verification executes no library or linker code, and does not prevent later binary replacement: the normal symbol guard remains necessary.

Extracted contracts remain trusted .plentyi source. Metadata matching detects contract mismatch, not malicious code, native bugs, or authenticity.

Runtime loading

A generated runtime interface exposes load(path: &str) -> Result[Library, LoadError] and a Library method for each export. Importing it does not load native code; the consumer needs no link argument for the loaded library. Generation currently requires the original export source and differs from a linked .plentyi interface. See commands.

Methods preserve supported numeric/unit inputs and results, scalar borrows, Result error types, and owned class handles. Borrowed input mutations remain visible on Err. The generated module is checked before publication.

Loading opens the requested library, compares exact discovery metadata, requires the fingerprint guard, and resolves every export/destructor before returning a usable table. Failed lookup never exposes a partial public table; its private lookup lease closes on every path. Calls use cached native addresses.

LoadError has allocation-free OutOfMemory, CapacityOverflow, InvalidPath, OpenFailed, InvalidSymbol, MissingSymbol, and IncompatibleContract variants. Path termination requires one fallible Plenty allocation. The private lease wrapper and returned method table are inline classes; they allocate nothing. Native constructors and the OS loader may allocate independently. Cached scalar calls and adapters need no allocation beyond effects of the native callee.

Generation accepts at most 128 functions including destructors, and 240 parameter slots per method; consuming owners require additional temporary slots. Method names beginning _, new, and self, and exported class names Library and load, are reserved. Native symbols must be shorter than 512 bytes; discovery metadata must fit 4 MiB.

Owned objects and code residency

Returned class wrappers store their handle, originating identity, and exact destructor address. Inline wrapper construction allocates nothing; the native factory’s owned-handle allocation remains fallible. Factory Err cleans the empty wrapper. Consuming arguments transfer on both Ok and Err; successful class results arm a new wrapper.

Objects may outlive the Library table. Successfully loaded code and native static state remain resident until process exit, so their originating destructor remains callable. Before any class argument crosses the native boundary, methods compare its origin with their table. A different loaded instance traps with a diagnostic, even if contracts match; it is not a LoadError. Repeated loads of the same resident instance are compatible. Calls and objects stay on their creating thread.

A path containing / is absolute or relative to the process working directory; bare names follow platform search rules. Native constructors can run during open before a later check fails. Loading trusts native executable code; metadata is compatibility checking, not authentication.

Trusted loading primitives

Trusted interfaces can implement their own runtime loader with these exact versioned declarations. They are checked exceptions to the reserved plenty_ symbol namespace; opaque type names are chosen by the binding author.

opaque Lease
opaque Address
extern def open_raw(path: &str as utf8, output: &mut Lease) -> u32 = "plenty_library_open_v1"
extern def symbol_raw(lease: Lease, name: &str as utf8, output: &mut Address) -> u32 = "plenty_library_symbol_v1"
extern def close_raw(lease: Lease) -> () = "plenty_library_close_v1"
extern def check_raw(lease: Lease, discovery: &str as utf8, expected: &str as utf8) -> u32 = "plenty_library_contract_v1"
extern def require_origin(expected: Address, actual: Address) -> () = "plenty_library_require_origin_v1"

Open and lookup return zero and write output only on success. Errors are 1 (OutOfMemory), 2 (CapacityOverflow), 3 (InvalidPath), 4 (OpenFailed), 5 (InvalidSymbol), 6 (MissingSymbol), and 7 (IncompatibleContract). Paths must be nonempty without NUL; symbols must be ASCII C identifiers shorter than 512 bytes. Path conversion is fallible; lookup and error handling allocate nothing. The OS loader has independent memory management.

Successful open grants one lookup lease. Close consumes it exactly once; null is a no-op. Other operations need a live lease, and wrappers must close it on every exit, including partial lookup failure. Closing never unmaps successfully loaded code or native static state; addresses remain valid until process exit. The GNU/Linux implementation uses eager local binding. Imports alone do not open libraries, and native constructors may run before later checks fail.

Private opaque addresses can select typed indirect extern calls. The binding must verify every lookup, exact signature, ownership, and thread contract. Low-level lookup does not automatically verify metadata.

check_raw resolves the named discovery function with C signature const uint8_t *(size_t *), then compares exact length and bytes with expected. Expected metadata must be nonempty and at most 4 MiB. Null, wrong-sized, or different discovery data returns IncompatibleContract; missing discovery returns MissingSymbol. Comparison allocates nothing. Discovery must supply valid readable memory; matching cannot prove memory safety or undo native constructor effects.

require_origin compares non-owning identities. Null expected identity or mismatch terminates with the instance-mismatch diagnostic.

Standard library

The standard library is the set of modules under the reserved std namespace. They are written in Plenty or expose existing compiler facilities, and are embedded in the compiler: importing one needs no source installation, network, Cargo, or rustc. Native compilation still requires the supported linker environment. Using std.dataframe additionally requires its separately installed optional native engine; unused imports need no engine.

A standard module must be imported. Names available without an import belong to the prelude and are specified under built-in facilities.

Modules

ModuleContentsError type
std.actorSupervised actors on one thread, each with its own heap, bounded mailbox and copied messages; a trap discards only that actor’s heap and files; timers, request-reply and restartStartError, Undelivered[M], TimerError, RequestError[M]
std.actor.netTCP for actors: a listener that starts one actor per connection with admission limits, connections with bounded buffers, back-pressure and deadlines, and nonblocking connect and lookupNetError, TimerError, BytesError
std.actor.tlsTLS for actor connections: a std.tls session over a std.actor.net connection, with plaintext back-pressure and the session in the connection actor’s heapTlsError, TimerError
std.actor.http.serverHTTP/1.1 and HTTPS on isolated connection actors, bounded request/response handlers or resumable streaming events, deadlines and drainingServerError
std.archiveTAR and ZIP readers and writers over std.io, and extraction into a new directory under explicit path, link, and size limitsArchiveError
std.atomicInline atomic integers and booleans with compile-time checked orderings, and fencesNone
std.bytesCompact owning byte buffers with checked ranges, explicit copies and validated text conversionBytesError
std.cache.lruFixed-capacity caches that evict the least recently used entry, with constant-time lookup, promotion and evictionAllocError
std.channelBounded channels, endpoints and selectionChannel/send/receive/selection errors
std.cliExplicit typed options/positionals, repeatable values, generated help and lossless path argumentsCliError
std.cli.fileinputBuffered fallible UTF-8 lines across native paths or command-line files, with stdin fallbackFileInputError
std.cli.progressTerminal-only stderr bars and spinners with counts, rates, ETA and throttled redrawsProgressError
std.collectionsOwned binary heaps ordered by < or by a stored comparatorAllocError
std.complexComplex numbers over f64 with arithmetic operators, polar form, exp, log, sqrt, pow, and a+bi displayNone
std.compressBounded gzip, zlib, DEFLATE and Brotli compression of buffers and std.io streams, and CRC-32CompressError
std.concurrentExecutors, futures and parallel operationsPool/submission/future/parallel errors
std.cryptoDigests, HMAC, BLAKE3 hashing/key derivation, constant-time comparison, AEAD encryption, Ed25519, X25519, and HKDF over std.bytesCryptoError, AdvancedError
std.csvStreaming CSV readers and writers over std.io with explicit dialects, size limits, and positioned errorsCsvError
std.dataframeOptional Polars analytical expression plans, typed nullable columns, schema inspection, grouping, and CSV/Parquet I/ODataFrameError
std.db.kvOrdered byte-key maps, atomic transactions and bounded scans over a versioned SQLite storeKvError
std.db.sqliteSQLite databases through prepared statements with typed parameters and reads, owned transactions that roll back on drop, and an explicit value-size limitDbError
std.dequeOwned generic ring buffers with constant-time removal at either endAllocError
std.diffLine, word, character, byte, and list differences as edit operations, with Myers and patience algorithms and unified-diff outputAllocError
std.encodingHex, base32/base64, MIME base64 and validated custom alphabets, plus fixed-width integer and IEEE 754 float packing with explicit byte order, over std.bytesEncodingError
std.envEnvironment variables as text or bytes, a detached snapshot, and the working directoryIoError
std.errorExplicit error mapping, operation/path context, descriptions, and standard-error reportsMapper-selected error, Context[E], AllocError
std.fmtWriter-generic value/report formatting, bounded output and fallible stringsFormatError
std.fsDirectories, metadata/walking, symbolic links and canonical paths, rooted mutation, owned temporary resources, bounded file/tree copy, metadata preservation, delete/globIoError, AllocError, WalkError, FsError, TreeCopyError
std.fs.dirsThe home directory and the XDG configuration, cache, data, and state directories, from the environmentIoError
std.fs.lockOwned Linux shared/exclusive advisory locks, nonblocking and cancellable deadline acquisitionLockError
std.fs.watchBounded Linux filesystem change batches, rename cookies, overflow and readiness waitsWatchError, WaitError
std.graphDirected and undirected graphs with node and edge data, stable handles, traversal, shortest paths, topological order, components and spanning treesAllocError, GraphError
std.httpBounded HTTP/1.1 message heads and streamed Content-Length, chunked, and close-delimited bodies over std.io streamsHttpError
std.http.decoderResumable bounded HTTP/1.x request framing over caller-supplied bytes, without I/OHttpError
std.http.bodyBounded request bodies for handlers: URL-encoded forms, JSON, and multipart uploads into unnamed temporary filesBodyError
std.http.clientHTTP/1.1 client: pooled connections, deadlines and cancellation, bounded redirects, an explicit proxy, and replay of idempotent requests onlyClientError
std.http.cookieCookie fields read leniently and Set-Cookie fields validated against injection, with secure defaultsCookieError
std.http.filesStatic files below a Root, opened one segment at a time without following links, with media types and conditional requestsFilesError
std.http.httpsstd.http.client for https URLs through std.tls, verified by default, with CONNECT tunnels through a proxyClientError
std.http.proxyThe client address and scheme behind explicitly trusted reverse proxies; deployment guidanceProxyError
std.http.serverHTTP/1.1 server: routing, a bounded worker pool and admission queue, idle connections parked off the workers, per-phase deadlines, and graceful shutdownServerError
std.http.sseServer-Sent Events from a server exchange: event, id, multi-line data, comments and retry, flushed per event, with disconnect and shutdown detectionSseError
std.http.tlsstd.http.server over std.tls, with handshake deadlinesServerError
std.http.typedTyped head parsing and writes, fixed/chunked/close-delimited bodies, and guarded connection recovery with concrete causes and progressFramingError[E], Partial[E]
std.http.websocketRFC 6455 WebSocket connections: the handshake as a server exchange or a ws client, bounded framed messages, pings and the closing handshakeWebSocketError
std.http.wssstd.http.websocket clients for wss URLs through std.tlsWebSocketError
std.imageOwned 8-bit pixel buffers, bounded PNG decoding and encoding, cropping, colour conversion and resizingImageError
std.ioPartial-progress byte stream contracts, retry helpers, fixed-capacity buffering, file and process-stream adapters, and checked sequential binary fields in an owned memory cursorStreamError, Partial, CopyError, CloseError, BinaryError
std.io.typedByte stream protocols that carry a caller-chosen error type, buffering, adapters for std.io streams, and exact read, write, and copy helpersStepError[E], Partial[E], CopyFailure[R, W]
std.jsonStrict JSON under depth, input, and output limits: resumable event decoder, validating encoder, and a value treeJsonError
std.jwtJSON Web Tokens: compact JWS signing and verification with the algorithm fixed by the key, strict decoding, and explicit-clock claim checksJwtError
std.logLeveled events with typed fields, delivered to caller-chosen sinks: logfmt lines, bounded memory, or a nonblocking channelLogError
std.markdownCommonMark 0.31.2 parsing into events and HTML rendering, under input, depth, and output limitsMarkdownError
std.templateBounded Jinja-style text templates with explicit JSON values, HTML escaping, conditions, loops and named inheritanceTemplateError
std.mathFloating-point functions, checked rounding, integer algorithms, integer and float helpers, and constantsMathError, OverflowError[T]
std.mimeMedia types: RFC 9110 parsing and formatting with parameters, media-range matching, and the file-extension tableParseError, AllocError
std.netTCP listeners and streams, UDP sockets, IP addresses and name lookup, with deadlines and cancellationNetError
std.pathLossless Linux paths, lexical operations, checked text conversion and escaped displayIoError, AllocError
std.processChildren spawned from an argument vector with no implicit shell, piped streams, bounded output collection, signals, and explicit reapingIoError, StreamError
std.randomCaller-owned seeded generators with unbiased ranges, shuffling, and index sampling; not for secretsRandomError
std.readyReadiness waits on pipes and other handles, bounded by a deadline and a cancellation tokenWaitError
std.regexLinear-time regular expressions over text and bytes: search, captures, iteration, split, and replacement under explicit pattern, memory, and work limitsRegexError
std.reportBounded human-readable report rendering with std.fmtFormatError
std.secretsOperating-system entropy: secret bytes, hex and base64url tokens, and unbiased integers, with no fallback sourceSecretsError
std.shlexShell-style word splitting and POSIX argument quoting, without execution or expansionShlexError, AllocError
std.sketchBounded distinct counts, sampled quantiles and frequent-item bounds with versioned checkpointsSketchError, AllocError, BytesError
std.sketch.xorStatic approximate membership over numeric identities with bounded constructionSketchError, AllocError, BytesError
std.sketch.cuckooScalable approximate membership with bounded growth and receipt-directed deletionSketchError, AllocError, BytesError
std.signalSIGTERM and SIGINT as an event to wait or select on, and a helper that cancels a token on oneSignalError, WaitError
std.simdPortable fixed-lane values, masks, checked and wrapping arithmetic, selection, and bounded list loads/storesNone
std.statDescriptive f64 statistics, inclusive quantiles, and normal, uniform and exponential distributions with explicit RNG samplingStatError
std.plotLayered charts from owned columns or record accessors, validated mappings, grouped statistics, and deterministic streaming SVGPlotError
std.linalgDense f64 matrices, products, solves, LU, QR, Cholesky, SVD and eigen decompositions through an optional faer engineAllocError, LinalgError
std.syncShared cooperative cancellation tokensAllocError
std.systemBounded Linux procfs snapshots: CPUs, memory, load, uptime, block devices, disk I/O and process statusIoError, SystemError
std.tempfileUnnamed and named temporary files, recursively cleaned directories, context managers and explicit ownership transferIoError, FsError, StreamError, AllocError
std.testAssertions that explain their failure on standard error, self-removing scratch directories, the seeded generator for plenty test, and fake clocks and streamsTestError
std.textBounded builders, joining, scalar/word wrapping, shortening, and indentationTextError
std.threadExplicit scoped thread and owned-job importsThreadError, SpawnError[F]
std.timeChecked durations, monotonic instants and deadlines, wall timestamps, and sleepRangeError
std.tlsTLS 1.2 and 1.3 client and server streams over std.net TCP, with verified certificates, client certificates, ALPN, deadlines, and cancellationTlsError
std.tomlStrict TOML 1.0 parsing, value-tree writing and format-preserving document edits, under depth, input, and output limitsTomlError
std.unicodeUnicode 17 normalization, combining classes, and Mark classificationTextError
std.unitsSI dimensions, prefixed and common units, physical constants, and dimension-aware square roots, sums and trigonometry over QuantityMathError, RangeError
std.units.catalogueEvery SI prefix on every SI unit and the other named units of NIST SP 811None
std.urlStrict RFC 3986 absolute URL parsing and formatting without normalization, component-aware percent encoding, and query pairsUrlError
std.uuidUUID values, canonical text and network bytes, random v4 and timestamp-prefixed v7 generationParseError, BytesError, SecretsError, RangeError
std.xmlBounded UTF-8 XML event readers and validating stream writers, without DTDs or external entitiesXmlError

The generated signature index lists every public declaration of every module on one page. Standard library conventions states the naming, error, parameter and type rules public modules follow, and where modules still differ.

Planned modules appear only in the backlog. The tutorial introduces the library in Use the standard math library, Enumerate and classify paths, and Prune before descending.

Importing a standard module

Standard modules use the ordinary import forms:

import std.math
import std.math as math
from std.math import sqrt, pi
from std.math import MathError as Error

An import binds only the names it lists. import std.math requires the full std.math.sqrt qualification; an alias shortens it.

std is reserved. Each of these is an error:

  • a std directory, std.plenty, or std.plentyi in the source root;
  • a declaration named std;
  • an import that binds a non-standard module as std.

An unknown standard module, including a bare import std, is an error that lists the available modules. It never falls back to a local file.

How standard modules are compiled

A standard module passes through ordinary parsing, visibility checks, type checking, specialization, and native emission. Its private helpers stay private. Repeated imports, through any alias or dependency path, share its nominal types, so a MathError from one module equals a MathError from another. The scope rationale records when a module uses a native library instead of Plenty source.

The four concurrency modules export canonical builtin identities instead of wrapper functions. Their imports reach the existing ownership and worker checks directly, including scoped joins and blocking-wait diagnostics. Their prelude spellings remain available. Importing one of these under its own builtin name is permitted only when it denotes the same builtin; unrelated name collisions remain errors. Generic constructor and error aliases retain the same types.

Embedded modules have toolchain identities such as std/math.plenty, independent of the installation path. A diagnostic located inside a standard module, such as a note on a generic helper instantiated with an unsupported type, uses that name. The source-only APIs (check_source, compile_source_*) have no source root, so they can import std modules but not project modules.

The runtime packaging page describes how the sources ship with the compiler.

Reading a module page

Each module page gives every public function’s signature and states:

  • whether it returns a plain value or a Result, and which error variants it reports;
  • whether it borrows or consumes its arguments;
  • whether it allocates;
  • the limits that differ from the Python module of the same name.

Recoverable failures use typed errors: std.math defines MathError, while std.path and std.fs share the native IoError and storage AllocError. The filesystem walker adds WalkError to retain attempted paths and operations. ? propagates an error to a caller returning the same error type or Failure. std.error states the conventions modules follow for typed causes, native codes, and allocation failure. Module functions accept named arguments and defaults; the signature index shows each default.

Acknowledgements

Standard library signatures

Every public declaration of each std module, one line each, so you can check that an operation exists and what it returns before reading its module page. The page states the contracts: ownership, allocation, error variants, and limits. pub is omitted. Fields, methods, and variants are indented under their owner. Summaries come from the module table.

After changing a standard module, regenerate this page with UPDATE_STDLIB_INDEX=1 cargo test --locked --test integration test_stdlib_index::.

Modules

Each section’s lines in this file, so one module can be read alone.

std.actor

Supervised actors on one thread, each with its own heap, bounded mailbox and copied messages; a trap discards only that actor’s heap and files; timers, request-reply and restart. Errors: StartError, Undelivered[M], TimerError, RequestError[M]. Reference.

const max_heap_limit: i64 = 67108864
const max_spawns: i64 = 16
const max_capacity: i64 = 1048576
class Address
    def key(self) -> u64
class Watch
    def key(self) -> u64
class Timer
    def key(self) -> u64
    def tag(self) -> i64
class Request
    def key(self) -> u64
enum Reply[M]
    Answer(M) | TimedOut | Gone
enum TimerError
    Capacity | Range
enum RequestError[M]
    Full(M) | Gone(M) | Capacity(M)
enum Exit[E]
    # Stopped by `Context.stop`, a `Stop` request, or a spawn hook that | Failed(E) | Trapped(TrapKind)
enum TrapKind
    Memory | Index | Arithmetic | Unwrap | Other
enum Mail[M, E]
    Message(M) | Ready(Watch, Readiness) | Down(Address, i64, Exit[E]) | Stop | Timer(Timer) | Request(Request, M) | Reply(Request, Reply[M])
enum Undelivered[M]
    Full(M) | Gone(M)
enum StageError
    Allocation(AllocError) | Wait(WaitError)
enum StartError
    Unsupported | Capacity | Heaps | Mailbox
class Stats
    started: i64
    stopped: i64
    failed: i64
    trapped: i64
    restarted: i64
    delivered: i64
    full: i64
    gone: i64
    lost: i64
def supported() -> bool
def heaps_live() -> i64
def heap_bytes_used() -> i64
def heap_bytes_committed() -> i64
class Context[M, E]
    def address(self) -> Address
    def stop(self: &mut Context[M, E]) -> ()
    def heap_used(self) -> i64
    def send(self: &mut Context[M, E], to: Address, message: M) -> Result[(), Undelivered[M]]
    def stop_actor(self: &mut Context[M, E], to: Address) -> bool
    def spawn(self: &mut Context[M, E], tag: i64, limit: Positive[i64]) -> Result[Address, StartError]
    def spawn_isolated(self: &mut Context[M, E], tag: i64, limit: Positive[i64]) -> Result[Address, StartError]
    def restart(self: &mut Context[M, E], child: Address, limit: i64, window: &Duration) -> bool
    def after(self: &mut Context[M, E], delay: &Duration, tag: i64) -> Result[Timer, TimerError]
    def every(self: &mut Context[M, E], period: &Duration, tag: i64) -> Result[Timer, TimerError]
    def cancel(self: &mut Context[M, E], timer: Timer) -> bool
    def request(self: &mut Context[M, E], to: Address, message: M, timeout: &Duration) -> Result[Request, RequestError[M]]
    def reply(self: &mut Context[M, E], request: Request, message: M) -> Result[(), Undelivered[M]]
    def watch[H: Watchable](self: &mut Context[M, E], handle: &H, interest: Interest) -> Result[Watch, WaitError]
    def unwatch[H: Watchable](self: &mut Context[M, E], handle: &H) -> Result[(), WaitError]
class Stage[N, M, E]
    def shutdown(self: &mut Stage[N, M, E]) -> ()
    def start(self: &mut Stage[N, M, E], node: N, limit: Positive[i64]) -> Result[Address, StartError]
    def send(self: &mut Stage[N, M, E], to: Address, message: M) -> Result[(), Undelivered[M]]
    def stop_actor(self: &mut Stage[N, M, E], to: Address) -> bool
    def add_timers(self: &mut Stage[N, M, E], count: i64) -> Result[(), AllocError]
    def live(self) -> i64
    def stats(self) -> Stats
    def node(self, actor: Address) -> Option[&N]
    def heap_used(self, actor: Address) -> Option[i64]
    def run(self: &mut Stage[N, M, E], deadline: &Deadline, cancel: &CancellationToken) -> Result[(), WaitError]
def stage[N, M, E](capacity: i64, mailbox: i64, events: i64) -> Result[Stage[N, M, E], StageError]

std.actor.net

TCP for actors: a listener that starts one actor per connection with admission limits, connections with bounded buffers, back-pressure and deadlines, and nonblocking connect and lookup. Errors: NetError, TimerError, BytesError. Reference.

class Limits
    input: Positive[i64]
    output: Positive[i64]
    idle: Duration
    stall: Duration
enum Expired
    Idle | Read | Write
class Events
    mine: bool
    received: i64
    ended: bool
    writable: bool
    drained: bool
    expired: Option[Expired]
class Connection
    def start[M, E](self: &mut Connection, context: &mut Context[M, E]) -> Result[(), NetError | TimerError]
    def handle[M, E](self: &mut Connection, mail: &Mail[M, E], context: &mut Context[M, E]) -> Result[Events, NetError | TimerError]
    def input(self) -> &Bytes
    def input_start(self) -> i64
    def input_stop(self) -> i64
    def available(self) -> i64
    def consume[M, E](self: &mut Connection, count: i64, context: &mut Context[M, E]) -> Result[(), NetError | TimerError]
    def pending(self) -> i64
    def space(self) -> i64
    def write[M, E](self: &mut Connection, source: &[u8], context: &mut Context[M, E]) -> Result[i64, NetError | TimerError]
    def reserve(self: &mut Connection, wanted: NonNegative[i64]) -> Result[i64, NetError]
    def output(self: &mut Connection) -> &mut Bytes
    def output_stop(self) -> i64
    def commit[M, E](self: &mut Connection, count: i64, context: &mut Context[M, E]) -> Result[(), NetError | TimerError]
    def forward[M, E](self: &mut Connection, count: i64, context: &mut Context[M, E]) -> Result[i64, NetError | TimerError]
    def read_within[M, E](self: &mut Connection, span: &Duration, context: &mut Context[M, E]) -> Result[(), NetError | TimerError]
    def clear_read_deadline(self: &mut Connection) -> ()
    def shutdown_write[M, E](self: &mut Connection, context: &mut Context[M, E]) -> Result[(), NetError | TimerError]
    def close[M, E](self: &mut Connection, context: &mut Context[M, E]) -> Result[(), NetError]
    def acknowledge_now(self: &mut Connection) -> ()
    def reading(self) -> bool
    def closed(self) -> bool
    def stream(self) -> &TcpStream
def connection(stream: TcpStream, limits: &Limits) -> Result[Connection, AllocError]
enum Admission
    Pause | Refuse
class Listener
    def start[M, E](self: &mut Listener, context: &mut Context[M, E]) -> Result[(), NetError]
    def handle[M, E](self: &mut Listener, mail: &Mail[M, E], context: &mut Context[M, E]) -> Result[bool, NetError]
    def accept(self) -> Result[Option[Connection], NetError | AllocError]
    def live(self) -> i64
    def refused(self) -> i64
    def tag(self) -> i64
    def local_address(self) -> Result[SocketAddress, NetError]
def listener(socket: TcpListener, limits: Limits, capacity: Positive[i64], heap: Positive[i64], tag: i64, admission: Admission) -> Result[Listener, NetError]
class Dialing
    def handle[M, E](self: &mut Dialing, mail: &Mail[M, E], context: &mut Context[M, E]) -> Result[Option[TcpStream], NetError]
    def abandon[M, E](self: &mut Dialing, context: &mut Context[M, E]) -> Result[(), NetError]
def dial[M, E](address: &SocketAddress, timeout: &Duration, context: &mut Context[M, E]) -> Result[Dialing, NetError | TimerError]
class Resolving
    def handle[M, E](self: &mut Resolving, mail: &Mail[M, E], context: &mut Context[M, E]) -> Result[Option[list[SocketAddress]], NetError]
    def abandon[M, E](self: &mut Resolving, context: &mut Context[M, E]) -> Result[(), NetError]
def lookup[M, E](host: &str, port: u16, timeout: &Duration, context: &mut Context[M, E]) -> Result[Resolving, NetError | TimerError]

std.actor.tls

TLS for actor connections: a std.tls session over a std.actor.net connection, with plaintext back-pressure and the session in the connection actor’s heap. Errors: TlsError, TimerError. Reference.

class TlsConnection
    def start[M, E](self: &mut TlsConnection, context: &mut Context[M, E]) -> Result[(), TlsError | TimerError]
    def handle[M, E](self: &mut TlsConnection, mail: &Mail[M, E], context: &mut Context[M, E]) -> Result[Events, TlsError | TimerError]
    def input(self) -> &Bytes
    def input_start(self) -> i64
    def input_stop(self) -> i64
    def available(self) -> i64
    def consume[M, E](self: &mut TlsConnection, count: i64, context: &mut Context[M, E]) -> Result[(), TlsError | TimerError]
    def pending(self) -> i64
    def space(self) -> i64
    def write[M, E](self: &mut TlsConnection, source: &[u8], context: &mut Context[M, E]) -> Result[i64, TlsError | TimerError]
    def forward[M, E](self: &mut TlsConnection, count: i64, context: &mut Context[M, E]) -> Result[i64, TlsError | TimerError]
    def read_within[M, E](self: &mut TlsConnection, span: &Duration, context: &mut Context[M, E]) -> Result[(), TlsError | TimerError]
    def clear_read_deadline(self: &mut TlsConnection) -> ()
    def shutdown_write[M, E](self: &mut TlsConnection, context: &mut Context[M, E]) -> Result[(), TlsError | TimerError]
    def close[M, E](self: &mut TlsConnection, context: &mut Context[M, E]) -> Result[(), TlsError]
    def established(self) -> bool
    def reading(self) -> bool
    def closed(self) -> bool
    def session(self) -> &Session
    def connection(self) -> &Connection
def over(net: Connection, session: Session, input: Positive[i64]) -> Result[TlsConnection, TlsError]
def server(net: Connection, config: &ServerConfig, input: Positive[i64]) -> Result[TlsConnection, TlsError]
def client(net: Connection, config: &ClientConfig, server_name: &str, input: Positive[i64]) -> Result[TlsConnection, TlsError]

std.actor.http.server

HTTP/1.1 and HTTPS on isolated connection actors, bounded request/response handlers or resumable streaming events, deadlines and draining. Errors: ServerError. Reference.

enum ServerError
    InvalidConfig | AdmissionStopped | Allocation(AllocError) | Bytes(BytesError) | Http(HttpError) | Net(NetError) | Tls(TlsError) | Timer(TimerError) | Wait(WaitError) | Actor(std.actor.StartError) | Range(RangeError) | Handler
enum Notice[M]
    Message(M) | Drain | Resume | Completed | CompletedBy(Address) | Rejected
enum Event[M]
    Head(RequestHead, RequestTarget) | Body(Bytes) | Trailers(list[std.http.Header]) | End | Writable | Actor(Mail[Notice[M], ServerError]) | Message(M) | Stopping
class Config
    capacity: i64
    heap_limit: i64
    input_buffer: i64
    output_buffer: i64
    pending_output: i64
    limits: Limits
    idle_timeout: Duration
    head_timeout: Duration
    body_timeout: Duration
    write_timeout: Duration
    response_timeout: Duration
    handshake_timeout: Duration
    drain_timeout: Duration
    max_requests: Positive[i64]
class Report
    actors: Stats
    requests: u64
    rejected: u64
    protocol_errors: u64
protocol Handler[M, E]
    def receive(self: &mut Handler[M, E], event: Event[M], response: &mut Response, context: &mut Context[Notice[M], ServerError]) -> Result[(), E]
class Response
    def is_stopping(self) -> bool
    def close_after(self: &mut Response) -> ()
    def shutdown_server(self: &mut Response) -> ()
    def is_started(self) -> bool
    def is_done(self) -> bool
    def pending(self) -> i64
    def space(self) -> i64
    def start(self: &mut Response, head: ResponseHead) -> Result[(), ServerError]
    def write(self: &mut Response, source: &[u8]) -> Result[i64, ServerError]
    def finish(self: &mut Response) -> Result[(), ServerError]
    def respond(self: &mut Response, head: ResponseHead, body: &[u8]) -> Result[(), ServerError]
    def send(self: &mut Response, status: i64, content_type: &str, body: &[u8]) -> Result[(), ServerError]
    def send_text(self: &mut Response, status: i64, text: &str) -> Result[(), ServerError]
def config() -> Config
class Node[F, H]
    def started[M](self: &mut Node[F, H], context: &mut Context[Notice[M], ServerError]) -> Result[(), ServerError]
    def receive[M](self: &mut Node[F, H], mail: Mail[Notice[M], ServerError], context: &mut Context[Notice[M], ServerError]) -> Result[(), ServerError]
    def spawn[M](self, tag: i64, child: Address, context: &mut Context[Notice[M], ServerError]) -> Result[Option[Node[F, H]], ServerError]
class Request
    head: RequestHead
    target: RequestTarget
    body: Bytes
    trailers: list[std.http.Header]
protocol RequestHandler[E]
    def handle(self: &mut RequestHandler[E], request: Request, response: &mut Response) -> Result[(), E]
def serve[F: Callable[[], Result[H, FE]], H: RequestHandler[HE], FE, HE](listener: TcpListener, factory: F, config: &Config, stop: &CancellationToken) -> Result[Report, ServerError]
def serve_tls[F: Callable[[], Result[H, FE]], H: RequestHandler[HE], FE, HE](listener: TcpListener, tls: ServerConfig, factory: F, config: &Config, stop: &CancellationToken) -> Result[Report, ServerError]
def serve_events[F: Callable[[], Result[H, FE]], H: Handler[M, HE], M, FE, HE](listener: TcpListener, factory: F, config: &Config, stop: &CancellationToken) -> Result[Report, ServerError]
def serve_tls_events[F: Callable[[], Result[H, FE]], H: Handler[M, HE], M, FE, HE](listener: TcpListener, tls: ServerConfig, factory: F, config: &Config, stop: &CancellationToken) -> Result[Report, ServerError]

std.archive

TAR and ZIP readers and writers over std.io, and extraction into a new directory under explicit path, link, and size limits. Errors: ArchiveError. Reference.

enum ArchiveError
    Stream(StreamError) | Io(IoError) | Fs(FsError) | Allocation(AllocError) | Corrupt | Checksum | Truncated | Unsupported | UnsafePath | Link | Special | Duplicate | Exists | Limit | Cancelled | InvalidInput
enum EntryKind
    File | Directory | Symlink | Hardlink | Other
class Entry
    name: Path
    kind: EntryKind
    size: i64
    compressed: i64
    mode: i64
    mtime: i64
    link: Path
class Policy
    max_entries: i64
    max_entry_size: i64
    max_total_size: i64
    max_ratio: i64
    max_depth: i64
    max_name: i64
    skip_links: bool
    skip_special: bool
def policy() -> Policy
class Summary
    entries: i64
    files: i64
    bytes: i64
    skipped: i64
def stream_error(error: ArchiveError) -> StreamError
class TarReader[R]
    inner: R
    def failure(self) -> Option[ArchiveError]
    def next(self: &mut TarReader[R]) -> Result[Option[Entry], ArchiveError]
    def read(self: &mut TarReader[R], buffer: &mut Bytes, start: i64, stop: i64) -> Result[i64, StreamError]
def tar_reader[R](inner: R) -> Result[TarReader[R], ArchiveError]
class TarWriter[W]
    inner: W
    def add_file(self: &mut TarWriter[W], name: &Path, data: &[u8], mode: i64, mtime: i64) -> Result[(), ArchiveError]
    def add_from[S](self: &mut TarWriter[W], name: &Path, size: i64, source: &mut S, mode: i64, mtime: i64) -> Result[(), ArchiveError]
    def add_dir(self: &mut TarWriter[W], name: &Path, mode: i64, mtime: i64) -> Result[(), ArchiveError]
    def add_symlink(self: &mut TarWriter[W], name: &Path, target: &Path, mtime: i64) -> Result[(), ArchiveError]
    def finish(self: &mut TarWriter[W]) -> Result[(), ArchiveError]
def tar_writer[W](inner: W) -> Result[TarWriter[W], ArchiveError]
class ZipReader[R]
    inner: R
    def len(self) -> i64
    def entries(self) -> &list[Entry]
    def failure(self) -> Option[ArchiveError]
    def open(self: &mut ZipReader[R], index: i64) -> Result[(), ArchiveError]
    def read(self: &mut ZipReader[R], buffer: &mut Bytes, start: i64, stop: i64) -> Result[i64, StreamError]
def zip_reader[R](inner: R, max_entries: NonNegative[i64]) -> Result[ZipReader[R], ArchiveError]
enum Method
    Stored | Deflate
class ZipWriter[W]
    inner: W
    def add_file(self: &mut ZipWriter[W], name: &Path, data: &[u8], method: Method, mode: i64, mtime: i64) -> Result[(), ArchiveError]
    def add_dir(self: &mut ZipWriter[W], name: &Path, mode: i64, mtime: i64) -> Result[(), ArchiveError]
    def add_symlink(self: &mut ZipWriter[W], name: &Path, target: &Path, mtime: i64) -> Result[(), ArchiveError]
    def finish(self: &mut ZipWriter[W]) -> Result[(), ArchiveError]
def zip_writer[W](inner: W) -> Result[ZipWriter[W], ArchiveError]
def extract_tar[R](archive: &mut TarReader[R], parent: &Root, destination: &Path, policy: &Policy, cancel: &CancellationToken) -> Result[Summary, ArchiveError]
def extract_zip[R](archive: &mut ZipReader[R], parent: &Root, destination: &Path, policy: &Policy, cancel: &CancellationToken) -> Result[Summary, ArchiveError]

std.atomic

Inline atomic integers and booleans with compile-time checked orderings, and fences. Reference.

Compiler-provided: Atomic, fence. Their signatures are on the reference page.

enum Ordering
    Relaxed | Release | Acquire | AcqRel | SeqCst

std.bytes

Compact owning byte buffers with checked ranges, explicit copies and validated text conversion. Errors: BytesError. Reference.

enum BytesError
    InvalidUtf8 | Allocation(AllocError)
class Bytes
    def len(self) -> i64
    def is_empty(self) -> bool
    def write_float(self: &mut Bytes, value: f64) -> i64
    def write_float_spec(self: &mut Bytes, value: f64, style: i64, precision: i64, offset: i64) -> i64
    def capacity(self) -> i64
    def get(self, index: i64) -> u8
    def set(self: &mut Bytes, index: i64, value: u8) -> ()
    def push(self: &mut Bytes, value: u8) -> Result[(), AllocError]
    def pop(self: &mut Bytes) -> Option[u8]
    def reserve(self: &mut Bytes, additional: i64) -> Result[(), AllocError]
    def resize(self: &mut Bytes, count: i64, fill: u8) -> Result[(), AllocError]
    def truncate(self: &mut Bytes, count: i64) -> ()
    def clear(self: &mut Bytes) -> ()
    def fill(self: &mut Bytes, start: i64, stop: i64, value: u8) -> ()
    def extend(self: &mut Bytes, source: &[u8]) -> Result[(), AllocError]
    def overwrite(self: &mut Bytes, offset: i64, source: &[u8]) -> ()
    def copy_within(self: &mut Bytes, start: i64, stop: i64, to: i64) -> ()
    def slice(self, start: i64, stop: i64) -> Result[Bytes, AllocError]
    def copy(self) -> Result[Bytes, AllocError]
    def find(self, value: u8, start: i64, stop: i64) -> Option[i64]
    def find_in(self, set: &[u8], start: i64, stop: i64) -> Option[i64]
    def find_not_in(self, set: &[u8], start: i64, stop: i64) -> Option[i64]
    def find_bytes(self, needle: &[u8], start: i64, stop: i64) -> Option[i64]
    def count(self, value: u8, start: i64, stop: i64) -> i64
    def matches(self, offset: i64, source: &[u8]) -> bool
    def matches_text(self, offset: i64, text: &str) -> bool
    def ascii_run(self, start: i64, stop: i64) -> i64
    def utf8_run(self, start: i64, stop: i64) -> i64
    def to_list(self) -> Result[list[u8], AllocError]
    def to_text(self) -> Result[str, BytesError]
    def to_text_range(self, start: i64, stop: i64) -> Result[str, BytesError]
    def read_file(self: &mut Bytes, file: &mut File, start: i64, stop: i64) -> Result[i64, IoError]
    def read_stdin(self: &mut Bytes, start: i64, stop: i64) -> Result[i64, IoError]
def empty() -> Result[Bytes, AllocError]
def with_capacity(capacity: i64) -> Result[Bytes, AllocError]
def from_list(values: &list[u8]) -> Result[Bytes, AllocError]
def from_text(text: &str) -> Result[Bytes, AllocError]
def from_bytes(source: &[u8]) -> Result[Bytes, AllocError]
def write_file(file: &mut File, source: &[u8]) -> Result[i64, IoError]

std.cache.lru

Fixed-capacity caches that evict the least recently used entry, with constant-time lookup, promotion and eviction. Errors: AllocError. Reference.

class LruCache[K, V]
    def __init__(self, capacity: i64) -> Result[(), AllocError]
    def capacity(self) -> i64
    def len(self) -> i64
    def is_empty(self) -> bool
    def contains(self, key: &K) -> bool
    def get(self: &mut LruCache[K, V], key: &K) -> Option[V]
    def peek(self, key: &K) -> Option[V]
    def get_ref(self: &mut LruCache[K, V], key: &K) -> Option[&V]
    def get_mut(self: &mut LruCache[K, V], key: &K) -> Option[&mut V]
    def peek_ref(self, key: &K) -> Option[&V]
    def peek_mut(self: &mut LruCache[K, V], key: &K) -> Option[&mut V]
    def promote(self: &mut LruCache[K, V], key: &K) -> bool
    def put(self: &mut LruCache[K, V], key: K, value: V) -> Result[Option[(K, V)], AllocError]
    def remove(self: &mut LruCache[K, V], key: &K) -> Option[V]
    def pop_lru(self: &mut LruCache[K, V]) -> Option[(K, V)]
    def clear(self: &mut LruCache[K, V]) -> ()

std.channel

Bounded channels, endpoints and selection. Errors: Channel/send/receive/selection errors. Reference.

Compiler-provided: channel, Sender, Receiver, SendError, RecvError, SendTimeoutError, RecvTimeoutError, Selected, SelectError, select_recv, select_recv_nowait, select_recv_timeout, select_recv_timeout_ms. Their signatures are on the reference page.

std.cli

Explicit typed options/positionals, repeatable values, generated help and lossless path arguments. Errors: CliError. Reference.

enum Kind
    Flag | Integer | Text | Path
class Spec
    name: str
    kind: Kind
    positional: bool
    required: bool
    repeatable: bool
    help: str
enum Value
    Flag | Integer(i64) | Text(str) | Path(Path)
class Entry
    spec: i64
    value: Value
class Parsed
    help_requested: bool
    entries: list[Entry]
enum CliError
    Schema(i64) | UnknownOption(i64) | UnexpectedArgument(i64) | MissingValue(i64) | Duplicate(i64) | Missing(i64) | InvalidInteger(i64) | InvalidUtf8(i64) | Allocation(AllocError)
def validate(schema: &list[Spec]) -> Result[(), CliError]
def parse(schema: &list[Spec], arguments: &list[list[u8]], start: i64) -> Result[Parsed, CliError]
def help(schema: &list[Spec], program: &str) -> Result[str, CliError]
def exit_status(error: &CliError) -> i32
def describe(error: &CliError) -> Result[str, AllocError]
def describe_for(error: &CliError, schema: &list[Spec]) -> Result[str, AllocError]

std.cli.fileinput

Buffered fallible UTF-8 lines across native paths or command-line files, with stdin fallback. Errors: FileInputError. Reference.

enum FileInputError
    Open(Option[Path], StreamError) | Read(Option[Path], StreamError) | Text(Option[Path], BytesError) | Close(Option[Path], StreamError)
def files(paths: list[Path]) -> Generator[Result[str, FileInputError]]
def lines() -> Result[Generator[Result[str, FileInputError]], IoError]

std.cli.progress

Terminal-only stderr bars and spinners with counts, rates, ETA and throttled redraws. Errors: ProgressError. Reference.

enum ProgressError
    InvalidPosition | Finished | Output(FormatError)
class Progress
    def position(self) -> u64
    def total(self) -> Option[u64]
    def is_finished(self) -> bool
    def rate(self, now: &Instant) -> Option[f64]
    def eta(self, now: &Instant) -> Option[f64]
    def render[W](self, now: &Instant, output: &mut Formatter[W]) -> Result[(), FormatError]
    def set_position_at(self: &mut Progress, position: u64, now: &Instant) -> Result[(), ProgressError]
    def set_position(self: &mut Progress, position: u64) -> Result[(), ProgressError]
    def advance(self: &mut Progress, count: u64 = 1u64) -> Result[(), ProgressError]
    def tick(self: &mut Progress) -> Result[(), ProgressError]
    def finish_at(self: &mut Progress, now: &Instant) -> Result[(), ProgressError]
    def finish(self: &mut Progress) -> Result[(), ProgressError]
def bar_at(total: u64, now: &Instant) -> Progress
def bar(total: u64) -> Progress
def spinner_at(now: &Instant) -> Progress
def spinner() -> Progress

std.collections

Owned binary heaps ordered by < or by a stored comparator. Errors: AllocError. Reference.

class Heap[T]
    def __init__(self) -> Result[(), AllocError]
    def len(self) -> i64
    def is_empty(self) -> bool
    def reserve(self: &mut Heap[T], additional: i64) -> Result[(), AllocError]
    def peek(self) -> Option[T]
    def peek_ref(self) -> Option[&T]
    def push(self: &mut Heap[T], value: T) -> Result[(), AllocError]
    def pop(self: &mut Heap[T]) -> Option[T]
    def clear(self: &mut Heap[T]) -> ()
class HeapBy[T, F: Callable[[&T, &T], bool]]
    def __init__(self, before: F) -> Result[(), AllocError]
    def len(self) -> i64
    def is_empty(self) -> bool
    def reserve(self: &mut HeapBy[T, F], additional: i64) -> Result[(), AllocError]
    def peek(self) -> Option[T]
    def peek_ref(self) -> Option[&T]
    def push(self: &mut HeapBy[T, F], value: T) -> Result[(), AllocError]
    def pop(self: &mut HeapBy[T, F]) -> Option[T]
    def clear(self: &mut HeapBy[T, F]) -> ()
def drain[T](heap: Heap[T]) -> Generator[T]
def drain_by[T, F: Callable[[&T, &T], bool]](heap: HeapBy[T, F]) -> Generator[T]

std.complex

Complex numbers over f64 with arithmetic operators, polar form, exp, log, sqrt, pow, and a+bi display. Reference.

class Complex
    re: f64
    im: f64
    def __add__(self, other: &Complex) -> Complex
    def __sub__(self, other: &Complex) -> Complex
    def __mul__(self, other: &Complex) -> Complex
    def __truediv__(self, other: &Complex) -> Complex
    def __neg__(self) -> Complex
    def scale(self, factor: f64) -> Complex
    def conj(self) -> Complex
    def norm(self) -> f64
    def abs(self) -> f64
    def arg(self) -> f64
    def to_polar(self) -> (f64, f64)
    def exp(self) -> Complex
    def log(self) -> Complex
    def sqrt(self) -> Complex
    def pow(self, exponent: &Complex) -> Complex
    def powi(self, n: i64) -> Complex
    def display[W](self, out: &mut Formatter[W]) -> Result[(), FormatError]
def from_polar(r: f64, theta: f64) -> Complex

std.compress

Bounded gzip, zlib, DEFLATE and Brotli compression of buffers and std.io streams, and CRC-32. Errors: CompressError. Reference.

enum Format
    Deflate | Zlib | Gzip | Brotli
enum CompressError
    OutOfRange | Corrupt | Checksum | Truncated | TrailingData | LimitExceeded | Allocation(AllocError)
def stream_error(error: CompressError) -> StreamError
class Inflater
    def step(self: &mut Inflater, input: &[u8]) -> Result[i64, CompressError]
    def complete(self) -> bool
    def output(self) -> &Bytes
    def clear(self: &mut Inflater) -> ()
def inflater(format: Format, limit: NonNegative[i64]) -> Result[Inflater, CompressError]
def compress(format: Format, data: &[u8], level: i64) -> Result[Bytes, CompressError]
def decompress(format: Format, data: &[u8], limit: NonNegative[i64]) -> Result[Bytes, CompressError]
def crc32(seed: i64, data: &[u8]) -> Result[i64, CompressError]
class Compressor[W]
    inner: W
    def write(self: &mut Compressor[W], source: &[u8]) -> Result[i64, StreamError]
    def flush(self: &mut Compressor[W]) -> Result[(), StreamError]
    def finish(self: &mut Compressor[W]) -> Result[(), StreamError]
def compressor[W](format: Format, inner: W, level: i64) -> Result[Compressor[W], CompressError]
class Decompressor[R]
    inner: R
    def failure(self) -> Option[CompressError]
    def read(self: &mut Decompressor[R], buffer: &mut Bytes, start: i64, stop: i64) -> Result[i64, StreamError]
def decompressor[R](format: Format, inner: R, limit: NonNegative[i64]) -> Result[Decompressor[R], CompressError]

std.concurrent

Executors, futures and parallel operations. Errors: Pool/submission/future/parallel errors. Reference.

Compiler-provided: ThreadPoolExecutor, Future, PoolError, SubmitError, FutureError, PoolMapError, ParallelError. Their signatures are on the reference page.

std.crypto

Digests, HMAC, BLAKE3 hashing/key derivation, constant-time comparison, AEAD encryption, Ed25519, X25519, and HKDF over std.bytes. Errors: CryptoError, AdvancedError. Reference.

enum CryptoError
    Finished | InvalidKeyLength | UnsupportedAlgorithm | Allocation(AllocError)
enum Algorithm
    Sha256 | Sha512 | Sha1 | Blake3
def output_size(algorithm: Algorithm) -> i64
class Digest
    def size(self) -> i64
    def update(self: &mut Digest, data: &[u8]) -> Result[(), CryptoError]
    def update_text(self: &mut Digest, text: &str) -> Result[(), CryptoError]
    def finish(self: &mut Digest) -> Result[Bytes, CryptoError]
    def finish_into(self: &mut Digest, target: &mut Bytes, offset: i64) -> Result[(), CryptoError]
class Mac
    def size(self) -> i64
    def update(self: &mut Mac, data: &[u8]) -> Result[(), CryptoError]
    def update_text(self: &mut Mac, text: &str) -> Result[(), CryptoError]
    def finish(self: &mut Mac) -> Result[Bytes, CryptoError]
    def finish_into(self: &mut Mac, target: &mut Bytes, offset: i64) -> Result[(), CryptoError]
    def verify(self: &mut Mac, expected: &[u8]) -> Result[bool, CryptoError]
def digest(algorithm: Algorithm) -> Result[Digest, CryptoError]
def hmac(algorithm: Algorithm, key: &[u8]) -> Result[Mac, CryptoError]
def sha256(data: &[u8]) -> Result[Bytes, CryptoError]
def sha512(data: &[u8]) -> Result[Bytes, CryptoError]
def blake3(data: &[u8]) -> Result[Bytes, CryptoError]
def blake3_keyed_hasher(key: &[u8]) -> Result[Mac, CryptoError]
def blake3_keyed(key: &[u8], data: &[u8]) -> Result[Bytes, CryptoError]
def blake3_derive_hasher(context: &str) -> Result[Digest, CryptoError]
def blake3_derive_key(context: &str, key_material: &[u8]) -> Result[Bytes, CryptoError]
def sha1(data: &[u8]) -> Result[Bytes, CryptoError]
def hmac_sha256(key: &[u8], data: &[u8]) -> Result[Bytes, CryptoError]
def hmac_sha512(key: &[u8], data: &[u8]) -> Result[Bytes, CryptoError]
def equal(a: &[u8], b: &[u8]) -> bool
enum AdvancedError
    InvalidLength | AuthenticationFailed | InvalidPeer | Finished | RandomFailure | InvalidKey | WeakKey | SmallOrderKey | Allocation(AllocError)
    def format[W](self, output: &mut Formatter[W]) -> Result[(), FormatError]
enum AeadAlgorithm
    Aes256Gcm | ChaCha20Poly1305
def seal_in_place(algorithm: AeadAlgorithm, key: &[u8], nonce: &[u8], aad: &[u8], data: &mut Bytes) -> Result[(), AdvancedError]
def open_in_place(algorithm: AeadAlgorithm, key: &[u8], nonce: &[u8], aad: &[u8], data: &mut Bytes) -> Result[(), AdvancedError]
def ed25519_public_key_into(seed: &[u8], target: &mut Bytes) -> Result[(), AdvancedError]
def ed25519_public_key(seed: &[u8]) -> Result[Bytes, AdvancedError]
def ed25519_sign_into(seed: &[u8], message: &[u8], target: &mut Bytes) -> Result[(), AdvancedError]
def ed25519_sign(seed: &[u8], message: &[u8]) -> Result[Bytes, AdvancedError]
def ed25519_verify(public_key: &[u8], message: &[u8], signature: &[u8]) -> Result[bool, AdvancedError]
def hkdf_sha256_into(salt: &[u8], key_material: &[u8], info: &[u8], target: &mut Bytes) -> Result[(), AdvancedError]
def hkdf_sha256(salt: &[u8], key_material: &[u8], info: &[u8], size: NonNegative[i64]) -> Result[Bytes, AdvancedError]
class X25519Key
    def public_key_into(self, target: &mut Bytes) -> Result[(), AdvancedError]
    def public_key(self) -> Result[Bytes, AdvancedError]
    def agree_into(self: &mut X25519Key, peer_public_key: &[u8], target: &mut Bytes) -> Result[(), AdvancedError]
    def agree(self: &mut X25519Key, peer_public_key: &[u8]) -> Result[Bytes, AdvancedError]
def x25519() -> Result[X25519Key, AdvancedError]
enum RsaScheme
    Pkcs1Sha256 | Pkcs1Sha384 | Pkcs1Sha512 | PssSha256 | PssSha384 | PssSha512
class RsaPublicKey
    def signature_size(self) -> i64
    def verify(self, scheme: RsaScheme, message: &[u8], signature: &[u8]) -> Result[bool, AdvancedError]
class RsaPrivateKey
    def signature_size(self) -> i64
    def public_key(self) -> Result[RsaPublicKey, AdvancedError]
    def sign_into(self, scheme: RsaScheme, message: &[u8], target: &mut Bytes) -> Result[(), AdvancedError]
    def sign(self, scheme: RsaScheme, message: &[u8]) -> Result[Bytes, AdvancedError]
class P256PublicKey
    def verify(self, message: &[u8], signature: &[u8]) -> Result[bool, AdvancedError]
class P256PrivateKey
    def public_key(self) -> Result[P256PublicKey, AdvancedError]
    def sign_into(self, message: &[u8], target: &mut Bytes) -> Result[(), AdvancedError]
    def sign(self, message: &[u8]) -> Result[Bytes, AdvancedError]
def rsa_public_key_from_spki(der: &[u8]) -> Result[RsaPublicKey, AdvancedError]
def rsa_public_key_from_components(n: &[u8], e: &[u8]) -> Result[RsaPublicKey, AdvancedError]
def rsa_private_key_from_pkcs8(der: &[u8]) -> Result[RsaPrivateKey, AdvancedError]
def p256_public_key_from_point(point: &[u8]) -> Result[P256PublicKey, AdvancedError]
def p256_public_key_from_coordinates(x: &[u8], y: &[u8]) -> Result[P256PublicKey, AdvancedError]
def p256_public_key_from_spki(der: &[u8]) -> Result[P256PublicKey, AdvancedError]
def p256_private_key_from_pkcs8(der: &[u8]) -> Result[P256PrivateKey, AdvancedError]
def ed25519_public_key_from_spki(der: &[u8]) -> Result[Bytes, AdvancedError]
def ed25519_seed_from_pkcs8(der: &[u8]) -> Result[Bytes, AdvancedError]

std.csv

Streaming CSV readers and writers over std.io with explicit dialects, size limits, and positioned errors. Errors: CsvError. Reference.

enum Fault
    BareQuote | AfterQuote | UnterminatedQuote | FieldTooLong | RecordTooLong | TooManyFields
class Position
    record: i64
    field: i64
    line: i64
    offset: i64
enum CsvError
    Invalid(Fault, Position) | Rejected(Fault, i64, i64) | Width(i64, i64, i64) | EmptyRecord(i64) | InvalidUtf8(i64, i64) | InvalidDialect | Stopped | Stream(StreamError) | Allocation(AllocError)
enum Width
    Any | First | Exactly(i64)
enum Newline
    Lf | CrLf
enum Quoting
    Needed | Always
class Dialect
    delimiter: u8
    quote: u8
    newline: Newline
    quoting: Quoting
    width: Width
    skip_bom: bool
    max_field: i64
    max_record: i64
    max_fields: i64
def rfc4180() -> Dialect
class Record
    def len(self) -> i64
    def number(self) -> i64
    def line(self) -> i64
    def data(self) -> &Bytes
    def start(self, index: i64) -> i64
    def stop(self, index: i64) -> i64
    def bytes(self, index: i64) -> Result[Bytes, CsvError]
    def append_to(self, index: i64, out: &mut Bytes) -> Result[(), CsvError]
    def text(self, index: i64) -> Result[str, CsvError]
    def texts(self) -> Result[list[str], CsvError]
    def clear(self: &mut Record) -> ()
def record() -> Result[Record, CsvError]
class Reader[R]
    inner: R
    def records(self) -> i64
    def line(self) -> i64
    def offset(self) -> i64
    def read(self: &mut Reader[R], record: &mut Record) -> Result[bool, CsvError]
def reader[R](inner: R, dialect: &Dialect, capacity: Positive[i64]) -> Result[Reader[R], CsvError]
class Writer[W]
    inner: W
    def records(self) -> i64
    def pending(self) -> i64
    def field(self: &mut Writer[W], source: &[u8]) -> Result[(), CsvError]
    def text(self: &mut Writer[W], value: &str) -> Result[(), CsvError]
    def discard(self: &mut Writer[W]) -> ()
    def end_record(self: &mut Writer[W]) -> Result[(), CsvError]
    def record(self: &mut Writer[W], record: &Record) -> Result[(), CsvError]
    def texts(self: &mut Writer[W], fields: &list[str]) -> Result[(), CsvError]
    def flush(self: &mut Writer[W]) -> Result[(), CsvError]
def writer[W](inner: W, dialect: &Dialect) -> Result[Writer[W], CsvError]

std.dataframe

Optional Polars analytical expression plans, typed nullable columns, schema inspection, grouping, and CSV/Parquet I/O. Errors: DataFrameError. Reference.

enum TimeUnit
    Milliseconds | Microseconds | Nanoseconds
enum DType
    I8 | I16 | I32 | U8 | U16 | U32 | U64 | F32 | I64 | F64 | Bool | String | Binary | Date | Datetime(TimeUnit, Option[str]) | List | Struct | Other(str)
class Field
    name: str
    dtype: DType
class SchemaNode
    parent: i64
    name: str
    dtype: DType
class ListParts
    values: Column
    offsets: list[i64]
    valid: list[bool]
class StructParts
    fields: DataFrame
    valid: list[bool]
enum JoinKind
    Inner | Left | Full | Semi | Anti
enum AsOfStrategy
    Backward | Forward | Nearest
class SortKey
    def __init__(self, expression: Expr, descending: bool = False, nulls_last: bool = True) -> ()
enum DataFrameError
    Schema(str) | Query(str) | Io(str) | NativePanic(str) | Null(str) | Bounds(str) | Allocation(AllocError) | Message(IoError)
class Column
    def len(self) -> Result[i64, DataFrameError]
    def to_frame(self) -> Result[DataFrame, DataFrameError]
    def list_parts(self) -> Result[ListParts, DataFrameError]
    def struct_parts(self) -> Result[StructParts, DataFrameError]
class DataFrame
    def column(self, name: &str) -> Result[Column, DataFrameError]
    def schema_tree(self) -> Result[list[SchemaNode], DataFrameError]
    def height(self) -> Result[i64, DataFrameError]
    def lazy(self) -> Result[Query, DataFrameError]
    def write_parquet(self, path: &str) -> Result[(), DataFrameError]
    def write_csv(self, path: &str, has_header: bool = True, separator: &str = ",", null_value: &str = "") -> Result[(), DataFrameError]
    def get_date_days(self, name: &str, row: i64) -> Result[Option[i32], DataFrameError]
    def get_datetime_ticks(self, name: &str, row: i64, unit: TimeUnit, utc: bool = False) -> Result[Option[i64], DataFrameError]
    def values_date_days(self, name: &str) -> Result[list[i32], DataFrameError]
    def values_date_days_nullable(self, name: &str) -> Result[list[Option[i32]], DataFrameError]
    def values_datetime_ticks(self, name: &str, unit: TimeUnit, utc: bool = False) -> Result[list[i64], DataFrameError]
    def values_datetime_ticks_nullable(self, name: &str, unit: TimeUnit, utc: bool = False) -> Result[list[Option[i64]], DataFrameError]
    def get_i8(self, name: &str, row: i64) -> Result[Option[i8], DataFrameError]
    def values_i8(self, name: &str) -> Result[list[i8], DataFrameError]
    def values_i8_nullable(self, name: &str) -> Result[list[Option[i8]], DataFrameError]
    def get_i16(self, name: &str, row: i64) -> Result[Option[i16], DataFrameError]
    def values_i16(self, name: &str) -> Result[list[i16], DataFrameError]
    def values_i16_nullable(self, name: &str) -> Result[list[Option[i16]], DataFrameError]
    def get_i32(self, name: &str, row: i64) -> Result[Option[i32], DataFrameError]
    def values_i32(self, name: &str) -> Result[list[i32], DataFrameError]
    def values_i32_nullable(self, name: &str) -> Result[list[Option[i32]], DataFrameError]
    def get_u8(self, name: &str, row: i64) -> Result[Option[u8], DataFrameError]
    def values_u8(self, name: &str) -> Result[list[u8], DataFrameError]
    def values_u8_nullable(self, name: &str) -> Result[list[Option[u8]], DataFrameError]
    def get_u16(self, name: &str, row: i64) -> Result[Option[u16], DataFrameError]
    def values_u16(self, name: &str) -> Result[list[u16], DataFrameError]
    def values_u16_nullable(self, name: &str) -> Result[list[Option[u16]], DataFrameError]
    def get_u32(self, name: &str, row: i64) -> Result[Option[u32], DataFrameError]
    def values_u32(self, name: &str) -> Result[list[u32], DataFrameError]
    def values_u32_nullable(self, name: &str) -> Result[list[Option[u32]], DataFrameError]
    def get_u64(self, name: &str, row: i64) -> Result[Option[u64], DataFrameError]
    def values_u64(self, name: &str) -> Result[list[u64], DataFrameError]
    def values_u64_nullable(self, name: &str) -> Result[list[Option[u64]], DataFrameError]
    def get_f32(self, name: &str, row: i64) -> Result[Option[f32], DataFrameError]
    def values_f32(self, name: &str) -> Result[list[f32], DataFrameError]
    def values_f32_nullable(self, name: &str) -> Result[list[Option[f32]], DataFrameError]
    def get_i64(self, name: &str, row: i64) -> Result[Option[i64], DataFrameError]
    def values_i64(self, name: &str) -> Result[list[i64], DataFrameError]
    def get_f64(self, name: &str, row: i64) -> Result[Option[f64], DataFrameError]
    def values_f64(self, name: &str) -> Result[list[f64], DataFrameError]
    def schema(self) -> Result[list[Field], DataFrameError]
    def get_bool(self, name: &str, row: i64) -> Result[Option[bool], DataFrameError]
    def get_binary(self, name: &str, row: i64) -> Result[Option[Bytes], DataFrameError]
    def values_binary(self, name: &str) -> Result[list[Bytes], DataFrameError]
    def values_binary_nullable(self, name: &str) -> Result[list[Option[Bytes]], DataFrameError]
    def get_str(self, name: &str, row: i64) -> Result[Option[str], DataFrameError]
    def values_bool(self, name: &str) -> Result[list[bool], DataFrameError]
    def values_str(self, name: &str) -> Result[list[str], DataFrameError]
    def values_str_nullable(self, name: &str) -> Result[list[Option[str]], DataFrameError]
    def values_i64_nullable(self, name: &str) -> Result[list[Option[i64]], DataFrameError]
    def values_f64_nullable(self, name: &str) -> Result[list[Option[f64]], DataFrameError]
    def values_bool_nullable(self, name: &str) -> Result[list[Option[bool]], DataFrameError]
class Expr
    def list_len(self) -> Result[Expr, DataFrameError]
    def list_get(self, index: i64) -> Result[Expr, DataFrameError]
    def struct_field(self, name: &str) -> Result[Expr, DataFrameError]
    def str_to_date(self, format: &str = "%Y-%m-%d", strict: bool = True) -> Result[Expr, DataFrameError]
    def str_to_datetime(self, format: &str, unit: TimeUnit, utc: bool = False, strict: bool = True) -> Result[Expr, DataFrameError]
    def dt_year(self) -> Result[Expr, DataFrameError]
    def dt_month(self) -> Result[Expr, DataFrameError]
    def dt_day(self) -> Result[Expr, DataFrameError]
    def dt_hour(self) -> Result[Expr, DataFrameError]
    def dt_minute(self) -> Result[Expr, DataFrameError]
    def dt_second(self) -> Result[Expr, DataFrameError]
    def dt_date(self) -> Result[Expr, DataFrameError]
    def dt_format(self, format: &str) -> Result[Expr, DataFrameError]
    def cast(self, dtype: DType, strict: bool = True) -> Result[Expr, DataFrameError]
    def str_lower(self) -> Result[Expr, DataFrameError]
    def str_upper(self) -> Result[Expr, DataFrameError]
    def str_len(self) -> Result[Expr, DataFrameError]
    def str_contains(self, pattern: &Expr) -> Result[Expr, DataFrameError]
    def str_starts_with(self, prefix: &Expr) -> Result[Expr, DataFrameError]
    def str_ends_with(self, suffix: &Expr) -> Result[Expr, DataFrameError]
    def n_unique(self) -> Result[Expr, DataFrameError]
    def over(self, partition_by: &list[Expr]) -> Result[Expr, DataFrameError]
    def gt(self, other: &Expr) -> Result[Expr, DataFrameError]
    def mul(self, other: &Expr) -> Result[Expr, DataFrameError]
    def add(self, other: &Expr) -> Result[Expr, DataFrameError]
    def sub(self, other: &Expr) -> Result[Expr, DataFrameError]
    def div(self, other: &Expr) -> Result[Expr, DataFrameError]
    def eq(self, other: &Expr) -> Result[Expr, DataFrameError]
    def lt(self, other: &Expr) -> Result[Expr, DataFrameError]
    def ge(self, other: &Expr) -> Result[Expr, DataFrameError]
    def le(self, other: &Expr) -> Result[Expr, DataFrameError]
    def and_(self, other: &Expr) -> Result[Expr, DataFrameError]
    def or_(self, other: &Expr) -> Result[Expr, DataFrameError]
    def fill_null(self, other: &Expr) -> Result[Expr, DataFrameError]
    def sum(self) -> Result[Expr, DataFrameError]
    def count(self) -> Result[Expr, DataFrameError]
    def mean(self) -> Result[Expr, DataFrameError]
    def min(self) -> Result[Expr, DataFrameError]
    def max(self) -> Result[Expr, DataFrameError]
    def is_null(self) -> Result[Expr, DataFrameError]
    def alias(self, name: &str) -> Result[Expr, DataFrameError]
class Query
    def schema_tree(self) -> Result[list[SchemaNode], DataFrameError]
    def schema(self) -> Result[list[Field], DataFrameError]
    def filter(self, predicate: &Expr) -> Result[Query, DataFrameError]
    def with_columns(self, expressions: &list[Expr]) -> Result[Query, DataFrameError]
    def select(self, expressions: &list[Expr]) -> Result[Query, DataFrameError]
    def group_by(self, keys: &list[Expr], aggregates: &list[Expr]) -> Result[Query, DataFrameError]
    def collect(self) -> Result[DataFrame, DataFrameError]
    def collect_streaming(self) -> Result[DataFrame, DataFrameError]
    def sink_parquet(self, path: &str) -> Result[(), DataFrameError]
    def join(self, other: &Query, left_keys: &list[Expr], right_keys: &list[Expr], how: JoinKind = JoinKind.Inner, suffix: &str = "_right", nulls_equal: bool = False) -> Result[Query, DataFrameError]
    def join_asof(self, other: &Query, left_on: &str, right_on: &str, strategy: AsOfStrategy = AsOfStrategy.Backward, tolerance: Option[i64] = Nothing, allow_exact_matches: bool = True, suffix: &str = "_right") -> Result[Query, DataFrameError]
    def sort(self, keys: &list[SortKey]) -> Result[Query, DataFrameError]
    def slice(self, offset: i64, length: i64) -> Result[Query, DataFrameError]
    def limit(self, count: i64) -> Result[Query, DataFrameError]
def column_list(name: &str, values: &Column, offsets: &list[i64]) -> Result[Column, DataFrameError]
def column_list_nullable(name: &str, values: &Column, offsets: &list[i64], valid: &list[bool]) -> Result[Column, DataFrameError]
def column_struct(name: &str, fields: &DataFrame) -> Result[Column, DataFrameError]
def column_struct_nullable(name: &str, fields: &DataFrame, valid: &list[bool]) -> Result[Column, DataFrameError]
def frame(columns: &list[Column]) -> Result[DataFrame, DataFrameError]
def column_i8(name: &str, values: &list[i8]) -> Result[Column, DataFrameError]
def column_i8_nullable(name: &str, values: &list[Option[i8]]) -> Result[Column, DataFrameError]
def lit_i8(value: i8) -> Result[Expr, DataFrameError]
def column_i16(name: &str, values: &list[i16]) -> Result[Column, DataFrameError]
def column_i16_nullable(name: &str, values: &list[Option[i16]]) -> Result[Column, DataFrameError]
def lit_i16(value: i16) -> Result[Expr, DataFrameError]
def column_i32(name: &str, values: &list[i32]) -> Result[Column, DataFrameError]
def column_i32_nullable(name: &str, values: &list[Option[i32]]) -> Result[Column, DataFrameError]
def lit_i32(value: i32) -> Result[Expr, DataFrameError]
def column_u8(name: &str, values: &list[u8]) -> Result[Column, DataFrameError]
def column_u8_nullable(name: &str, values: &list[Option[u8]]) -> Result[Column, DataFrameError]
def lit_u8(value: u8) -> Result[Expr, DataFrameError]
def column_u16(name: &str, values: &list[u16]) -> Result[Column, DataFrameError]
def column_u16_nullable(name: &str, values: &list[Option[u16]]) -> Result[Column, DataFrameError]
def lit_u16(value: u16) -> Result[Expr, DataFrameError]
def column_u32(name: &str, values: &list[u32]) -> Result[Column, DataFrameError]
def column_u32_nullable(name: &str, values: &list[Option[u32]]) -> Result[Column, DataFrameError]
def lit_u32(value: u32) -> Result[Expr, DataFrameError]
def column_u64(name: &str, values: &list[u64]) -> Result[Column, DataFrameError]
def column_u64_nullable(name: &str, values: &list[Option[u64]]) -> Result[Column, DataFrameError]
def lit_u64(value: u64) -> Result[Expr, DataFrameError]
def column_f32(name: &str, values: &list[f32]) -> Result[Column, DataFrameError]
def column_f32_nullable(name: &str, values: &list[Option[f32]]) -> Result[Column, DataFrameError]
def lit_f32(value: f32) -> Result[Expr, DataFrameError]
def column_i64(name: &str, values: &list[i64]) -> Result[Column, DataFrameError]
def column_date(name: &str, days: &list[i32]) -> Result[Column, DataFrameError]
def column_date_nullable(name: &str, days: &list[Option[i32]]) -> Result[Column, DataFrameError]
def column_datetime(name: &str, ticks: &list[i64], unit: TimeUnit, utc: bool = False) -> Result[Column, DataFrameError]
def column_datetime_nullable(name: &str, ticks: &list[Option[i64]], unit: TimeUnit, utc: bool = False) -> Result[Column, DataFrameError]
def column_f64(name: &str, values: &list[f64]) -> Result[Column, DataFrameError]
def col(name: &str) -> Result[Expr, DataFrameError]
def lit(value: f64) -> Result[Expr, DataFrameError]
def lit_i64(value: i64) -> Result[Expr, DataFrameError]
def lit_str(value: &str) -> Result[Expr, DataFrameError]
def lit_bool(value: bool) -> Result[Expr, DataFrameError]
def lit_binary(value: &[u8]) -> Result[Expr, DataFrameError]
def when(predicate: &Expr, then_value: &Expr, otherwise: &Expr) -> Result[Expr, DataFrameError]
def null_f64() -> Result[Expr, DataFrameError]
def scan_parquet(path: &str) -> Result[Query, DataFrameError]
def scan_csv(path: &str, has_header: bool = True, separator: &str = ",", infer_rows: i64 = 100, null_value: Option[str] = Nothing, schema_overrides: Option[list[Field]] = Nothing) -> Result[Query, DataFrameError]
def column_bool(name: &str, values: &list[bool]) -> Result[Column, DataFrameError]
def column_binary(name: &str, values: &list[Bytes]) -> Result[Column, DataFrameError]
def column_binary_nullable(name: &str, values: &list[Option[Bytes]]) -> Result[Column, DataFrameError]
def column_str(name: &str, values: &list[str]) -> Result[Column, DataFrameError]
def column_i64_nullable(name: &str, values: &list[Option[i64]]) -> Result[Column, DataFrameError]
def column_f64_nullable(name: &str, values: &list[Option[f64]]) -> Result[Column, DataFrameError]
def column_bool_nullable(name: &str, values: &list[Option[bool]]) -> Result[Column, DataFrameError]
def column_str_nullable(name: &str, values: &list[Option[str]]) -> Result[Column, DataFrameError]

std.db.kv

Ordered byte-key maps, atomic transactions and bounded scans over a versioned SQLite store. Errors: KvError. Reference.

enum Mode
    ReadOnly | ReadWrite | Create
enum KvError
    Database(sqlite.DbError) | Allocation(AllocError) | InvalidName | ForeignDatabase | UnsupportedVersion(i64)
const format_version: i64 = 1
class Entry
    key: Bytes
    value: Bytes
class Scan
    def next(self: &mut Scan) -> Result[Option[Entry], KvError]
class Transaction
    def commit(self: &mut Transaction) -> Result[(), KvError]
    def rollback(self: &mut Transaction) -> Result[(), KvError]
class Store
    def begin(self: &mut Store, immediate: bool = True) -> Result[Transaction, KvError]
    def set_busy_timeout(self: &mut Store, budget: &Duration) -> Result[(), KvError]
    def close(self: &mut Store) -> Result[(), KvError]
    def put(self: &mut Store, table: &str, key: &[u8], value: &[u8]) -> Result[(), KvError]
    def get(self: &mut Store, table: &str, key: &[u8]) -> Result[Option[Bytes], KvError]
    def contains(self: &mut Store, table: &str, key: &[u8]) -> Result[bool, KvError]
    def remove(self: &mut Store, table: &str, key: &[u8]) -> Result[bool, KvError]
    def clear(self: &mut Store, table: &str) -> Result[i64, KvError]
    def len(self: &mut Store, table: &str) -> Result[i64, KvError]
    def scan(self: &mut Store, table: &str, start: &[u8], stop: &[u8], max_rows: NonNegative[i64], after: bool = False, reverse: bool = False) -> Result[Scan, KvError]
    def scan_all(self: &mut Store, table: &str, max_rows: NonNegative[i64], reverse: bool = False) -> Result[Scan, KvError]
def connect(path: &str, mode: Mode = Mode.Create, max_length: i64 = 1048576) -> Result[Store, KvError]
def memory(max_length: i64 = 1048576) -> Result[Store, KvError]

std.db.sqlite

SQLite databases through prepared statements with typed parameters and reads, owned transactions that roll back on drop, and an explicit value-size limit. Errors: DbError. Reference.

enum Mode
    ReadOnly | ReadWrite | Create
enum Begin
    Deferred | Immediate | Exclusive
enum ValueType
    Integer | Real | Text | Blob | Null
class SqliteError
    code: i64
    message: str
    def primary(self) -> i64
enum DbError
    Sqlite(SqliteError) | Closed | InUse | EmptySql | TrailingSql | TransactionActive | TransactionFinished | InvalidPath | InvalidExtensionEntry | OutOfRange | NoRow | TypeMismatch(ValueType) | TooLarge | InvalidUtf8 | Allocation(AllocError)
const error: i64 = 1
const busy: i64 = 5
const locked: i64 = 6
const read_only: i64 = 8
const cant_open: i64 = 14
const constraint: i64 = 19
const mismatch: i64 = 20
const min_length_limit: i64 = 1024
const max_length_limit: i64 = 1000000000
class Connection
    def execute(self: &mut Connection, sql: &str) -> Result[(), DbError]
    def prepare(self: &mut Connection, sql: &str) -> Result[Statement, DbError]
    def begin(self: &mut Connection, mode: Begin) -> Result[Transaction, DbError]
    def changes(self) -> i64
    def last_insert_rowid(self) -> i64
    def set_busy_timeout(self: &mut Connection, budget: &Duration) -> Result[(), DbError]
    def load_extension(self: &mut Connection, path: &str, entrypoint: &str = "") -> Result[(), DbError]
    def close(self: &mut Connection) -> Result[(), DbError]
def connect(path: &str, mode: Mode, max_length: i64) -> Result[Connection, DbError]
def memory(max_length: i64) -> Result[Connection, DbError]
class Statement
    def parameter_count(self) -> i64
    def parameter_index(self, name: &str) -> Result[i64, DbError]
    def bind_null(self: &mut Statement, index: i64) -> Result[(), DbError]
    def bind_integer(self: &mut Statement, index: i64, value: i64) -> Result[(), DbError]
    def bind_real(self: &mut Statement, index: i64, value: f64) -> Result[(), DbError]
    def bind_text(self: &mut Statement, index: i64, value: &str) -> Result[(), DbError]
    def bind_blob(self: &mut Statement, index: i64, value: &[u8]) -> Result[(), DbError]
    def clear_bindings(self: &mut Statement) -> Result[(), DbError]
    def step(self: &mut Statement) -> Result[bool, DbError]
    def reset(self: &mut Statement) -> ()
    def run(self: &mut Statement) -> Result[(), DbError]
    def column_count(self) -> i64
    def column_name(self, column: i64) -> Result[str, DbError]
    def kind(self, column: i64) -> Result[ValueType, DbError]
    def is_null(self, column: i64) -> Result[bool, DbError]
    def integer(self, column: i64) -> Result[i64, DbError]
    def real(self, column: i64) -> Result[f64, DbError]
    def text(self, column: i64) -> Result[str, DbError]
    def blob(self, column: i64) -> Result[Bytes, DbError]
    def optional_integer(self, column: i64) -> Result[Option[i64], DbError]
    def optional_real(self, column: i64) -> Result[Option[f64], DbError]
    def optional_text(self, column: i64) -> Result[Option[str], DbError]
    def optional_blob(self, column: i64) -> Result[Option[Bytes], DbError]
class Transaction
    def commit(self: &mut Transaction) -> Result[(), DbError]
    def rollback(self: &mut Transaction) -> Result[(), DbError]

std.deque

Owned generic ring buffers with constant-time removal at either end. Errors: AllocError. Reference.

class Deque[T]
    def __init__(self) -> Result[(), AllocError]
    def len(self) -> i64
    def is_empty(self) -> bool
    def append(self: &mut Deque[T], value: T) -> Result[(), AllocError]
    def appendleft(self: &mut Deque[T], value: T) -> Result[(), AllocError]
    def pop(self: &mut Deque[T]) -> Option[T]
    def popleft(self: &mut Deque[T]) -> Option[T]
    def clear(self: &mut Deque[T]) -> ()

std.diff

Line, word, character, byte, and list differences as edit operations, with Myers and patience algorithms and unified-diff output. Errors: AllocError. Reference.

enum Algorithm
    Myers | Patience
enum Kind
    Equal | Delete | Insert
class Op
    kind: Kind
    old_start: i64
    old_end: i64
    new_start: i64
    new_end: i64
    def old_len(self) -> Result[i64, OverflowError[i64]]
    def new_len(self) -> Result[i64, OverflowError[i64]]
    def is_equal(self) -> bool
def diff[T](old: &list[T], new: &list[T], algorithm: Algorithm = Algorithm.Myers, max_cost: Option[i64] = Nothing) -> Result[list[Op], AllocError]
def diff_bytes(old: &[u8], new: &[u8], algorithm: Algorithm = Algorithm.Myers, max_cost: Option[i64] = Nothing) -> Result[list[Op], AllocError]
class TextDiff
    def ops(self) -> &list[Op]
    def old_tokens(self) -> &list[str]
    def new_tokens(self) -> &list[str]
    def has_changes(self) -> bool
    def unified(self, context: i64, old_name: &str, new_name: &str) -> Result[str, AllocError]
def lines(old: &str, new: &str, algorithm: Algorithm = Algorithm.Myers, max_cost: Option[i64] = Nothing) -> Result[TextDiff, AllocError]
def words(old: &str, new: &str, algorithm: Algorithm = Algorithm.Myers, max_cost: Option[i64] = Nothing) -> Result[TextDiff, AllocError]
def chars(old: &str, new: &str, algorithm: Algorithm = Algorithm.Myers, max_cost: Option[i64] = Nothing) -> Result[TextDiff, AllocError]

std.encoding

Hex, base32/base64, MIME base64 and validated custom alphabets, plus fixed-width integer and IEEE 754 float packing with explicit byte order, over std.bytes. Errors: EncodingError. Reference.

enum EncodingError
    InvalidLength | InvalidByte(i64) | InvalidPadding(i64) | NonCanonical(i64) | InvalidAlphabet | InvalidPem(i64) | PemLabel | Allocation(AllocError)
enum Endian
    Little | Big
class Encoding
    def encode(self, source: &[u8]) -> Result[Bytes, EncodingError]
    def decode(self, source: &[u8]) -> Result[Bytes, EncodingError]
    def text(self, source: &[u8]) -> Result[str, EncodingError]
    def from_text(self, text: &str) -> Result[Bytes, EncodingError]
def custom(symbols: &str, padding: Option[u8] = Some(61u8)) -> Result[Encoding, EncodingError]
def base32_encode(source: &[u8]) -> Result[Bytes, EncodingError]
def base32_decode(source: &[u8]) -> Result[Bytes, EncodingError]
def base32_text(source: &[u8]) -> Result[str, EncodingError]
def base32_from_text(text: &str) -> Result[Bytes, EncodingError]
def base32hex_encode(source: &[u8]) -> Result[Bytes, EncodingError]
def base32hex_decode(source: &[u8]) -> Result[Bytes, EncodingError]
def base32hex_text(source: &[u8]) -> Result[str, EncodingError]
def base32hex_from_text(text: &str) -> Result[Bytes, EncodingError]
def base64mime_encode(source: &[u8]) -> Result[Bytes, EncodingError]
def base64mime_decode(source: &[u8]) -> Result[Bytes, EncodingError]
def base64mime_text(source: &[u8]) -> Result[str, EncodingError]
def base64mime_from_text(text: &str) -> Result[Bytes, EncodingError]
def pem_encode(source: &[u8], label: &[u8]) -> Result[Bytes, EncodingError]
def pem_text(source: &[u8], label: &[u8]) -> Result[str, EncodingError]
def pem_decode(source: &[u8], label: &[u8]) -> Result[Bytes, EncodingError]
def pem_from_text(text: &str, label: &[u8]) -> Result[Bytes, EncodingError]
def hex_encode(source: &[u8]) -> Result[Bytes, EncodingError]
def hex_decode(source: &[u8]) -> Result[Bytes, EncodingError]
def hex_text(source: &[u8]) -> Result[str, EncodingError]
def hex_from_text(text: &str) -> Result[Bytes, EncodingError]
def base64_encode(source: &[u8]) -> Result[Bytes, EncodingError]
def base64_decode(source: &[u8]) -> Result[Bytes, EncodingError]
def base64_text(source: &[u8]) -> Result[str, EncodingError]
def base64_from_text(text: &str) -> Result[Bytes, EncodingError]
def base64url_encode(source: &[u8]) -> Result[Bytes, EncodingError]
def base64url_decode(source: &[u8]) -> Result[Bytes, EncodingError]
def base64url_text(source: &[u8]) -> Result[str, EncodingError]
def base64url_from_text(text: &str) -> Result[Bytes, EncodingError]
def pack_u16(target: &mut Bytes, offset: i64, value: u16, order: Endian) -> ()
def pack_u32(target: &mut Bytes, offset: i64, value: u32, order: Endian) -> ()
def pack_u64(target: &mut Bytes, offset: i64, value: u64, order: Endian) -> ()
def pack_i16(target: &mut Bytes, offset: i64, value: i16, order: Endian) -> ()
def pack_i32(target: &mut Bytes, offset: i64, value: i32, order: Endian) -> ()
def pack_i64(target: &mut Bytes, offset: i64, value: i64, order: Endian) -> ()
def append_u16(target: &mut Bytes, value: u16, order: Endian) -> Result[(), EncodingError]
def append_u32(target: &mut Bytes, value: u32, order: Endian) -> Result[(), EncodingError]
def append_u64(target: &mut Bytes, value: u64, order: Endian) -> Result[(), EncodingError]
def append_i16(target: &mut Bytes, value: i16, order: Endian) -> Result[(), EncodingError]
def append_i32(target: &mut Bytes, value: i32, order: Endian) -> Result[(), EncodingError]
def append_i64(target: &mut Bytes, value: i64, order: Endian) -> Result[(), EncodingError]
def unpack_u16(source: &[u8], offset: i64, order: Endian) -> u16
def unpack_u32(source: &[u8], offset: i64, order: Endian) -> u32
def unpack_u64(source: &[u8], offset: i64, order: Endian) -> u64
def unpack_i16(source: &[u8], offset: i64, order: Endian) -> i16
def unpack_i32(source: &[u8], offset: i64, order: Endian) -> i32
def unpack_i64(source: &[u8], offset: i64, order: Endian) -> i64
def pack_f32(target: &mut Bytes, offset: i64, value: f32, order: Endian) -> ()
def pack_f64(target: &mut Bytes, offset: i64, value: f64, order: Endian) -> ()
def append_f32(target: &mut Bytes, value: f32, order: Endian) -> Result[(), EncodingError]
def append_f64(target: &mut Bytes, value: f64, order: Endian) -> Result[(), EncodingError]
def unpack_f32(source: &[u8], offset: i64, order: Endian) -> f32
def unpack_f64(source: &[u8], offset: i64, order: Endian) -> f64

std.env

Environment variables as text or bytes, a detached snapshot, and the working directory. Errors: IoError. Reference.

class Variable
    name: list[u8]
    value: list[u8]
def valid_name(name: &list[u8]) -> bool
def get_bytes(name: &list[u8]) -> Result[Option[list[u8]], IoError]
def get(name: &str) -> Result[Option[str], IoError]
def vars() -> Result[list[Variable], IoError]
def current_dir() -> Result[Path, IoError]
def set_current_dir(path: &Path) -> Result[(), IoError]

std.error

Explicit error mapping, operation/path context, descriptions, and standard-error reports. Errors: Mapper-selected error, Context[E], AllocError. Reference.

def map_error[T, E, G, F: OnceCallable[[E], G]](result: Result[T, E], mapper: F) -> Result[T, G]
enum Subject
    Absent | At(Path) | Unavailable(AllocError)
class Context[E]
    operation: str
    subject: Subject
    cause: E
    def format[W](self, output: &mut Formatter[W]) -> Result[(), FormatError]
    def describe(self) -> Result[str, AllocError]
    def describe_with(self, cause: &str) -> Result[str, AllocError]
    def report(self) -> ()
    def report_with(self, description: Result[str, AllocError]) -> ()
def context[T, E](result: Result[T, E], operation: str) -> Result[T, Context[E]]
def path_context[T, E](result: Result[T, E], operation: str, path: &Path) -> Result[T, Context[E]]
def native_code(error: IoError) -> Option[i32]

std.fmt

Writer-generic value/report formatting, bounded output and fallible strings. Errors: FormatError. Reference.

enum FormatError
    InvalidSpec | LimitExceeded | RecursionLimit | Allocation(AllocError) | Output(StreamError) | OutputClosed
class Spec
    fill: str
    align: str
    sign: str
    coerce_zero: bool
    alternate: bool
    zero: bool
    width: i64
    grouping: str
    precision: i64
    kind: str
def spec(fill: str = " ", align: str = "", sign: str = "", coerce_zero: bool = False, alternate: bool = False, zero: bool = False, width: i64 = 0, grouping: str = "", precision: i64 = -1, kind: str = "") -> Spec
def spec_fill(fill: &str, align: &str) -> Result[str, FormatError]
def spec_align(align: &str) -> Result[str, FormatError]
def spec_count(value: i64) -> Result[i64, FormatError]
class Formatter[W]
    writer: W
    def completed(self) -> i64
    def has_failed(self) -> bool
    def text(self: &mut Formatter[W], value: &str) -> Result[(), FormatError]
    def substr(self: &mut Formatter[W], value: &substr) -> Result[(), FormatError]
    def boolean(self: &mut Formatter[W], value: bool) -> Result[(), FormatError]
    def floating(self: &mut Formatter[W], value: f64, width: NonNegative[i64]) -> Result[(), FormatError]
    def unsigned(self: &mut Formatter[W], value: u64, base: i64, width: NonNegative[i64] = 0) -> Result[(), FormatError]
    def integer(self: &mut Formatter[W], value: i64, width: NonNegative[i64] = 0) -> Result[(), FormatError]
    def debug_text(self: &mut Formatter[W], value: &str) -> Result[(), FormatError]
    def debug_substr(self: &mut Formatter[W], value: &substr) -> Result[(), FormatError]
    def ascii_text(self: &mut Formatter[W], value: &str) -> Result[(), FormatError]
    def char_spec(self: &mut Formatter[W], code: i64, spec: &Spec) -> Result[(), FormatError]
    def text_spec(self: &mut Formatter[W], value: &str, spec: &Spec) -> Result[(), FormatError]
    def substr_spec(self: &mut Formatter[W], value: &substr, spec: &Spec) -> Result[(), FormatError]
    def signed_spec(self: &mut Formatter[W], value: i64, spec: &Spec) -> Result[(), FormatError]
    def unsigned_spec(self: &mut Formatter[W], value: u64, spec: &Spec) -> Result[(), FormatError]
    def float_spec(self: &mut Formatter[W], value: f64, spec: &Spec) -> Result[(), FormatError]
    def float32_spec(self: &mut Formatter[W], value: f32, spec: &Spec) -> Result[(), FormatError]
    def enter(self: &mut Formatter[W]) -> Result[(), FormatError]
    def leave(self: &mut Formatter[W]) -> ()
def formatter[W: Writer](writer: W, scratch: Bytes, max_bytes: NonNegative[i64], max_depth: NonNegative[i64]) -> Result[Formatter[W], FormatError]
def stderr_formatter(max_bytes: NonNegative[i64], max_depth: NonNegative[i64]) -> Result[Formatter[Stderr], FormatError]
def string_formatter(max_bytes: NonNegative[i64], max_depth: NonNegative[i64]) -> Result[Formatter[Cursor], FormatError]
def finish(output: &Formatter[Cursor]) -> Result[str, FormatError]
def display[T, W](value: &T, output: &mut Formatter[W]) -> Result[(), FormatError]
def debug[T, W](value: &T, output: &mut Formatter[W]) -> Result[(), FormatError]
def to_string[T](value: &T, max_bytes: NonNegative[i64], max_depth: NonNegative[i64]) -> Result[str, FormatError]
def to_debug[T](value: &T, max_bytes: NonNegative[i64], max_depth: NonNegative[i64]) -> Result[str, FormatError]
def represent[T, W](value: &T, output: &mut Formatter[W]) -> Result[(), FormatError]
def repr_string[T](value: &T) -> Result[str, FormatError]
def represent_view[T, W](value: &[T], output: &mut Formatter[W]) -> Result[(), FormatError]
def repr_view_string[T](value: &[T]) -> Result[str, FormatError]
def repr_text[W](spelled: Result[str, AllocError | FormatError], output: &mut Formatter[W]) -> Result[(), FormatError]
def repr_result(spelled: Result[str, AllocError | FormatError]) -> Result[str, FormatError]
def allocation_only(built: Result[str, FormatError]) -> Result[str, AllocError]
def allocation_or_format(built: Result[str, FormatError]) -> Result[str, AllocError | FormatError]
def debug_string(value: &str) -> Result[str, FormatError]
def repr_substr_string(value: &substr) -> Result[str, FormatError]
def ascii_string(value: &str) -> Result[str, FormatError]
def list_with[T, W, F: Callable[[&T, &mut Formatter[W]], Result[(), FormatError]]](values: &list[T], output: &mut Formatter[W], render: F) -> Result[(), FormatError]
def pair_with[A, B, W, F: Callable[[&A, &mut Formatter[W]], Result[(), FormatError]], G: Callable[[&B, &mut Formatter[W]], Result[(), FormatError]]](value: &(A, B), output: &mut Formatter[W], first: F, second: G) -> Result[(), FormatError]
protocol Formattable
    def format[W](self, output: &mut Formatter[W]) -> Result[(), FormatError]

std.fs

Directories, metadata/walking, symbolic links and canonical paths, rooted mutation, owned temporary resources, bounded file/tree copy, metadata preservation, delete/glob. Errors: IoError, AllocError, WalkError, FsError, TreeCopyError. Reference.

enum FsError
    Io(IoError) | Stream(StreamError) | Allocation(AllocError) | Entropy(SecretsError) | InvalidInput | Limit | Cancelled | Replaced | Collisions | UnsupportedType | Overlap | InvalidLimits
class Root
    def open_dir(self, name: &Path) -> Result[Root, IoError]
    def read_file(self, name: &Path) -> Result[FileStream, IoError]
    def create_dir(self, name: &Path, unix_mode: u32 = 0o700u32) -> Result[(), IoError]
    def create_file(self, name: &Path, unix_mode: u32 = 0o600u32) -> Result[FileStream, IoError]
    def set_unix_mode(self, unix_mode: u32) -> Result[(), IoError]
    def remove_file(self, name: &Path) -> Result[(), IoError]
    def remove_dir(self, name: &Path) -> Result[(), IoError]
    def move(self, source: &Path, destination: &Path) -> Result[(), IoError]
    def names(self) -> Result[Names, IoError]
def set_file_unix_mode(stream: &FileStream, unix_mode: u32) -> Result[(), IoError]
def root(path: &Path) -> Result[Root, IoError]
class Names
    def next(self: &mut Names) -> Result[Option[Path], IoError]
    def close(self: &mut Names) -> Result[(), IoError]
class TempFile
    stream: FileStream
    def publish(self: &mut TempFile, name: &Path) -> Result[(), IoError]
    def close(self: &mut TempFile) -> Result[(), StreamError]
    def remove_published(self: &mut TempFile, name: &Path) -> Result[(), FsError]
def temporary_file(parent: &Root) -> Result[TempFile, IoError]
class TempDir
    def name(self) -> &Path
    def root(self) -> &Root
    def cleanup(self: &mut TempDir) -> Result[(), FsError]
    def keep(self: &mut TempDir) -> ()
    def cleanup_tree(self: &mut TempDir, max_entries: Positive[i64], max_depth: Positive[i64], cancel: &CancellationToken) -> Result[(), FsError]
def temporary_dir_named(parent: &Root, name: &Path) -> Result[TempDir, FsError]
def temporary_dir(parent: &Root, prefix: &Path, attempts: Positive[i64]) -> Result[TempDir, FsError]
enum CopyMetadata
    ContentsOnly | PermissionsAndTimes
def copy_from[R: Reader](source: &mut R, parent: &Root, destination: &Path, max_bytes: NonNegative[i64], cancel: &CancellationToken) -> Result[i64, FsError]
def copy_file(parent: &Root, source: &Path, destination: &Path, max_bytes: NonNegative[i64], cancel: &CancellationToken, preserve: CopyMetadata = CopyMetadata.ContentsOnly) -> Result[i64, FsError]
class CopyLimits
    max_entries: i64
    max_depth: i64
    max_bytes: i64
    max_open_handles: i64
def copy_limits() -> CopyLimits
enum TreeCopyError
    Setup(FsError) | Partial(FsError) | Entry(Path, i64, FsError)
def copy_tree(source_parent: &Root, source: &Path, destination_parent: &Root, destination: &Path, limits: &CopyLimits, cancel: &CancellationToken, preserve: CopyMetadata = CopyMetadata.ContentsOnly) -> Result[i64, TreeCopyError]
def remove_tree(parent: &Root, name: &Path, max_entries: Positive[i64], max_depth: Positive[i64], cancel: &CancellationToken) -> Result[(), FsError]
def glob(parent: &Root, pattern: &Path, max_entries: NonNegative[i64], max_matches: NonNegative[i64], max_steps: NonNegative[i64]) -> Result[list[Path], FsError]
enum FileType
    File | Directory | Symlink | Other
class Permissions
    readonly: bool
    unix_mode: u32
class Metadata
    def file_type(self) -> FileType
    def is_file(self) -> bool
    def is_dir(self) -> bool
    def is_symlink(self) -> bool
    def size(self) -> u64
    def permissions(self) -> Permissions
    def modified(self) -> Option[Timestamp]
    def accessed(self) -> Option[Timestamp]
    def created(self) -> Option[Timestamp]
    def unix_changed(self) -> Timestamp
    def unix_uid(self) -> u32
    def unix_gid(self) -> u32
    def unix_device(self) -> u64
    def unix_inode(self) -> u64
class DirEntry
    def file_name(self) -> Result[Path, IoError]
    def path(self) -> Result[Path, IoError]
    def file_type(self) -> Result[FileType, IoError]
    def metadata(self) -> Result[Metadata, IoError]
class ReadDir
    def next(self: &mut ReadDir) -> Result[Option[DirEntry], IoError]
    def close(self: &mut ReadDir) -> Result[(), IoError]
    def closed(self) -> bool
def read_dir(path: &Path) -> Result[ReadDir, IoError]
def metadata(path: &Path) -> Result[Metadata, IoError]
def symlink_metadata(path: &Path) -> Result[Metadata, IoError]
def file_metadata(stream: &FileStream) -> Result[Metadata, IoError]
def read_link(path: &Path) -> Result[Path, IoError]
def canonicalize(path: &Path) -> Result[Path, IoError]
def symlink(target: &Path, link: &Path) -> Result[(), IoError]
def entries(owner: ReadDir) -> Generator[Result[DirEntry, IoError]]
enum WalkError
    Io(Path, str, IoError) | Allocation(AllocError)
class WalkDir
    path: Path
    directories: list[Path]
    files: list[DirEntry]
class Walk
    current: WalkDir
    def close(self: &mut Walk) -> ()
    def next(self: &mut Walk) -> Result[bool, WalkError]
def walk(root: &Path, top_down: bool) -> Result[Walk, AllocError]

std.fs.dirs

The home directory and the XDG configuration, cache, data, and state directories, from the environment. Errors: IoError. Reference.

def home_dir() -> Result[Option[Path], IoError]
def config_dir() -> Result[Option[Path], IoError]
def cache_dir() -> Result[Option[Path], IoError]
def data_dir() -> Result[Option[Path], IoError]
def state_dir() -> Result[Option[Path], IoError]

std.fs.lock

Owned Linux shared/exclusive advisory locks, nonblocking and cancellable deadline acquisition. Errors: LockError. Reference.

enum Mode
    Shared | Exclusive
enum LockError
    WouldBlock | TimedOut | Cancelled | InvalidPath | Io(IoError) | Allocation(AllocError)
class Lock
    def close(self: &mut Lock) -> Result[(), LockError]
    def is_closed(self) -> bool
def acquire_now(path: &Path, mode: Mode) -> Result[Lock, LockError]
def acquire_before(path: &Path, mode: Mode, deadline: &Deadline, cancel: &CancellationToken) -> Result[Lock, LockError]

std.fs.watch

Bounded Linux filesystem change batches, rename cookies, overflow and readiness waits. Errors: WatchError, WaitError. Reference.

enum WatchError
    OutOfRange | InvalidData | Io(IoError) | Wait(WaitError) | Allocation(AllocError)
class Event
    watch: i32
    cookie: u32
    name: Path
    def created(self) -> bool
    def deleted(self) -> bool
    def modified(self) -> bool
    def metadata_changed(self) -> bool
    def closed_after_write(self) -> bool
    def moved_from(self) -> bool
    def moved_to(self) -> bool
    def deleted_self(self) -> bool
    def moved_self(self) -> bool
    def unmounted(self) -> bool
    def overflowed(self) -> bool
    def ignored(self) -> bool
    def is_directory(self) -> bool
class Watcher
    def add(self: &mut Watcher, path: &Path) -> Result[i32, WatchError]
    def remove(self: &mut Watcher, watch: i32) -> Result[(), WatchError]
    def poll(self: &mut Watcher) -> Result[i64, WatchError]
    def wait(self: &mut Watcher, deadline: &Deadline, cancel: &CancellationToken) -> Result[i64, WatchError]
    def count(self) -> i64
    def event(self, index: i64) -> Result[Event, WatchError]
    def register(self, selector: &mut Selector, key: u64) -> Result[(), WaitError]
    def unregister(self, selector: &mut Selector) -> Result[(), WaitError]
    def close(self: &mut Watcher) -> Result[(), WatchError]
def watcher(batch_bytes: i64 = 4096) -> Result[Watcher, WatchError]

std.graph

Directed and undirected graphs with node and edge data, stable handles, traversal, shortest paths, topological order, components and spanning trees. Errors: AllocError, GraphError. Reference.

type NodeId = i64
type EdgeId = i64
enum GraphError
    Cycle(list[NodeId]) | NegativeCycle(list[NodeId]) | InvalidWeight(EdgeId) | InvalidEstimate(NodeId) | Allocation(AllocError)
class Walk
class Paths
    def distance(self, node: NodeId) -> Option[f64]
    def edge_to(self, node: NodeId) -> Option[EdgeId]
    def path(self, node: NodeId) -> Result[list[NodeId], AllocError]
class DiGraph[N, E]
    def __init__(self) -> Result[(), AllocError]
    def node_count(self) -> i64
    def edge_count(self) -> i64
    def node_bound(self) -> i64
    def edge_bound(self) -> i64
    def reserve(self: &mut DiGraph[N, E], nodes: i64, edges: i64) -> Result[(), AllocError]
    def add_node(self: &mut DiGraph[N, E], data: N) -> Result[NodeId, AllocError]
    def add_edge(self: &mut DiGraph[N, E], source: NodeId, target: NodeId, data: E) -> Result[EdgeId, AllocError]
    def remove_node(self: &mut DiGraph[N, E], node: NodeId) -> Option[N]
    def remove_edge(self: &mut DiGraph[N, E], edge: EdgeId) -> Option[E]
    def contains_node(self, node: NodeId) -> bool
    def contains_edge(self, edge: EdgeId) -> bool
    def node(self, node: NodeId) -> &N
    def node_mut(self: &mut DiGraph[N, E], node: NodeId) -> &mut N
    def edge(self, edge: EdgeId) -> &E
    def edge_mut(self: &mut DiGraph[N, E], edge: EdgeId) -> &mut E
    def get_node_ref(self, node: NodeId) -> Option[&N]
    def get_node_mut(self: &mut DiGraph[N, E], node: NodeId) -> Option[&mut N]
    def get_edge_ref(self, edge: EdgeId) -> Option[&E]
    def get_edge_mut(self: &mut DiGraph[N, E], edge: EdgeId) -> Option[&mut E]
    def source(self, edge: EdgeId) -> NodeId
    def target(self, edge: EdgeId) -> NodeId
    def find_edge(self, source: NodeId, target: NodeId) -> Option[EdgeId]
    def nodes(self) -> Result[list[NodeId], AllocError]
    def edges(self) -> Result[list[EdgeId], AllocError]
    def successors(self, node: NodeId) -> Result[list[NodeId], AllocError]
    def predecessors(self, node: NodeId) -> Result[list[NodeId], AllocError]
    def outgoing(self, node: NodeId) -> Walk
    def incoming(self, node: NodeId) -> Walk
    def next_edge(self, walk: &mut Walk) -> Option[EdgeId]
    def bfs(self, start: NodeId) -> Result[list[NodeId], AllocError]
    def dfs(self, start: NodeId) -> Result[list[NodeId], AllocError]
    def toposort(self) -> Result[list[NodeId], GraphError]
    def strongly_connected_components(self) -> Result[list[list[NodeId]], AllocError]
    def dijkstra[F: Callable[[&E], f64]](self, start: NodeId, weight: &F) -> Result[Paths, GraphError]
    def astar[F: Callable[[&E], f64], H: Callable[[NodeId], f64]](self, start: NodeId, goal: NodeId, weight: &F, estimate: &H) -> Result[Option[(f64, list[NodeId])], GraphError]
    def bellman_ford[F: Callable[[&E], f64]](self, start: NodeId, weight: &F) -> Result[Paths, GraphError]
class UnGraph[N, E]
    def __init__(self) -> Result[(), AllocError]
    def node_count(self) -> i64
    def edge_count(self) -> i64
    def node_bound(self) -> i64
    def edge_bound(self) -> i64
    def reserve(self: &mut UnGraph[N, E], nodes: i64, edges: i64) -> Result[(), AllocError]
    def add_node(self: &mut UnGraph[N, E], data: N) -> Result[NodeId, AllocError]
    def add_edge(self: &mut UnGraph[N, E], a: NodeId, b: NodeId, data: E) -> Result[EdgeId, AllocError]
    def remove_node(self: &mut UnGraph[N, E], node: NodeId) -> Option[N]
    def remove_edge(self: &mut UnGraph[N, E], edge: EdgeId) -> Option[E]
    def contains_node(self, node: NodeId) -> bool
    def contains_edge(self, edge: EdgeId) -> bool
    def node(self, node: NodeId) -> &N
    def node_mut(self: &mut UnGraph[N, E], node: NodeId) -> &mut N
    def edge(self, edge: EdgeId) -> &E
    def edge_mut(self: &mut UnGraph[N, E], edge: EdgeId) -> &mut E
    def get_node_ref(self, node: NodeId) -> Option[&N]
    def get_node_mut(self: &mut UnGraph[N, E], node: NodeId) -> Option[&mut N]
    def get_edge_ref(self, edge: EdgeId) -> Option[&E]
    def get_edge_mut(self: &mut UnGraph[N, E], edge: EdgeId) -> Option[&mut E]
    def endpoints(self, edge: EdgeId) -> (NodeId, NodeId)
    def opposite(self, edge: EdgeId, node: NodeId) -> NodeId
    def find_edge(self, a: NodeId, b: NodeId) -> Option[EdgeId]
    def nodes(self) -> Result[list[NodeId], AllocError]
    def edges(self) -> Result[list[EdgeId], AllocError]
    def neighbors(self, node: NodeId) -> Result[list[NodeId], AllocError]
    def incident(self, node: NodeId) -> Walk
    def next_edge(self, walk: &mut Walk) -> Option[EdgeId]
    def bfs(self, start: NodeId) -> Result[list[NodeId], AllocError]
    def dfs(self, start: NodeId) -> Result[list[NodeId], AllocError]
    def connected_components(self) -> Result[list[list[NodeId]], AllocError]
    def dijkstra[F: Callable[[&E], f64]](self, start: NodeId, weight: &F) -> Result[Paths, GraphError]
    def astar[F: Callable[[&E], f64], H: Callable[[NodeId], f64]](self, start: NodeId, goal: NodeId, weight: &F, estimate: &H) -> Result[Option[(f64, list[NodeId])], GraphError]
    def bellman_ford[F: Callable[[&E], f64]](self, start: NodeId, weight: &F) -> Result[Paths, GraphError]
    def minimum_spanning_tree[F: Callable[[&E], f64]](self, weight: &F) -> Result[list[EdgeId], GraphError]

std.http

Bounded HTTP/1.1 message heads and streamed Content-Length, chunked, and close-delimited bodies over std.io streams. Errors: HttpError. Reference.

enum HttpError
    Stream(StreamError) | Allocation(AllocError) | InvalidLimit | Truncated | LineTooLong | HeadTooLarge | TooManyHeaders | BareLineFeed | InvalidRequestLine | InvalidStatusLine | UnsupportedVersion | InvalidHeader | InvalidContentLength | DuplicateFraming | ConflictingFraming | UnsupportedTransferEncoding | InvalidChunk | BodyTooLarge | LengthMismatch | Unfinished | Broken | InvalidTarget | MissingHost | DuplicateHost | InvalidHost
class Limits
    max_line: i64
    max_head: i64
    max_headers: NonNegative[i64]
    max_body: NonNegative[i64]
def limits() -> Limits
class Header
    name: str
    value: str
class RequestHead
    method: str
    target: str
    minor: i64
    headers: list[Header]
    def add(self: &mut RequestHead, name: str, value: str) -> Result[(), HttpError]
class ResponseHead
    minor: i64
    status: i64
    reason: str
    headers: list[Header]
    def add(self: &mut ResponseHead, name: str, value: str) -> Result[(), HttpError]
enum Framing
    Empty | Length(i64) | Chunked | UntilClose
def request(method: str, target: str) -> Result[RequestHead, HttpError]
def response(status: i64, reason: str) -> Result[ResponseHead, HttpError]
def find(headers: &list[Header], name: &str) -> Option[i64]
def same_name(left: &str, right: &str) -> bool
def parse_field(line: &[u8]) -> Result[Header, HttpError]
def parse_request_line(line: &[u8]) -> Result[RequestHead, HttpError]
def parse_status_line(line: &[u8]) -> Result[ResponseHead, HttpError]
def read_request[R: Reader](conn: &mut BufReader[R], limits: &Limits) -> Result[Option[RequestHead], HttpError]
def read_response[R: Reader](conn: &mut BufReader[R], limits: &Limits) -> Result[Option[ResponseHead], HttpError]
def request_framing(head: &RequestHead) -> Result[Framing, HttpError]
def response_framing(head: &ResponseHead, method: &str) -> Result[Framing, HttpError]
def request_keeps_alive(head: &RequestHead) -> bool
class RequestTarget
    authority: str
    path: str
    query: str
def request_target(head: &RequestHead) -> Result[RequestTarget, HttpError]
def response_keeps_alive(head: &ResponseHead, framing: &Framing) -> bool
def parse_chunk_size(line: &[u8]) -> Result[i64, HttpError]
class BodyReader[R]
    trailers: list[Header]
    def total(self) -> i64
    def is_done(self) -> bool
    def take_failure(self: &mut BodyReader[R]) -> Option[HttpError]
    def read_body(self: &mut BodyReader[R], buffer: &mut Bytes, start: i64, stop: i64) -> Result[i64, HttpError]
    def read(self: &mut BodyReader[R], buffer: &mut Bytes, start: i64, stop: i64) -> Result[i64, StreamError]
def body_reader[R: Reader](conn: BufReader[R], framing: &Framing, limits: &Limits) -> Result[BodyReader[R], HttpError]
def read_to_end[R: Reader](body: &mut BodyReader[R], out: &mut Bytes) -> Result[i64, HttpError]
def discard[R: Reader](body: &mut BodyReader[R]) -> Result[i64, HttpError]
def finish[R](body: &mut BodyReader[R]) -> Result[BufReader[R], HttpError]
def release[R](body: &mut BodyReader[R]) -> Option[BufReader[R]]
def write_request[W: Writer](writer: &mut W, head: &RequestHead) -> Result[(), HttpError]
def encode_request(head: &RequestHead) -> Result[Bytes, HttpError]
def write_response[W: Writer](writer: &mut W, head: &ResponseHead) -> Result[(), HttpError]
def encode_response(head: &ResponseHead) -> Result[Bytes, HttpError]
def write_chunk[W: Writer](writer: &mut W, source: &[u8]) -> Result[(), HttpError]
def encode_chunk_size(length: NonNegative[i64]) -> Result[Bytes, HttpError]
def write_last_chunk[W: Writer](writer: &mut W, trailers: &list[Header]) -> Result[(), HttpError]
def encode_last_chunk(trailers: &list[Header]) -> Result[Bytes, HttpError]
def write_body[W: Writer](writer: &mut W, framing: &Framing, source: &[u8]) -> Result[(), HttpError]

std.http.decoder

Resumable bounded HTTP/1.x request framing over caller-supplied bytes, without I/O. Errors: HttpError. Reference.

const feed_budget: i64 = 4096
enum Event
    Head(RequestHead, RequestTarget, Framing) | Body(Bytes) | Trailers(list[Header]) | End
class Decoder
    def is_started(self) -> bool
    def has_head(self) -> bool
    def is_done(self) -> bool
    def total(self) -> i64
    def reset(self: &mut Decoder) -> Result[(), HttpError]
    def finish(self: &mut Decoder) -> Result[(), HttpError]
    def feed(self: &mut Decoder, source: &[u8]) -> Result[(i64, Option[Event]), HttpError]
def decoder(limits: &Limits) -> Result[Decoder, HttpError]

std.http.body

Bounded request bodies for handlers: URL-encoded forms, JSON, and multipart uploads into unnamed temporary files. Errors: BodyError. Reference.

enum BodyError
    UnsupportedMediaType | TooLarge | TooManyFields | Malformed | InvalidUtf8 | Http(HttpError) | Json(JsonError) | Storage(StreamError) | Allocation(AllocError)
def status(error: &BodyError) -> i64
def reject[T: Transport](exchange: &mut Exchange[T], error: &BodyError) -> Result[(), HttpError]
class FormLimits
    max_bytes: NonNegative[i64]
    max_fields: NonNegative[i64]
def form_limits() -> FormLimits
def parse_form(source: &[u8], limits: &FormLimits) -> Result[list[Pair], BodyError]
def parse_form_text(text: &str, limits: &FormLimits) -> Result[list[Pair], BodyError]
def find(pairs: &list[Pair], key: &str) -> Option[i64]
def read_form[T: Transport](exchange: &mut Exchange[T], limits: &FormLimits) -> Result[list[Pair], BodyError]
def read_json[T: Transport](exchange: &mut Exchange[T], limits: &std.json.Limits) -> Result[Json, BodyError]
def send_json[T: Transport](exchange: &mut Exchange[T], status: i64, value: &Json, limits: &std.json.Limits) -> Result[(), BodyError]
class MultipartLimits
    max_parts: NonNegative[i64]
    max_head: NonNegative[i64]
    max_field: NonNegative[i64]
    max_file: NonNegative[i64]
    max_body: NonNegative[i64]
def multipart_limits() -> MultipartLimits
enum Content
    Text(str) | File(TempFile)
class Part
    name: str
    filename: Option[str]
    content_type: str
    size: i64
    content: Content
def read_multipart[T: Transport](exchange: &mut Exchange[T], limits: &MultipartLimits, storage: &Root) -> Result[list[Part], BodyError]

std.http.client

HTTP/1.1 client: pooled connections, deadlines and cancellation, bounded redirects, an explicit proxy, and replay of idempotent requests only. Errors: ClientError. Reference.

enum ClientError
    Net(NetError) | Http(HttpError) | Url(UrlError) | Allocation(AllocError) | InvalidConfig | InvalidUrl | UnsupportedScheme | InvalidRequest | InvalidRedirect | TooManyRedirects | UnexpectedInterim | Tls(TlsFailure) | TunnelRefused(i64) | Body(StreamError) | WrongBodyLength
enum TlsFailure
    UntrustedCertificate | ExpiredCertificate | WrongHost | InvalidCertificate | RevokedCertificate | Rejected | Protocol | InvalidName | InvalidConfiguration | NoTrustRoots | CertificateRequired | Incompatible
class Tunnel
    host: str
    port: u16
    authorization: Option[str]
    limits: Limits
class Endpoint
    scheme: str
    host: str
    port: u16
    tunnel: Option[Tunnel]
protocol Transport
    def read_before(self: &mut Transport, buffer: &mut Bytes, start: i64, stop: i64, deadline: &Deadline, cancel: &CancellationToken) -> Result[i64, ClientError]
    def write_before(self: &mut Transport, source: &[u8], deadline: &Deadline, cancel: &CancellationToken) -> Result[i64, ClientError]
protocol Connector[T]
    def open(self, endpoint: &Endpoint, deadline: &Deadline, cancel: &CancellationToken) -> Result[T, ClientError]
class Plain
    def read_before(self: &mut Plain, buffer: &mut Bytes, start: i64, stop: i64, deadline: &Deadline, cancel: &CancellationToken) -> Result[i64, ClientError]
    def write_before(self: &mut Plain, source: &[u8], deadline: &Deadline, cancel: &CancellationToken) -> Result[i64, ClientError]
class TcpConnector
    def open(self, endpoint: &Endpoint, deadline: &Deadline, cancel: &CancellationToken) -> Result[Plain, ClientError]
def dial(endpoint: &Endpoint, deadline: &Deadline, cancel: &CancellationToken) -> Result[TcpStream, ClientError]
class Dialled
    def read(self: &mut Dialled, buffer: &mut Bytes, start: i64, stop: i64) -> Result[i64, StepError[ClientError]]
    def write(self: &mut Dialled, source: &[u8]) -> Result[i64, StepError[ClientError]]
class Proxy
    endpoint: Endpoint
    authorization: Option[str]
def proxy(url: &str) -> Result[Proxy, ClientError]
class Config
    limits: Limits
    connect_timeout: Duration
    write_timeout: Duration
    head_timeout: Duration
    read_timeout: Duration
    idle_timeout: Duration
    continue_timeout: Duration
    max_idle: i64
    max_idle_per_origin: i64
    redirects: i64
    retries: i64
    sensitive: list[str]
    proxy: Option[Proxy]
def config() -> Result[Config, ClientError]
class Upload
    length: Option[i64]
    expect_continue: bool
def sized(length: i64) -> Upload
def chunked() -> Upload
class Unstreamed
    def read(self: &mut Unstreamed, buffer: &mut Bytes, start: i64, stop: i64) -> Result[i64, StreamError]
class Stats
    opened: u64
    reused: u64
    retried: u64
    redirected: u64
    discarded: u64
class Request
    method: str
    url: str
    headers: list[Header]
    body: Bytes
    def add(self: &mut Request, name: str, value: str) -> Result[(), ClientError]
def request(method: &str, url: &str) -> Result[Request, ClientError]
class Fetched
    status: i64
    head: ResponseHead
    url: str
    redirects: i64
    body: Bytes
class Wire[T]
    def read(self: &mut Wire[T], buffer: &mut Bytes, start: i64, stop: i64) -> Result[i64, StepError[ClientError]]
    def write(self: &mut Wire[T], source: &[u8]) -> Result[i64, StepError[ClientError]]
class Response[T]
    status: i64
    head: ResponseHead
    url: str
    redirects: i64
    def is_done(self) -> bool
    def read_body(self: &mut Response[T], buffer: &mut Bytes, start: i64, stop: i64) -> Result[i64, ClientError]
    def read_all(self: &mut Response[T], out: &mut Bytes) -> Result[i64, ClientError]
    def read_all_progress(self: &mut Response[T], out: &mut Bytes) -> Result[i64, Partial[ClientError]]
    def total(self) -> i64
    def trailers(self) -> &list[Header]
    def read_typed(self: &mut Response[T], buffer: &mut Bytes, start: i64, stop: i64) -> Result[i64, StepError[ClientError]]
    def read(self: &mut Response[T], buffer: &mut Bytes, start: i64, stop: i64) -> Result[i64, StreamError]
    def take_failure(self: &mut Response[T]) -> Option[ClientError]
class TypedResponse[T]
    def read(self: &mut TypedResponse[T], buffer: &mut Bytes, start: i64, stop: i64) -> Result[i64, StepError[ClientError]]
def typed_response[T](response: Response[T]) -> TypedResponse[T]
def unwrap_response[T](response: TypedResponse[T]) -> Response[T]
class Client[T, C]
    def stats(self) -> Stats
    def idle(self) -> i64
    def close_idle(self) -> ()
    def send(self, request: Request, deadline: &Deadline, cancel: &CancellationToken) -> Result[Response[T], ClientError]
    def send_from[R: Reader](self, request: Request, body: &mut R, upload: &Upload, deadline: &Deadline, cancel: &CancellationToken) -> Result[Response[T], ClientError]
    def fetch(self, request: Request, deadline: &Deadline, cancel: &CancellationToken) -> Result[Fetched, ClientError]
    def get(self, url: &str, deadline: &Deadline, cancel: &CancellationToken) -> Result[Fetched, ClientError]
    def finish(self, response: Response[T]) -> ()
def client(config: Config) -> Result[Client[Plain, TcpConnector], ClientError]
def connector[T: Transport, C: Connector[T]](opener: C, config: Config) -> Result[Client[T, C], ClientError]

std.http.cookie

Cookie fields read leniently and Set-Cookie fields validated against injection, with secure defaults. Errors: CookieError. Reference.

enum CookieError
    InvalidName | InvalidValue | InvalidAttribute | TooManyCookies | InvalidKey | Http(HttpError) | Crypto(CryptoError) | Allocation(AllocError)
class Cookie
    name: str
    value: str
def cookies(headers: &list[Header], limit: i64) -> Result[list[Cookie], CookieError]
def find(cookies: &list[Cookie], name: &str) -> Option[i64]
enum SameSite
    Strict | Lax | Unrestricted
class SetCookie
    name: str
    value: str
    path: str
    domain: str
    max_age: Option[Duration]
    secure: bool
    http_only: bool
    same_site: SameSite
    def to_string(self) -> Result[str, CookieError]
def set_cookie(name: &str, value: &str) -> Result[SetCookie, CookieError]
def removal(name: &str) -> Result[SetCookie, CookieError]
def add(head: &mut ResponseHead, cookie: &SetCookie) -> Result[(), CookieError]
const min_key: i64 = 32
def sign(keys: &list[Bytes], name: &str, value: &str) -> Result[str, CookieError]
def verify(keys: &list[Bytes], name: &str, signed: &str) -> Result[Option[str], CookieError]
def find_signed(cookies: &list[Cookie], keys: &list[Bytes], name: &str) -> Result[Option[str], CookieError]
def signed_cookie(keys: &list[Bytes], name: &str, value: &str) -> Result[SetCookie, CookieError]

std.http.files

Static files below a Root, opened one segment at a time without following links, with media types and conditional requests. Errors: FilesError. Reference.

enum FilesError
    Io(IoError) | Stream(StreamError) | Http(HttpError) | Truncated | Allocation(AllocError)
const max_path: i64 = 4096
def open_file(root: &Root, path: &str) -> Result[Option[FileStream], FilesError]
def media_type(name: &str) -> str
def http_date(timestamp: &Timestamp) -> Result[Option[str], FilesError]
def parse_http_date(text: &str) -> Result[Option[Timestamp], FilesError]
def serve[T: Transport](exchange: &mut Exchange[T], root: &Root, path: &str) -> Result[i64, FilesError]

std.http.https

std.http.client for https URLs through std.tls, verified by default, with CONNECT tunnels through a proxy. Errors: ClientError. Reference.

class Connection
    def read_before(self: &mut Connection, buffer: &mut Bytes, start: i64, stop: i64, deadline: &Deadline, cancel: &CancellationToken) -> Result[i64, ClientError]
    def write_before(self: &mut Connection, source: &[u8], deadline: &Deadline, cancel: &CancellationToken) -> Result[i64, ClientError]
def secure_error(error: TlsError) -> ClientError
class HttpsConnector
    def open(self, endpoint: &Endpoint, deadline: &Deadline, cancel: &CancellationToken) -> Result[Connection, ClientError]
def connector(tls: ClientConfig) -> HttpsConnector
def client(config: Config, tls: ClientConfig) -> Result[Client[Connection, HttpsConnector], ClientError]
def system_client(config: Config) -> Result[Client[Connection, HttpsConnector], ClientError]

std.http.proxy

The client address and scheme behind explicitly trusted reverse proxies; deployment guidance. Errors: ProxyError. Reference.

enum ProxyError
    InvalidNetwork | UnknownPeer | InvalidForwarded | TooManyHops | Allocation(AllocError)
enum Forwarding
    # X-Forwarded-For, with the scheme from X-Forwarded-Proto. | XForwardedFor
class Proxies
    header: Forwarding
    max_hops: i64
    def trust(self: &mut Proxies, network: &str) -> Result[(), ProxyError]
    def trusts(self, ip: &IpAddress) -> bool
def proxies(header: Forwarding) -> Result[Proxies, ProxyError]
class Client
    address: IpAddress
    scheme: Option[str]
    proxied: bool
def client(peer: &SocketAddress, headers: &list[Header], proxies: &Proxies) -> Result[Client, ProxyError]
def client_of[T: Transport](exchange: &Exchange[T], proxies: &Proxies) -> Result[Client, ProxyError]

std.http.server

HTTP/1.1 server: routing, a bounded worker pool and admission queue, idle connections parked off the workers, per-phase deadlines, and graceful shutdown. Errors: ServerError. Reference.

enum ServerError
    InvalidConfig | InvalidPattern | DuplicateRoute | Allocation(AllocError) | Thread(SpawnError[()]) | Net(NetError)
protocol Transport
    def read_before(self: &mut Transport, buffer: &mut Bytes, start: i64, stop: i64, deadline: &Deadline, cancel: &CancellationToken) -> Result[i64, NetError]
    def write_before(self: &mut Transport, source: &[u8], deadline: &Deadline, cancel: &CancellationToken) -> Result[i64, NetError]
    def shutdown(self: &mut Transport, direction: Shutdown) -> Result[(), NetError]
protocol Handler[T, R, E]
    def handle(self, route: &R, exchange: &mut Exchange[T]) -> Result[(), E]
class Config
    workers: i64
    queue: i64
    limits: Limits
    idle_timeout: Duration
    head_timeout: Duration
    body_timeout: Duration
    write_timeout: Duration
    response_timeout: Duration
    drain_timeout: Duration
    max_requests: i64
    events: Option[ChannelSink]
    handshake_timeout: Duration
    idle_connections: i64
def config() -> Config
class Report
    connections: u64
    rejected: u64
    requests: u64
    handler_errors: u64
    protocol_errors: u64
    timeouts: u64
    cancelled: u64
    accept_errors: u64
    dropped_events: u64
enum Match
    Found(i64, list[str]) | NotFound | MethodNotAllowed(str)
class Router[R]
    def add(self: &mut Router[R], method: &str, pattern: &str, route: R) -> Result[(), ServerError]
    def find(self, method: &str, path: &str) -> Result[Match, ServerError]
    def route(self, index: i64) -> &R
def router[R]() -> Result[Router[R], ServerError]
class Conn[T]
    def read(self: &mut Conn[T], buffer: &mut Bytes, start: i64, stop: i64) -> Result[i64, StreamError]
    def write(self: &mut Conn[T], source: &[u8]) -> Result[i64, StreamError]
    def flush(self: &mut Conn[T]) -> Result[(), StreamError]
class Exchange[T]
    request: RequestHead
    authority: str
    path: str
    query: str
    params: list[str]
    peer: Option[SocketAddress]
    def stopping(self) -> bool
    def cancelled(self) -> bool
    def stop_token(self) -> CancellationToken
    def cancel_token(self) -> CancellationToken
    def status(self) -> i64
    def sent(self) -> i64
    def read_body(self: &mut Exchange[T], buffer: &mut Bytes, start: i64, stop: i64) -> Result[i64, HttpError]
    def read_all(self: &mut Exchange[T], out: &mut Bytes) -> Result[i64, HttpError]
    def respond(self: &mut Exchange[T], head: ResponseHead, body: &[u8]) -> Result[(), HttpError]
    def send(self: &mut Exchange[T], status: i64, content_type: &str, body: &[u8]) -> Result[(), HttpError]
    def send_text(self: &mut Exchange[T], status: i64, text: &str) -> Result[(), HttpError]
    def start(self: &mut Exchange[T], head: ResponseHead) -> Result[(), HttpError]
    def write(self: &mut Exchange[T], source: &[u8]) -> Result[(), HttpError]
    def end(self: &mut Exchange[T]) -> Result[(), HttpError]
    def switch_protocols(self: &mut Exchange[T], head: ResponseHead) -> Result[(T, Bytes), HttpError]
    def detach(self: &mut Exchange[T], head: ResponseHead) -> Result[T, HttpError]
def reason(status: i64) -> str
def serve_connection[T: Transport, R, E, H: Handler[T, R, E]](stream: T, router: &Router[R], handler: &H, config: &Config, stop: &CancellationToken, cancel: &CancellationToken) -> Report
def serve_connection_from[T: Transport, R, E, H: Handler[T, R, E]](stream: T, peer: Option[SocketAddress], router: &Router[R], handler: &H, config: &Config, stop: &CancellationToken, cancel: &CancellationToken) -> Report
enum OpenError
    TimedOut | Cancelled | Failed | Gone
protocol Opener[T]
    def open(self, tcp: TcpStream) -> Result[T, OpenError]
    def advance(self, stream: &mut T, deadline: &Deadline, cancel: &CancellationToken) -> Result[(), OpenError]
    def handshakes(self) -> bool
    def watch(self, stream: &T, selector: &mut Selector, key: u64) -> bool
    def unwatch(self, stream: &T, selector: &mut Selector) -> ()
def serve[R, E, H: Handler[TcpStream, R, E]](listener: TcpListener, router: &Router[R], handler: &H, config: &Config, stop: &CancellationToken) -> Result[Report, ServerError]
def serve_with[T: Transport, O: Opener[T], R, E, H: Handler[T, R, E]](listener: TcpListener, opener: &O, router: &Router[R], handler: &H, config: &Config, stop: &CancellationToken) -> Result[Report, ServerError]

std.http.sse

Server-Sent Events from a server exchange: event, id, multi-line data, comments and retry, flushed per event, with disconnect and shutdown detection. Errors: SseError. Reference.

enum SseError
    Net(NetError) | Allocation(AllocError) | Http(HttpError) | InvalidField | InvalidText | Disconnected | Closed | Stopping
class EventStream[T]
    last_event_id: Option[str]
    def is_open(self) -> bool
    def send(self: &mut EventStream[T], data: &str, deadline: &Deadline, cancel: &CancellationToken, event: str = "", id: Option[str] = Nothing) -> Result[(), SseError]
    def send_bytes(self: &mut EventStream[T], data: &[u8], deadline: &Deadline, cancel: &CancellationToken, event: str = "", id: Option[str] = Nothing) -> Result[(), SseError]
    def comment(self: &mut EventStream[T], text: &str, deadline: &Deadline, cancel: &CancellationToken) -> Result[(), SseError]
    def retry(self: &mut EventStream[T], span: &Duration, deadline: &Deadline, cancel: &CancellationToken) -> Result[(), SseError]
    def wait(self: &mut EventStream[T], deadline: &Deadline, cancel: &CancellationToken) -> Result[(), SseError]
    def close(self: &mut EventStream[T]) -> ()
def head() -> Result[ResponseHead, HttpError]
def accept[T: Transport](exchange: &mut Exchange[T]) -> Result[EventStream[T], SseError]
def accept_with[T: Transport](exchange: &mut Exchange[T], head: ResponseHead) -> Result[EventStream[T], SseError]

std.http.tls

std.http.server over std.tls, with handshake deadlines. Errors: ServerError. Reference.

class TlsTransport
    stream: TlsStream
    def read_before(self: &mut TlsTransport, buffer: &mut Bytes, start: i64, stop: i64, deadline: &Deadline, cancel: &CancellationToken) -> Result[i64, NetError]
    def write_before(self: &mut TlsTransport, source: &[u8], deadline: &Deadline, cancel: &CancellationToken) -> Result[i64, NetError]
    def shutdown(self: &mut TlsTransport, direction: Shutdown) -> Result[(), NetError]
class TlsOpener
    def open(self, tcp: TcpStream) -> Result[TlsTransport, OpenError]
    def advance(self, stream: &mut TlsTransport, deadline: &Deadline, cancel: &CancellationToken) -> Result[(), OpenError]
    def handshakes(self) -> bool
    def watch(self, stream: &TlsTransport, selector: &mut Selector, key: u64) -> bool
    def unwatch(self, stream: &TlsTransport, selector: &mut Selector) -> ()
def serve[R, E, H: Handler[TlsTransport, R, E]](listener: TcpListener, tls: ServerConfig, router: &Router[R], handler: &H, config: &Config, stop: &CancellationToken) -> Result[Report, ServerError]

std.http.typed

Typed head parsing and writes, fixed/chunked/close-delimited bodies, and guarded connection recovery with concrete causes and progress. Errors: FramingError[E], Partial[E]. Reference.

enum FramingError[E]
    Transport(E) | Truncated | InvalidProgress | OutOfRange | Broken | NotReusable | Protocol(HttpError)
def read_request[E, R: Reader[E]](reader: &mut R, limits: &Limits) -> Result[Option[RequestHead], Partial[FramingError[E]]]
def read_response[E, R: Reader[E]](reader: &mut R, limits: &Limits) -> Result[Option[ResponseHead], Partial[FramingError[E]]]
def write_request[E, W: Writer[E]](writer: &mut W, head: &RequestHead) -> Result[(), Partial[FramingError[E]]]
def write_response[E, W: Writer[E]](writer: &mut W, head: &ResponseHead) -> Result[(), Partial[FramingError[E]]]
def write_last_chunk[E, W: Writer[E]](writer: &mut W, trailers: &list[Header]) -> Result[(), Partial[FramingError[E]]]
def write_chunk[E, W: Writer[E]](writer: &mut W, source: &[u8]) -> Result[(), Partial[FramingError[E]]]
def fixed_body[E, R: Reader[E]](inner: R, length: NonNegative[i64]) -> Result[FixedBody[E, R], FramingError[E]]
class FixedBody[E, R]
    def read(self: &mut FixedBody[E, R], buffer: &mut Bytes, start: i64, stop: i64) -> Result[i64, StepError[FramingError[E]]]
def chunked_body[E, R: Reader[E]](inner: R, limits: &Limits) -> Result[ChunkedBody[E, R], FramingError[E]]
class ChunkedBody[E, R]
    trailers: list[Header]
    def total(self) -> i64
    def is_done(self) -> bool
    def read(self: &mut ChunkedBody[E, R], buffer: &mut Bytes, start: i64, stop: i64) -> Result[i64, StepError[FramingError[E]]]
def close_body[E, R: Reader[E]](inner: R) -> CloseBody[E, R]
class CloseBody[E, R]
    def read(self: &mut CloseBody[E, R], buffer: &mut Bytes, start: i64, stop: i64) -> Result[i64, StepError[FramingError[E]]]
def request_body[E, R: Reader[E]](inner: R, head: &RequestHead, limits: &Limits) -> Result[Body[E, R], FramingError[E]]
def response_body[E, R: Reader[E]](inner: R, head: &ResponseHead, method: &str, request_keep_alive: bool, limits: &Limits) -> Result[Body[E, R], FramingError[E]]
class Body[E, R]
    def total(self) -> i64
    def is_done(self) -> bool
    def trailers(self) -> &list[Header]
    def read(self: &mut Body[E, R], buffer: &mut Bytes, start: i64, stop: i64) -> Result[i64, StepError[FramingError[E]]]
    def finish(self: &mut Body[E, R]) -> Result[R, FramingError[E]]

std.http.websocket

RFC 6455 WebSocket connections: the handshake as a server exchange or a ws client, bounded framed messages, pings and the closing handshake. Errors: WebSocketError. Reference.

enum WebSocketError
    Net(NetError) | Allocation(AllocError) | Http(HttpError) | Connect(ClientError) | Entropy(StreamError) | InvalidConfig | InvalidUrl | UnsupportedScheme | OutOfRange | Refused(i64) | Rejected(i64) | InvalidHandshake | Protocol | TooLarge | InvalidText | Closed | Stopping
class Config
    max_frame: Positive[i64]
    max_message: Positive[i64]
    subprotocol: Option[str]
def config() -> Config
enum Message
    Text | Binary | Close(i64)
enum Kind
    Text | Binary
class WebSocket[T]
    subprotocol: Option[str]
    def is_open(self) -> bool
    def receive(self: &mut WebSocket[T], out: &mut Bytes, deadline: &Deadline, cancel: &CancellationToken) -> Result[Message, WebSocketError]
    def send(self: &mut WebSocket[T], kind: Kind, data: &[u8], deadline: &Deadline, cancel: &CancellationToken) -> Result[(), WebSocketError]
    def send_text(self: &mut WebSocket[T], text: &str, deadline: &Deadline, cancel: &CancellationToken) -> Result[(), WebSocketError]
    def ping(self: &mut WebSocket[T], data: &[u8], deadline: &Deadline, cancel: &CancellationToken) -> Result[(), WebSocketError]
    def close(self: &mut WebSocket[T], code: i64, reason: &str, deadline: &Deadline, cancel: &CancellationToken) -> Result[(), WebSocketError]
def accept[T: Transport](exchange: &mut Exchange[T], config: &Config) -> Result[WebSocket[T], WebSocketError]
class Target
    secure: bool
    host: str
    port: u16
def target(url: &str) -> Result[Target, WebSocketError]
def dial(target: &Target, deadline: &Deadline, cancel: &CancellationToken) -> Result[TcpStream, WebSocketError]
class Wire[T]
    def read(self: &mut Wire[T], buffer: &mut Bytes, start: i64, stop: i64) -> Result[i64, StreamError]
def handshake[T: Transport](stream: T, target: &Target, config: &Config, deadline: &Deadline, cancel: &CancellationToken) -> Result[WebSocket[T], WebSocketError]
def connect(url: &str, config: &Config, deadline: &Deadline, cancel: &CancellationToken) -> Result[WebSocket[TcpStream], WebSocketError]

std.http.wss

std.http.websocket clients for wss URLs through std.tls. Errors: WebSocketError. Reference.

def connect(url: &str, tls: &ClientConfig, config: &Config, deadline: &Deadline, cancel: &CancellationToken) -> Result[WebSocket[TlsTransport], WebSocketError]

std.image

Owned 8-bit pixel buffers, bounded PNG decoding and encoding, cropping, colour conversion and resizing. Errors: ImageError. Reference.

enum ImageError
    OutOfRange | Corrupt | Checksum | Truncated | Unsupported | LimitExceeded | Allocation(AllocError)
enum ColorType
    Gray | GrayAlpha | Rgb | Rgba
    def channels(self) -> i64
    def has_alpha(self) -> bool
enum Format
    Png | Jpeg | Gif | WebP
enum Filter
    Nearest | Bilinear
class Limits
    max_width: NonNegative[i64]
    max_height: NonNegative[i64]
    max_bytes: NonNegative[i64]
    def check(self, width: i64, height: i64, color: ColorType) -> Result[(), ImageError]
def limits() -> Limits
class Info
    format: Format
    width: i64
    height: i64
    color: ColorType
    depth: i64
    interlaced: bool
class Image
    def width(self) -> i64
    def height(self) -> i64
    def color(self) -> ColorType
    def pixels(self) -> &Bytes
    def into_pixels(self: Image) -> Bytes
    def copy(self) -> Result[Image, ImageError]
    def get(self, x: i64, y: i64, channel: i64) -> Result[u8, ImageError]
    def set(self: &mut Image, x: i64, y: i64, channel: i64, value: u8) -> Result[(), ImageError]
    def crop(self, x: i64, y: i64, width: i64, height: i64) -> Result[Image, ImageError]
    def convert(self, color: ColorType) -> Result[Image, ImageError]
    def resize(self, width: i64, height: i64, filter: Filter) -> Result[Image, ImageError]
def new(width: Positive[i64], height: Positive[i64], color: ColorType) -> Result[Image, ImageError]
def from_pixels(width: Positive[i64], height: Positive[i64], color: ColorType, pixels: Bytes) -> Result[Image, ImageError]
def detect(data: &[u8]) -> Option[Format]
def info(data: &[u8]) -> Result[Info, ImageError]
def decode(data: &[u8], limits: &Limits) -> Result[Image, ImageError]
def encode_png(image: &Image, level: i64 = 6) -> Result[Bytes, ImageError]

std.io

Partial-progress byte stream contracts, retry helpers, fixed-capacity buffering, file and process-stream adapters, and checked sequential binary fields in an owned memory cursor. Errors: StreamError, Partial, CopyError, CloseError, BinaryError. Reference.

enum StreamError
    Interrupted | UnexpectedEof | WriteZero | OutOfRange | InvalidProgress | Io(IoError) | Allocation(AllocError)
enum BinaryError
    Truncated(i64, i64, i64) | TrailingData(i64, i64) | OutOfRange | LimitExceeded | Allocation(AllocError)
class Partial
    completed: i64
    cause: StreamError
enum CopyError
    Read(i64, StreamError) | Write(i64, StreamError)
enum CloseError
    Flush(StreamError) | Close(StreamError) | FlushAndClose(StreamError, StreamError)
enum SeekFrom
    Start(i64) | Current(i64) | End(i64)
protocol Reader
    def read(self: &mut Reader, buffer: &mut Bytes, start: i64, stop: i64) -> Result[i64, StreamError]
protocol Writer
    def write(self: &mut Writer, source: &[u8]) -> Result[i64, StreamError]
    def flush(self: &mut Writer) -> Result[(), StreamError]
protocol Seeker
    def seek(self: &mut Seeker, target: SeekFrom) -> Result[i64, StreamError]
protocol Closer
    def close(self: &mut Closer) -> Result[(), StreamError]
def classify(error: IoError) -> StreamError
def read_exact[R: Reader](reader: &mut R, buffer: &mut Bytes, start: i64, stop: i64) -> Result[(), Partial]
def write_all[W: Writer](writer: &mut W, source: &[u8]) -> Result[(), Partial]
def read_up_to[R: Reader](reader: &mut R, out: &mut Bytes, limit: NonNegative[i64]) -> Result[i64, Partial]
def copy_all[R: Reader, W: Writer](reader: &mut R, writer: &mut W, scratch: &mut Bytes) -> Result[i64, CopyError]
class BufReader[R]
    inner: R
    def capacity(self) -> i64
    def buffered(self) -> i64
    def read(self: &mut BufReader[R], buffer: &mut Bytes, start: i64, stop: i64) -> Result[i64, StreamError]
    def read_until(self: &mut BufReader[R], delimiter: u8, out: &mut Bytes, limit: Positive[i64]) -> Result[i64, StreamError]
def buffered_reader[R](inner: R, capacity: Positive[i64]) -> Result[BufReader[R], StreamError]
def seek_reader[R: Seeker](reader: &mut BufReader[R], target: SeekFrom) -> Result[i64, StreamError]
class BufWriter[W]
    inner: W
    def capacity(self) -> i64
    def pending(self) -> i64
    def write(self: &mut BufWriter[W], source: &[u8]) -> Result[i64, StreamError]
    def flush(self: &mut BufWriter[W]) -> Result[(), StreamError]
def buffered_writer[W](inner: W, capacity: Positive[i64]) -> Result[BufWriter[W], StreamError]
def seek_writer[W: Seeker](writer: &mut BufWriter[W], target: SeekFrom) -> Result[i64, StreamError]
def close_buffered[W: Closer](writer: &mut BufWriter[W]) -> Result[(), CloseError]
class Cursor
    data: Bytes
    def position(self) -> i64
    def remaining(self) -> i64
    def require_end(self) -> Result[(), BinaryError]
    def read_bytes(self: &mut Cursor, count: i64, max_bytes: i64 = 1048576) -> Result[Bytes, BinaryError]
    def write_bytes(self: &mut Cursor, source: &[u8], max_bytes: i64 = 1048576) -> Result[(), BinaryError]
    def read_u8(self: &mut Cursor) -> Result[u8, BinaryError]
    def write_u8(self: &mut Cursor, value: u8, max_bytes: i64 = 1048576) -> Result[(), BinaryError]
    def read_i8(self: &mut Cursor) -> Result[i8, BinaryError]
    def write_i8(self: &mut Cursor, value: i8, max_bytes: i64 = 1048576) -> Result[(), BinaryError]
    def read_u16(self: &mut Cursor, order: Endian) -> Result[u16, BinaryError]
    def write_u16(self: &mut Cursor, value: u16, order: Endian, max_bytes: i64 = 1048576) -> Result[(), BinaryError]
    def read_i16(self: &mut Cursor, order: Endian) -> Result[i16, BinaryError]
    def write_i16(self: &mut Cursor, value: i16, order: Endian, max_bytes: i64 = 1048576) -> Result[(), BinaryError]
    def read_u32(self: &mut Cursor, order: Endian) -> Result[u32, BinaryError]
    def write_u32(self: &mut Cursor, value: u32, order: Endian, max_bytes: i64 = 1048576) -> Result[(), BinaryError]
    def read_i32(self: &mut Cursor, order: Endian) -> Result[i32, BinaryError]
    def write_i32(self: &mut Cursor, value: i32, order: Endian, max_bytes: i64 = 1048576) -> Result[(), BinaryError]
    def read_u64(self: &mut Cursor, order: Endian) -> Result[u64, BinaryError]
    def write_u64(self: &mut Cursor, value: u64, order: Endian, max_bytes: i64 = 1048576) -> Result[(), BinaryError]
    def read_i64(self: &mut Cursor, order: Endian) -> Result[i64, BinaryError]
    def write_i64(self: &mut Cursor, value: i64, order: Endian, max_bytes: i64 = 1048576) -> Result[(), BinaryError]
    def read_f32(self: &mut Cursor, order: Endian) -> Result[f32, BinaryError]
    def write_f32(self: &mut Cursor, value: f32, order: Endian, max_bytes: i64 = 1048576) -> Result[(), BinaryError]
    def read_f64(self: &mut Cursor, order: Endian) -> Result[f64, BinaryError]
    def write_f64(self: &mut Cursor, value: f64, order: Endian, max_bytes: i64 = 1048576) -> Result[(), BinaryError]
    def read(self: &mut Cursor, buffer: &mut Bytes, start: i64, stop: i64) -> Result[i64, StreamError]
    def write(self: &mut Cursor, source: &[u8]) -> Result[i64, StreamError]
    def flush(self: &mut Cursor) -> Result[(), StreamError]
    def seek(self: &mut Cursor, target: SeekFrom) -> Result[i64, StreamError]
def cursor(data: Bytes) -> Cursor
class FileStream
    file: File
    def read(self: &mut FileStream, buffer: &mut Bytes, start: i64, stop: i64) -> Result[i64, StreamError]
    def write(self: &mut FileStream, source: &[u8]) -> Result[i64, StreamError]
    def flush(self: &mut FileStream) -> Result[(), StreamError]
    def sync(self: &mut FileStream) -> Result[(), StreamError]
    def seek(self: &mut FileStream, target: SeekFrom) -> Result[i64, StreamError]
    def close(self: &mut FileStream) -> Result[(), StreamError]
def open_file(path: &str, mode: &str) -> Result[FileStream, StreamError]
def open_path(path: &Path, mode: &str) -> Result[FileStream, StreamError]
def from_file(file: File) -> FileStream
class Stdin
    def is_terminal(self) -> bool
    def read(self: &mut Stdin, buffer: &mut Bytes, start: i64, stop: i64) -> Result[i64, StreamError]
class Stdout
    def is_terminal(self) -> bool
    def write(self: &mut Stdout, source: &[u8]) -> Result[i64, StreamError]
    def flush(self: &mut Stdout) -> Result[(), StreamError]
class Stderr
    def is_terminal(self) -> bool
    def write(self: &mut Stderr, source: &[u8]) -> Result[i64, StreamError]
    def flush(self: &mut Stderr) -> Result[(), StreamError]
def stdin() -> Stdin
def stdout() -> Stdout
def stderr() -> Stderr

std.io.typed

Byte stream protocols that carry a caller-chosen error type, buffering, adapters for std.io streams, and exact read, write, and copy helpers. Errors: StepError[E], Partial[E], CopyFailure[R, W]. Reference.

enum StepError[E]
    Interrupted | Source(E)
class LegacyReader[R]
    def read(self: &mut LegacyReader[R], buffer: &mut Bytes, start: i64, stop: i64) -> Result[i64, StepError[StreamError]]
def reader[R: std.io.Reader](inner: R) -> LegacyReader[R]
def unwrap_reader[R](reader: LegacyReader[R]) -> R
class LegacyWriter[W]
    def write(self: &mut LegacyWriter[W], buffer: &[u8]) -> Result[i64, StepError[StreamError]]
def writer[W: std.io.Writer](inner: W) -> LegacyWriter[W]
def unwrap_writer[W](writer: LegacyWriter[W]) -> W
protocol Reader[E]
    def read(self: &mut Reader[E], buffer: &mut Bytes, start: i64, stop: i64) -> Result[i64, StepError[E]]
protocol Writer[E]
    def write(self: &mut Writer[E], buffer: &[u8]) -> Result[i64, StepError[E]]
enum ReadFailure[E]
    Source(E) | UnexpectedEof | InvalidProgress | OutOfRange
class BufReader[E, R]
    inner: R
    def capacity(self) -> i64
    def buffered(self) -> i64
    def read(self: &mut BufReader[E, R], buffer: &mut Bytes, start: i64, stop: i64) -> Result[i64, StepError[ReadFailure[E]]]
def buffered_reader[E, R: Reader[E]](inner: R, capacity: Positive[i64]) -> Result[BufReader[E, R], StreamError]
enum WriteFailure[E]
    Source(E) | WriteZero | InvalidProgress
class Partial[E]
    completed: i64
    cause: E
enum CopyFailure[R, W]
    Read(i64, ReadFailure[R]) | Write(i64, WriteFailure[W])
def within(start: i64, stop: i64, size: i64) -> bool
def read_exact[E, R: Reader[E]](reader: &mut R, buffer: &mut Bytes, start: i64, stop: i64) -> Result[(), Partial[ReadFailure[E]]]
def write_all[E, W: Writer[E]](writer: &mut W, buffer: &[u8]) -> Result[(), Partial[WriteFailure[E]]]
def copy_all[E, G, R: Reader[E], W: Writer[G]](reader: &mut R, writer: &mut W, scratch: &mut Bytes) -> Result[i64, CopyFailure[E, G]]

std.json

Strict JSON under depth, input, and output limits: resumable event decoder, validating encoder, and a value tree. Errors: JsonError. Reference.

enum JsonFault
    UnexpectedByte | UnexpectedEnd | TrailingData | InvalidLiteral | InvalidNumber | NumberOutOfRange | InvalidEscape | InvalidSurrogate | ControlCharacter | InvalidUtf8 | DuplicateKey | DepthExceeded | InputTooLarge | OutputTooLarge | NonFiniteNumber | InvalidState | Stream(StreamError) | Allocation(AllocError)
class JsonError
    fault: JsonFault
    offset: i64
    line: i64
    column: i64
enum DuplicateKeys
    Reject | Keep
class Limits
    max_depth: NonNegative[i64]
    max_input: NonNegative[i64]
    max_output: NonNegative[i64]
    duplicate_keys: DuplicateKeys
enum Json
    Null | Bool(bool) | Int(i64) | Float(f64) | Text(str) | Array(list[Json]) | Object(list[Member])
class Member
    key: str
    value: Json
enum Event
    Pending | End | Null | Bool(bool) | Number(str) | Text(str) | Key(str) | StartArray | EndArray | StartObject | EndObject
def limits() -> Limits
class Decoder
    def offset(self) -> i64
    def depth(self) -> i64
    def reject(self, fault: JsonFault) -> JsonError
    def feed(self: &mut Decoder, source: &[u8]) -> Result[(), JsonError]
    def finish(self: &mut Decoder) -> ()
    def next(self: &mut Decoder) -> Result[Event, JsonError]
def decoder(bounds: &Limits) -> Result[Decoder, JsonError]
def read_event[R: Reader](source: &mut Decoder, reader: &mut R) -> Result[Event, JsonError]
def parse(source: &[u8], bounds: &Limits) -> Result[Json, JsonError]
def parse_text(text: &str, bounds: &Limits) -> Result[Json, JsonError]
def read_value[R: Reader](reader: &mut R, bounds: &Limits) -> Result[Json, JsonError]
def find(members: &list[Member], key: &str) -> Option[i64]
class Encoder
    def pending(self) -> i64
    def written(self) -> i64
    def is_complete(self) -> bool
    def null(self: &mut Encoder) -> Result[(), JsonError]
    def boolean(self: &mut Encoder, value: bool) -> Result[(), JsonError]
    def integer(self: &mut Encoder, value: i64) -> Result[(), JsonError]
    def unsigned(self: &mut Encoder, value: u64) -> Result[(), JsonError]
    def decimal(self: &mut Encoder, value: f64) -> Result[(), JsonError]
    def number(self: &mut Encoder, text: &str) -> Result[(), JsonError]
    def text(self: &mut Encoder, value: &str) -> Result[(), JsonError]
    def key(self: &mut Encoder, name: &str) -> Result[(), JsonError]
    def start_array(self: &mut Encoder) -> Result[(), JsonError]
    def end_array(self: &mut Encoder) -> Result[(), JsonError]
    def start_object(self: &mut Encoder) -> Result[(), JsonError]
    def end_object(self: &mut Encoder) -> Result[(), JsonError]
    def event(self: &mut Encoder, event: &Event) -> Result[(), JsonError]
    def value(self: &mut Encoder, value: &Json) -> Result[(), JsonError]
    def take(self: &mut Encoder) -> Result[Bytes, JsonError]
def encoder(bounds: &Limits) -> Result[Encoder, JsonError]
def write_pending[W: Writer](source: &mut Encoder, writer: &mut W) -> Result[(), JsonError]
def encode(value: &Json, bounds: &Limits) -> Result[Bytes, JsonError]
def encode_text(value: &Json, bounds: &Limits) -> Result[str, JsonError]
def write_value[W: Writer](writer: &mut W, value: &Json, bounds: &Limits) -> Result[(), JsonError]

std.jwt

JSON Web Tokens: compact JWS signing and verification with the algorithm fixed by the key, strict decoding, and explicit-clock claim checks. Errors: JwtError. Reference.

enum Algorithm
    Hs256 | Hs512 | Rs256 | Rs384 | Rs512 | Ps256 | Ps384 | Ps512 | Es256 | EdDsa
    def name(self) -> str
enum Claim
    Expiry | NotBefore | IssuedAt | Issuer | Audience
enum JwtError
    Malformed | NotObject | TooLarge | UnsupportedAlgorithm | AlgorithmMismatch | CriticalHeader | InvalidSignature | Expired | NotYetValid | MissingClaim(Claim) | InvalidClaim(Claim) | WrongIssuer | WrongAudience | InvalidKey | InvalidValidation | RandomFailure | Allocation(AllocError)
    def format[W](self, output: &mut Formatter[W]) -> Result[(), FormatError]
class SigningKey
    def algorithm(self) -> Algorithm
    def verifying_key(self) -> Result[VerifyingKey, JwtError]
class VerifyingKey
    def algorithm(self) -> Algorithm
def hmac_signing_key(algorithm: Algorithm, secret: &[u8]) -> Result[SigningKey, JwtError]
def hmac_verifying_key(algorithm: Algorithm, secret: &[u8]) -> Result[VerifyingKey, JwtError]
def rsa_signing_key(algorithm: Algorithm, key: RsaPrivateKey) -> Result[SigningKey, JwtError]
def rsa_verifying_key(algorithm: Algorithm, key: RsaPublicKey) -> Result[VerifyingKey, JwtError]
def es256_signing_key(key: P256PrivateKey) -> SigningKey
def es256_verifying_key(key: P256PublicKey) -> VerifyingKey
def ed25519_signing_key(seed: &[u8]) -> Result[SigningKey, JwtError]
def ed25519_verifying_key(public_key: &[u8]) -> Result[VerifyingKey, JwtError]
const default_max_size: i64 = 8192
const max_leeway: i64 = 86400
const max_token_size: i64 = 1048576
class Validation
    now: Timestamp
    leeway: Duration
    require_expiry: bool
    issuer: Option[str]
    audience: Option[str]
    max_size: Positive[i64]
def validation(now: &Timestamp) -> Validation
class Header
    algorithm: Algorithm
    key_id: Option[str]
    token_type: Option[str]
class Token
    header: Header
    claims: list[Member]
def decode_header(token: &str, max_size: Positive[i64] = 8192) -> Result[Header, JwtError]
def verify(token: &str, key: &VerifyingKey, rules: &Validation) -> Result[Token, JwtError]
def sign(key: &SigningKey, claims: &Json, key_id: Option[str] = Nothing) -> Result[str, JwtError]

std.log

Leveled events with typed fields, delivered to caller-chosen sinks: logfmt lines, bounded memory, or a nonblocking channel. Errors: LogError. Reference.

enum LogError
    InvalidSpec | InvalidKey | UnknownLevel | TooLarge | Full | Closed | Format(FormatError) | Output(StreamError) | Allocation(AllocError)
enum Level
    Trace | Debug | Info | Warn | Error
def level_name(level: &Level) -> str
def parse_level(name: &str) -> Result[Level, LogError]
enum Value
    Text(str) | Integer(i64) | Unsigned(u64) | Floating(f64) | Boolean(bool)
class Field
    key: str
    value: Value
def text(key: str, value: str) -> Field
def integer(key: str, value: i64) -> Field
def unsigned(key: str, value: u64) -> Field
def floating(key: str, value: f64) -> Field
def boolean(key: str, value: bool) -> Field
class Event
    level: Level
    message: str
    fields: list[Field]
    time: Option[Timestamp]
def event(level: Level, message: str, fields: list[Field]) -> Event
protocol Sink
    def emit(self: &mut Sink, event: Event) -> Result[(), LogError]
enum Clock
    Off | Wall | Fixed(Timestamp)
class Logger[S]
    sink: S
    def enabled(self, level: &Level) -> bool
    def emitted(self) -> u64
    def filtered(self) -> u64
    def failed(self) -> u64
    def emit(self: &mut Logger[S], event: Event) -> Result[(), LogError]
    def log(self: &mut Logger[S], level: Level, message: str, fields: list[Field]) -> Result[(), LogError]
    def debug(self: &mut Logger[S], message: str, fields: list[Field]) -> Result[(), LogError]
    def info(self: &mut Logger[S], message: str, fields: list[Field]) -> Result[(), LogError]
    def warn(self: &mut Logger[S], message: str, fields: list[Field]) -> Result[(), LogError]
    def error(self: &mut Logger[S], message: str, fields: list[Field]) -> Result[(), LogError]
def logger[S: Sink](sink: S, threshold: Level, clock: Clock) -> Logger[S]
class LineBuffer
    data: Bytes
    def write(self: &mut LineBuffer, source: &[u8]) -> Result[i64, StreamError]
    def flush(self: &mut LineBuffer) -> Result[(), StreamError]
class TextSink[W]
    writer: W
    def emit(self: &mut TextSink[W], event: Event) -> Result[(), LogError]
def text_sink[W: Writer](writer: W, max_line: Positive[i64]) -> Result[TextSink[W], LogError]
class MemorySink
    events: list[Event]
    def emit(self: &mut MemorySink, event: Event) -> Result[(), LogError]
def memory_sink(capacity: NonNegative[i64]) -> Result[MemorySink, LogError]
class Discard
    def emit(self: &mut Discard, event: Event) -> Result[(), LogError]
def discard() -> Discard
class ChannelSink
    def share(self) -> ChannelSink
    def emit(self: &mut ChannelSink, event: Event) -> Result[(), LogError]
def channel_sink(sender: Sender[Event]) -> ChannelSink
class Drained
    delivered: u64
    failed: u64
def drain[S: Sink](receiver: &Receiver[Event], sink: &mut S) -> Drained

std.markdown

CommonMark 0.31.2 parsing into events and HTML rendering, under input, depth, and output limits. Errors: MarkdownError. Reference.

enum MarkdownFault
    InvalidUtf8 | InputTooLarge | OutputTooLarge | DepthExceeded | InvalidState | Stream(StreamError) | Allocation(AllocError)
class MarkdownError
    fault: MarkdownFault
    offset: i64
    line: i64
class Limits
    max_input: NonNegative[i64]
    max_depth: NonNegative[i64]
    max_output: NonNegative[i64]
enum Tag
    Paragraph | Heading(i64) | BlockQuote | CodeBlock(str) | BulletList | OrderedList(i64) | Item | Emphasis | Strong | Link(str, str) | Image(str, str)
enum Event
    Start(Tag) | End(Tag) | Text(str) | Code(str) | Html(str) | InlineHtml(str) | SoftBreak | HardBreak | Rule | Done
def limits() -> Limits
class Parser
    def next(self: &mut Parser) -> Result[Event, MarkdownError]
def parser(source: &[u8], bounds: &Limits) -> Result[Parser, MarkdownError]
def parser_text(text: &str, bounds: &Limits) -> Result[Parser, MarkdownError]
class HtmlRenderer
    def pending(self) -> i64
    def written(self) -> i64
    def event(self: &mut HtmlRenderer, event: &Event) -> Result[(), MarkdownError]
    def take(self: &mut HtmlRenderer) -> Result[Bytes, MarkdownError]
def renderer(bounds: &Limits) -> Result[HtmlRenderer, MarkdownError]
def write_pending[W: Writer](source: &mut HtmlRenderer, writer: &mut W) -> Result[(), MarkdownError]
def write_html[W: Writer](source: &mut Parser, writer: &mut W, bounds: &Limits) -> Result[(), MarkdownError]
def render(source: &[u8], bounds: &Limits) -> Result[Bytes, MarkdownError]
def to_html(text: &str, bounds: &Limits) -> Result[str, MarkdownError]

std.template

Bounded Jinja-style text templates with explicit JSON values, HTML escaping, conditions, loops and named inheritance. Errors: TemplateError. Reference.

enum TemplateError
    LimitExceeded | Syntax(i64) | UnknownTemplate(str) | DuplicateTemplate(str) | MissingValue(str) | TypeMismatch | InheritanceCycle | Allocation(AllocError) | Bytes(BytesError)
enum HtmlContext
    Text | DoubleQuotedAttribute | SingleQuotedAttribute | TextAndAttributes
class Limits
    max_source: NonNegative[i64]
    max_tokens: NonNegative[i64]
    max_templates: NonNegative[i64]
    max_context_nodes: Positive[i64]
    max_context_bytes: NonNegative[i64]
    max_depth: Positive[i64]
    max_steps: NonNegative[i64]
    max_output: NonNegative[i64]
def limits() -> Limits
def escape(text: &str, context: HtmlContext, max_bytes: NonNegative[i64] = 16777216) -> Result[str, TemplateError]
class Engine
    def add(self: &mut Engine, name: &str, source: &str) -> Result[(), TemplateError]
    def render(self, name: &str, context: &Json, html: bool = True) -> Result[str, TemplateError]
def engine(bounds: &Limits) -> Result[Engine, TemplateError]

std.math

Floating-point functions, checked rounding, integer algorithms, integer and float helpers, and constants. Errors: MathError, OverflowError[T]. Reference.

enum MathError
    Domain | Overflow
const pi: f64 = f64.PI
const e: f64 = f64.E
const tau: f64 = f64.TAU
const inf: f64 = f64.INF
const nan: f64 = f64.NAN
def isnan(x: f64) -> bool
def isinf(x: f64) -> bool
def isfinite(x: f64) -> bool
def fabs(x: f64) -> f64
def copysign(x: f64, y: f64) -> f64
def isclose(a: f64, b: f64, rel_tol: f64, abs_tol: f64) -> Result[bool, MathError]
def degrees(x: f64) -> f64
def radians(x: f64) -> f64
def sqrt(x: f64) -> Result[f64, MathError]
def exp(x: f64) -> Result[f64, MathError]
def exp2(x: f64) -> Result[f64, MathError]
def expm1(x: f64) -> Result[f64, MathError]
def log(x: f64) -> Result[f64, MathError]
def log2(x: f64) -> Result[f64, MathError]
def log10(x: f64) -> Result[f64, MathError]
def log1p(x: f64) -> Result[f64, MathError]
def log_base(x: f64, base: f64) -> Result[f64, MathError]
def sin(x: f64) -> Result[f64, MathError]
def cos(x: f64) -> Result[f64, MathError]
def tan(x: f64) -> Result[f64, MathError]
def asin(x: f64) -> Result[f64, MathError]
def acos(x: f64) -> Result[f64, MathError]
def atan(x: f64) -> Result[f64, MathError]
def atan2(y: f64, x: f64) -> Result[f64, MathError]
def sinh(x: f64) -> Result[f64, MathError]
def cosh(x: f64) -> Result[f64, MathError]
def tanh(x: f64) -> Result[f64, MathError]
def asinh(x: f64) -> Result[f64, MathError]
def acosh(x: f64) -> Result[f64, MathError]
def atanh(x: f64) -> Result[f64, MathError]
def cbrt(x: f64) -> Result[f64, MathError]
def pow(x: f64, y: f64) -> Result[f64, MathError]
def fmod(x: f64, y: f64) -> Result[f64, MathError]
def floor(x: f64) -> Result[i64, MathError]
def ceil(x: f64) -> Result[i64, MathError]
def trunc(x: f64) -> Result[i64, MathError]
def round(x: f64) -> Result[i64, MathError]
def divmod[T: IntType](a: T, b: T) -> Result[(T, T), MathError]
def divmod_float(a: f64, b: f64) -> Result[(f64, f64), MathError]
def factorial(n: NonNegative[i64]) -> Result[u64, MathError]
def gcd(a: u64, b: u64) -> u64
def lcm(a: u64, b: u64) -> Result[u64, MathError]
def isqrt(n: u64) -> u64
def perm(n: u64, k: u64) -> Result[u64, MathError]
def comb(n: u64, k: u64) -> Result[u64, MathError]
def abs[T: IntType](x: T) -> Result[T, OverflowError[T]]
def abs_float[T: FloatType](x: T) -> T
def min[T](a: T, b: T) -> T
def max[T](a: T, b: T) -> T
def min_all[T](items: &list[T]) -> Option[T]
def max_all[T](items: &list[T]) -> Option[T]
def sum[T: IntType](items: &list[T], start: T) -> Result[T, OverflowError[T]]
def prod[T: IntType](items: &list[T], start: T) -> Result[T, OverflowError[T]]
def sum_iter[I, T: IntType](items: &I, start: T) -> Result[T, OverflowError[T]]
def prod_iter[I, T: IntType](items: &I, start: T) -> Result[T, OverflowError[T]]
def sum_float[T: FloatType](items: &list[T], start: T) -> T
def prod_float[T: FloatType](items: &list[T], start: T) -> T
def sum_iter_float[I, T: FloatType](items: &I, start: T) -> T
def prod_iter_float[I, T: FloatType](items: &I, start: T) -> T

std.mime

Media types: RFC 9110 parsing and formatting with parameters, media-range matching, and the file-extension table. Errors: ParseError, AllocError. Reference.

enum MimeError
    Invalid | Allocation(AllocError)
class Param
    name: str
    value: str
class MediaType
    def type_name(self) -> &str
    def subtype(self) -> &str
    def params(self) -> &list[Param]
    def param(self, name: &str) -> Option[str]
    def suffix(self) -> Result[Option[str], AllocError]
    def essence(self) -> Result[str, AllocError]
    def is_wildcard(self) -> bool
    def matches(self, pattern: &MediaType) -> bool
    def set_param(self: &mut MediaType, name: &str, value: &str) -> Result[(), MimeError]
    def remove_param(self: &mut MediaType, name: &str) -> bool
    def to_string(self) -> Result[str, AllocError]
def parse(text: &str) -> Result[MediaType, MimeError]
def new(type_name: &str, subtype: &str) -> Result[MediaType, MimeError]
def from_extension(extension: &str) -> Option[str]
def from_file_name(name: &str) -> str

std.net

TCP listeners and streams, UDP sockets, IP addresses and name lookup, with deadlines and cancellation. Errors: NetError. Reference.

enum NetError
    TimedOut | ConnectionTimedOut | WouldBlock | Cancelled | Refused | Reset | AddressInUse | AddressUnavailable | Unreachable | NameNotFound | TryAgain | InvalidAddress | HandleLimit | Stream(StreamError)
enum IpAddress
    V4(u32) | V6(u64, u64)
class SocketAddress
    ip: IpAddress
    port: u16
    scope: u32
    def to_string(self) -> Result[str, NetError]
def socket_address(ip: IpAddress, port: u16) -> SocketAddress
def ipv4(a: u8, b: u8, c: u8, d: u8) -> IpAddress
def loopback_v4(port: u16) -> SocketAddress
def loopback_v6(port: u16) -> SocketAddress
def any_v4(port: u16) -> SocketAddress
def any_v6(port: u16) -> SocketAddress
def format_ip(ip: IpAddress) -> Result[str, NetError]
def parse_ip(text: &str) -> Result[IpAddress, NetError]
def parse_address(text: &str) -> Result[SocketAddress, NetError]
def resolve(host: &str, port: u16, deadline: &Deadline, cancel: &CancellationToken) -> Result[list[SocketAddress], NetError]
class Lookup
    def descriptor(self) -> &File
    def watch(self, selector: &mut Selector, key: u64) -> Result[(), NetError]
    def unwatch(self, selector: &mut Selector) -> Result[(), NetError]
    def finish_now(self: &mut Lookup) -> Result[list[SocketAddress], NetError]
def start_lookup(host: &str, port: u16) -> Result[Lookup, NetError]
enum Shutdown
    Read | Write | Both
class TcpStream
    def read(self: &mut TcpStream, buffer: &mut Bytes, start: i64, stop: i64) -> Result[i64, StreamError]
    def write(self: &mut TcpStream, source: &[u8]) -> Result[i64, StreamError]
    def flush(self: &mut TcpStream) -> Result[(), StreamError]
    def close(self: &mut TcpStream) -> Result[(), StreamError]
    def read_now(self: &mut TcpStream, buffer: &mut Bytes, start: i64, stop: i64) -> Result[i64, NetError]
    def write_now(self: &mut TcpStream, source: &[u8]) -> Result[i64, NetError]
    def read_before(self: &mut TcpStream, buffer: &mut Bytes, start: i64, stop: i64, deadline: &Deadline, cancel: &CancellationToken) -> Result[i64, NetError]
    def write_before(self: &mut TcpStream, source: &[u8], deadline: &Deadline, cancel: &CancellationToken) -> Result[i64, NetError]
    def write_all_before(self: &mut TcpStream, source: &[u8], deadline: &Deadline, cancel: &CancellationToken) -> Result[(), NetError]
    def local_address(self) -> Result[SocketAddress, NetError]
    def peer_address(self) -> Result[SocketAddress, NetError]
    def shutdown(self: &mut TcpStream, direction: Shutdown) -> Result[(), NetError]
    def descriptor(self) -> &File
    def watch(self, selector: &mut Selector, key: u64, interest: Interest) -> Result[(), NetError]
    def unwatch(self, selector: &mut Selector) -> Result[(), NetError]
    def set_no_delay(self: &mut TcpStream, enabled: bool) -> Result[(), NetError]
    def acknowledge_now(self: &mut TcpStream) -> Result[(), NetError]
class TcpListener
    def local_address(self) -> Result[SocketAddress, NetError]
    def descriptor(self) -> &File
    def watch(self, selector: &mut Selector, key: u64) -> Result[(), NetError]
    def unwatch(self, selector: &mut Selector) -> Result[(), NetError]
    def accept_now(self) -> Result[TcpStream, NetError]
    def accept(self, deadline: &Deadline, cancel: &CancellationToken) -> Result[TcpStream, NetError]
    def close(self: &mut TcpListener) -> Result[(), NetError]
def listen(address: &SocketAddress, backlog: i64) -> Result[TcpListener, NetError]
def connect(address: &SocketAddress, deadline: &Deadline, cancel: &CancellationToken) -> Result[TcpStream, NetError]
class Connecting
    def descriptor(self) -> &File
    def watch(self, selector: &mut Selector, key: u64) -> Result[(), NetError]
    def unwatch(self, selector: &mut Selector) -> Result[(), NetError]
    def finish_now(self: &mut Connecting) -> Result[(), NetError]
    def stream(self: Connecting) -> TcpStream
def start_connect(address: &SocketAddress) -> Result[Connecting, NetError]
class Datagram
    size: i64
    stored: i64
    source: SocketAddress
    def truncated(self) -> bool
class UdpSocket
    def local_address(self) -> Result[SocketAddress, NetError]
    def descriptor(self) -> &File
    def watch(self, selector: &mut Selector, key: u64, interest: Interest) -> Result[(), NetError]
    def unwatch(self, selector: &mut Selector) -> Result[(), NetError]
    def set_broadcast(self: &mut UdpSocket, enabled: bool) -> Result[(), NetError]
    def send_to(self: &mut UdpSocket, source: &[u8], target: &SocketAddress, deadline: &Deadline, cancel: &CancellationToken) -> Result[i64, NetError]
    def receive_from(self: &mut UdpSocket, buffer: &mut Bytes, start: i64, stop: i64, deadline: &Deadline, cancel: &CancellationToken) -> Result[Datagram, NetError]
    def close(self: &mut UdpSocket) -> Result[(), NetError]
def bind_udp(address: &SocketAddress) -> Result[UdpSocket, NetError]

std.path

Lossless Linux paths, lexical operations, checked text conversion and escaped display. Errors: IoError, AllocError. Reference.

class Path
    def as_bytes(self) -> &list[u8]
    def to_text(self) -> Result[str, IoError]
    def to_string(self) -> Result[str, IoError]
    def copy(self) -> Result[Path, AllocError]
    def join(self, child: &Path) -> Result[Path, IoError]
    def is_absolute(self) -> bool
    def parent(self) -> Result[Option[Path], IoError]
    def file_name(self) -> Result[Option[Path], IoError]
    def components(self) -> Result[list[list[u8]], IoError]
def from_bytes(bytes: list[u8]) -> Path
def from_text(text: &str) -> Result[Path, IoError]

std.process

Children spawned from an argument vector with no implicit shell, piped streams, bounded output collection, signals, and explicit reaping. Errors: IoError, StreamError. Reference.

enum Stdio
    Inherit | Null | Piped
class ExitStatus
    def success(self) -> bool
    def code(self) -> Option[i32]
    def signal(self) -> Option[i32]
class Output
    status: ExitStatus
    stdout: Bytes
    stderr: Bytes
class Command
    def arg_bytes(self: &mut Command, value: &list[u8]) -> Result[(), IoError]
    def arg(self: &mut Command, value: &str) -> Result[(), IoError]
    def arg_path(self: &mut Command, value: &Path) -> Result[(), IoError]
    def env_bytes(self: &mut Command, name: &list[u8], value: &list[u8]) -> Result[(), IoError]
    def env(self: &mut Command, name: &str, value: &str) -> Result[(), IoError]
    def env_remove_bytes(self: &mut Command, name: &list[u8]) -> Result[(), IoError]
    def env_remove(self: &mut Command, name: &str) -> Result[(), IoError]
    def env_clear(self: &mut Command) -> ()
    def current_dir(self: &mut Command, path: &Path) -> Result[(), IoError]
    def stdin(self: &mut Command, mode: Stdio) -> ()
    def stdout(self: &mut Command, mode: Stdio) -> ()
    def stderr(self: &mut Command, mode: Stdio) -> ()
    def spawn(self) -> Result[Child, IoError]
def command_bytes(program: &list[u8]) -> Result[Command, IoError]
def command(program: &str) -> Result[Command, IoError]
def shell(script: &str) -> Result[Command, IoError]
class Child
    def id(self) -> i64
    def take_stdin(self: &mut Child) -> Option[File]
    def take_stdout(self: &mut Child) -> Option[File]
    def take_stderr(self: &mut Child) -> Option[File]
    def close_stdin(self: &mut Child) -> Result[(), IoError]
    def wait(self: &mut Child) -> Result[ExitStatus, IoError]
    def try_wait(self: &mut Child) -> Result[Option[ExitStatus], IoError]
    def wait_timeout(self: &mut Child, budget: &Duration, cancel: &CancellationToken) -> Result[Option[ExitStatus], WaitError]
    def wait_before(self: &mut Child, deadline: &Deadline, cancel: &CancellationToken) -> Result[Option[ExitStatus], WaitError]
    def signal(self, number: i32) -> Result[(), IoError]
    def terminate(self) -> Result[(), IoError]
    def kill(self) -> Result[(), IoError]
    def communicate(self: &mut Child, input: &[u8], limit: NonNegative[i64]) -> Result[Output, StreamError]
    def output(self: &mut Child, limit: NonNegative[i64]) -> Result[Output, StreamError]
enum WaitError
    Io(IoError) | Cancelled

std.random

Caller-owned seeded generators with unbiased ranges, shuffling, and index sampling; not for secrets. Errors: RandomError. Reference.

enum RandomError
    EmptyRange | ZeroState
class Rng
    def next_u64(self: &mut Rng) -> u64
    def next_u32(self: &mut Rng) -> u32
    def next_bool(self: &mut Rng) -> bool
    def next_f64(self: &mut Rng) -> f64
    def below(self: &mut Rng, bound: u64) -> Result[u64, RandomError]
    def between(self: &mut Rng, low: i64, high: i64) -> Result[i64, RandomError]
    def index(self: &mut Rng, count: i64) -> Result[i64, RandomError]
    def shuffle[T](self: &mut Rng, items: &mut list[T]) -> ()
    def sample_indices(self: &mut Rng, count: i64, amount: i64) -> Result[list[i64], AllocError]
    def fill(self: &mut Rng, target: &mut Bytes, start: i64, stop: i64) -> ()
    def state(self) -> (u64, u64, u64, u64)
def seeded(seed: u64) -> Rng
def from_state(a: u64, b: u64, c: u64, d: u64) -> Result[Rng, RandomError]

std.ready

Readiness waits on pipes and other handles, bounded by a deadline and a cancellation token. Errors: WaitError. Reference.

enum Interest
    Read | Write | ReadWrite
class Readiness
    readable: bool
    writable: bool
    hangup: bool
    error: bool
enum WaitError
    TimedOut | Cancelled | Stream(StreamError)
def wait_now(file: &File, interest: Interest) -> Result[Readiness, WaitError]
def wait_until(file: &File, interest: Interest, deadline: &Deadline, cancel: &CancellationToken) -> Result[Readiness, WaitError]
def wait(file: &File, interest: Interest, deadline: &Deadline, cancel: &CancellationToken) -> Result[Readiness, WaitError]
def read_before(stream: &mut FileStream, buffer: &mut Bytes, start: i64, stop: i64, deadline: &Deadline, cancel: &CancellationToken) -> Result[i64, WaitError]
def write_before(stream: &mut FileStream, source: &[u8], deadline: &Deadline, cancel: &CancellationToken) -> Result[i64, WaitError]
protocol Watchable
    def descriptor(self) -> &File
class Event
    key: u64
    readiness: Readiness
class Selector
    def watch(self: &mut Selector, file: &File, key: u64, interest: Interest) -> Result[(), WaitError]
    def unwatch(self: &mut Selector, file: &File) -> Result[(), WaitError]
    def wait(self: &mut Selector, deadline: &Deadline, cancel: &CancellationToken) -> Result[i64, WaitError]
    def event(self, index: i64) -> Result[Event, WaitError]
    def close(self: &mut Selector) -> Result[(), WaitError]
def selector(capacity: i64) -> Result[Selector, WaitError]

std.regex

Linear-time regular expressions over text and bytes: search, captures, iteration, split, and replacement under explicit pattern, memory, and work limits. Errors: RegexError. Reference.

enum Limit
    PatternBytes | Nesting | Repetition | Instructions | Memory | Work | Output | Matches
class SyntaxError
    offset: i64
    message: str
enum RegexError
    Syntax(SyntaxError) | LimitExceeded(Limit) | OutOfRange | Allocation(AllocError)
class Limits
    max_pattern_bytes: NonNegative[i64]
    max_nesting: NonNegative[i64]
    max_repeat: NonNegative[i64]
    max_instructions: NonNegative[i64]
    max_memory: NonNegative[i64]
    max_work: NonNegative[i64]
def limits() -> Limits
class Match
    start: i64
    end: i64
    def len(self) -> Result[i64, OverflowError[i64]]
    def is_empty(self) -> bool
    def text(self, haystack: &str) -> Result[str, RegexError]
    def bytes(self, data: &[u8]) -> Result[Bytes, RegexError]
class Captures
    def len(self) -> i64
    def get(self, index: i64) -> Option[Match]
    def name(self, name: &str) -> Option[Match]
class Cursor
class Regex
    def group_count(self) -> i64
    def group_index(self, name: &str) -> Option[i64]
    def instructions(self) -> i64
    def is_match(self, text: &str) -> Result[bool, RegexError]
    def is_match_bytes(self, data: &[u8]) -> Result[bool, RegexError]
    def find(self, text: &str) -> Result[Option[Match], RegexError]
    def find_at(self, text: &str, start: i64) -> Result[Option[Match], RegexError]
    def find_bytes(self, data: &[u8], start: i64) -> Result[Option[Match], RegexError]
    def captures(self, text: &str) -> Result[Option[Captures], RegexError]
    def captures_at(self, text: &str, start: i64) -> Result[Option[Captures], RegexError]
    def captures_bytes(self, data: &[u8], start: i64) -> Result[Option[Captures], RegexError]
    def cursor(self) -> Result[Cursor, RegexError]
    def next_match(self, text: &str, cursor: &mut Cursor) -> Result[Option[Match], RegexError]
    def next_match_bytes(self, data: &[u8], cursor: &mut Cursor) -> Result[Option[Match], RegexError]
    def find_all(self, text: &str, max_matches: i64) -> Result[list[Match], RegexError]
    def find_all_bytes(self, data: &[u8], max_matches: i64) -> Result[list[Match], RegexError]
    def split(self, text: &str, max_pieces: i64) -> Result[list[str], RegexError]
    def replace(self, text: &str, template: &str, max_bytes: i64) -> Result[str, RegexError]
    def replace_all(self, text: &str, template: &str, max_bytes: i64) -> Result[str, RegexError]
def compile(pattern: &str, bounds: &Limits) -> Result[Regex, RegexError]

std.report

Bounded human-readable report rendering with std.fmt. Errors: FormatError. Reference.

def render[T: Formattable, W](value: &T, output: &mut Formatter[W]) -> Result[(), FormatError]
def to_string[T: Formattable](value: &T, max_bytes: NonNegative[i64], max_depth: NonNegative[i64]) -> Result[str, FormatError]

std.secrets

Operating-system entropy: secret bytes, hex and base64url tokens, and unbiased integers, with no fallback source. Errors: SecretsError. Reference.

enum SecretsError
    OutOfRange | EmptyRange | Entropy(StreamError) | Allocation(AllocError)
class OsEntropy
    def read(self: &mut OsEntropy, buffer: &mut Bytes, start: i64, stop: i64) -> Result[i64, StreamError]
def os_entropy() -> OsEntropy
class Source[R]
    inner: R
    def fill(self: &mut Source[R], target: &mut Bytes, start: i64, stop: i64) -> Result[(), SecretsError]
    def token_bytes(self: &mut Source[R], count: i64) -> Result[Bytes, SecretsError]
    def token_hex(self: &mut Source[R], count: i64) -> Result[str, SecretsError]
    def token_urlsafe(self: &mut Source[R], count: i64) -> Result[str, SecretsError]
    def next_u64(self: &mut Source[R]) -> Result[u64, SecretsError]
    def below(self: &mut Source[R], bound: u64) -> Result[u64, SecretsError]
def source_from[R: Reader](inner: R) -> Source[R]
def os() -> Source[OsEntropy]
def fill(target: &mut Bytes, start: i64, stop: i64) -> Result[(), SecretsError]
def token_bytes(count: i64) -> Result[Bytes, SecretsError]
def token_hex(count: i64) -> Result[str, SecretsError]
def token_urlsafe(count: i64) -> Result[str, SecretsError]
def random_u64() -> Result[u64, SecretsError]
def below(bound: u64) -> Result[u64, SecretsError]

std.shlex

Shell-style word splitting and POSIX argument quoting, without execution or expansion. Errors: ShlexError, AllocError. Reference.

enum ShlexError
    UnclosedQuote(i64) | TrailingEscape(i64) | Nul(i64) | Allocation(AllocError)
def split(text: &str, comments: bool = False) -> Result[list[str], ShlexError]
def quote(text: &str) -> Result[str, ShlexError]

std.sketch

Bounded distinct counts, sampled quantiles and frequent-item bounds with versioned checkpoints. Errors: SketchError, AllocError, BytesError. Reference.

enum SketchError
    InvalidArgument | InvalidData | Incompatible | Empty | Limit | ConstructionFailed | Allocation(AllocError)
def hash_u64(key: u64, seed: u64 = 0u64) -> u64
class HyperLogLog
    def precision(self) -> i64
    def add(self: &mut HyperLogLog, key: u64) -> ()
    def estimate(self) -> f64
    def relative_standard_error(self) -> f64
    def merge(self: &mut HyperLogLog, other: &HyperLogLog) -> Result[(), SketchError]
    def to_bytes(self) -> Result[Bytes, AllocError]
def hyperloglog(precision: i64 = 12, seed: u64 = 0u64) -> Result[HyperLogLog, SketchError]
def hyperloglog_from_bytes(data: &[u8], max_precision: i64 = 18) -> Result[HyperLogLog, SketchError]
class Quantiles
    def count(self) -> u64
    def retained(self) -> i64
    def add(self: &mut Quantiles, value: f64) -> Result[(), SketchError]
    def quantile(self, fraction: f64) -> Result[f64, SketchError]
    def to_bytes(self) -> Result[Bytes, AllocError]
def quantiles(capacity: Positive[i64] = 1024, seed: u64 = 0u64) -> Result[Quantiles, SketchError]
def quantiles_from_bytes(data: &[u8], max_capacity: Positive[i64] = 1048576) -> Result[Quantiles, SketchError]
class Frequency
    key: u64
    lower: u64
    upper: u64
class FrequentItems
    def count(self) -> u64
    def error_bound(self) -> u64
    def add(self: &mut FrequentItems, key: u64) -> Result[(), SketchError]
    def frequency(self, key: u64) -> Frequency
    def candidates(self) -> Result[list[Frequency], AllocError]
    def to_bytes(self) -> Result[Bytes, AllocError]
def frequent_items(capacity: Positive[i64] = 128) -> Result[FrequentItems, SketchError]
def frequent_items_from_bytes(data: &[u8], max_capacity: Positive[i64] = 65536) -> Result[FrequentItems, SketchError]

std.sketch.xor

Static approximate membership over numeric identities with bounded construction. Errors: SketchError, AllocError, BytesError. Reference.

class XorFilter
    def len(self) -> i64
    def contains(self, key: u64) -> bool
    def to_bytes(self) -> Result[Bytes, AllocError]
def build(keys: &list[u64], max_keys: NonNegative[i64], seed: u64 = 0u64) -> Result[XorFilter, SketchError]
def from_bytes(data: &[u8], max_keys: NonNegative[i64]) -> Result[XorFilter, SketchError]

std.sketch.cuckoo

Scalable approximate membership with bounded growth and receipt-directed deletion. Errors: SketchError, AllocError, BytesError. Reference.

class CuckooFilter
    def len(self) -> u64
    def layers(self) -> i64
    def buckets(self) -> i64
    def contains(self, key: u64) -> bool
    def insert(self: &mut CuckooFilter, key: u64) -> Result[i64, SketchError]
    def remove(self: &mut CuckooFilter, key: u64, layer_index: i64) -> Result[bool, SketchError]
    def to_bytes(self) -> Result[Bytes, AllocError]
def cuckoo(initial_buckets: i64 = 64, max_buckets: Positive[i64] = 1048576, seed: u64 = 0u64) -> Result[CuckooFilter, SketchError]
def from_bytes(data: &[u8], max_buckets: Positive[i64]) -> Result[CuckooFilter, SketchError]

std.signal

SIGTERM and SIGINT as an event to wait or select on, and a helper that cancels a token on one. Errors: SignalError, WaitError. Reference.

enum Signal
    Interrupt | Terminate
enum SignalError
    AlreadyListening | Stream(StreamError)
class Signals
    def wait(self: &mut Signals, deadline: &Deadline, cancel: &CancellationToken) -> Result[Signal, WaitError]
    def received(self: &mut Signals) -> Result[Option[Signal], WaitError]
    def watch(self, selector: &mut Selector, key: u64) -> Result[(), WaitError]
    def unwatch(self, selector: &mut Selector) -> Result[(), WaitError]
    def close(self: &mut Signals) -> Result[(), WaitError]
def listen() -> Result[Signals, SignalError]
def cancel_on(signals: Signals, stop: CancellationToken) -> Result[Option[Signal], WaitError]

std.simd

Portable fixed-lane values, masks, checked and wrapping arithmetic, selection, and bounded list loads/stores. Reference.

enum i32x4
    Lanes(i32, i32, i32, i32)
    def lane(self, index: i64) -> Option[i32]
    def to_tuple(self) -> (i32, i32, i32, i32)
    def add(self, other: &i32x4) -> Result[i32x4, OverflowError[i32]]
    def sub(self, other: &i32x4) -> Result[i32x4, OverflowError[i32]]
    def mul(self, other: &i32x4) -> Result[i32x4, OverflowError[i32]]
    def wrapping_add(self, other: &i32x4) -> i32x4
    def wrapping_sub(self, other: &i32x4) -> i32x4
    def wrapping_mul(self, other: &i32x4) -> i32x4
    def simd_eq(self, other: &i32x4) -> mask32x4
    def simd_ne(self, other: &i32x4) -> mask32x4
    def simd_lt(self, other: &i32x4) -> mask32x4
    def simd_le(self, other: &i32x4) -> mask32x4
    def simd_gt(self, other: &i32x4) -> mask32x4
    def simd_ge(self, other: &i32x4) -> mask32x4
    def wrapping_sum(self) -> i32
    def store(self, values: &mut list[i32], offset: i64) -> bool
def splat_i32x4(value: i32) -> i32x4
def load_i32x4(values: &list[i32], offset: i64 = 0) -> Option[i32x4]
enum u32x4
    Lanes(u32, u32, u32, u32)
    def lane(self, index: i64) -> Option[u32]
    def to_tuple(self) -> (u32, u32, u32, u32)
    def add(self, other: &u32x4) -> Result[u32x4, OverflowError[u32]]
    def sub(self, other: &u32x4) -> Result[u32x4, OverflowError[u32]]
    def mul(self, other: &u32x4) -> Result[u32x4, OverflowError[u32]]
    def wrapping_add(self, other: &u32x4) -> u32x4
    def wrapping_sub(self, other: &u32x4) -> u32x4
    def wrapping_mul(self, other: &u32x4) -> u32x4
    def simd_eq(self, other: &u32x4) -> mask32x4
    def simd_ne(self, other: &u32x4) -> mask32x4
    def simd_lt(self, other: &u32x4) -> mask32x4
    def simd_le(self, other: &u32x4) -> mask32x4
    def simd_gt(self, other: &u32x4) -> mask32x4
    def simd_ge(self, other: &u32x4) -> mask32x4
    def wrapping_sum(self) -> u32
    def store(self, values: &mut list[u32], offset: i64) -> bool
def splat_u32x4(value: u32) -> u32x4
def load_u32x4(values: &list[u32], offset: i64 = 0) -> Option[u32x4]
enum f32x4
    Lanes(f32, f32, f32, f32)
    def lane(self, index: i64) -> Option[f32]
    def to_tuple(self) -> (f32, f32, f32, f32)
    def add(self, other: &f32x4) -> f32x4
    def sub(self, other: &f32x4) -> f32x4
    def mul(self, other: &f32x4) -> f32x4
    def div(self, other: &f32x4) -> f32x4
    def simd_eq(self, other: &f32x4) -> mask32x4
    def simd_ne(self, other: &f32x4) -> mask32x4
    def simd_lt(self, other: &f32x4) -> mask32x4
    def simd_le(self, other: &f32x4) -> mask32x4
    def simd_gt(self, other: &f32x4) -> mask32x4
    def simd_ge(self, other: &f32x4) -> mask32x4
    def sum_ordered(self, initial: f32) -> f32
    def store(self, values: &mut list[f32], offset: i64) -> bool
def splat_f32x4(value: f32) -> f32x4
def load_f32x4(values: &list[f32], offset: i64 = 0) -> Option[f32x4]
enum f64x2
    Lanes(f64, f64)
    def lane(self, index: i64) -> Option[f64]
    def to_tuple(self) -> (f64, f64)
    def add(self, other: &f64x2) -> f64x2
    def sub(self, other: &f64x2) -> f64x2
    def mul(self, other: &f64x2) -> f64x2
    def div(self, other: &f64x2) -> f64x2
    def simd_eq(self, other: &f64x2) -> mask64x2
    def simd_ne(self, other: &f64x2) -> mask64x2
    def simd_lt(self, other: &f64x2) -> mask64x2
    def simd_le(self, other: &f64x2) -> mask64x2
    def simd_gt(self, other: &f64x2) -> mask64x2
    def simd_ge(self, other: &f64x2) -> mask64x2
    def sum_ordered(self, initial: f64) -> f64
    def store(self, values: &mut list[f64], offset: i64) -> bool
def splat_f64x2(value: f64) -> f64x2
def load_f64x2(values: &list[f64], offset: i64 = 0) -> Option[f64x2]
enum mask32x4
    Lanes(bool, bool, bool, bool)
    def any(self) -> bool
    def all(self) -> bool
    def count(self) -> i64
    def select[T](self, yes: &T, no: &T) -> T
    def invert(self) -> mask32x4
    def and_mask(self, other: &mask32x4) -> mask32x4
    def or_mask(self, other: &mask32x4) -> mask32x4
    def xor_mask(self, other: &mask32x4) -> mask32x4
enum mask64x2
    Lanes(bool, bool)
    def any(self) -> bool
    def all(self) -> bool
    def count(self) -> i64
    def select[T](self, yes: &T, no: &T) -> T
    def invert(self) -> mask64x2
    def and_mask(self, other: &mask64x2) -> mask64x2
    def or_mask(self, other: &mask64x2) -> mask64x2
    def xor_mask(self, other: &mask64x2) -> mask64x2

std.stat

Descriptive f64 statistics, inclusive quantiles, and normal, uniform and exponential distributions with explicit RNG sampling. Errors: StatError. Reference.

enum StatError
    Empty | InsufficientData | NonFinite | Domain | Overflow | Allocation(AllocError)
def mean(values: &list[f64]) -> Result[f64, StatError]
def pstdev(values: &list[f64]) -> Result[f64, StatError]
def stdev(values: &list[f64]) -> Result[f64, StatError]
def pvariance(values: &list[f64]) -> Result[f64, StatError]
def variance(values: &list[f64]) -> Result[f64, StatError]
def quantile(values: &list[f64], p: f64) -> Result[f64, StatError]
def median(values: &list[f64]) -> Result[f64, StatError]
def quantiles(values: &list[f64], n: Positive[i64] = 4) -> Result[list[f64], StatError]
class Uniform
    def pdf(self, x: f64) -> Result[f64, StatError]
    def cdf(self, x: f64) -> Result[f64, StatError]
    def inverse_cdf(self, p: f64) -> Result[f64, StatError]
    def sample(self, rng: &mut random.Rng) -> f64
def uniform(low: f64 = 0.0, high: f64 = 1.0) -> Result[Uniform, StatError]
class Exponential
    def pdf(self, x: f64) -> Result[f64, StatError]
    def cdf(self, x: f64) -> Result[f64, StatError]
    def inverse_cdf(self, p: f64) -> Result[f64, StatError]
    def sample(self, rng: &mut random.Rng) -> Result[f64, StatError]
def exponential(rate: f64 = 1.0) -> Result[Exponential, StatError]
class Normal
    def pdf(self, x: f64) -> Result[f64, StatError]
    def cdf(self, x: f64) -> Result[f64, StatError]
    def inverse_cdf(self, p: f64) -> Result[f64, StatError]
    def sample(self, rng: &mut random.Rng) -> Result[f64, StatError]
def normal(mean: f64 = 0.0, sigma: f64 = 1.0) -> Result[Normal, StatError]

std.plot

Layered charts from owned columns or record accessors, validated mappings, grouped statistics, and deterministic streaming SVG. Errors: PlotError. Reference.

enum PlotError
    Allocation(AllocError) | Write(i64, StreamError) | Io(StreamError) | UnknownColumn(str) | DuplicateColumn(str) | ShapeMismatch(str) | MissingAesthetic(str) | WrongKind(str) | Conflict(str) | InvalidSetting(str)
enum Column
    Num(list[f64]) | Int(list[i64]) | Text(list[str]) | Bool(list[bool]) | MaybeNum(list[Option[f64]]) | MaybeText(list[Option[str]])
class Table
    def share(self) -> Table
    def height(self) -> i64
    def names(self) -> Result[list[str], PlotError]
def table(values: list[(str, Column)]) -> Result[Table, PlotError]
def columns(values: &dict[str, list[f64]]) -> Result[Table, PlotError]
enum Field[T]
    Num(str, BoxedCallable[[&T], f64]) | Int(str, BoxedCallable[[&T], i64]) | Text(str, BoxedCallable[[&T], str]) | Bool(str, BoxedCallable[[&T], bool])
def num[T, F: Callable[[&T], f64]](name: str, read: F) -> Result[Field[T], PlotError]
def integer[T, F: Callable[[&T], i64]](name: str, read: F) -> Result[Field[T], PlotError]
def text[T, F: Callable[[&T], str]](name: str, read: F) -> Result[Field[T], PlotError]
def flag[T, F: Callable[[&T], bool]](name: str, read: F) -> Result[Field[T], PlotError]
def rows[T](values: &list[T], fields: list[Field[T]]) -> Result[Table, PlotError]
class Factor
def factor(name: str) -> Factor
class Mapping
def aes(x: str | Factor = "", y: str | Factor = "", colour: str | Factor = "", color: str | Factor = "", fill: str | Factor = "", group: str = "", ymin: str = "", ymax: str = "") -> Mapping
enum Position
    Identity | Stack | Dodge | Fill
def position_identity() -> Position
def position_stack() -> Position
def position_dodge() -> Position
def position_fill() -> Position
enum Units
    In | Cm | Mm | Px
enum Legend
    Right | Hidden
class Layer
class Labels
class Theme
enum Component
    Layer(Layer) | Labels(Labels) | Theme(Theme) | Cartesian(Option[(f64, f64)], Option[(f64, f64)]) | Flip | ColourScale(bool, ColourScale) | PositionScale(bool, PositionScale) | Facet(Facet)
enum Scales
    Fixed | Free | FreeX | FreeY
class Facet
def facet_wrap(facets: str, nrow: i64 = 0, ncol: i64 = 0, scales: Scales = Scales.Fixed) -> Component
def facet_grid(rows: str = "", cols: str = "", scales: Scales = Scales.Fixed) -> Component
enum ColourScale
    Default | Manual(dict[str, str]) | Gradient(str, str)
def scale_colour_manual(values: dict[str, str]) -> Component
def scale_color_manual(values: dict[str, str]) -> Component
def scale_fill_manual(values: dict[str, str]) -> Component
def scale_colour_gradient(low: str = "#132B43", high: str = "#56B1F7") -> Component
def scale_color_gradient(low: str = "#132B43", high: str = "#56B1F7") -> Component
def scale_fill_gradient(low: str = "#132B43", high: str = "#56B1F7") -> Component
enum Unset
    Unset
enum Transform
    Identity | Log10 | Sqrt | Reverse
class PositionScale
def scale_x_continuous(name: str = "", breaks: Option[list[f64]] = Nothing, labels: Option[list[str]] = Nothing, limits: (f64, f64) | Unset = Unset.Unset, transform: Transform = Transform.Identity) -> Component
def scale_y_continuous(name: str = "", breaks: Option[list[f64]] = Nothing, labels: Option[list[str]] = Nothing, limits: (f64, f64) | Unset = Unset.Unset, transform: Transform = Transform.Identity) -> Component
def scale_x_discrete(name: str = "", limits: Option[list[str]] = Nothing, labels: Option[list[str]] = Nothing) -> Component
def scale_y_discrete(name: str = "", limits: Option[list[str]] = Nothing, labels: Option[list[str]] = Nothing) -> Component
def scale_x_log10(name: str = "", breaks: Option[list[f64]] = Nothing, labels: Option[list[str]] = Nothing, limits: (f64, f64) | Unset = Unset.Unset) -> Component
def scale_y_log10(name: str = "", breaks: Option[list[f64]] = Nothing, labels: Option[list[str]] = Nothing, limits: (f64, f64) | Unset = Unset.Unset) -> Component
def scale_x_sqrt(name: str = "", breaks: Option[list[f64]] = Nothing, labels: Option[list[str]] = Nothing, limits: (f64, f64) | Unset = Unset.Unset) -> Component
def scale_x_reverse(name: str = "", breaks: Option[list[f64]] = Nothing, labels: Option[list[str]] = Nothing, limits: (f64, f64) | Unset = Unset.Unset) -> Component
def scale_y_sqrt(name: str = "", breaks: Option[list[f64]] = Nothing, labels: Option[list[str]] = Nothing, limits: (f64, f64) | Unset = Unset.Unset) -> Component
def scale_y_reverse(name: str = "", breaks: Option[list[f64]] = Nothing, labels: Option[list[str]] = Nothing, limits: (f64, f64) | Unset = Unset.Unset) -> Component
def geom_point(colour: str = "", color: str = "", size: f64 | i64 = 1.5, alpha: f64 | i64 = 1.0, na_rm: bool = False, data: Table | Unset = Unset.Unset, mapping: Mapping | Unset = Unset.Unset, inherit_aes: bool = True) -> Component
def geom_line(colour: str = "", color: str = "", linewidth: f64 | i64 = 0.5, alpha: f64 | i64 = 1.0, na_rm: bool = False, data: Table | Unset = Unset.Unset, mapping: Mapping | Unset = Unset.Unset, inherit_aes: bool = True) -> Component
def geom_path(colour: str = "", color: str = "", linewidth: f64 | i64 = 0.5, alpha: f64 | i64 = 1.0, na_rm: bool = False, data: Table | Unset = Unset.Unset, mapping: Mapping | Unset = Unset.Unset, inherit_aes: bool = True) -> Component
def geom_col(colour: str = "", color: str = "", fill: str = "", width: f64 | i64 = 0.9, alpha: f64 | i64 = 1.0, position: Position = Position.Stack, na_rm: bool = False, data: Table | Unset = Unset.Unset, mapping: Mapping | Unset = Unset.Unset, inherit_aes: bool = True) -> Component
def geom_bar(colour: str = "", color: str = "", fill: str = "", width: f64 | i64 = 0.9, alpha: f64 | i64 = 1.0, position: Position = Position.Stack, na_rm: bool = False, data: Table | Unset = Unset.Unset, mapping: Mapping | Unset = Unset.Unset, inherit_aes: bool = True) -> Component
def geom_histogram(colour: str = "", color: str = "", fill: str = "", bins: i64 = 30, alpha: f64 | i64 = 1.0, position: Position = Position.Stack, na_rm: bool = False, data: Table | Unset = Unset.Unset, mapping: Mapping | Unset = Unset.Unset, inherit_aes: bool = True) -> Component
def geom_errorbar(colour: str = "", color: str = "", linewidth: f64 | i64 = 0.5, width: f64 | i64 = 0.2, alpha: f64 | i64 = 1.0, na_rm: bool = False, data: Table | Unset = Unset.Unset, mapping: Mapping | Unset = Unset.Unset, inherit_aes: bool = True) -> Component
def geom_linerange(colour: str = "", color: str = "", linewidth: f64 | i64 = 0.5, alpha: f64 | i64 = 1.0, na_rm: bool = False, data: Table | Unset = Unset.Unset, mapping: Mapping | Unset = Unset.Unset, inherit_aes: bool = True) -> Component
def geom_ribbon(colour: str = "", color: str = "", fill: str = "", linewidth: f64 | i64 = 0.5, alpha: f64 | i64 = 0.3, na_rm: bool = False, data: Table | Unset = Unset.Unset, mapping: Mapping | Unset = Unset.Unset, inherit_aes: bool = True) -> Component
def labs(title: str = "", x: str = "", y: str = "", colour: str = "", fill: str = "") -> Component
def theme_grey(base_size: f64 | i64 = 11, legend_position: Legend = Legend.Right) -> Component
def theme_minimal(base_size: f64 | i64 = 11, legend_position: Legend = Legend.Right) -> Component
def theme_classic(base_size: f64 | i64 = 11, legend_position: Legend = Legend.Right) -> Component
def coord_flip() -> Component
def coord_cartesian(xlim: (f64, f64) | Unset = Unset.Unset, ylim: (f64, f64) | Unset = Unset.Unset) -> Component
class Plot
    def plus(self, components: list[Component]) -> Result[Plot, PlotError]
    def write_svg[W: Writer](self, writer: &mut W, width: f64 = 6.0, height: f64 = 4.0, units: Units = Units.In, dpi: f64 = 300.0) -> Result[list[str], PlotError]
    def save(self, path: &str, width: f64 = 6.0, height: f64 = 4.0, units: Units = Units.In, dpi: f64 = 300.0) -> Result[list[str], PlotError]
def ggplot(data: Table, mapping: Mapping, components: list[Component]) -> Result[Plot, PlotError]

std.linalg

Dense f64 matrices, products, solves, LU, QR, Cholesky, SVD and eigen decompositions through an optional faer engine. Errors: AllocError, LinalgError. Reference.

enum LinalgError
    Shape | Bounds | NonFinite | Singular | NotSymmetric | NotPositiveDefinite | NoConvergence | Overflow | NativePanic | Allocation(AllocError)
class Matrix
    def rows(self) -> i64
    def cols(self) -> i64
    def get(self, row: i64, col: i64) -> Result[f64, LinalgError]
    def set(self: &mut Matrix, row: i64, col: i64, value: f64) -> Result[(), LinalgError]
    def to_list(self) -> Result[list[f64], AllocError]
    def transpose(self) -> Result[Matrix, AllocError]
    def add(self, rhs: &Matrix) -> Result[Matrix, LinalgError]
    def subtract(self, rhs: &Matrix) -> Result[Matrix, LinalgError]
    def scale(self, factor: f64) -> Result[Matrix, LinalgError]
    def negate(self) -> Result[Matrix, LinalgError]
    def __add__(self, rhs: &Matrix) -> Result[Matrix, LinalgError]
    def __sub__(self, rhs: &Matrix) -> Result[Matrix, LinalgError]
    def __neg__(self) -> Result[Matrix, LinalgError]
    def __mul__(self, rhs: &Matrix) -> Result[Matrix, LinalgError]
    def __rmul__(self, factor: f64) -> Result[Matrix, LinalgError]
    def matmul(self, rhs: &Matrix) -> Result[Matrix, LinalgError]
    def lu(self) -> Result[Lu, LinalgError]
    def solve(self, rhs: &Matrix) -> Result[Matrix, LinalgError]
    def qr(self) -> Result[(Matrix, Matrix), LinalgError]
    def cholesky(self) -> Result[Matrix, LinalgError]
    def svd(self) -> Result[(Matrix, list[f64], Matrix), LinalgError]
    def eigh(self) -> Result[(list[f64], Matrix), LinalgError]
    def eig(self) -> Result[(list[f64], list[f64], Matrix, Matrix), LinalgError]
class Lu
    def lower(self) -> Result[Matrix, LinalgError]
    def upper(self) -> Result[Matrix, LinalgError]
    def permutation(self) -> Result[list[i64], AllocError]
    def solve(self, rhs: &Matrix) -> Result[Matrix, LinalgError]
def matrix(rows: i64, cols: i64, values: &list[f64]) -> Result[Matrix, LinalgError]
def vector(values: &list[f64]) -> Result[Matrix, LinalgError]
def zeros(rows: i64, cols: i64) -> Result[Matrix, LinalgError]
def identity(n: i64) -> Result[Matrix, LinalgError]
def diagonal(values: &list[f64]) -> Result[Matrix, LinalgError]

std.sync

Shared cooperative cancellation tokens. Errors: AllocError. Reference.

Compiler-provided: CancellationToken. Their signatures are on the reference page.

class Mutex[T]
    def __init__(self, value: T) -> ()
enum WaitError
    TimedOut | Cancelled | Disconnected
enum SendBeforeError[T]
    TimedOut(T) | Cancelled(T) | Disconnected(T)
def recv_before[T](receiver: &Receiver[T], deadline: &Deadline, cancel: &CancellationToken) -> Result[T, WaitError]
def send_before[T](sender: &Sender[T], value: T, deadline: &Deadline, cancel: &CancellationToken) -> Result[(), SendBeforeError[T]]
def future_before[T](future: &Future[T], deadline: &Deadline, cancel: &CancellationToken) -> Result[(), WaitError]
def cancelled_before(token: &CancellationToken, deadline: &Deadline, cancel: &CancellationToken) -> Result[(), WaitError]
def select_recv_before[A, B](first: &Receiver[A], second: &Receiver[B], deadline: &Deadline, cancel: &CancellationToken) -> Result[Selected[A, B], WaitError]

std.system

Bounded Linux procfs snapshots: CPUs, memory, load, uptime, block devices, disk I/O and process status. Errors: IoError, SystemError. Reference.

enum SystemError
    Io(IoError) | Stream(StreamError) | Allocation(AllocError) | Limit | Malformed(str)
class Memory
    total_bytes: u64
    free_bytes: u64
    available_bytes: Option[u64]
    swap_total_bytes: u64
    swap_free_bytes: u64
class LoadAverage
    one_minute: f64
    five_minutes: f64
    fifteen_minutes: f64
    runnable_tasks: u64
    total_tasks: u64
    last_pid: u32
class Uptime
    seconds: f64
    idle_cpu_seconds: f64
class BlockDevice
    major: u32
    minor: u32
    name: str
    size_bytes: u64
class DiskIo
    major: u32
    minor: u32
    name: str
    reads_completed: u64
    bytes_read: u64
    writes_completed: u64
    bytes_written: u64
    in_flight: u64
    busy_milliseconds: u64
class Process
    pid: u32
    parent_pid: u32
    name: str
    state: str
    real_uid: u32
    threads: u64
    virtual_bytes: Option[u64]
    resident_bytes: Option[u64]
class Procfs
    def cpu_count(self, max_bytes: Positive[i64] = 1048576) -> Result[i64, SystemError]
    def memory(self, max_bytes: Positive[i64] = 65536) -> Result[Memory, SystemError]
    def load_average(self, max_bytes: Positive[i64] = 4096) -> Result[LoadAverage, SystemError]
    def uptime(self, max_bytes: Positive[i64] = 4096) -> Result[Uptime, SystemError]
    def block_devices(self, max_bytes: Positive[i64] = 1048576, max_devices: NonNegative[i64] = 4096) -> Result[list[BlockDevice], SystemError]
    def disk_io(self, max_bytes: Positive[i64] = 1048576, max_devices: NonNegative[i64] = 4096) -> Result[list[DiskIo], SystemError]
    def process_ids(self, max_entries: NonNegative[i64] = 65536) -> Result[list[u32], SystemError]
    def process(self, pid: u32, max_bytes: Positive[i64] = 65536) -> Result[Process, SystemError]
def open_procfs(path: &Path) -> Result[Procfs, IoError]
def local() -> Result[Procfs, IoError]

std.tempfile

Unnamed and named temporary files, recursively cleaned directories, context managers and explicit ownership transfer. Errors: IoError, FsError, StreamError, AllocError. Reference.

def default_dir() -> Result[Path, IoError]
class TemporaryFile
    def stream(self: &mut TemporaryFile) -> &mut FileStream
    def close(self: &mut TemporaryFile) -> Result[(), StreamError]
    def __enter__(self: &mut TemporaryFile) -> &mut FileStream
    def __exit__(self: &mut TemporaryFile) -> ()
def file(parent: &Path = default_dir()?) -> Result[TemporaryFile, IoError]
class NamedTemporaryFile
    def path(self) -> &Path
    def read(self: &mut NamedTemporaryFile, buffer: &mut Bytes, start: i64, stop: i64) -> Result[i64, StreamError]
    def write(self: &mut NamedTemporaryFile, source: &[u8]) -> Result[i64, StreamError]
    def flush(self: &mut NamedTemporaryFile) -> Result[(), StreamError]
    def sync(self: &mut NamedTemporaryFile) -> Result[(), StreamError]
    def seek(self: &mut NamedTemporaryFile, target: SeekFrom) -> Result[i64, StreamError]
    def close(self: &mut NamedTemporaryFile) -> Result[(), FsError]
    def keep(self: &mut NamedTemporaryFile) -> Result[Path, AllocError]
    def __enter__(self: &mut NamedTemporaryFile) -> &mut NamedTemporaryFile
    def __exit__(self: &mut NamedTemporaryFile) -> ()
def named_file(parent: &Path = io_error(default_dir())?, prefix: &str = "plenty-", suffix: &str = "") -> Result[NamedTemporaryFile, FsError]
class TemporaryDirectory
    def path(self) -> &Path
    def root(self) -> &Root
    def cleanup(self: &mut TemporaryDirectory) -> Result[(), FsError]
    def keep(self: &mut TemporaryDirectory) -> Result[Path, AllocError]
    def __enter__(self: &mut TemporaryDirectory) -> &mut TemporaryDirectory
    def __exit__(self: &mut TemporaryDirectory) -> ()
def directory(parent: &Path = io_error(default_dir())?, prefix: &str = "plenty-", max_entries: Positive[i64] = 100000, max_depth: Positive[i64] = 64) -> Result[TemporaryDirectory, FsError]

std.test

Assertions that explain their failure on standard error, self-removing scratch directories, the seeded generator for plenty test, and fake clocks and streams. Errors: TestError. Reference.

enum TestError
    Assertion | Fixture(FsError) | Allocation(AllocError) | InvalidSeed | Unavailable
def fail(message: &str) -> Result[(), TestError]
def check(condition: bool, message: &str) -> Result[(), TestError]
def equal[T](left: &T, right: &T) -> Result[(), TestError]
def not_equal[T](left: &T, right: &T) -> Result[(), TestError]
def less[T](left: &T, right: &T) -> Result[(), TestError]
def close(left: f64, right: f64, tolerance: f64) -> Result[(), TestError]
def is_ok[T, E](result: &Result[T, E]) -> Result[(), TestError]
def is_err[T, E](result: &Result[T, E]) -> Result[(), TestError]
def fail_allocations_after(count: u32) -> Result[(), TestError]
def restore_allocations() -> Result[(), TestError]
def fail_files_after(count: u32, code: i32) -> Result[(), TestError]
def restore_files() -> Result[(), TestError]
class Scratch
    def path(self) -> &Path
    def root(self) -> &Root
    def file(self, name: &str) -> Result[str, TestError]
    def cleanup(self: &mut Scratch) -> Result[(), TestError]
def scratch() -> Result[Scratch, TestError]
def seed() -> Result[u64, TestError]
def rng() -> Result[Rng, TestError]
class FakeClock
    def now(self) -> Instant
    def advance(self: &mut FakeClock, span: &Duration) -> Result[(), RangeError]
    def set(self: &mut FakeClock, nanoseconds_after_origin: u64) -> ()
def fake_clock(nanoseconds_after_origin: u64) -> FakeClock
class FakeStream
    input: Bytes
    output: Bytes
    def calls(self) -> i64
    def limit(self: &mut FakeStream, bytes: i64) -> ()
    def fail_after(self: &mut FakeStream, after: i64, error: StreamError) -> Result[(), OverflowError[i64]]
    def read(self: &mut FakeStream, buffer: &mut Bytes, start: i64, stop: i64) -> Result[i64, StreamError]
    def write(self: &mut FakeStream, source: &[u8]) -> Result[i64, StreamError]
    def flush(self: &mut FakeStream) -> Result[(), StreamError]
def fake_stream(input: Bytes) -> Result[FakeStream, AllocError]

std.text

Bounded builders, joining, scalar/word wrapping, shortening, and indentation. Errors: TextError. Reference.

def find_in(text: &str, set: &[u8]) -> Option[i64]
def find_not_in(text: &str, set: &[u8]) -> Option[i64]
enum TextError
    InvalidLimit | LimitExceeded | Allocation(AllocError) | Bytes(BytesError)
class Builder
    def byte_len(self) -> i64
    def limit(self) -> i64
    def append(self: &mut Builder, text: &str) -> Result[(), TextError]
    def clear(self: &mut Builder) -> ()
    def to_text(self) -> Result[str, TextError]
    def finish(self: Builder) -> Result[str, TextError]
def builder(max_bytes: NonNegative[i64]) -> Result[Builder, TextError]
def join(pieces: &list[str], separator: &str, max_bytes: NonNegative[i64]) -> Result[str, TextError]
def hard_wrap(text: &str, width: Positive[i64], max_bytes: NonNegative[i64]) -> Result[str, TextError]
class TextWrapper
    width: Positive[i64]
    initial_indent: str
    subsequent_indent: str
    expand_tabs: bool
    replace_whitespace: bool
    fix_sentence_endings: bool
    break_long_words: bool
    drop_whitespace: bool
    break_on_hyphens: bool
    tabsize: NonNegative[i64]
    max_lines: Option[Positive[i64]]
    placeholder: str
    max_bytes: NonNegative[i64]
    max_input_bytes: NonNegative[i64]
    def wrap(self, text: &str) -> Result[list[str], TextError]
    def fill(self, text: &str) -> Result[str, TextError]
def text_wrapper(width: Positive[i64] = 70, max_bytes: NonNegative[i64] = 1048576, max_input_bytes: NonNegative[i64] = 1048576) -> TextWrapper
def wrap(text: &str, width: Positive[i64] = 70, max_bytes: NonNegative[i64] = 1048576, max_input_bytes: NonNegative[i64] = 1048576) -> Result[list[str], TextError]
def fill(text: &str, width: Positive[i64] = 70, max_bytes: NonNegative[i64] = 1048576, max_input_bytes: NonNegative[i64] = 1048576) -> Result[str, TextError]
def shorten(text: &str, width: Positive[i64], placeholder: &str = " [...]", max_bytes: NonNegative[i64] = 1048576, max_input_bytes: NonNegative[i64] = 1048576, fix_sentence_endings: bool = False, break_long_words: bool = True, break_on_hyphens: bool = True) -> Result[str, TextError]
def dedent(text: &str, max_bytes: NonNegative[i64] = 1048576, max_input_bytes: NonNegative[i64] = 1048576) -> Result[str, TextError]
def indent(text: &str, prefix: &str, max_bytes: NonNegative[i64] = 1048576, max_input_bytes: NonNegative[i64] = 1048576) -> Result[str, TextError]
def indent_with[F: Callable[[&str], bool]](text: &str, prefix: &str, predicate: F, max_bytes: NonNegative[i64] = 1048576, max_input_bytes: NonNegative[i64] = 1048576) -> Result[str, TextError]

std.thread

Explicit scoped thread and owned-job imports. Errors: ThreadError, SpawnError[F]. Reference.

Compiler-provided: spawn, ThreadError, SpawnError. Their signatures are on the reference page.

std.time

Checked durations, monotonic instants and deadlines, wall timestamps, and sleep. Errors: RangeError. Reference.

class Duration
    def as_nanoseconds(self) -> u64
    def as_microseconds(self) -> u64
    def as_milliseconds(self) -> u64
    def as_seconds(self) -> u64
    def as_timeout_ms(self) -> u64
    def is_zero(self) -> bool
    def checked_add(self, other: &Duration) -> Result[Duration, RangeError]
    def checked_sub(self, other: &Duration) -> Result[Duration, RangeError]
    def saturating_sub(self, other: &Duration) -> Duration
    def checked_mul(self, factor: u64) -> Result[Duration, RangeError]
def nanoseconds(count: u64) -> Duration
def microseconds(count: u32) -> Duration
def milliseconds(count: u32) -> Duration
def seconds(count: u32) -> Duration
class Instant
    def checked_add(self, span: &Duration) -> Result[Instant, RangeError]
    def duration_since(self, earlier: &Instant) -> Result[Duration, RangeError]
    def saturating_duration_since(self, earlier: &Instant) -> Duration
def monotonic_now() -> Instant
def instant_at(nanoseconds_after_origin: u64) -> Instant
protocol Clock
    def now(self) -> Instant
class MonotonicClock
    def now(self) -> Instant
def monotonic_clock() -> MonotonicClock
class Deadline
    def expired(self, now: &Instant) -> bool
    def remaining(self, now: &Instant) -> Duration
def deadline_at(at: &Instant) -> Deadline
def deadline_after(start: &Instant, budget: &Duration) -> Result[Deadline, RangeError]
class Timestamp
    seconds: i64
    nanoseconds: u32
    def duration_since(self, earlier: &Timestamp) -> Result[Duration, RangeError]
    def checked_add(self, span: &Duration) -> Result[Timestamp, RangeError]
def wall_now() -> Timestamp
def sleep(span: &Duration) -> ()
def sleep_until(deadline: &Deadline) -> ()

std.tls

TLS 1.2 and 1.3 client and server streams over std.net TCP, with verified certificates, client certificates, ALPN, deadlines, and cancellation. Errors: TlsError. Reference.

enum TlsError
    Net(NetError) | UntrustedCertificate | ExpiredCertificate | WrongHost | InvalidCertificate | Rejected | Protocol | Truncated | InvalidName | InvalidConfiguration | NoTrustRoots | Closed | Allocation(AllocError) | RevokedCertificate | CertificateRequired | Incompatible
enum TlsVersion
    Tls12 | Tls13
enum CipherSuite
    Tls13Aes256GcmSha384 | Tls13Aes128GcmSha256 | Tls13Chacha20Poly1305Sha256 | EcdheEcdsaAes256GcmSha384 | EcdheEcdsaAes128GcmSha256 | EcdheEcdsaChacha20Poly1305Sha256 | EcdheRsaAes256GcmSha384 | EcdheRsaAes128GcmSha256 | EcdheRsaChacha20Poly1305Sha256
enum ClientCertificates
    Required | Optional
class TrustRoots
    def len(self) -> i64
def roots_from_pem(pem: &str) -> Result[TrustRoots, TlsError]
def system_roots() -> Result[TrustRoots, TlsError]
class ClientConfig
class ServerConfig
class ClientBuilder
    def certificate(self: &mut ClientBuilder, chain_pem: &str, key_pem: &str) -> Result[(), TlsError]
    def application_protocol(self: &mut ClientBuilder, name: &str) -> Result[(), TlsError]
    def minimum_version(self: &mut ClientBuilder, version: TlsVersion) -> ()
    def cipher_suite(self: &mut ClientBuilder, suite: CipherSuite) -> ()
    def revocation_list(self: &mut ClientBuilder, crl_pem: &str) -> Result[(), TlsError]
    def build(self) -> Result[ClientConfig, TlsError]
def client_builder(roots: &TrustRoots) -> Result[ClientBuilder, TlsError]
def client_config(roots: &TrustRoots) -> Result[ClientConfig, TlsError]
class ServerBuilder
    def certificate(self: &mut ServerBuilder, chain_pem: &str, key_pem: &str) -> Result[(), TlsError]
    def verify_clients(self: &mut ServerBuilder, roots: &TrustRoots, certificates: ClientCertificates) -> ()
    def application_protocol(self: &mut ServerBuilder, name: &str) -> Result[(), TlsError]
    def minimum_version(self: &mut ServerBuilder, version: TlsVersion) -> ()
    def cipher_suite(self: &mut ServerBuilder, suite: CipherSuite) -> ()
    def revocation_list(self: &mut ServerBuilder, crl_pem: &str) -> Result[(), TlsError]
    def build(self) -> Result[ServerConfig, TlsError]
def server_builder(chain_pem: &str, key_pem: &str) -> Result[ServerBuilder, TlsError]
def server_config(chain_pem: &str, key_pem: &str) -> Result[ServerConfig, TlsError]
class TlsStream
    def handshake_before(self: &mut TlsStream, deadline: &Deadline, cancel: &CancellationToken) -> Result[(), TlsError]
    def try_handshake(self: &mut TlsStream) -> Result[bool, TlsError]
    def try_flush(self: &mut TlsStream) -> Result[bool, TlsError]
    def try_read(self: &mut TlsStream, buffer: &mut Bytes, start: i64, stop: i64) -> Result[Option[i64], TlsError]
    def try_write(self: &mut TlsStream, source: &[u8]) -> Result[Option[i64], TlsError]
    def interest(self) -> Interest
    def watch(self, selector: &mut Selector, key: u64, interest: Interest) -> Result[(), TlsError]
    def unwatch(self, selector: &mut Selector) -> Result[(), TlsError]
    def read_before(self: &mut TlsStream, buffer: &mut Bytes, start: i64, stop: i64, deadline: &Deadline, cancel: &CancellationToken) -> Result[i64, TlsError]
    def write_before(self: &mut TlsStream, source: &[u8], deadline: &Deadline, cancel: &CancellationToken) -> Result[i64, TlsError]
    def write_all_before(self: &mut TlsStream, source: &[u8], deadline: &Deadline, cancel: &CancellationToken) -> Result[(), TlsError]
    def close_before(self: &mut TlsStream, deadline: &Deadline, cancel: &CancellationToken) -> Result[(), TlsError]
    def shutdown(self: &mut TlsStream, direction: Shutdown) -> Result[(), TlsError]
    def read(self: &mut TlsStream, buffer: &mut Bytes, start: i64, stop: i64) -> Result[i64, StreamError]
    def write(self: &mut TlsStream, source: &[u8]) -> Result[i64, StreamError]
    def flush(self: &mut TlsStream) -> Result[(), StreamError]
    def close(self: &mut TlsStream) -> Result[(), StreamError]
    def application_protocol(self) -> Result[Option[str], TlsError]
    def server_name(self) -> Result[Option[str], TlsError]
    def peer_certificate(self) -> Result[Option[Bytes], TlsError]
    def version(self) -> Option[TlsVersion]
    def cipher_suite(self) -> Option[CipherSuite]
def start_connect(tcp: TcpStream, config: &ClientConfig, server_name: &str) -> Result[TlsStream, TlsError]
def start_accept(tcp: TcpStream, config: &ServerConfig) -> Result[TlsStream, TlsError]
def connect(tcp: TcpStream, config: &ClientConfig, server_name: &str, deadline: &Deadline, cancel: &CancellationToken) -> Result[TlsStream, TlsError]
def accept(tcp: TcpStream, config: &ServerConfig, deadline: &Deadline, cancel: &CancellationToken) -> Result[TlsStream, TlsError]
class Session
    def feed(self: &mut Session, source: &[u8]) -> Result[i64, TlsError]
    def feed_end(self: &mut Session) -> Result[(), TlsError]
    def take(self: &mut Session, target: &mut Bytes, start: i64, stop: i64) -> Result[i64, TlsError]
    def read(self: &mut Session, target: &mut Bytes, start: i64, stop: i64) -> Result[Option[i64], TlsError]
    def write(self: &mut Session, source: &[u8]) -> Result[i64, TlsError]
    def close(self: &mut Session) -> ()
    def handshaking(self) -> bool
    def wants_write(self) -> bool
    def wants_read(self) -> bool
    def application_protocol(self) -> Result[Option[str], TlsError]
    def server_name(self) -> Result[Option[str], TlsError]
    def peer_certificate(self) -> Result[Option[Bytes], TlsError]
    def version(self) -> Option[TlsVersion]
    def cipher_suite(self) -> Option[CipherSuite]
def client_session(config: &ClientConfig, server_name: &str) -> Result[Session, TlsError]
def server_session(config: &ServerConfig) -> Result[Session, TlsError]

std.toml

Strict TOML 1.0 parsing, value-tree writing and format-preserving document edits, under depth, input, and output limits. Errors: TomlError. Reference.

enum TomlFault
    UnexpectedByte | UnexpectedEnd | InvalidUtf8 | ControlCharacter | InvalidEscape | InvalidNumber | NumberOutOfRange | InvalidDateTime | DuplicateKey | DuplicateTable | KeyConflict | DepthExceeded | InputTooLarge | OutputTooLarge | NotATable | Stream(StreamError) | Allocation(AllocError)
class TomlError
    fault: TomlFault
    offset: i64
    line: i64
    column: i64
class Limits
    max_depth: NonNegative[i64]
    max_input: NonNegative[i64]
    max_output: NonNegative[i64]
enum DateTimeKind
    OffsetDateTime | LocalDateTime | LocalDate | LocalTime
class DateTime
    kind: DateTimeKind
    year: i64
    month: i64
    day: i64
    hour: i64
    minute: i64
    second: i64
    nanosecond: i64
    offset_minutes: i64
enum Toml
    Bool(bool) | Int(i64) | Float(f64) | Text(str) | DateTime(DateTime) | Array(list[Toml]) | Table(list[Entry])
class Entry
    key: str
    value: Toml
def limits() -> Limits
def parse(source: &[u8], bounds: &Limits) -> Result[Toml, TomlError]
def parse_text(text: &str, bounds: &Limits) -> Result[Toml, TomlError]
def read_value[R: Reader](reader: &mut R, bounds: &Limits) -> Result[Toml, TomlError]
def find(entries: &list[Entry], key: &str) -> Option[i64]
def encode(value: &Toml, bounds: &Limits) -> Result[Bytes, TomlError]
def encode_text(value: &Toml, bounds: &Limits) -> Result[str, TomlError]
def write_value[W: Writer](writer: &mut W, value: &Toml, bounds: &Limits) -> Result[(), TomlError]
class Document
    def text(self) -> Result[str, TomlError]
    def get(self, path: &list[str]) -> Result[Option[Toml], TomlError]
    def set(self: &mut Document, path: &list[str], value: &Toml) -> Result[(), TomlError]
    def insert(self: &mut Document, path: &list[str], value: &Toml) -> Result[(), TomlError]
    def remove(self: &mut Document, path: &list[str]) -> Result[bool, TomlError]
def document(source: &[u8], bounds: &Limits) -> Result[Document, TomlError]
def document_text(text: &str, bounds: &Limits) -> Result[Document, TomlError]

std.unicode

Unicode 17 normalization, combining classes, and Mark classification. Errors: TextError. Reference.

def version() -> str
def combining_class(scalar: &str) -> Option[u32]
def is_mark(scalar: &str) -> Option[bool]
def nfd(text: &str, max_scalars: NonNegative[i64]) -> Result[str, TextError]
def nfc(text: &str, max_scalars: NonNegative[i64]) -> Result[str, TextError]
def nfkd(text: &str, max_scalars: NonNegative[i64]) -> Result[str, TextError]
def nfkc(text: &str, max_scalars: NonNegative[i64]) -> Result[str, TextError]

std.units

SI dimensions, prefixed and common units, physical constants, and dimension-aware square roots, sums and trigonometry over Quantity. Errors: MathError, RangeError. Reference.

dimension Mass("kg")
dimension Length("m")
dimension Time("s")
dimension Current("A")
dimension Temperature("K")
dimension Amount("mol")
dimension Luminosity("cd")
dimension Area = Pow[Length, 2]
dimension Volume = Pow[Length, 3]
dimension Wavenumber = One / Length
dimension Velocity = Length / Time
dimension Acceleration = Length / Pow[Time, 2]
dimension Frequency = One / Time
dimension Force = Mass * Acceleration
dimension Pressure = Force / Area
dimension Energy = Force * Length
dimension Power = Energy / Time
dimension Momentum = Mass * Velocity
dimension Action = Energy * Time
dimension Density = Mass / Volume
dimension MassFlowRate = Mass / Time
dimension VolumeFlowRate = Volume / Time
dimension DynamicViscosity = Pressure * Time
dimension KinematicViscosity = Area / Time
dimension SurfaceTension = Force / Length
dimension HeatFlux = Power / Area
dimension SpecificEnergy = Energy / Mass
dimension Entropy = Energy / Temperature
dimension SpecificHeatCapacity = SpecificEnergy / Temperature
dimension ThermalConductivity = Power / (Length * Temperature)
dimension MolarMass = Mass / Amount
dimension MolarEnergy = Energy / Amount
dimension MolarHeatCapacity = Entropy / Amount
dimension Concentration = Amount / Volume
dimension PerAmount = One / Amount
dimension CatalyticActivity = Amount / Time
dimension Charge = Current * Time
dimension Voltage = Power / Current
dimension ElectricField = Voltage / Length
dimension Capacitance = Charge / Voltage
dimension Permittivity = Capacitance / Length
dimension Resistance = Voltage / Current
dimension Conductance = Current / Voltage
dimension MagneticFlux = Voltage * Time
dimension MagneticFluxDensity = MagneticFlux / Area
dimension Inductance = MagneticFlux / Current
dimension Permeability = Inductance / Length
dimension Illuminance = Luminosity / Area
dimension LuminousEfficacy = Luminosity / Power
dimension StefanBoltzmann = HeatFlux / Pow[Temperature, 4]
dimension Gravitation = Volume / (Mass * Pow[Time, 2])
const kg: Quantity[f64, Mass] = Quantity[f64, Mass](1.0)
const m: Quantity[f64, Length] = Quantity[f64, Length](1.0)
const s: Quantity[f64, Time] = Quantity[f64, Time](1.0)
const A: Quantity[f64, Current] = Quantity[f64, Current](1.0)
const K: Quantity[f64, Temperature] = Quantity[f64, Temperature](1.0)
const mol: Quantity[f64, Amount] = Quantity[f64, Amount](1.0)
const cd: Quantity[f64, Luminosity] = Quantity[f64, Luminosity](1.0)
const rad: Quantity[f64, One] = Quantity[f64, One](1.0)
const sr: Quantity[f64, One] = Quantity[f64, One](1.0)
const degree: Quantity[f64, One] = (f64.PI / 180.0) * rad
const arcminute: Quantity[f64, One] = degree / 60.0
const arcsecond: Quantity[f64, One] = degree / 3600.0
const Hz: Quantity[f64, Frequency] = 1.0 / s
const N: Quantity[f64, Force] = kg * m / (s * s)
const Pa: Quantity[f64, Pressure] = N / (m * m)
const J: Quantity[f64, Energy] = N * m
const W: Quantity[f64, Power] = J / s
const C: Quantity[f64, Charge] = A * s
const V: Quantity[f64, Voltage] = W / A
const F: Quantity[f64, Capacitance] = C / V
const ohm: Quantity[f64, Resistance] = V / A
const S: Quantity[f64, Conductance] = A / V
const Wb: Quantity[f64, MagneticFlux] = V * s
const T: Quantity[f64, MagneticFluxDensity] = Wb / (m * m)
const H: Quantity[f64, Inductance] = Wb / A
const lm: Quantity[f64, Luminosity] = cd * sr
const lx: Quantity[f64, Illuminance] = lm / (m * m)
const Bq: Quantity[f64, Frequency] = 1.0 / s
const Gy: Quantity[f64, SpecificEnergy] = J / kg
const Sv: Quantity[f64, SpecificEnergy] = J / kg
const kat: Quantity[f64, CatalyticActivity] = mol / s
const ug: Quantity[f64, Mass] = 1e-9 * kg
const mg: Quantity[f64, Mass] = 1e-6 * kg
const g: Quantity[f64, Mass] = 1e-3 * kg
const pm: Quantity[f64, Length] = 1e-12 * m
const nm: Quantity[f64, Length] = 1e-9 * m
const um: Quantity[f64, Length] = 1e-6 * m
const mm: Quantity[f64, Length] = 1e-3 * m
const cm: Quantity[f64, Length] = 1e-2 * m
const km: Quantity[f64, Length] = 1e3 * m
const ps: Quantity[f64, Time] = 1e-12 * s
const ns: Quantity[f64, Time] = 1e-9 * s
const us: Quantity[f64, Time] = 1e-6 * s
const ms: Quantity[f64, Time] = 1e-3 * s
const uA: Quantity[f64, Current] = 1e-6 * A
const mA: Quantity[f64, Current] = 1e-3 * A
const kA: Quantity[f64, Current] = 1e3 * A
const mK: Quantity[f64, Temperature] = 1e-3 * K
const umol: Quantity[f64, Amount] = 1e-6 * mol
const mmol: Quantity[f64, Amount] = 1e-3 * mol
const kmol: Quantity[f64, Amount] = 1e3 * mol
const kHz: Quantity[f64, Frequency] = 1e3 * Hz
const MHz: Quantity[f64, Frequency] = 1e6 * Hz
const GHz: Quantity[f64, Frequency] = 1e9 * Hz
const mN: Quantity[f64, Force] = 1e-3 * N
const kN: Quantity[f64, Force] = 1e3 * N
const MN: Quantity[f64, Force] = 1e6 * N
const hPa: Quantity[f64, Pressure] = 1e2 * Pa
const kPa: Quantity[f64, Pressure] = 1e3 * Pa
const MPa: Quantity[f64, Pressure] = 1e6 * Pa
const GPa: Quantity[f64, Pressure] = 1e9 * Pa
const mJ: Quantity[f64, Energy] = 1e-3 * J
const kJ: Quantity[f64, Energy] = 1e3 * J
const MJ: Quantity[f64, Energy] = 1e6 * J
const GJ: Quantity[f64, Energy] = 1e9 * J
const uW: Quantity[f64, Power] = 1e-6 * W
const mW: Quantity[f64, Power] = 1e-3 * W
const kW: Quantity[f64, Power] = 1e3 * W
const MW: Quantity[f64, Power] = 1e6 * W
const GW: Quantity[f64, Power] = 1e9 * W
const Wh: Quantity[f64, Energy] = 3600.0 * J
const kWh: Quantity[f64, Energy] = 3.6e6 * J
const MWh: Quantity[f64, Energy] = 3.6e9 * J
const GWh: Quantity[f64, Energy] = 3.6e12 * J
const nC: Quantity[f64, Charge] = 1e-9 * C
const uC: Quantity[f64, Charge] = 1e-6 * C
const mC: Quantity[f64, Charge] = 1e-3 * C
const uV: Quantity[f64, Voltage] = 1e-6 * V
const mV: Quantity[f64, Voltage] = 1e-3 * V
const kV: Quantity[f64, Voltage] = 1e3 * V
const MV: Quantity[f64, Voltage] = 1e6 * V
const pF: Quantity[f64, Capacitance] = 1e-12 * F
const nF: Quantity[f64, Capacitance] = 1e-9 * F
const uF: Quantity[f64, Capacitance] = 1e-6 * F
const mF: Quantity[f64, Capacitance] = 1e-3 * F
const kohm: Quantity[f64, Resistance] = 1e3 * ohm
const Mohm: Quantity[f64, Resistance] = 1e6 * ohm
const uS: Quantity[f64, Conductance] = 1e-6 * S
const mS: Quantity[f64, Conductance] = 1e-3 * S
const uT: Quantity[f64, MagneticFluxDensity] = 1e-6 * T
const mT: Quantity[f64, MagneticFluxDensity] = 1e-3 * T
const nH: Quantity[f64, Inductance] = 1e-9 * H
const uH: Quantity[f64, Inductance] = 1e-6 * H
const mH: Quantity[f64, Inductance] = 1e-3 * H
const delta_nu_Cs: Quantity[f64, Frequency] = 9192631770.0 * Hz
const c: Quantity[f64, Velocity] = 299792458.0 * (m / s)
const h: Quantity[f64, Action] = 6.62607015e-34 * (J * s)
const e: Quantity[f64, Charge] = 1.602176634e-19 * C
const k_B: Quantity[f64, Entropy] = 1.380649e-23 * (J / K)
const N_A: Quantity[f64, PerAmount] = 6.02214076e23 / mol
const K_cd: Quantity[f64, LuminousEfficacy] = 683.0 * (lm / W)
const hbar: Quantity[f64, Action] = h / f64.TAU
const R: Quantity[f64, MolarHeatCapacity] = N_A * k_B
const sigma: Quantity[f64, StefanBoltzmann] = 5.670374419e-8 * (W / (m * m * K * K * K * K))
const g_n: Quantity[f64, Acceleration] = 9.80665 * (m / (s * s))
const atm: Quantity[f64, Pressure] = 101325.0 * Pa
const G: Quantity[f64, Gravitation] = 6.67430e-11 * (m * m * m / (kg * s * s))
const m_e: Quantity[f64, Mass] = 9.1093837139e-31 * kg
const m_p: Quantity[f64, Mass] = 1.67262192595e-27 * kg
const m_u: Quantity[f64, Mass] = 1.66053906892e-27 * kg
const epsilon_0: Quantity[f64, Permittivity] = 8.8541878188e-12 * (F / m)
const mu_0: Quantity[f64, Permeability] = 1.25663706127e-6 * (H / m)
const minute: Quantity[f64, Time] = 60.0 * s
const hour: Quantity[f64, Time] = 3600.0 * s
const day: Quantity[f64, Time] = 86400.0 * s
const hectare: Quantity[f64, Area] = 1e4 * (m * m)
const litre: Quantity[f64, Volume] = 1e-3 * (m * m * m)
const millilitre: Quantity[f64, Volume] = 1e-6 * (m * m * m)
const cubic_metre: Quantity[f64, Volume] = m * m * m
const tonne: Quantity[f64, Mass] = 1e3 * kg
const dalton: Quantity[f64, Mass] = m_u
const eV: Quantity[f64, Energy] = e * V
const keV: Quantity[f64, Energy] = 1e3 * eV
const MeV: Quantity[f64, Energy] = 1e6 * eV
const GeV: Quantity[f64, Energy] = 1e9 * eV
const au: Quantity[f64, Length] = 149597870700.0 * m
const bar: Quantity[f64, Pressure] = 1e5 * Pa
const mbar: Quantity[f64, Pressure] = 1e2 * Pa
const rankine: Quantity[f64, Temperature] = K * 5.0 / 9.0
const inch: Quantity[f64, Length] = 0.0254 * m
const foot: Quantity[f64, Length] = 0.3048 * m
const yard: Quantity[f64, Length] = 0.9144 * m
const mile: Quantity[f64, Length] = 1609.344 * m
const nautical_mile: Quantity[f64, Length] = 1852.0 * m
const angstrom: Quantity[f64, Length] = 1e-10 * m
const light_year: Quantity[f64, Length] = 9460730472580800.0 * m
const parsec: Quantity[f64, Length] = (648000.0 / f64.PI) * au
const acre: Quantity[f64, Area] = 4046.8564224 * (m * m)
const gallon: Quantity[f64, Volume] = 3.785411784e-3 * (m * m * m)
const pound: Quantity[f64, Mass] = 0.45359237 * kg
const ounce: Quantity[f64, Mass] = pound / 16.0
const pound_force: Quantity[f64, Force] = 0.45359237 * 9.80665 * N
const knot: Quantity[f64, Velocity] = nautical_mile / hour
const km_per_hour: Quantity[f64, Velocity] = km / hour
const mph: Quantity[f64, Velocity] = mile / hour
const psi: Quantity[f64, Pressure] = pound_force / (inch * inch)
const torr: Quantity[f64, Pressure] = atm / 760.0
const mmHg: Quantity[f64, Pressure] = 133.322387415 * Pa
const calorie: Quantity[f64, Energy] = 4.184 * J
const kilocalorie: Quantity[f64, Energy] = 4184.0 * J
const btu: Quantity[f64, Energy] = 1055.05585262 * J
const horsepower: Quantity[f64, Power] = 550.0 * foot * pound_force / s
const week: Quantity[f64, Time] = 7.0 * day
const year: Quantity[f64, Time] = 365.0 * day
def abs[T: IntType, D: Dimension](x: Quantity[T, D]) -> Result[Quantity[T, D], OverflowError[T]]
def abs_float[T: FloatType, D: Dimension](x: Quantity[T, D]) -> Quantity[T, D]
def clamp[Num, D: Dimension](x: Quantity[Num, D], low: Quantity[Num, D], high: Quantity[Num, D]) -> Quantity[Num, D]
def sum[D: Dimension](items: &list[Quantity[f64, D]]) -> Quantity[f64, D]
def mean[D: Dimension](items: &list[Quantity[f64, D]]) -> Option[Quantity[f64, D]]
def wrap_list[D: Dimension](values: list[f64]) -> list[Quantity[f64, D]]
def strip_list[D: Dimension](items: list[Quantity[f64, D]]) -> list[f64]
def sqrt[D: Dimension](x: Quantity[f64, D]) -> Result[Quantity[f64, Root[D, 2]], MathError]
def cbrt[D: Dimension](x: Quantity[f64, D]) -> Result[Quantity[f64, Root[D, 3]], MathError]
def hypot[D: Dimension](a: Quantity[f64, D], b: Quantity[f64, D]) -> Quantity[f64, D]
def exp(x: Quantity[f64, One]) -> Result[Quantity[f64, One], MathError]
def log(x: Quantity[f64, One]) -> Result[Quantity[f64, One], MathError]
def log2(x: Quantity[f64, One]) -> Result[Quantity[f64, One], MathError]
def log10(x: Quantity[f64, One]) -> Result[Quantity[f64, One], MathError]
def sin(angle: Quantity[f64, One]) -> Result[Quantity[f64, One], MathError]
def cos(angle: Quantity[f64, One]) -> Result[Quantity[f64, One], MathError]
def tan(angle: Quantity[f64, One]) -> Result[Quantity[f64, One], MathError]
def asin(x: Quantity[f64, One]) -> Result[Quantity[f64, One], MathError]
def acos(x: Quantity[f64, One]) -> Result[Quantity[f64, One], MathError]
def atan(x: Quantity[f64, One]) -> Result[Quantity[f64, One], MathError]
def atan2[D: Dimension](y: Quantity[f64, D], x: Quantity[f64, D]) -> Result[Quantity[f64, One], MathError]
def from_celsius(degrees: f64) -> Quantity[f64, Temperature]
def to_celsius(t: Quantity[f64, Temperature]) -> f64
def from_fahrenheit(degrees: f64) -> Quantity[f64, Temperature]
def to_fahrenheit(t: Quantity[f64, Temperature]) -> f64
def from_duration(span: &Duration) -> Quantity[f64, Time]
def to_duration(t: Quantity[f64, Time]) -> Result[Duration, RangeError]

std.units.catalogue

Every SI prefix on every SI unit and the other named units of NIST SP 811. Reference.

dimension LinearDensity = u.Mass / u.Length
dimension ElectricDipoleMoment = u.Charge * u.Length
dimension MagneticFieldStrength = u.Current / u.Length
dimension Exposure = u.Charge / u.Mass
dimension Luminance = u.Luminosity / u.Area
const qg: Quantity[f64, u.Mass] = 1e-33 * u.kg
const rg: Quantity[f64, u.Mass] = 1e-30 * u.kg
const yg: Quantity[f64, u.Mass] = 1e-27 * u.kg
const zg: Quantity[f64, u.Mass] = 1e-24 * u.kg
const ag: Quantity[f64, u.Mass] = 1e-21 * u.kg
const fg: Quantity[f64, u.Mass] = 1e-18 * u.kg
const pg: Quantity[f64, u.Mass] = 1e-15 * u.kg
const ng: Quantity[f64, u.Mass] = 1e-12 * u.kg
const ug: Quantity[f64, u.Mass] = 1e-9 * u.kg
const mg: Quantity[f64, u.Mass] = 1e-6 * u.kg
const cg: Quantity[f64, u.Mass] = 1e-5 * u.kg
const dg: Quantity[f64, u.Mass] = 1e-4 * u.kg
const dag: Quantity[f64, u.Mass] = 1e-2 * u.kg
const hg: Quantity[f64, u.Mass] = 1e-1 * u.kg
const kg: Quantity[f64, u.Mass] = 1e0 * u.kg
const Mg: Quantity[f64, u.Mass] = 1e3 * u.kg
const Gg: Quantity[f64, u.Mass] = 1e6 * u.kg
const Tg: Quantity[f64, u.Mass] = 1e9 * u.kg
const Pg: Quantity[f64, u.Mass] = 1e12 * u.kg
const Eg: Quantity[f64, u.Mass] = 1e15 * u.kg
const Zg: Quantity[f64, u.Mass] = 1e18 * u.kg
const Yg: Quantity[f64, u.Mass] = 1e21 * u.kg
const Rg: Quantity[f64, u.Mass] = 1e24 * u.kg
const Qg: Quantity[f64, u.Mass] = 1e27 * u.kg
const qm: Quantity[f64, u.Length] = 1e-30 * u.m
const rm: Quantity[f64, u.Length] = 1e-27 * u.m
const ym: Quantity[f64, u.Length] = 1e-24 * u.m
const zm: Quantity[f64, u.Length] = 1e-21 * u.m
const am: Quantity[f64, u.Length] = 1e-18 * u.m
const fm: Quantity[f64, u.Length] = 1e-15 * u.m
const pm: Quantity[f64, u.Length] = 1e-12 * u.m
const nm: Quantity[f64, u.Length] = 1e-9 * u.m
const um: Quantity[f64, u.Length] = 1e-6 * u.m
const mm: Quantity[f64, u.Length] = 1e-3 * u.m
const cm: Quantity[f64, u.Length] = 1e-2 * u.m
const dm: Quantity[f64, u.Length] = 1e-1 * u.m
const dam: Quantity[f64, u.Length] = 1e1 * u.m
const hm: Quantity[f64, u.Length] = 1e2 * u.m
const km: Quantity[f64, u.Length] = 1e3 * u.m
const Mm: Quantity[f64, u.Length] = 1e6 * u.m
const Gm: Quantity[f64, u.Length] = 1e9 * u.m
const Tm: Quantity[f64, u.Length] = 1e12 * u.m
const Pm: Quantity[f64, u.Length] = 1e15 * u.m
const Em: Quantity[f64, u.Length] = 1e18 * u.m
const Zm: Quantity[f64, u.Length] = 1e21 * u.m
const Ym: Quantity[f64, u.Length] = 1e24 * u.m
const Rm: Quantity[f64, u.Length] = 1e27 * u.m
const Qm: Quantity[f64, u.Length] = 1e30 * u.m
const qs: Quantity[f64, u.Time] = 1e-30 * u.s
const rs: Quantity[f64, u.Time] = 1e-27 * u.s
const ys: Quantity[f64, u.Time] = 1e-24 * u.s
const zs: Quantity[f64, u.Time] = 1e-21 * u.s
const attosecond: Quantity[f64, u.Time] = 1e-18 * u.s
const fs: Quantity[f64, u.Time] = 1e-15 * u.s
const ps: Quantity[f64, u.Time] = 1e-12 * u.s
const ns: Quantity[f64, u.Time] = 1e-9 * u.s
const us: Quantity[f64, u.Time] = 1e-6 * u.s
const ms: Quantity[f64, u.Time] = 1e-3 * u.s
const cs: Quantity[f64, u.Time] = 1e-2 * u.s
const ds: Quantity[f64, u.Time] = 1e-1 * u.s
const das: Quantity[f64, u.Time] = 1e1 * u.s
const hs: Quantity[f64, u.Time] = 1e2 * u.s
const ks: Quantity[f64, u.Time] = 1e3 * u.s
const Ms: Quantity[f64, u.Time] = 1e6 * u.s
const Gs: Quantity[f64, u.Time] = 1e9 * u.s
const Ts: Quantity[f64, u.Time] = 1e12 * u.s
const Ps: Quantity[f64, u.Time] = 1e15 * u.s
const Es: Quantity[f64, u.Time] = 1e18 * u.s
const Zs: Quantity[f64, u.Time] = 1e21 * u.s
const Ys: Quantity[f64, u.Time] = 1e24 * u.s
const Rs: Quantity[f64, u.Time] = 1e27 * u.s
const Qs: Quantity[f64, u.Time] = 1e30 * u.s
const qA: Quantity[f64, u.Current] = 1e-30 * u.A
const rA: Quantity[f64, u.Current] = 1e-27 * u.A
const yA: Quantity[f64, u.Current] = 1e-24 * u.A
const zA: Quantity[f64, u.Current] = 1e-21 * u.A
const aA: Quantity[f64, u.Current] = 1e-18 * u.A
const fA: Quantity[f64, u.Current] = 1e-15 * u.A
const pA: Quantity[f64, u.Current] = 1e-12 * u.A
const nA: Quantity[f64, u.Current] = 1e-9 * u.A
const uA: Quantity[f64, u.Current] = 1e-6 * u.A
const mA: Quantity[f64, u.Current] = 1e-3 * u.A
const cA: Quantity[f64, u.Current] = 1e-2 * u.A
const dA: Quantity[f64, u.Current] = 1e-1 * u.A
const daA: Quantity[f64, u.Current] = 1e1 * u.A
const hA: Quantity[f64, u.Current] = 1e2 * u.A
const kA: Quantity[f64, u.Current] = 1e3 * u.A
const MA: Quantity[f64, u.Current] = 1e6 * u.A
const GA: Quantity[f64, u.Current] = 1e9 * u.A
const TA: Quantity[f64, u.Current] = 1e12 * u.A
const PA: Quantity[f64, u.Current] = 1e15 * u.A
const EA: Quantity[f64, u.Current] = 1e18 * u.A
const ZA: Quantity[f64, u.Current] = 1e21 * u.A
const YA: Quantity[f64, u.Current] = 1e24 * u.A
const RA: Quantity[f64, u.Current] = 1e27 * u.A
const QA: Quantity[f64, u.Current] = 1e30 * u.A
const qK: Quantity[f64, u.Temperature] = 1e-30 * u.K
const rK: Quantity[f64, u.Temperature] = 1e-27 * u.K
const yK: Quantity[f64, u.Temperature] = 1e-24 * u.K
const zK: Quantity[f64, u.Temperature] = 1e-21 * u.K
const aK: Quantity[f64, u.Temperature] = 1e-18 * u.K
const fK: Quantity[f64, u.Temperature] = 1e-15 * u.K
const pK: Quantity[f64, u.Temperature] = 1e-12 * u.K
const nK: Quantity[f64, u.Temperature] = 1e-9 * u.K
const uK: Quantity[f64, u.Temperature] = 1e-6 * u.K
const mK: Quantity[f64, u.Temperature] = 1e-3 * u.K
const cK: Quantity[f64, u.Temperature] = 1e-2 * u.K
const dK: Quantity[f64, u.Temperature] = 1e-1 * u.K
const daK: Quantity[f64, u.Temperature] = 1e1 * u.K
const hK: Quantity[f64, u.Temperature] = 1e2 * u.K
const kK: Quantity[f64, u.Temperature] = 1e3 * u.K
const MK: Quantity[f64, u.Temperature] = 1e6 * u.K
const GK: Quantity[f64, u.Temperature] = 1e9 * u.K
const TK: Quantity[f64, u.Temperature] = 1e12 * u.K
const PK: Quantity[f64, u.Temperature] = 1e15 * u.K
const EK: Quantity[f64, u.Temperature] = 1e18 * u.K
const ZK: Quantity[f64, u.Temperature] = 1e21 * u.K
const YK: Quantity[f64, u.Temperature] = 1e24 * u.K
const RK: Quantity[f64, u.Temperature] = 1e27 * u.K
const QK: Quantity[f64, u.Temperature] = 1e30 * u.K
const qmol: Quantity[f64, u.Amount] = 1e-30 * u.mol
const rmol: Quantity[f64, u.Amount] = 1e-27 * u.mol
const ymol: Quantity[f64, u.Amount] = 1e-24 * u.mol
const zmol: Quantity[f64, u.Amount] = 1e-21 * u.mol
const amol: Quantity[f64, u.Amount] = 1e-18 * u.mol
const fmol: Quantity[f64, u.Amount] = 1e-15 * u.mol
const pmol: Quantity[f64, u.Amount] = 1e-12 * u.mol
const nmol: Quantity[f64, u.Amount] = 1e-9 * u.mol
const umol: Quantity[f64, u.Amount] = 1e-6 * u.mol
const mmol: Quantity[f64, u.Amount] = 1e-3 * u.mol
const cmol: Quantity[f64, u.Amount] = 1e-2 * u.mol
const dmol: Quantity[f64, u.Amount] = 1e-1 * u.mol
const damol: Quantity[f64, u.Amount] = 1e1 * u.mol
const hmol: Quantity[f64, u.Amount] = 1e2 * u.mol
const kmol: Quantity[f64, u.Amount] = 1e3 * u.mol
const Mmol: Quantity[f64, u.Amount] = 1e6 * u.mol
const Gmol: Quantity[f64, u.Amount] = 1e9 * u.mol
const Tmol: Quantity[f64, u.Amount] = 1e12 * u.mol
const Pmol: Quantity[f64, u.Amount] = 1e15 * u.mol
const Emol: Quantity[f64, u.Amount] = 1e18 * u.mol
const Zmol: Quantity[f64, u.Amount] = 1e21 * u.mol
const Ymol: Quantity[f64, u.Amount] = 1e24 * u.mol
const Rmol: Quantity[f64, u.Amount] = 1e27 * u.mol
const Qmol: Quantity[f64, u.Amount] = 1e30 * u.mol
const qcd: Quantity[f64, u.Luminosity] = 1e-30 * u.cd
const rcd: Quantity[f64, u.Luminosity] = 1e-27 * u.cd
const ycd: Quantity[f64, u.Luminosity] = 1e-24 * u.cd
const zcd: Quantity[f64, u.Luminosity] = 1e-21 * u.cd
const acd: Quantity[f64, u.Luminosity] = 1e-18 * u.cd
const fcd: Quantity[f64, u.Luminosity] = 1e-15 * u.cd
const pcd: Quantity[f64, u.Luminosity] = 1e-12 * u.cd
const ncd: Quantity[f64, u.Luminosity] = 1e-9 * u.cd
const ucd: Quantity[f64, u.Luminosity] = 1e-6 * u.cd
const mcd: Quantity[f64, u.Luminosity] = 1e-3 * u.cd
const ccd: Quantity[f64, u.Luminosity] = 1e-2 * u.cd
const dcd: Quantity[f64, u.Luminosity] = 1e-1 * u.cd
const dacd: Quantity[f64, u.Luminosity] = 1e1 * u.cd
const hcd: Quantity[f64, u.Luminosity] = 1e2 * u.cd
const kcd: Quantity[f64, u.Luminosity] = 1e3 * u.cd
const Mcd: Quantity[f64, u.Luminosity] = 1e6 * u.cd
const Gcd: Quantity[f64, u.Luminosity] = 1e9 * u.cd
const Tcd: Quantity[f64, u.Luminosity] = 1e12 * u.cd
const Pcd: Quantity[f64, u.Luminosity] = 1e15 * u.cd
const Ecd: Quantity[f64, u.Luminosity] = 1e18 * u.cd
const Zcd: Quantity[f64, u.Luminosity] = 1e21 * u.cd
const Ycd: Quantity[f64, u.Luminosity] = 1e24 * u.cd
const Rcd: Quantity[f64, u.Luminosity] = 1e27 * u.cd
const Qcd: Quantity[f64, u.Luminosity] = 1e30 * u.cd
const qrad: Quantity[f64, One] = 1e-30 * u.rad
const rrad: Quantity[f64, One] = 1e-27 * u.rad
const yrad: Quantity[f64, One] = 1e-24 * u.rad
const zrad: Quantity[f64, One] = 1e-21 * u.rad
const arad: Quantity[f64, One] = 1e-18 * u.rad
const frad: Quantity[f64, One] = 1e-15 * u.rad
const prad: Quantity[f64, One] = 1e-12 * u.rad
const nrad: Quantity[f64, One] = 1e-9 * u.rad
const urad: Quantity[f64, One] = 1e-6 * u.rad
const mrad: Quantity[f64, One] = 1e-3 * u.rad
const crad: Quantity[f64, One] = 1e-2 * u.rad
const drad: Quantity[f64, One] = 1e-1 * u.rad
const darad: Quantity[f64, One] = 1e1 * u.rad
const hrad: Quantity[f64, One] = 1e2 * u.rad
const krad: Quantity[f64, One] = 1e3 * u.rad
const Mrad: Quantity[f64, One] = 1e6 * u.rad
const Grad: Quantity[f64, One] = 1e9 * u.rad
const Trad: Quantity[f64, One] = 1e12 * u.rad
const Prad: Quantity[f64, One] = 1e15 * u.rad
const Erad: Quantity[f64, One] = 1e18 * u.rad
const Zrad: Quantity[f64, One] = 1e21 * u.rad
const Yrad: Quantity[f64, One] = 1e24 * u.rad
const Rrad: Quantity[f64, One] = 1e27 * u.rad
const Qrad: Quantity[f64, One] = 1e30 * u.rad
const qHz: Quantity[f64, u.Frequency] = 1e-30 * u.Hz
const rHz: Quantity[f64, u.Frequency] = 1e-27 * u.Hz
const yHz: Quantity[f64, u.Frequency] = 1e-24 * u.Hz
const zHz: Quantity[f64, u.Frequency] = 1e-21 * u.Hz
const aHz: Quantity[f64, u.Frequency] = 1e-18 * u.Hz
const fHz: Quantity[f64, u.Frequency] = 1e-15 * u.Hz
const pHz: Quantity[f64, u.Frequency] = 1e-12 * u.Hz
const nHz: Quantity[f64, u.Frequency] = 1e-9 * u.Hz
const uHz: Quantity[f64, u.Frequency] = 1e-6 * u.Hz
const mHz: Quantity[f64, u.Frequency] = 1e-3 * u.Hz
const cHz: Quantity[f64, u.Frequency] = 1e-2 * u.Hz
const dHz: Quantity[f64, u.Frequency] = 1e-1 * u.Hz
const daHz: Quantity[f64, u.Frequency] = 1e1 * u.Hz
const hHz: Quantity[f64, u.Frequency] = 1e2 * u.Hz
const kHz: Quantity[f64, u.Frequency] = 1e3 * u.Hz
const MHz: Quantity[f64, u.Frequency] = 1e6 * u.Hz
const GHz: Quantity[f64, u.Frequency] = 1e9 * u.Hz
const THz: Quantity[f64, u.Frequency] = 1e12 * u.Hz
const PHz: Quantity[f64, u.Frequency] = 1e15 * u.Hz
const EHz: Quantity[f64, u.Frequency] = 1e18 * u.Hz
const ZHz: Quantity[f64, u.Frequency] = 1e21 * u.Hz
const YHz: Quantity[f64, u.Frequency] = 1e24 * u.Hz
const RHz: Quantity[f64, u.Frequency] = 1e27 * u.Hz
const QHz: Quantity[f64, u.Frequency] = 1e30 * u.Hz
const qN: Quantity[f64, u.Force] = 1e-30 * u.N
const rN: Quantity[f64, u.Force] = 1e-27 * u.N
const yN: Quantity[f64, u.Force] = 1e-24 * u.N
const zN: Quantity[f64, u.Force] = 1e-21 * u.N
const aN: Quantity[f64, u.Force] = 1e-18 * u.N
const fN: Quantity[f64, u.Force] = 1e-15 * u.N
const pN: Quantity[f64, u.Force] = 1e-12 * u.N
const nN: Quantity[f64, u.Force] = 1e-9 * u.N
const uN: Quantity[f64, u.Force] = 1e-6 * u.N
const mN: Quantity[f64, u.Force] = 1e-3 * u.N
const cN: Quantity[f64, u.Force] = 1e-2 * u.N
const dN: Quantity[f64, u.Force] = 1e-1 * u.N
const daN: Quantity[f64, u.Force] = 1e1 * u.N
const hN: Quantity[f64, u.Force] = 1e2 * u.N
const kN: Quantity[f64, u.Force] = 1e3 * u.N
const MN: Quantity[f64, u.Force] = 1e6 * u.N
const GN: Quantity[f64, u.Force] = 1e9 * u.N
const TN: Quantity[f64, u.Force] = 1e12 * u.N
const PN: Quantity[f64, u.Force] = 1e15 * u.N
const EN: Quantity[f64, u.Force] = 1e18 * u.N
const ZN: Quantity[f64, u.Force] = 1e21 * u.N
const YN: Quantity[f64, u.Force] = 1e24 * u.N
const RN: Quantity[f64, u.Force] = 1e27 * u.N
const QN: Quantity[f64, u.Force] = 1e30 * u.N
const qPa: Quantity[f64, u.Pressure] = 1e-30 * u.Pa
const rPa: Quantity[f64, u.Pressure] = 1e-27 * u.Pa
const yPa: Quantity[f64, u.Pressure] = 1e-24 * u.Pa
const zPa: Quantity[f64, u.Pressure] = 1e-21 * u.Pa
const aPa: Quantity[f64, u.Pressure] = 1e-18 * u.Pa
const fPa: Quantity[f64, u.Pressure] = 1e-15 * u.Pa
const pPa: Quantity[f64, u.Pressure] = 1e-12 * u.Pa
const nPa: Quantity[f64, u.Pressure] = 1e-9 * u.Pa
const uPa: Quantity[f64, u.Pressure] = 1e-6 * u.Pa
const mPa: Quantity[f64, u.Pressure] = 1e-3 * u.Pa
const cPa: Quantity[f64, u.Pressure] = 1e-2 * u.Pa
const dPa: Quantity[f64, u.Pressure] = 1e-1 * u.Pa
const daPa: Quantity[f64, u.Pressure] = 1e1 * u.Pa
const hPa: Quantity[f64, u.Pressure] = 1e2 * u.Pa
const kPa: Quantity[f64, u.Pressure] = 1e3 * u.Pa
const MPa: Quantity[f64, u.Pressure] = 1e6 * u.Pa
const GPa: Quantity[f64, u.Pressure] = 1e9 * u.Pa
const TPa: Quantity[f64, u.Pressure] = 1e12 * u.Pa
const PPa: Quantity[f64, u.Pressure] = 1e15 * u.Pa
const EPa: Quantity[f64, u.Pressure] = 1e18 * u.Pa
const ZPa: Quantity[f64, u.Pressure] = 1e21 * u.Pa
const YPa: Quantity[f64, u.Pressure] = 1e24 * u.Pa
const RPa: Quantity[f64, u.Pressure] = 1e27 * u.Pa
const QPa: Quantity[f64, u.Pressure] = 1e30 * u.Pa
const qJ: Quantity[f64, u.Energy] = 1e-30 * u.J
const rJ: Quantity[f64, u.Energy] = 1e-27 * u.J
const yJ: Quantity[f64, u.Energy] = 1e-24 * u.J
const zJ: Quantity[f64, u.Energy] = 1e-21 * u.J
const aJ: Quantity[f64, u.Energy] = 1e-18 * u.J
const fJ: Quantity[f64, u.Energy] = 1e-15 * u.J
const pJ: Quantity[f64, u.Energy] = 1e-12 * u.J
const nJ: Quantity[f64, u.Energy] = 1e-9 * u.J
const uJ: Quantity[f64, u.Energy] = 1e-6 * u.J
const mJ: Quantity[f64, u.Energy] = 1e-3 * u.J
const cJ: Quantity[f64, u.Energy] = 1e-2 * u.J
const dJ: Quantity[f64, u.Energy] = 1e-1 * u.J
const daJ: Quantity[f64, u.Energy] = 1e1 * u.J
const hJ: Quantity[f64, u.Energy] = 1e2 * u.J
const kJ: Quantity[f64, u.Energy] = 1e3 * u.J
const MJ: Quantity[f64, u.Energy] = 1e6 * u.J
const GJ: Quantity[f64, u.Energy] = 1e9 * u.J
const TJ: Quantity[f64, u.Energy] = 1e12 * u.J
const PJ: Quantity[f64, u.Energy] = 1e15 * u.J
const EJ: Quantity[f64, u.Energy] = 1e18 * u.J
const ZJ: Quantity[f64, u.Energy] = 1e21 * u.J
const YJ: Quantity[f64, u.Energy] = 1e24 * u.J
const RJ: Quantity[f64, u.Energy] = 1e27 * u.J
const QJ: Quantity[f64, u.Energy] = 1e30 * u.J
const qW: Quantity[f64, u.Power] = 1e-30 * u.W
const rW: Quantity[f64, u.Power] = 1e-27 * u.W
const yW: Quantity[f64, u.Power] = 1e-24 * u.W
const zW: Quantity[f64, u.Power] = 1e-21 * u.W
const aW: Quantity[f64, u.Power] = 1e-18 * u.W
const fW: Quantity[f64, u.Power] = 1e-15 * u.W
const pW: Quantity[f64, u.Power] = 1e-12 * u.W
const nW: Quantity[f64, u.Power] = 1e-9 * u.W
const uW: Quantity[f64, u.Power] = 1e-6 * u.W
const mW: Quantity[f64, u.Power] = 1e-3 * u.W
const cW: Quantity[f64, u.Power] = 1e-2 * u.W
const dW: Quantity[f64, u.Power] = 1e-1 * u.W
const daW: Quantity[f64, u.Power] = 1e1 * u.W
const hW: Quantity[f64, u.Power] = 1e2 * u.W
const kW: Quantity[f64, u.Power] = 1e3 * u.W
const MW: Quantity[f64, u.Power] = 1e6 * u.W
const GW: Quantity[f64, u.Power] = 1e9 * u.W
const TW: Quantity[f64, u.Power] = 1e12 * u.W
const PW: Quantity[f64, u.Power] = 1e15 * u.W
const EW: Quantity[f64, u.Power] = 1e18 * u.W
const ZW: Quantity[f64, u.Power] = 1e21 * u.W
const YW: Quantity[f64, u.Power] = 1e24 * u.W
const RW: Quantity[f64, u.Power] = 1e27 * u.W
const QW: Quantity[f64, u.Power] = 1e30 * u.W
const qC: Quantity[f64, u.Charge] = 1e-30 * u.C
const rC: Quantity[f64, u.Charge] = 1e-27 * u.C
const yC: Quantity[f64, u.Charge] = 1e-24 * u.C
const zC: Quantity[f64, u.Charge] = 1e-21 * u.C
const aC: Quantity[f64, u.Charge] = 1e-18 * u.C
const fC: Quantity[f64, u.Charge] = 1e-15 * u.C
const pC: Quantity[f64, u.Charge] = 1e-12 * u.C
const nC: Quantity[f64, u.Charge] = 1e-9 * u.C
const uC: Quantity[f64, u.Charge] = 1e-6 * u.C
const mC: Quantity[f64, u.Charge] = 1e-3 * u.C
const cC: Quantity[f64, u.Charge] = 1e-2 * u.C
const dC: Quantity[f64, u.Charge] = 1e-1 * u.C
const daC: Quantity[f64, u.Charge] = 1e1 * u.C
const hC: Quantity[f64, u.Charge] = 1e2 * u.C
const kC: Quantity[f64, u.Charge] = 1e3 * u.C
const MC: Quantity[f64, u.Charge] = 1e6 * u.C
const GC: Quantity[f64, u.Charge] = 1e9 * u.C
const TC: Quantity[f64, u.Charge] = 1e12 * u.C
const PC: Quantity[f64, u.Charge] = 1e15 * u.C
const EC: Quantity[f64, u.Charge] = 1e18 * u.C
const ZC: Quantity[f64, u.Charge] = 1e21 * u.C
const YC: Quantity[f64, u.Charge] = 1e24 * u.C
const RC: Quantity[f64, u.Charge] = 1e27 * u.C
const QC: Quantity[f64, u.Charge] = 1e30 * u.C
const qV: Quantity[f64, u.Voltage] = 1e-30 * u.V
const rV: Quantity[f64, u.Voltage] = 1e-27 * u.V
const yV: Quantity[f64, u.Voltage] = 1e-24 * u.V
const zV: Quantity[f64, u.Voltage] = 1e-21 * u.V
const aV: Quantity[f64, u.Voltage] = 1e-18 * u.V
const fV: Quantity[f64, u.Voltage] = 1e-15 * u.V
const pV: Quantity[f64, u.Voltage] = 1e-12 * u.V
const nV: Quantity[f64, u.Voltage] = 1e-9 * u.V
const uV: Quantity[f64, u.Voltage] = 1e-6 * u.V
const mV: Quantity[f64, u.Voltage] = 1e-3 * u.V
const cV: Quantity[f64, u.Voltage] = 1e-2 * u.V
const dV: Quantity[f64, u.Voltage] = 1e-1 * u.V
const daV: Quantity[f64, u.Voltage] = 1e1 * u.V
const hV: Quantity[f64, u.Voltage] = 1e2 * u.V
const kV: Quantity[f64, u.Voltage] = 1e3 * u.V
const MV: Quantity[f64, u.Voltage] = 1e6 * u.V
const GV: Quantity[f64, u.Voltage] = 1e9 * u.V
const TV: Quantity[f64, u.Voltage] = 1e12 * u.V
const PV: Quantity[f64, u.Voltage] = 1e15 * u.V
const EV: Quantity[f64, u.Voltage] = 1e18 * u.V
const ZV: Quantity[f64, u.Voltage] = 1e21 * u.V
const YV: Quantity[f64, u.Voltage] = 1e24 * u.V
const RV: Quantity[f64, u.Voltage] = 1e27 * u.V
const QV: Quantity[f64, u.Voltage] = 1e30 * u.V
const qF: Quantity[f64, u.Capacitance] = 1e-30 * u.F
const rF: Quantity[f64, u.Capacitance] = 1e-27 * u.F
const yF: Quantity[f64, u.Capacitance] = 1e-24 * u.F
const zF: Quantity[f64, u.Capacitance] = 1e-21 * u.F
const aF: Quantity[f64, u.Capacitance] = 1e-18 * u.F
const fF: Quantity[f64, u.Capacitance] = 1e-15 * u.F
const pF: Quantity[f64, u.Capacitance] = 1e-12 * u.F
const nF: Quantity[f64, u.Capacitance] = 1e-9 * u.F
const uF: Quantity[f64, u.Capacitance] = 1e-6 * u.F
const mF: Quantity[f64, u.Capacitance] = 1e-3 * u.F
const cF: Quantity[f64, u.Capacitance] = 1e-2 * u.F
const dF: Quantity[f64, u.Capacitance] = 1e-1 * u.F
const daF: Quantity[f64, u.Capacitance] = 1e1 * u.F
const hF: Quantity[f64, u.Capacitance] = 1e2 * u.F
const kF: Quantity[f64, u.Capacitance] = 1e3 * u.F
const MF: Quantity[f64, u.Capacitance] = 1e6 * u.F
const GF: Quantity[f64, u.Capacitance] = 1e9 * u.F
const TF: Quantity[f64, u.Capacitance] = 1e12 * u.F
const PF: Quantity[f64, u.Capacitance] = 1e15 * u.F
const EF: Quantity[f64, u.Capacitance] = 1e18 * u.F
const ZF: Quantity[f64, u.Capacitance] = 1e21 * u.F
const YF: Quantity[f64, u.Capacitance] = 1e24 * u.F
const RF: Quantity[f64, u.Capacitance] = 1e27 * u.F
const QF: Quantity[f64, u.Capacitance] = 1e30 * u.F
const qohm: Quantity[f64, u.Resistance] = 1e-30 * u.ohm
const rohm: Quantity[f64, u.Resistance] = 1e-27 * u.ohm
const yohm: Quantity[f64, u.Resistance] = 1e-24 * u.ohm
const zohm: Quantity[f64, u.Resistance] = 1e-21 * u.ohm
const aohm: Quantity[f64, u.Resistance] = 1e-18 * u.ohm
const fohm: Quantity[f64, u.Resistance] = 1e-15 * u.ohm
const pohm: Quantity[f64, u.Resistance] = 1e-12 * u.ohm
const nohm: Quantity[f64, u.Resistance] = 1e-9 * u.ohm
const uohm: Quantity[f64, u.Resistance] = 1e-6 * u.ohm
const milliohm: Quantity[f64, u.Resistance] = 1e-3 * u.ohm
const cohm: Quantity[f64, u.Resistance] = 1e-2 * u.ohm
const dohm: Quantity[f64, u.Resistance] = 1e-1 * u.ohm
const daohm: Quantity[f64, u.Resistance] = 1e1 * u.ohm
const hohm: Quantity[f64, u.Resistance] = 1e2 * u.ohm
const kohm: Quantity[f64, u.Resistance] = 1e3 * u.ohm
const Mohm: Quantity[f64, u.Resistance] = 1e6 * u.ohm
const Gohm: Quantity[f64, u.Resistance] = 1e9 * u.ohm
const Tohm: Quantity[f64, u.Resistance] = 1e12 * u.ohm
const Pohm: Quantity[f64, u.Resistance] = 1e15 * u.ohm
const Eohm: Quantity[f64, u.Resistance] = 1e18 * u.ohm
const Zohm: Quantity[f64, u.Resistance] = 1e21 * u.ohm
const Yohm: Quantity[f64, u.Resistance] = 1e24 * u.ohm
const Rohm: Quantity[f64, u.Resistance] = 1e27 * u.ohm
const Qohm: Quantity[f64, u.Resistance] = 1e30 * u.ohm
const qS: Quantity[f64, u.Conductance] = 1e-30 * u.S
const rS: Quantity[f64, u.Conductance] = 1e-27 * u.S
const yS: Quantity[f64, u.Conductance] = 1e-24 * u.S
const zS: Quantity[f64, u.Conductance] = 1e-21 * u.S
const aS: Quantity[f64, u.Conductance] = 1e-18 * u.S
const fS: Quantity[f64, u.Conductance] = 1e-15 * u.S
const pS: Quantity[f64, u.Conductance] = 1e-12 * u.S
const nS: Quantity[f64, u.Conductance] = 1e-9 * u.S
const uS: Quantity[f64, u.Conductance] = 1e-6 * u.S
const mS: Quantity[f64, u.Conductance] = 1e-3 * u.S
const cS: Quantity[f64, u.Conductance] = 1e-2 * u.S
const dS: Quantity[f64, u.Conductance] = 1e-1 * u.S
const daS: Quantity[f64, u.Conductance] = 1e1 * u.S
const hS: Quantity[f64, u.Conductance] = 1e2 * u.S
const kS: Quantity[f64, u.Conductance] = 1e3 * u.S
const MS: Quantity[f64, u.Conductance] = 1e6 * u.S
const GS: Quantity[f64, u.Conductance] = 1e9 * u.S
const TS: Quantity[f64, u.Conductance] = 1e12 * u.S
const PS: Quantity[f64, u.Conductance] = 1e15 * u.S
const ES: Quantity[f64, u.Conductance] = 1e18 * u.S
const ZS: Quantity[f64, u.Conductance] = 1e21 * u.S
const YS: Quantity[f64, u.Conductance] = 1e24 * u.S
const RS: Quantity[f64, u.Conductance] = 1e27 * u.S
const QS: Quantity[f64, u.Conductance] = 1e30 * u.S
const qWb: Quantity[f64, u.MagneticFlux] = 1e-30 * u.Wb
const rWb: Quantity[f64, u.MagneticFlux] = 1e-27 * u.Wb
const yWb: Quantity[f64, u.MagneticFlux] = 1e-24 * u.Wb
const zWb: Quantity[f64, u.MagneticFlux] = 1e-21 * u.Wb
const aWb: Quantity[f64, u.MagneticFlux] = 1e-18 * u.Wb
const fWb: Quantity[f64, u.MagneticFlux] = 1e-15 * u.Wb
const pWb: Quantity[f64, u.MagneticFlux] = 1e-12 * u.Wb
const nWb: Quantity[f64, u.MagneticFlux] = 1e-9 * u.Wb
const uWb: Quantity[f64, u.MagneticFlux] = 1e-6 * u.Wb
const mWb: Quantity[f64, u.MagneticFlux] = 1e-3 * u.Wb
const cWb: Quantity[f64, u.MagneticFlux] = 1e-2 * u.Wb
const dWb: Quantity[f64, u.MagneticFlux] = 1e-1 * u.Wb
const daWb: Quantity[f64, u.MagneticFlux] = 1e1 * u.Wb
const hWb: Quantity[f64, u.MagneticFlux] = 1e2 * u.Wb
const kWb: Quantity[f64, u.MagneticFlux] = 1e3 * u.Wb
const MWb: Quantity[f64, u.MagneticFlux] = 1e6 * u.Wb
const GWb: Quantity[f64, u.MagneticFlux] = 1e9 * u.Wb
const TWb: Quantity[f64, u.MagneticFlux] = 1e12 * u.Wb
const PWb: Quantity[f64, u.MagneticFlux] = 1e15 * u.Wb
const EWb: Quantity[f64, u.MagneticFlux] = 1e18 * u.Wb
const ZWb: Quantity[f64, u.MagneticFlux] = 1e21 * u.Wb
const YWb: Quantity[f64, u.MagneticFlux] = 1e24 * u.Wb
const RWb: Quantity[f64, u.MagneticFlux] = 1e27 * u.Wb
const QWb: Quantity[f64, u.MagneticFlux] = 1e30 * u.Wb
const qT: Quantity[f64, u.MagneticFluxDensity] = 1e-30 * u.T
const rT: Quantity[f64, u.MagneticFluxDensity] = 1e-27 * u.T
const yT: Quantity[f64, u.MagneticFluxDensity] = 1e-24 * u.T
const zT: Quantity[f64, u.MagneticFluxDensity] = 1e-21 * u.T
const aT: Quantity[f64, u.MagneticFluxDensity] = 1e-18 * u.T
const fT: Quantity[f64, u.MagneticFluxDensity] = 1e-15 * u.T
const pT: Quantity[f64, u.MagneticFluxDensity] = 1e-12 * u.T
const nT: Quantity[f64, u.MagneticFluxDensity] = 1e-9 * u.T
const uT: Quantity[f64, u.MagneticFluxDensity] = 1e-6 * u.T
const mT: Quantity[f64, u.MagneticFluxDensity] = 1e-3 * u.T
const cT: Quantity[f64, u.MagneticFluxDensity] = 1e-2 * u.T
const dT: Quantity[f64, u.MagneticFluxDensity] = 1e-1 * u.T
const daT: Quantity[f64, u.MagneticFluxDensity] = 1e1 * u.T
const hT: Quantity[f64, u.MagneticFluxDensity] = 1e2 * u.T
const kT: Quantity[f64, u.MagneticFluxDensity] = 1e3 * u.T
const MT: Quantity[f64, u.MagneticFluxDensity] = 1e6 * u.T
const GT: Quantity[f64, u.MagneticFluxDensity] = 1e9 * u.T
const TT: Quantity[f64, u.MagneticFluxDensity] = 1e12 * u.T
const PT: Quantity[f64, u.MagneticFluxDensity] = 1e15 * u.T
const ET: Quantity[f64, u.MagneticFluxDensity] = 1e18 * u.T
const ZT: Quantity[f64, u.MagneticFluxDensity] = 1e21 * u.T
const YT: Quantity[f64, u.MagneticFluxDensity] = 1e24 * u.T
const RT: Quantity[f64, u.MagneticFluxDensity] = 1e27 * u.T
const QT: Quantity[f64, u.MagneticFluxDensity] = 1e30 * u.T
const qH: Quantity[f64, u.Inductance] = 1e-30 * u.H
const rH: Quantity[f64, u.Inductance] = 1e-27 * u.H
const yH: Quantity[f64, u.Inductance] = 1e-24 * u.H
const zH: Quantity[f64, u.Inductance] = 1e-21 * u.H
const aH: Quantity[f64, u.Inductance] = 1e-18 * u.H
const fH: Quantity[f64, u.Inductance] = 1e-15 * u.H
const pH: Quantity[f64, u.Inductance] = 1e-12 * u.H
const nH: Quantity[f64, u.Inductance] = 1e-9 * u.H
const uH: Quantity[f64, u.Inductance] = 1e-6 * u.H
const mH: Quantity[f64, u.Inductance] = 1e-3 * u.H
const cH: Quantity[f64, u.Inductance] = 1e-2 * u.H
const dH: Quantity[f64, u.Inductance] = 1e-1 * u.H
const daH: Quantity[f64, u.Inductance] = 1e1 * u.H
const hH: Quantity[f64, u.Inductance] = 1e2 * u.H
const kH: Quantity[f64, u.Inductance] = 1e3 * u.H
const MH: Quantity[f64, u.Inductance] = 1e6 * u.H
const GH: Quantity[f64, u.Inductance] = 1e9 * u.H
const TH: Quantity[f64, u.Inductance] = 1e12 * u.H
const PH: Quantity[f64, u.Inductance] = 1e15 * u.H
const EH: Quantity[f64, u.Inductance] = 1e18 * u.H
const ZH: Quantity[f64, u.Inductance] = 1e21 * u.H
const YH: Quantity[f64, u.Inductance] = 1e24 * u.H
const RH: Quantity[f64, u.Inductance] = 1e27 * u.H
const QH: Quantity[f64, u.Inductance] = 1e30 * u.H
const qlm: Quantity[f64, u.Luminosity] = 1e-30 * u.lm
const rlm: Quantity[f64, u.Luminosity] = 1e-27 * u.lm
const ylm: Quantity[f64, u.Luminosity] = 1e-24 * u.lm
const zlm: Quantity[f64, u.Luminosity] = 1e-21 * u.lm
const alm: Quantity[f64, u.Luminosity] = 1e-18 * u.lm
const flm: Quantity[f64, u.Luminosity] = 1e-15 * u.lm
const plm: Quantity[f64, u.Luminosity] = 1e-12 * u.lm
const nlm: Quantity[f64, u.Luminosity] = 1e-9 * u.lm
const ulm: Quantity[f64, u.Luminosity] = 1e-6 * u.lm
const mlm: Quantity[f64, u.Luminosity] = 1e-3 * u.lm
const clm: Quantity[f64, u.Luminosity] = 1e-2 * u.lm
const dlm: Quantity[f64, u.Luminosity] = 1e-1 * u.lm
const dalm: Quantity[f64, u.Luminosity] = 1e1 * u.lm
const hlm: Quantity[f64, u.Luminosity] = 1e2 * u.lm
const klm: Quantity[f64, u.Luminosity] = 1e3 * u.lm
const Mlm: Quantity[f64, u.Luminosity] = 1e6 * u.lm
const Glm: Quantity[f64, u.Luminosity] = 1e9 * u.lm
const Tlm: Quantity[f64, u.Luminosity] = 1e12 * u.lm
const Plm: Quantity[f64, u.Luminosity] = 1e15 * u.lm
const Elm: Quantity[f64, u.Luminosity] = 1e18 * u.lm
const Zlm: Quantity[f64, u.Luminosity] = 1e21 * u.lm
const Ylm: Quantity[f64, u.Luminosity] = 1e24 * u.lm
const Rlm: Quantity[f64, u.Luminosity] = 1e27 * u.lm
const Qlm: Quantity[f64, u.Luminosity] = 1e30 * u.lm
const qlx: Quantity[f64, u.Illuminance] = 1e-30 * u.lx
const rlx: Quantity[f64, u.Illuminance] = 1e-27 * u.lx
const ylx: Quantity[f64, u.Illuminance] = 1e-24 * u.lx
const zlx: Quantity[f64, u.Illuminance] = 1e-21 * u.lx
const alx: Quantity[f64, u.Illuminance] = 1e-18 * u.lx
const flx: Quantity[f64, u.Illuminance] = 1e-15 * u.lx
const plx: Quantity[f64, u.Illuminance] = 1e-12 * u.lx
const nlx: Quantity[f64, u.Illuminance] = 1e-9 * u.lx
const ulx: Quantity[f64, u.Illuminance] = 1e-6 * u.lx
const mlx: Quantity[f64, u.Illuminance] = 1e-3 * u.lx
const clx: Quantity[f64, u.Illuminance] = 1e-2 * u.lx
const dlx: Quantity[f64, u.Illuminance] = 1e-1 * u.lx
const dalx: Quantity[f64, u.Illuminance] = 1e1 * u.lx
const hlx: Quantity[f64, u.Illuminance] = 1e2 * u.lx
const klx: Quantity[f64, u.Illuminance] = 1e3 * u.lx
const Mlx: Quantity[f64, u.Illuminance] = 1e6 * u.lx
const Glx: Quantity[f64, u.Illuminance] = 1e9 * u.lx
const Tlx: Quantity[f64, u.Illuminance] = 1e12 * u.lx
const Plx: Quantity[f64, u.Illuminance] = 1e15 * u.lx
const Elx: Quantity[f64, u.Illuminance] = 1e18 * u.lx
const Zlx: Quantity[f64, u.Illuminance] = 1e21 * u.lx
const Ylx: Quantity[f64, u.Illuminance] = 1e24 * u.lx
const Rlx: Quantity[f64, u.Illuminance] = 1e27 * u.lx
const Qlx: Quantity[f64, u.Illuminance] = 1e30 * u.lx
const qBq: Quantity[f64, u.Frequency] = 1e-30 * u.Bq
const rBq: Quantity[f64, u.Frequency] = 1e-27 * u.Bq
const yBq: Quantity[f64, u.Frequency] = 1e-24 * u.Bq
const zBq: Quantity[f64, u.Frequency] = 1e-21 * u.Bq
const aBq: Quantity[f64, u.Frequency] = 1e-18 * u.Bq
const fBq: Quantity[f64, u.Frequency] = 1e-15 * u.Bq
const pBq: Quantity[f64, u.Frequency] = 1e-12 * u.Bq
const nBq: Quantity[f64, u.Frequency] = 1e-9 * u.Bq
const uBq: Quantity[f64, u.Frequency] = 1e-6 * u.Bq
const mBq: Quantity[f64, u.Frequency] = 1e-3 * u.Bq
const cBq: Quantity[f64, u.Frequency] = 1e-2 * u.Bq
const dBq: Quantity[f64, u.Frequency] = 1e-1 * u.Bq
const daBq: Quantity[f64, u.Frequency] = 1e1 * u.Bq
const hBq: Quantity[f64, u.Frequency] = 1e2 * u.Bq
const kBq: Quantity[f64, u.Frequency] = 1e3 * u.Bq
const MBq: Quantity[f64, u.Frequency] = 1e6 * u.Bq
const GBq: Quantity[f64, u.Frequency] = 1e9 * u.Bq
const TBq: Quantity[f64, u.Frequency] = 1e12 * u.Bq
const PBq: Quantity[f64, u.Frequency] = 1e15 * u.Bq
const EBq: Quantity[f64, u.Frequency] = 1e18 * u.Bq
const ZBq: Quantity[f64, u.Frequency] = 1e21 * u.Bq
const YBq: Quantity[f64, u.Frequency] = 1e24 * u.Bq
const RBq: Quantity[f64, u.Frequency] = 1e27 * u.Bq
const QBq: Quantity[f64, u.Frequency] = 1e30 * u.Bq
const qGy: Quantity[f64, u.SpecificEnergy] = 1e-30 * u.Gy
const rGy: Quantity[f64, u.SpecificEnergy] = 1e-27 * u.Gy
const yGy: Quantity[f64, u.SpecificEnergy] = 1e-24 * u.Gy
const zGy: Quantity[f64, u.SpecificEnergy] = 1e-21 * u.Gy
const aGy: Quantity[f64, u.SpecificEnergy] = 1e-18 * u.Gy
const fGy: Quantity[f64, u.SpecificEnergy] = 1e-15 * u.Gy
const pGy: Quantity[f64, u.SpecificEnergy] = 1e-12 * u.Gy
const nGy: Quantity[f64, u.SpecificEnergy] = 1e-9 * u.Gy
const uGy: Quantity[f64, u.SpecificEnergy] = 1e-6 * u.Gy
const mGy: Quantity[f64, u.SpecificEnergy] = 1e-3 * u.Gy
const cGy: Quantity[f64, u.SpecificEnergy] = 1e-2 * u.Gy
const dGy: Quantity[f64, u.SpecificEnergy] = 1e-1 * u.Gy
const daGy: Quantity[f64, u.SpecificEnergy] = 1e1 * u.Gy
const hGy: Quantity[f64, u.SpecificEnergy] = 1e2 * u.Gy
const kGy: Quantity[f64, u.SpecificEnergy] = 1e3 * u.Gy
const MGy: Quantity[f64, u.SpecificEnergy] = 1e6 * u.Gy
const GGy: Quantity[f64, u.SpecificEnergy] = 1e9 * u.Gy
const TGy: Quantity[f64, u.SpecificEnergy] = 1e12 * u.Gy
const PGy: Quantity[f64, u.SpecificEnergy] = 1e15 * u.Gy
const EGy: Quantity[f64, u.SpecificEnergy] = 1e18 * u.Gy
const ZGy: Quantity[f64, u.SpecificEnergy] = 1e21 * u.Gy
const YGy: Quantity[f64, u.SpecificEnergy] = 1e24 * u.Gy
const RGy: Quantity[f64, u.SpecificEnergy] = 1e27 * u.Gy
const QGy: Quantity[f64, u.SpecificEnergy] = 1e30 * u.Gy
const qSv: Quantity[f64, u.SpecificEnergy] = 1e-30 * u.Sv
const rSv: Quantity[f64, u.SpecificEnergy] = 1e-27 * u.Sv
const ySv: Quantity[f64, u.SpecificEnergy] = 1e-24 * u.Sv
const zSv: Quantity[f64, u.SpecificEnergy] = 1e-21 * u.Sv
const aSv: Quantity[f64, u.SpecificEnergy] = 1e-18 * u.Sv
const fSv: Quantity[f64, u.SpecificEnergy] = 1e-15 * u.Sv
const pSv: Quantity[f64, u.SpecificEnergy] = 1e-12 * u.Sv
const nSv: Quantity[f64, u.SpecificEnergy] = 1e-9 * u.Sv
const uSv: Quantity[f64, u.SpecificEnergy] = 1e-6 * u.Sv
const mSv: Quantity[f64, u.SpecificEnergy] = 1e-3 * u.Sv
const cSv: Quantity[f64, u.SpecificEnergy] = 1e-2 * u.Sv
const dSv: Quantity[f64, u.SpecificEnergy] = 1e-1 * u.Sv
const daSv: Quantity[f64, u.SpecificEnergy] = 1e1 * u.Sv
const hSv: Quantity[f64, u.SpecificEnergy] = 1e2 * u.Sv
const kSv: Quantity[f64, u.SpecificEnergy] = 1e3 * u.Sv
const MSv: Quantity[f64, u.SpecificEnergy] = 1e6 * u.Sv
const GSv: Quantity[f64, u.SpecificEnergy] = 1e9 * u.Sv
const TSv: Quantity[f64, u.SpecificEnergy] = 1e12 * u.Sv
const PSv: Quantity[f64, u.SpecificEnergy] = 1e15 * u.Sv
const ESv: Quantity[f64, u.SpecificEnergy] = 1e18 * u.Sv
const ZSv: Quantity[f64, u.SpecificEnergy] = 1e21 * u.Sv
const YSv: Quantity[f64, u.SpecificEnergy] = 1e24 * u.Sv
const RSv: Quantity[f64, u.SpecificEnergy] = 1e27 * u.Sv
const QSv: Quantity[f64, u.SpecificEnergy] = 1e30 * u.Sv
const qkat: Quantity[f64, u.CatalyticActivity] = 1e-30 * u.kat
const rkat: Quantity[f64, u.CatalyticActivity] = 1e-27 * u.kat
const ykat: Quantity[f64, u.CatalyticActivity] = 1e-24 * u.kat
const zkat: Quantity[f64, u.CatalyticActivity] = 1e-21 * u.kat
const akat: Quantity[f64, u.CatalyticActivity] = 1e-18 * u.kat
const fkat: Quantity[f64, u.CatalyticActivity] = 1e-15 * u.kat
const pkat: Quantity[f64, u.CatalyticActivity] = 1e-12 * u.kat
const nkat: Quantity[f64, u.CatalyticActivity] = 1e-9 * u.kat
const ukat: Quantity[f64, u.CatalyticActivity] = 1e-6 * u.kat
const mkat: Quantity[f64, u.CatalyticActivity] = 1e-3 * u.kat
const ckat: Quantity[f64, u.CatalyticActivity] = 1e-2 * u.kat
const dkat: Quantity[f64, u.CatalyticActivity] = 1e-1 * u.kat
const dakat: Quantity[f64, u.CatalyticActivity] = 1e1 * u.kat
const hkat: Quantity[f64, u.CatalyticActivity] = 1e2 * u.kat
const kkat: Quantity[f64, u.CatalyticActivity] = 1e3 * u.kat
const Mkat: Quantity[f64, u.CatalyticActivity] = 1e6 * u.kat
const Gkat: Quantity[f64, u.CatalyticActivity] = 1e9 * u.kat
const Tkat: Quantity[f64, u.CatalyticActivity] = 1e12 * u.kat
const Pkat: Quantity[f64, u.CatalyticActivity] = 1e15 * u.kat
const Ekat: Quantity[f64, u.CatalyticActivity] = 1e18 * u.kat
const Zkat: Quantity[f64, u.CatalyticActivity] = 1e21 * u.kat
const Ykat: Quantity[f64, u.CatalyticActivity] = 1e24 * u.kat
const Rkat: Quantity[f64, u.CatalyticActivity] = 1e27 * u.kat
const Qkat: Quantity[f64, u.CatalyticActivity] = 1e30 * u.kat
const microlitre: Quantity[f64, u.Volume] = 1e-9 * (u.m * u.m * u.m)
const centilitre: Quantity[f64, u.Volume] = 1e-5 * (u.m * u.m * u.m)
const decilitre: Quantity[f64, u.Volume] = 1e-4 * (u.m * u.m * u.m)
const hectolitre: Quantity[f64, u.Volume] = 1e-1 * (u.m * u.m * u.m)
const kilolitre: Quantity[f64, u.Volume] = u.m * u.m * u.m
const megalitre: Quantity[f64, u.Volume] = 1e3 * (u.m * u.m * u.m)
const meV: Quantity[f64, u.Energy] = 1e-3 * u.eV
const TeV: Quantity[f64, u.Energy] = 1e12 * u.eV
const TWh: Quantity[f64, u.Energy] = 1e12 * u.Wh
const fermi: Quantity[f64, u.Length] = 1e-15 * u.m
const micron: Quantity[f64, u.Length] = 1e-6 * u.m
const mil: Quantity[f64, u.Length] = 1e-3 * u.inch
const microinch: Quantity[f64, u.Length] = 1e-6 * u.inch
const point: Quantity[f64, u.Length] = u.inch / 72.0
const pica: Quantity[f64, u.Length] = u.inch / 6.0
const printers_point: Quantity[f64, u.Length] = 0.013837 * u.inch
const printers_pica: Quantity[f64, u.Length] = 12.0 * printers_point
const survey_foot: Quantity[f64, u.Length] = (1200.0 / 3937.0) * u.m
const survey_mile: Quantity[f64, u.Length] = 5280.0 * survey_foot
const chain: Quantity[f64, u.Length] = 66.0 * survey_foot
const rod: Quantity[f64, u.Length] = 16.5 * survey_foot
const fathom: Quantity[f64, u.Length] = 6.0 * survey_foot
const are: Quantity[f64, u.Area] = 1e2 * (u.m * u.m)
const barn: Quantity[f64, u.Area] = 1e-28 * (u.m * u.m)
const circular_mil: Quantity[f64, u.Area] = (f64.PI / 4.0) * (mil * mil)
const survey_acre: Quantity[f64, u.Area] = 43560.0 * (survey_foot * survey_foot)
const darcy: Quantity[f64, u.Area] = u.cm * (1e-3 * u.Pa * u.s) * (u.cm * u.cm * u.cm / u.s) / (u.cm * u.cm * u.atm)
const stere: Quantity[f64, u.Volume] = u.m * u.m * u.m
const acre_foot: Quantity[f64, u.Volume] = survey_acre * survey_foot
const barrel: Quantity[f64, u.Volume] = 42.0 * u.gallon
const quart: Quantity[f64, u.Volume] = u.gallon / 4.0
const pint: Quantity[f64, u.Volume] = u.gallon / 8.0
const cup: Quantity[f64, u.Volume] = u.gallon / 16.0
const gill: Quantity[f64, u.Volume] = u.gallon / 32.0
const fluid_ounce: Quantity[f64, u.Volume] = u.gallon / 128.0
const tablespoon: Quantity[f64, u.Volume] = fluid_ounce / 2.0
const teaspoon: Quantity[f64, u.Volume] = fluid_ounce / 6.0
const bushel: Quantity[f64, u.Volume] = 2150.42 * (u.inch * u.inch * u.inch)
const peck: Quantity[f64, u.Volume] = bushel / 4.0
const dry_quart: Quantity[f64, u.Volume] = bushel / 32.0
const dry_pint: Quantity[f64, u.Volume] = bushel / 64.0
const imperial_gallon: Quantity[f64, u.Volume] = 4.54609e-3 * (u.m * u.m * u.m)
const imperial_gill: Quantity[f64, u.Volume] = imperial_gallon / 32.0
const imperial_fluid_ounce: Quantity[f64, u.Volume] = imperial_gallon / 160.0
const cord: Quantity[f64, u.Volume] = 128.0 * (u.foot * u.foot * u.foot)
const register_ton: Quantity[f64, u.Volume] = 100.0 * (u.foot * u.foot * u.foot)
const carat: Quantity[f64, u.Mass] = 200.0 * u.mg
const grain: Quantity[f64, u.Mass] = 64.79891 * u.mg
const pennyweight: Quantity[f64, u.Mass] = 24.0 * grain
const troy_ounce: Quantity[f64, u.Mass] = 480.0 * grain
const troy_pound: Quantity[f64, u.Mass] = 5760.0 * grain
const short_hundredweight: Quantity[f64, u.Mass] = 100.0 * u.pound
const long_hundredweight: Quantity[f64, u.Mass] = 112.0 * u.pound
const short_ton: Quantity[f64, u.Mass] = 2000.0 * u.pound
const long_ton: Quantity[f64, u.Mass] = 2240.0 * u.pound
const assay_ton: Quantity[f64, u.Mass] = (short_ton / troy_ounce) * u.mg
const slug: Quantity[f64, u.Mass] = u.pound_force * u.s * u.s / u.foot
const shake: Quantity[f64, u.Time] = 1e-8 * u.s
const sidereal_day: Quantity[f64, u.Time] = 86164.0905 * u.s
const sidereal_hour: Quantity[f64, u.Time] = sidereal_day / 24.0
const sidereal_minute: Quantity[f64, u.Time] = sidereal_hour / 60.0
const sidereal_second: Quantity[f64, u.Time] = sidereal_minute / 60.0
const sidereal_year: Quantity[f64, u.Time] = 365.256363004 * u.day
const tropical_year: Quantity[f64, u.Time] = 365.24219 * u.day
const dyne: Quantity[f64, u.Force] = 1e-5 * u.N
const kilogram_force: Quantity[f64, u.Force] = u.kg * u.g_n
const ounce_force: Quantity[f64, u.Force] = u.pound_force / 16.0
const poundal: Quantity[f64, u.Force] = u.pound * u.foot / (u.s * u.s)
const kip: Quantity[f64, u.Force] = 1e3 * u.pound_force
const ton_force: Quantity[f64, u.Force] = 2000.0 * u.pound_force
const technical_atmosphere: Quantity[f64, u.Pressure] = kilogram_force / (u.cm * u.cm)
const ksi: Quantity[f64, u.Pressure] = 1e3 * u.psi
const cmHg: Quantity[f64, u.Pressure] = 10.0 * u.mmHg
const inHg: Quantity[f64, u.Pressure] = 25.4 * u.mmHg
const ftHg: Quantity[f64, u.Pressure] = 304.8 * u.mmHg
const mmH2O: Quantity[f64, u.Pressure] = water_density * u.g_n * u.mm
const cmH2O: Quantity[f64, u.Pressure] = water_density * u.g_n * u.cm
const inH2O: Quantity[f64, u.Pressure] = water_density * u.g_n * u.inch
const ftH2O: Quantity[f64, u.Pressure] = water_density * u.g_n * u.foot
const erg: Quantity[f64, u.Energy] = 1e-7 * u.J
const calorie_IT: Quantity[f64, u.Energy] = 4.1868 * u.J
const kilocalorie_IT: Quantity[f64, u.Energy] = 4186.8 * u.J
const btu_th: Quantity[f64, u.Energy] = u.calorie / (u.g * u.K) * u.pound * u.rankine
const therm_EC: Quantity[f64, u.Energy] = 1.05506e8 * u.J
const therm_US: Quantity[f64, u.Energy] = 1.054804e8 * u.J
const quad: Quantity[f64, u.Energy] = 1e15 * u.btu
const ton_of_tnt: Quantity[f64, u.Energy] = 4.184e9 * u.J
const horsepower_electric: Quantity[f64, u.Power] = 746.0 * u.W
const horsepower_metric: Quantity[f64, u.Power] = 75.0 * kilogram_force * u.m / u.s
const ton_of_refrigeration: Quantity[f64, u.Power] = 12000.0 * u.btu / u.hour
const poise: Quantity[f64, u.DynamicViscosity] = 0.1 * u.Pa * u.s
const centipoise: Quantity[f64, u.DynamicViscosity] = 1e-3 * u.Pa * u.s
const rhe: Quantity[f64, One / u.DynamicViscosity] = 1.0 / poise
const stokes: Quantity[f64, u.KinematicViscosity] = 1e-4 * (u.m * u.m) / u.s
const centistokes: Quantity[f64, u.KinematicViscosity] = 1e-6 * (u.m * u.m) / u.s
const ampere_hour: Quantity[f64, u.Charge] = u.A * u.hour
const milliampere_hour: Quantity[f64, u.Charge] = u.mA * u.hour
const faraday: Quantity[f64, u.Charge] = u.e * u.N_A * u.mol
const mho: Quantity[f64, u.Conductance] = u.S
const debye: Quantity[f64, ElectricDipoleMoment] = 1e-21 * u.C * u.m * u.m / (u.c * u.s)
const gauss: Quantity[f64, u.MagneticFluxDensity] = 1e-4 * u.T
const gamma: Quantity[f64, u.MagneticFluxDensity] = 1e-9 * u.T
const maxwell: Quantity[f64, u.MagneticFlux] = 1e-8 * u.Wb
const oersted: Quantity[f64, MagneticFieldStrength] = (1e3 / (4.0 * f64.PI)) * u.A / u.m
const curie: Quantity[f64, u.Frequency] = 3.7e10 * u.Bq
const rad_dose: Quantity[f64, u.SpecificEnergy] = 1e-2 * u.Gy
const rem: Quantity[f64, u.SpecificEnergy] = 1e-2 * u.Sv
const roentgen: Quantity[f64, Exposure] = 2.58e-4 * u.C / u.kg
const footcandle: Quantity[f64, u.Illuminance] = u.lm / (u.foot * u.foot)
const phot: Quantity[f64, u.Illuminance] = u.lm / (u.cm * u.cm)
const stilb: Quantity[f64, Luminance] = u.cd / (u.cm * u.cm)
const lambert: Quantity[f64, Luminance] = u.cd / (f64.PI * u.cm * u.cm)
const footlambert: Quantity[f64, Luminance] = u.cd / (f64.PI * u.foot * u.foot)
const revolution: Quantity[f64, One] = 2.0 * f64.PI * u.rad
const gon: Quantity[f64, One] = (f64.PI / 200.0) * u.rad
const angular_mil: Quantity[f64, One] = (2.0 * f64.PI / 6400.0) * u.rad
const gal: Quantity[f64, u.Acceleration] = 1e-2 * u.m / (u.s * u.s)
const kayser: Quantity[f64, u.Wavenumber] = 1e2 / u.m
const tex: Quantity[f64, LinearDensity] = u.g / u.km
const denier: Quantity[f64, LinearDensity] = u.g / (9000.0 * u.m)

std.url

Strict RFC 3986 absolute URL parsing and formatting without normalization, component-aware percent encoding, and query pairs. Errors: UrlError. Reference.

enum UrlError
    LimitExceeded | MissingScheme | MissingHost | InvalidByte(i64) | InvalidEscape(i64) | InvalidHost(i64) | InvalidPort(i64) | InvalidUtf8 | Allocation(AllocError)
enum Component
    Userinfo | Segment | Path | Query | QueryPart | Fragment
class Url
    scheme: str
    userinfo: Option[str]
    host: Option[str]
    port: Option[i64]
    path: str
    query: Option[str]
    fragment: Option[str]
    def to_string(self) -> Result[str, UrlError]
class Pair
    key: str
    value: str
def parse(text: &str, max_bytes: NonNegative[i64]) -> Result[Url, UrlError]
def encode_bytes(source: &[u8], component: Component) -> Result[Bytes, UrlError]
def decode_bytes(source: &[u8]) -> Result[Bytes, UrlError]
def encode(text: &str, component: Component) -> Result[str, UrlError]
def decode(text: &str) -> Result[str, UrlError]
def query_pairs(query: &str) -> Result[list[Pair], UrlError]
def query_text(pairs: &list[Pair]) -> Result[str, UrlError]

std.uuid

UUID values, canonical text and network bytes, random v4 and timestamp-prefixed v7 generation. Errors: ParseError, BytesError, SecretsError, RangeError. Reference.

class Uuid
    def as_u64_pair(self) -> (u64, u64)
    def compare(self, other: &Uuid) -> i64
    def is_nil(self) -> bool
    def is_rfc_variant(self) -> bool
    def version(self) -> Option[u8]
    def unix_milliseconds(self) -> Option[u64]
    def write_bytes(self, target: &mut Bytes, offset: i64) -> ()
    def to_bytes(self) -> Result[Bytes, BytesError]
    def to_string(self) -> Result[str, BytesError]
def nil() -> Uuid
def from_u64_pair(high: u64, low: u64) -> Uuid
def parse(text: &str) -> Result[Uuid, ParseError]
def from_bytes(source: &[u8]) -> Result[Uuid, ParseError]
def v4_from[R: Reader](source: &mut Source[R]) -> Result[Uuid, SecretsError]
def v4() -> Result[Uuid, SecretsError]
def v7_from[R: Reader](at: &Timestamp, source: &mut Source[R]) -> Result[Uuid, RangeError | SecretsError]
def v7() -> Result[Uuid, RangeError | SecretsError]

std.xml

Bounded UTF-8 XML event readers and validating stream writers, without DTDs or external entities. Errors: XmlError. Reference.

enum Fault
    Syntax | UnexpectedEnd | InvalidName | InvalidCharacter | InvalidUtf8 | InvalidReference | UnknownEntity | DuplicateAttribute | MismatchedEnd | DoctypeForbidden | UnsupportedDeclaration | DepthExceeded | InputTooLarge | OutputTooLarge | TokenTooLarge | TooManyAttributes | InvalidState | Stopped | Allocation(AllocError) | Stream(StreamError)
class XmlError
    fault: Fault
    offset: i64
class Limits
    max_depth: NonNegative[i64]
    max_input: NonNegative[i64]
    max_output: NonNegative[i64]
    max_token: Positive[i64]
    max_attributes: NonNegative[i64]
def limits() -> Limits
class Attribute
    name: str
    value: str
enum Event
    Declaration(Option[bool]) | Start(str, list[Attribute]) | Empty(str, list[Attribute]) | End(str) | Text(str) | CData(str) | Comment(str) | ProcessingInstruction(str, str) | Eof
class Reader[R]
    inner: R
    def offset(self) -> i64
    def next(self: &mut Reader[R]) -> Result[Event, XmlError]
def reader[R](inner: R, bounds: &Limits) -> Result[Reader[R], XmlError]
class Writer[W]
    inner: W
    def offset(self) -> i64
    def write(self: &mut Writer[W], event: &Event) -> Result[(), XmlError]
    def finish(self: &mut Writer[W]) -> Result[(), XmlError]
def writer[W](inner: W, bounds: &Limits) -> Result[Writer[W], XmlError]

Standard library conventions

These are the rules public std APIs follow. A new module, function, or method follows them; a change that breaks one explains why on its module page. Each rule records the majority or most recent practice in the signature index. Where existing modules differ, the rule says so and names the issue or backlog item that tracks the difference; those modules are not examples to copy.

Packages outside std may use the same rules so their APIs read like the standard library.

Implementation and visibility

  • Write a module in Plenty when the functionality is reasonably small and straightforward to express. Wrap a Rust or C library only for a concrete reason, such as substantial complexity (SQLite, Polars) or specialist security requirements (cryptography). The wrapper keeps the same allocation-failure and error contracts as Plenty code.
  • Everything declared pub is public API: it appears in the signature index and is documented on the module page. Helpers stay private.
  • Native hooks are std-only compiler intrinsics, not methods or functions of a std module. A std module calls the intrinsic (__crypto_update, __compress_new) with &[u8] views for inputs and &mut Bytes output buffers whose ranges it checks itself; user code calling one is rejected as reserved for the standard library.
  • A module’s error types and protocols are imported from that module. The prelude holds builtin names, plus std.fmt.FormatError, which user formatting methods return (issue #632).
  • Each module page states ownership, allocation, error variants and limits for every public function, as described in Reading a module page, and ends with ## Acknowledgements when there is something to credit.

Naming

Constructors

FormUseExamples
Lower-case noun of the productBuild a value or adapter from partsstd.io.cursor(data), std.json.decoder(&limits), std.fs.watch.watcher()
limits(), config()Default settings valuestd.json.limits(), std.http.client.config()
Verb for an operating-system resourceAcquire a file, socket or processopen_file, connect, listen, dial
from_<source>Convert from another representationstd.bytes.from_text, std.complex.from_polar, std.io.from_file
parse / parse_textDecode a value from bytes / from strstd.json.parse, std.json.parse_text
with_capacity(capacity)Empty container with reserved storagestd.bytes.with_capacity
__init__Containers built with the class nameDeque[T](), LruCache(capacity)

A format defined over text alone takes parse(text: &str) (std.url, std.uuid, std.mime). new is used only by std.image and std.mime; do not add it.

Conversions and rendering

NameMeaning
to_<type>()New owned value; may allocate and fail
as_<type>()Cheap view or reinterpretation; never allocates
to_text()Validated conversion of byte content to str (Bytes, Path)
to_string()Canonical text of a structured value, as an owned str
encode / encode_textSerialize to Bytes / to str
describe()One-line error description as Result[str, AllocError]
display[W](self, out)Write to a std.fmt.Formatter[W]; print and f-string fields call it
debug[W](self, out)Write a diagnostic form; str.repr, !r and aggregate elements call it
format[W](self, output)Write to a std.fmt.Formatter[W]; the Formattable protocol for reports

format is reserved for writing to a formatter. Url, MediaType, SocketAddress and SetCookie return owned canonical text through to_string(). display and debug are reserved for the formatter form: formatting a type whose method of either name has another signature is a compile error. See Formatting.

A _text suffix names the str form of an operation whose main form takes bytes (parse_text, encode_text, from_text). A _bytes suffix names the byte form of an operation whose main form takes text (get_bytes, command_bytes, send_bytes). The suffix does not yet predict the type: std.path.from_bytes takes list[u8] and std.bytes.from_bytes takes bytes (see Bytes).

Predicates

A method returning bool about the value’s state or contents starts with is_ or has_: is_empty, is_terminal, is_done, is_cancelled. Deadline.expired(now) is an established exception. Flags that report what an event was (std.fs.watch created(), deleted()) are read like fields and need no prefix.

Existing deviations: closed() on std.actor.net, std.actor.tls and std.fs readers; is_finished() in std.cli.progress; is_complete() on the JSON encoder; complete() on the inflater; cancelled() in std.http.server. New code uses is_done() for “no further work or output”.

Sizes and counts

NameMeaning
len()Elements the value holds, as i64
is_empty()len() == 0
capacity()Elements storable without reallocating, as i64
count(value, ...)Occurrences of value, as in Python’s list.count

count() without an argument also reports a size in std.fs.watch (events in the batch), std.simd masks (set lanes) and std.sketch (values observed). New code uses len() for held elements and a descriptive noun for totals. len(container) calls its accessible shared len(self) -> i64 method. in does not dispatch to container methods; use explicit search operations (issue #623; remaining protocol scope is B49).

Ending work

MethodContract
flush(&mut self)Write buffered output; the object stays usable
finish(&mut self) -> Result[P, E]Complete an encoder, writer or digest and return its product (Bytes, str, or ()); later writes fail
close(&mut self) -> Result[(), E]Release the resource and report the failure that release can produce

close returns the type’s own error, except that a type implementing std.io.Closer returns StreamError (issue #533 records why TcpStream.close differs from TcpListener.close). Walk.close and the TLS Session.close return ().

An adapter whose protocol cannot carry its own error records it for later: failure(self) -> Option[E] returns a copy, and take_failure(&mut self) -> Option[E] moves out an error that owns data. Do not add a third spelling.

Readiness

A selectable type has watch(self, selector: &mut Selector, key: u64, interest: Interest) -> Result[(), E] (omitting interest when only one direction applies) and unwatch(self, selector: &mut Selector) -> Result[(), E]. std.fs.watch spells these register/unregister, HTTP server transports return bool, and the actor Context.watch returns a Watch handle.

Errors

Domain error types

A module with failures of its own defines one public error named <Module>Error (JsonError, NetError): an enum, or a class carrying an enum plus position. Variants name the fault (Truncated, LimitExceeded, InvalidUtf8). A cause from another module is kept intact in a variant named after its source: Stream(StreamError), Io(IoError), Net(NetError), Http(HttpError). std.error covers context and mapping.

Allocation failure

Module failuresError type
Allocation onlyAllocError (std.deque, std.cache.lru, std.diff)
Allocation and domain faultsThe domain error with an Allocation(AllocError) variant

Do not return a union such as AllocError | GraphError; add the variant instead. A function that only forwards another module’s error keeps that error, as std.bytes constructors keep AllocError. An operation with two unrelated causes names both in its own enum: std.actor.StageError is Allocation(AllocError) or Wait(WaitError).

Invalid arguments

A size that cannot be allocated, negative included, is AllocError.CapacityOverflow, or Allocation(CapacityOverflow) in a domain error: std.bytes.with_capacity(-1), std.secrets.token_bytes(-1).

A position the caller computes against a value it holds, such as an index or range into a buffer, or a count that no call could accept, traps when invalid, as an out-of-range list index does. The message names the operation and the argument (Bytes.get: index is outside 0..len()), and the trap report names the calling line. The operation’s result then names only what can still fail: Bytes.get returns u8, Rng.sample_indices returns Result[list[i64], AllocError]. std.bytes range methods, their read_file and read_stdin ranges, and std.random counts and ranges follow this rule. Standard-library code reports these with the std-only __trap statement; a handle from the same object that has become invalid may trap the same way (std.graph).

A limit, size or count whose fixed lower bound is 0 or 1 is a NonNegative[i64] or Positive[i64] parameter or field, so the caller checks it once with try_from and a literal is checked at compile time (integer domains). A fixed ceiling on such a value is checked at a direct std call, constructor, or field update when its input is a literal or a module constant (including constant arithmetic the compiler can evaluate). A value above it is a compile error. Dynamic values still trap when the operation uses them, naming the operation and the argument. Ceilings are declaration metadata: they do not change Positive[i64] or NonNegative[i64], so passing a value through an ordinary local, function, reference, or erased callable does not prove a ceiling. Runtime guards remain necessary. See std ceiling checking for the internal declaration mechanism and its scope. Another run-time value the caller supplies (a limit or configuration) that the operation cannot accept is an Err with a named variant: OutOfRange, InvalidLimits, InvalidConfig. It is never clamped or ignored. Buffer ranges, encoding and UUID offsets, crypto output offsets, and CSV field indices trap. Builtins follow several policies today (issue #629); see the consistency proposal.

Returning moved inputs

An operation that takes ownership of a value and can fail before using it (send, submit, enqueue, admit) returns the value in its error, generic over the value’s type: Undelivered[M] and RequestError[M] in std.actor, SendError[T] for channels, SubmitError[F] for executors, SpawnError[F] for thread starts. Exception: a scoped spawn with borrowed arguments reports the job-free SpawnError[()] and drops its moved arguments, because an error cannot carry a loan.

Parameters

Ownership

Borrow (&T) what the call only reads. Use &mut T for state or a buffer it updates. Take ownership (T) only when the result or a queue keeps the value: std.io.cursor(data: Bytes), std.xml.reader(inner: R, ...). An adapter that owns its inner stream exposes it as a public inner field so the caller can recover it.

Time and cancellation

QuantityType
Span&Duration
Monotonic point&Instant
Wall-clock point&Timestamp
Blocking waitdeadline: &Deadline, cancel: &CancellationToken, the last two parameters
Server or loop shutdownstop: &CancellationToken

A method that also has a deadline-free form adds the deadline under a _before suffix (read_before, handshake_before); a nonblocking form uses _now (read_now, accept_now). An operation that reads the clock accepts now: &Instant, or offers an _at variant that does, so tests can supply the time with the fake clocks in std.test.

Typed spans and wall-clock values use Duration and Timestamp in SQLite, key-value storage, JWT validation, cookies, SSE retry hints and HTTP dates. The typed waits in std.sync provide deadline and cancellation forms for compiler-owned channels, futures and tokens. Low-level _timeout_ms forms remain available; their old _timeout names are compatibility aliases. SSE’s optional event and id follow the required deadline, cancel pair because required parameters precede defaults.

Limits

Every input-driven allocation has a cap.

  • One or two caps: named max_<thing>: NonNegative[i64] parameters with defaults (std.url.parse(text, max_bytes), std.db.kv.connect(..., max_length)).
  • Several caps: a Limits class with public max_<thing> fields, a limits() function returning documented defaults, and a limits: &Limits parameter. A module with several such classes qualifies both names (FormLimits, form_limits()).
  • Fields with a lower bound of 0 or 1 use NonNegative[i64] or Positive[i64]; direct construction or field updates reject a statically known value above a fixed ceiling, and dynamic values trap when the operation starts. A relation between fields that no type states remains InvalidLimits. An input that exceeds a limit is LimitExceeded or a specific variant (TooLarge).

std.json, std.toml, std.markdown, std.template, std.regex and std.xml name the parameter bounds: &Limits; std.http and std.image use limits:. A parameter follows its type’s name, as config: &Config and dialect: &Dialect do. std.actor.net.Limits has no limits() function and no max_ prefixes, and InvalidLimit appears beside InvalidLimits.

Bytes

RoleType
Owned buffer, argument or resultBytes
Read-only inputsource: &[u8], accepting &buffer, &buffer[a:b], &list and b"..." literals
Caller’s output bufferbuffer: &mut Bytes, start: i64, stop: i64, a half-open range
A list of byte-sized numberslist[u8]

A &[u8] input carries its own range, so it takes no start and stop and has no out-of-range error: the caller passes &buffer[a:b], whose bounds clamp. Slice views describe the conversion.

std.path, std.env, std.process and std.cli carry operating-system strings as list[u8] (B39).

Streams

Byte streams implement the std.io protocols Reader, Writer, Seeker and Closer: read(buffer, start, stop) -> Result[i64, StreamError] reports partial progress. Read ranges outside 0 <= start <= stop <= buffer.len() trap before transport access, including empty ranges and EOF; read_exact validates the same bounds. A function borrows a stream as reader: &mut R with R: Reader; an adapter takes inner: R. std.io.typed is for callers that must recover the transport’s own error type (std.http.typed); other modules do not add a second protocol. Network streams add deadline and nonblocking forms (read_before, read_now) that return the module’s error.

Booleans and enums

A choice between modes is an enum: Shutdown, Interest, Endian, DuplicateKeys. A bool parameter is acceptable when its name says what True means (enabled, descending, html), it has a default, and callers pass it by name. Two bool parameters that combine into modes are an enum. Exception: std.fs.walk(root, top_down) requires a positional bool. An operation that offers a mode by a separate method instead, such as ThreadPoolExecutor.cancel_pending() before shutdown(), needs neither.

Integer types

QuantityType
Length, index, offset, sizei64, matching len() and indexing
Capacity, limit or count with a fixed lower boundNonNegative[i64] (>= 0) or Positive[i64] (>= 1)
Hash, key, seed, bit pattern, selector keyu64
Value fixed by a format or the OS (file size, inode, wire integer)Its own width
Running statistics counteru64

Builtin capacities and worker/chunk counts also use Positive[i64]: channel(capacity), ThreadPoolExecutor(max_workers, queue_capacity) and .par(pool, chunks=...). Allocation sizes (with_capacity, reserve) stay i64 and report a negative size as CapacityOverflow. Results such as len() stay i64. CuckooFilter.len() returns u64; low-level timed waits still take u64 milliseconds.

Formatting

std.fmt.Formatter[W] and its FormatError serve value formatting and reports. Import types, constructors, and the writer-generic Formattable protocol from std.fmt; std.report provides rendering helpers.

  • a value that f-strings display implements display[W];
  • an error that renders into a report implements format[W](self, output: &mut Formatter[W]);
  • an owned canonical string is to_string() -> Result[str, E]; reserve format for writing to a formatter (including compiler-handled f-strings).
  • print and f-strings call display; str.repr, !r and elements of formatted aggregates call debug. A type without display uses its debug, and one without debug uses its structure. When a user method can run, print returns Result[(), IoError | FormatError], and str.repr and f-strings return Result[str, AllocError | FormatError]; otherwise print reports IoError and both others AllocError. See formatting.
  • std.log.Logger.debug and std.cli.Progress.display have other signatures, so print rejects those types; call their methods instead.

Deviations

AreaDeviationTracking
Invalid argumentsBuiltin policies differIssue #629
Containersin does not dispatch to std container methodsB49; issue #623 records the length/diagnostic fix
Byteslist[u8] OS stringsB39
StreamsTcpStream.close returns StreamErrorIssue #533 (intentional)
Limitsbounds: spelling; std.actor.net.Limits shapeNot yet tracked
NamesPredicates without is_; count() as size; watch shapes; newNot yet tracked

std.actor

Supervised actors on one thread. Every actor owns a heap: its handlers allocate there, and a trap in a handler discards that heap, closes the files the actor opened, and tells the actor’s parent. The rest of the program keeps running. Messages are copied into the receiver’s heap, mailboxes are bounded and allocated when the actor starts, and a send that cannot be delivered returns the message.

Implemented targets: x86-64 and AArch64 GNU/Linux. Elsewhere supported() is False and starting an actor returns StartError.Unsupported. The ARM heap and AAPCS64 recovery path are part of ARM qualification; native hardware validation remains a release gate.

A boss starts two workers. One answers; the other has a bug and traps. The boss receives the answer and both exits:

from std.actor import Address, Context, Exit, Mail, heap_bytes_used, heaps_live, StageError, stage
from std.ready import WaitError
from std.sync import CancellationToken
from std.time import deadline_after, monotonic_now, seconds

type Fault = AllocError | IoError | OverflowError[i64]

pub enum Job:
    Square(i64)
    Answer(i64)

pub class Boss:
    pub me: Option[Address]
    pub ended: i64

pub class Worker:
    pub boss: Address

pub enum Node:
    Leading(Boss)
    Working(Worker)

    pub def started(self: &mut Node, context: &mut Context[Job, Fault]) -> Result[(), Fault]:
        match self:
            case Node.Leading(boss):
                boss.me = Some(context.address())
                good = context.spawn(1, 65536).unwrap()
                bad = context.spawn(2, 65536).unwrap()
                context.send(good, Job.Square(7)).unwrap()
                context.send(bad, Job.Square(-1)).unwrap()
            case Node.Working(_):
                pass
        Ok(())

    pub def receive(self: &mut Node, mail: Mail[Job, Fault], context: &mut Context[Job, Fault]) -> Result[(), Fault]:
        match self:
            case Node.Leading(boss):
                match mail:
                    case Mail.Message(Job.Answer(value)):
                        print("answer", value)?
                    case Mail.Down(_, _, Exit.Stopped):
                        print("worker stopped")?
                        boss.ended += 1
                    case Mail.Down(_, _, Exit.Trapped(kind)):
                        print("worker trapped", kind)?
                        boss.ended += 1
                    case _:
                        pass
                if boss.ended == 2:
                    context.stop()
            case Node.Working(worker):
                match mail:
                    case Mail.Message(Job.Square(value)):
                        if value < 0:
                            # A bug: unwrapping Nothing traps.
                            missing: Option[i64] = Nothing
                            missing.unwrap()
                        context.send(worker.boss, Job.Answer(value * value)).unwrap()
                        context.stop()
                    case _:
                        pass
        Ok(())

    # Runs with the new child's heap current; see "The spawn hook".
    pub def spawn(self, tag: i64, child: Address, context: &mut Context[Job, Fault]) -> Result[Option[Node], Fault]:
        match self:
            case Node.Leading(boss):
                Ok(Some(Node.Working(Worker(boss.me.unwrap()))))
            case Node.Working(_):
                Ok(Nothing)

def main() -> Result[(), AllocError | IoError | StageError | WaitError]:
    mut actors = stage[Node, Job, Fault](8, 4, 4)?
    root = Node.Leading(Boss(Nothing, 0))
    actors.start(root, 65536).unwrap()
    actors.run(&deadline_after(&monotonic_now(), &seconds(5)).unwrap(), &CancellationToken()?)?
    stats = actors.stats()
    print(stats.started, stats.stopped, stats.trapped, heaps_live(), heap_bytes_used())?
    Ok(())
answer 49
worker stopped
worker trapped std.actor.TrapKind.Unwrap
3 2 1 0 0

The trap also prints its usual message to standard error, followed by note: contained in an actor; its heap is discarded.

For TCP servers and clients, std.actor.net provides a listener with one actor per connection, bounded connection buffers, deadlines, and nonblocking connect and lookup. The repository’s examples/actor_echo.plenty is an echo server built on it; a connection whose first byte is X traps, and only that connection closes.

Actors and stages

Stage.node(address) borrows a live node shared between calls to run. Use it to read counters or copy an owned snapshot outside the actor’s heap. It returns Nothing after the actor stops or traps, including when another actor has reused its slot. End the borrow before running the stage, sending mail, stopping an actor or dropping the stage; the ordinary borrowing rules prevent those operations while the node is borrowed. The accessor does not switch heaps or run actor code.

run checks its deadline between scheduling rounds even when mail remains queued. It does not preempt a running handler: a blocking call or an infinite loop still prevents the stage from making progress.

Stage.shutdown() immediately stops every live actor, releases queued mail, timers and tracked handles, and discards their heaps. It does not deliver Stop or Down callbacks or restart actors, and does not need mailbox space or timer capacity. Node destructors run under containment. It is idempotent; the empty stage can start new actors afterward, with its statistics retained and old addresses invalidated. Dropping a stage calls shutdown automatically, including on early returns after cancellation or errors. Application-level draining belongs before this final cleanup. Like other actor code, a destructor that blocks or loops is not preempted.

A Stage[N, M, E] runs actors of one node type N, message type M and error type E. N is a public class or enum with three public methods:

MethodCalled
started(self: &mut N, context: &mut Context[M, E]) -> Result[(), E]Once, before any mail
receive(self: &mut N, mail: Mail[M, E], context: &mut Context[M, E]) -> Result[(), E]For each delivered mail
spawn(self, tag: i64, child: Address, context: &mut Context[M, E]) -> Result[Option[N], E]On the parent, after a handler that called Context.spawn returns; builds the child with the child’s heap current. Nothing stops the child at once. See the spawn hook

M and E must be copyable with copy; delivery copies them into another heap. Use an enum node type to run several kinds of actor on one stage, as in the example.

The spawn hook

The spawn hook runs the parent’s code while the child’s heap is current, so everything it allocates belongs to the child. Two rules keep the parent’s heap and the child’s apart:

  • The hook takes self as a shared borrow. It cannot store a value in the parent’s node, so nothing it allocates outlives the child’s heap there, and it cannot move a value out of the parent’s node into the child.
  • The node type holds only values that a shared borrow neither aliases nor changes: numbers, bool, bytes, Bytes, files and sockets, functions, std.tls configurations and sessions, and lists, dicts, sets, boxes, classes and enums of these. A str copy shares its storage and Mutex.lock gives &mut through a shared borrow, so the compiler rejects a node holding str, Shared, a lock, an atomic, a channel, a closure or a task, naming the type and the field’s holder:
error: an actor node cannot hold `str` (in `Node`): its `spawn` hook reads the node while the child's heap is current; keep text in `Bytes` and pass shared state in messages

The hook builds the child’s node from copies of what it reads, and creates values the child owns, such as an accepted socket, inside the hook. A hook declared self: &mut N is rejected with cannot borrow shared reference as mutable in the standard library’s run_spawn.

In the hook, Context.spawn and Context.spawn_isolated return StartError.Capacity, and Context.stop has no effect. With ordinary spawn, an error or trap in the hook ends the parent: the hook is the parent’s code. An error is copied to the parent’s heap before the child’s heap is discarded.

Use Context.spawn_isolated(tag, heap_limit) when construction itself belongs to the child’s failure boundary, such as admitting a network connection. A hook error or trap ends only the attempted child and delivers Down(child, tag, Exit.Failed(error)) or Exit.Trapped(kind) to its parent. Returning Nothing delivers Exit.Stopped. The notification does not require room in the parent’s mailbox. A failed setup releases the child’s heap, tracked handles and timers before another attempt can reuse its slot. Stats.started includes these admitted attempts, even if they never reach their started method.

Child exits follow the mail already queued for the parent when the child stopped, including the child’s final messages. Newer mail cannot postpone Down indefinitely: an acceptor can recover retired connection slots even while new readiness keeps arriving.

Isolation also applies when the stage rebuilds an isolated child under a restart policy. A replacement hook that fails or traps ends the child and notifies its parent immediately; it is not retried recursively. A replacement hook returning Nothing reports Stopped. Ordinary child handler failures continue to follow the configured restart policy.

ExportContract
stage[N, M, E](capacity, mailbox, events)Result[Stage[N, M, E], StageError]: at most capacity live actors, mailbox messages per actor, events ready handles per wait. Allocates every slot table now
Stage.start(node, limit: Positive[i64])Result[Address, StartError]: a root actor with a heap of limit bytes. Allocates its mailbox now
Stage.send(to, message)Result[(), Undelivered[M]]
Stage.stop_actor(to)bool: queues Mail.Stop; False when the actor has stopped or its mailbox is full
Stage.run(&deadline, &cancel)Result[(), WaitError]: delivers mail and readiness until no actor is live; fails when the deadline passes or cancel is cancelled first
Stage.live()Live actors
Stage.stats()Stats
Stage.heap_used(actor)Option[i64]: bytes in the actor’s heap, Nothing once it has stopped
Stage.add_timers(count)Result[(), AllocError]: allocates count more records for timers and pending requests; a stage starts with none
Context.address()The running actor’s Address
Context.send(to, message)Result[(), Undelivered[M]]
Context.stop()Stops the running actor after this handler returns
Context.stop_actor(to)As Stage.stop_actor
Context.spawn(tag, limit: Positive[i64])Result[Address, StartError]: reserves a child slot, heap and mailbox now; the parent’s spawn hook builds the child after the handler returns
Context.restart(child, limit, &window)bool: restarts the child when it fails or traps, at most limit times per window; see Restart
Context.after(&delay, tag), every(&period, tag)Result[Timer, TimerError]: delivers Mail.Timer once after delay, or every period
Context.cancel(timer)bool: no Mail.Timer for it follows; False when it has already ended
Context.request(to, message, &timeout)Result[Request, RequestError[M]]: queues Mail.Request; exactly one Mail.Reply follows
Context.reply(request, message)Result[(), Undelivered[M]]: answers a request this actor received
Context.heap_used()Bytes in the running actor’s heap
Context.watch(&handle, interest)Result[Watch, WaitError]: delivers the handle’s readiness as Mail.Ready(watch, readiness); see Watching handles
Context.unwatch(&handle)Result[(), WaitError]: stops delivering it
supported()Whether this target has actor heaps
heaps_live(), heap_bytes_used(), heap_bytes_committed()Process totals over all actor heaps
max_heap_limit64 MiB, the largest heap limit; Stage.start, Context.spawn and Context.spawn_isolated trap above it
max_spawns16 spawn requests per handler
max_capacity1,048,576, the largest stage capacity; stage returns StageError.Allocation(CapacityOverflow) for a negative capacity or one above it
TypeValues
AddressA slot and a generation. A stopped actor’s address is never reused; key() returns it as u64
Mail[M, E]Message(M), Ready(Watch, Readiness), Down(Address, i64, Exit[E]) (the child, its spawn tag, its exit), Stop, Timer(Timer), Request(Request, M), Reply(Request, Reply[M])
Timertag(), and key() as u64
Watchkey() as u64, to tell watched handles apart; copyable
Requestkey() as u64, to match a reply to its request
Reply[M]Answer(M), TimedOut, Gone (the receiver stopped or restarted first)
TimerErrorCapacity (every timer record is in use), Range (a zero period, or a time out of range)
RequestError[M]Full(M), Gone(M) as for Undelivered; Capacity(M): no free timer record
Exit[E]Stopped, Failed(E), Trapped(TrapKind)
TrapKindMemory, Index, Arithmetic, Unwrap, Other; see Traps
Undelivered[M]Full(M): the mailbox is full or the receiver’s heap cannot hold the copy. Gone(M): the receiver has stopped
StageErrorAllocation(AllocError) (slot tables, or a capacity that is negative or over max_capacity: CapacityOverflow), Wait(WaitError) (the readiness selector)
StartErrorUnsupported, Capacity (no free slot, or max_spawns reached), Heaps (1024 heaps live or the reservation failed), Mailbox (its allocation failed)
Statsstarted, stopped, failed, trapped, restarted, delivered, full (refused sends, and readiness deferred by a full mailbox), gone (sends and readiness for stopped actors), lost (Down for a parent that had stopped)

Delivery

Each round delivers one mail to every actor that was runnable when the round began, then waits on the stage’s std.ready selector without blocking if any actor is still runnable. Readiness is level-triggered: an actor has at most one Mail.Ready queued per handle, and readiness that finds a full mailbox is reported again on a later wait. Due timers and ended requests wait outside the mailbox and alternate with mailbox deliveries, so they never need mailbox room and neither kind delays the other indefinitely.

Stop stops the actor after its handler returns. An Err(error) from any handler stops the actor and gives its parent Exit.Failed(error), copied into the parent’s heap; if that copy does not fit, the parent sees Exit.Trapped(TrapKind.Memory). Destructors run for a stopped or failed actor’s node.

Supervision

Every actor started by Context.spawn has a parent; the parent receives Mail.Down(child, tag, exit) after the child’s last message, so a reply always arrives before the Down that follows it. tag is the one the child was spawned with, so a parent can tell kinds of children apart without recording their addresses. Down never needs mailbox room and is never refused: the child’s slot holds the exit until the parent receives it, so a stopped child occupies one unit of capacity until then. When the parent has already stopped, the exit is dropped and counted in Stats.lost. A stopping actor’s undelivered mail is dropped, and so are its children’s pending exits.

A parent can also call Context.spawn again on Down, or ask the stage to restart the child.

Watching handles

Context.watch(&handle, interest) delivers a handle’s readiness to the running actor as Mail.Ready(watch, readiness). It takes any std.ready.Watchable handle: std.net’s TcpStream, TcpListener, UdpSocket, Connecting and Lookup. The returned Watch is the one the mail carries, so an actor watching several handles compares watch.key() to find which is ready. Watching a watched handle again changes its interest and returns the same Watch. Context.unwatch(&handle) stops delivery, and so does closing the handle.

The key is built from the handle’s descriptor number and the actor’s slot, so watching allocates nothing. A closed handle’s number can be reused by a later handle, which then has the same key; readiness already queued for the closed one can reach the new one, and its nonblocking calls return WouldBlock.

Arithmetic limits

Observation-only Stats totals saturate at i64.MAX; saturation never changes mail delivery, admission, errors or cleanup. Slot and timer/request generations never wrap: after an identity with generation u32.MAX ends, its slot or record is permanently retired. A pending Down still retains its payload and is delivered (or dropped during parent cleanup) before the exhausted slot is forgotten. Later admission uses the existing capacity errors when no reusable slot or record remains. Timer-record table sizes beyond their 32-bit key index range return AllocError.CapacityOverflow before appending.

Mailbox ordering counters are not lifetime totals. Before their representation would overflow, the stage subtracts the delivered prefix from both counters and all of that mailbox’s pending Down fences, clamping only already-passed fences to zero. Ready exits stay ready and future exits retain their exact remaining prefix distance. No callback or allocation runs during rebasing. Clearing an actor drains its queued messages and child exits before counters are reset on slot reuse; restarting also clears those queues.

Timers, requests and restart

A lead actor asks a counting service for its count, crashes it, and asks again. The request queued behind the crash is answered Gone, since the restart drops the old actor’s mail; the retry reaches the restarted service, whose count starts again:

from std.actor import Address, Context, Mail, Reply, StageError, stage
from std.ready import WaitError
from std.sync import CancellationToken
from std.time import deadline_after, milliseconds, monotonic_now, seconds

type Fault = AllocError | IoError | OverflowError[i64]

pub enum Job:
    Get
    Count(i64)
    Crash

pub class Lead:
    pub service: Option[Address]
    pub answers: i64

pub enum Node:
    Leading(Lead)
    Counting(i64)

    pub def started(self: &mut Node, context: &mut Context[Job, Fault]) -> Result[(), Fault]:
        match self:
            case Node.Leading(lead):
                service = context.spawn(0, 65536).unwrap()
                # Up to 3 restarts a minute.
                context.restart(service, 3, &seconds(60))
                context.request(service, Job.Get, &seconds(1)).unwrap()
                lead.service = Some(service)
            case Node.Counting(_):
                pass
        Ok(())

    pub def receive(self: &mut Node, mail: Mail[Job, Fault], context: &mut Context[Job, Fault]) -> Result[(), Fault]:
        match self:
            case Node.Leading(lead):
                service = lead.service.unwrap()
                match mail:
                    case Mail.Reply(_, Reply.Answer(Job.Count(count))):
                        print("count", count)?
                        lead.answers += 1
                        if lead.answers == 1:
                            context.send(service, Job.Crash).unwrap()
                            context.request(service, Job.Get, &seconds(1)).unwrap()
                        else:
                            context.after(&milliseconds(10), 0).unwrap()
                    case Mail.Reply(_, Reply.Gone):
                        print("gone")?
                        context.request(service, Job.Get, &seconds(1)).unwrap()
                    case Mail.Timer(_):
                        context.stop_actor(service)
                    case Mail.Down(_, _, exit):
                        print("service", exit)?
                        context.stop()
                    case _:
                        pass
            case Node.Counting(count):
                match mail:
                    case Mail.Request(request, Job.Get):
                        *count += 1
                        context.reply(request, Job.Count(*count)).unwrap()
                    case Mail.Message(Job.Crash):
                        missing: Option[i64] = Nothing
                        missing.unwrap()
                    case _:
                        pass
        Ok(())

    pub def spawn(self, tag: i64, child: Address, context: &mut Context[Job, Fault]) -> Result[Option[Node], Fault]:
        Ok(Some(Node.Counting(0)))

def main() -> Result[(), AllocError | IoError | StageError | WaitError]:
    mut actors = stage[Node, Job, Fault](2, 4, 1)?
    # Records for timers and pending requests.
    actors.add_timers(4)?
    actors.start(Node.Leading(Lead(Nothing, 0)), 65536).unwrap()
    actors.run(&deadline_after(&monotonic_now(), &seconds(5)).unwrap(), &CancellationToken()?)?
    stats = actors.stats()
    print(stats.trapped, stats.restarted)?
    Ok(())
count 1
gone
count 1
service std.actor.Exit[AllocError | IoError | OverflowError[i64]].Stopped
1 1

Timers

Context.after(&delay, tag) delivers Mail.Timer(timer) once, and Context.every(&period, tag) every period starting one period from now; timer.tag() is the caller’s tag. A tick still undelivered when the next falls due absorbs it, and ticks missed while the stage was busy are skipped rather than delivered late in a burst. Context.cancel(timer) ends a timer, and no Mail.Timer for it is delivered afterwards. An actor’s timers end when it stops.

Timers need records, which Stage.add_timers(count) allocates outside every actor heap; a stage starts with none, and after, every and request return Capacity when every record is in use. Starting, firing and cancelling a timer allocate nothing. Stage.run waits on the selector until the earlier of its deadline and the next due timer; the timer’s wake-up is not a timeout.

An unrepresentable now + delay returns TimerError.Range before reserving or scheduling a record. Repeating timers skip missed ticks and clamp their next deadline to the last representable instant. Mailbox/runnable ring indices and timer-heap indices use bounded arithmetic, including at representation limits, rather than relying on a potentially overflowing index sum.

Request and reply

Context.request(to, message, &timeout) copies message into the receiver’s heap like send, queues it as Mail.Request(request, message), and returns the Request. The receiver answers with Context.reply(request, answer), which copies the answer into the requester’s heap and queues it as Mail.Reply(request, Reply.Answer(answer)) in the requester’s mailbox, after the messages the receiver sent before it. Exactly one Mail.Reply reaches the requester for each request:

  • Answer(M) when the receiver replies in time;
  • TimedOut when timeout passes first;
  • Gone when the receiver stops or restarts first.

A late or second reply, a reply to a request sent to another actor, and a reply after the requester stopped return Undelivered.Gone. A pending request holds a timer record until its reply. Handlers do not block, so the requester handles other mail while it waits. Compare request.key() to match replies when several requests are pending.

A request timeout beyond the representable clock range is clamped to the last instant, rather than returning an arithmetic error or consuming its message on an overflow path.

Restart

Context.restart(child, limit, &window) makes the stage restart a child of the running actor when it fails or traps. Restart is opt-in per child; limit 0 turns it off. On a restart:

  1. The child’s queued mail, timers and child exits are dropped, requests sent to it are answered Gone, and its heap is discarded.
  2. A heap of the child’s original limit is reserved, and the parent’s spawn hook builds the child again with its original tag. The address stays, so other actors can keep sending to it.
  3. The new child runs started before any mail. Stats.restarted counts it.

The parent receives no mail for a restart. At most limit restarts happen per window, counted from the first restart in the window. After that, or when the hook returns Nothing or no heap can be reserved, the child stops and the parent receives Down with its exit. When the hook fails or traps, the parent ends as for any hook error, and the child’s exit is counted in Stats.lost.

Restart suits service actors that the hook can rebuild from the tag. An actor built around a resource the hook took once, such as an accepted connection, cannot be rebuilt: its socket was closed with its heap. Children that a restarted actor had started keep running without a parent.

Memory accounting

Each heap reserves a 64 MiB address range and commits pages as its size classes grow. A heap’s limit counts the bytes of its live blocks, each rounded up to its size class; an allocation over the limit fails like any other allocation, as AllocError where the program handles it. A handler that returns that error fails its actor:

from std.actor import Address, Context, Mail, heap_bytes_used, heaps_live, StageError, stage
from std.ready import WaitError
from std.sync import CancellationToken
from std.time import deadline_after, monotonic_now, seconds

type Fault = AllocError | IoError | OverflowError[i64]

pub class Greedy:
    pub size: i64

    pub def started(self: &mut Greedy, context: &mut Context[i64, Fault]) -> Result[(), Fault]:
        # 8 bytes for each of a million values do not fit in 64 KiB.
        values: list[i64] = [0 for _ in range(self.size)]?
        print("unreachable", len(values))?
        Ok(())

    pub def receive(self: &mut Greedy, mail: Mail[i64, Fault], context: &mut Context[i64, Fault]) -> Result[(), Fault]:
        Ok(())

    pub def spawn(self, tag: i64, child: Address, context: &mut Context[i64, Fault]) -> Result[Option[Greedy], Fault]:
        Ok(Nothing)

def main() -> Result[(), AllocError | IoError | StageError | WaitError]:
    mut actors = stage[Greedy, i64, Fault](1, 1, 1)?
    actors.start(Greedy(1000000), 65536).unwrap()
    actors.run(&deadline_after(&monotonic_now(), &seconds(5)).unwrap(), &CancellationToken()?)?
    print(actors.stats().failed, heaps_live(), heap_bytes_used())?
    Ok(())
1 0 0

An allocation failure the handler does not handle, such as one inside a runtime operation that cannot report it, is contained like a trap. std.crypto’s RSA operations reserve their memory in the heap first and return Allocation when it does not fit; see RSA allocation.

heap_bytes_used() is the sum of every heap’s used bytes and heap_bytes_committed() the pages made writable; discarding a heap returns both. At most 1024 heaps are live per process.

The stage’s own tables, every mailbox, and Context live outside all actor heaps and hold no pointer into one, so discarding a heap cannot leave a dangling pointer in them. The root node passed to Stage.start was built outside its heap: whatever it already owns stays outside and leaks if the actor traps. Build large state in started instead.

Traps

A trap in a handler, including an allocation failure that cannot be returned:

  • discards the actor’s heap, returning its memory;
  • closes up to 4096 files and sockets the actor opened, which also removes them from the stage’s selector;
  • gives the parent Exit.Trapped(kind);
  • does not run destructors for the trapped actor’s node or the handler’s locals, since they lived in the discarded heap.

kind is a TrapKind:

KindCause
MemoryAn allocation failure the program cannot handle, or an exit that does not fit the parent’s heap
IndexA list, string, byte or slice index out of bounds, or a missing dictionary key
ArithmeticInteger overflow, division by zero, an invalid shift or a lossy cast
Unwrapunwrap on Nothing or Err
OtherAny other runtime error, such as a failed assertion

The message appears on standard error only; reporting the kind allocates nothing. These are not contained and still end the process:

  • stack overflow;
  • internal runtime panics;
  • an allocation failure while no handler is running.

A lock or other process-wide state held at the moment of a trap stays as it was. Allocations made by C libraries through FFI, such as SQLite’s, are outside actor heaps and are neither limited nor freed by a trap.

Limits

  • One thread. Heaps belong to the thread that created the first one, and heap operations from another thread fail. Starting a thread in a handler traps the actor, since the thread could outlive the heap holding its task.
  • An actor node cannot hold str; use Bytes, which also accepts text.
  • Copying a message copies owned values. Where copy would share storage instead, a str is duplicated and any other shared value stops the program with a runtime error.
  • One message type per stage; a request and its reply are both M.
  • Restart is one-for-one: a child’s restart does not stop or restart its siblings, and the parent is not stopped when the restart limit is reached.
  • Files beyond 4096 per process are not tracked and stay open after a trap.
  • Handlers are not preempted: a handler that loops forever stalls the stage.
  • A stage holds at most max_capacity (1,048,576) actors, since a watch key holds a 20-bit slot.

Acknowledgements

  • Parents receiving their children’s exits follow Erlang/OTP supervisors; no Erlang code is used.
  • Uses the Rust libc crate for the Linux page-size query used by actor heaps (MIT OR Apache-2.0).

std.actor.net

TCP for std.actor: a listener that starts one actor per connection, a connection with bounded buffers and deadlines, and connect and name lookup that do not block the stage. A connection’s buffers are allocated in its actor’s heap when it is admitted; handling readiness and timers allocates nothing. A trap in a connection actor closes that connection only.

Supported target: x86-64 Linux, as for std.actor.

Connection cursor arithmetic is bounded by the allocated input/output buffers; invalid consume/commit/forward counts retain their existing NetError causes. Output reservation compares the requested count with remaining space without first adding it to the cursor, so even an i64.MAX request cannot overflow that comparison. Deadlines beyond the clock range clamp to the last instant. Listener.refused() is an observation-only total and saturates at i64.MAX; the live-child count remains an exact admission count. A matching Down when no child is live returns NetError.Stream(StreamError.Io(IoError.InvalidInput)) before changing that count.

A server echoes one connection. It also starts a client as its own child, under a different tag, which dials it, sends a request and prints the reply:

from std.actor import Address, Context, Mail, StartError, TimerError, heaps_live, StageError, stage
from std.actor.net import Admission, Connection, Dialing, Limits, Listener, connection, dial, listener
from std.bytes import BytesError
from std.net import NetError, SocketAddress, listen, loopback_v4
from std.ready import WaitError
from std.sync import CancellationToken
from std.time import deadline_after, monotonic_now, seconds

type Fault = AllocError | NetError | BytesError | TimerError | StartError | IoError | OverflowError[i64]

# Spawn tags: the listener's connections and the client.
const connection_tag: i64 = 0
const client_tag: i64 = 1

pub enum Step:
    Waiting(SocketAddress)
    Dialing(Dialing)
    Talking(Connection)

# Connects, sends one request, and prints the reply.
pub class Client:
    pub step: Step

    pub def started(self: &mut Client, context: &mut Context[i64, Fault]) -> Result[(), Fault]:
        match &self.step:
            case Step.Waiting(address):
                self.step = Step.Dialing(dial(address, &seconds(5), context)?)
            case _:
                pass
        Ok(())

    pub def receive(self: &mut Client, mail: &Mail[i64, Fault], context: &mut Context[i64, Fault]) -> Result[(), Fault]:
        match &mut self.step:
            case Step.Dialing(attempt):
                match attempt.handle(mail, context)?:
                    case Some(stream):
                        mut call = connection(stream, &Limits(64, 64, seconds(5), seconds(5)))?
                        call.start(context)?
                        call.write(b"hello, actors", context)?
                        call.shutdown_write(context)?
                        self.step = Step.Talking(call)
                    case Nothing:
                        pass
            case Step.Talking(call):
                events = call.handle(mail, context)?
                if events.ended:
                    print("reply", call.input().to_text_range(call.input_start(), call.input_stop())?)?
                    call.close(context)?
                    context.stop()
            case Step.Waiting(_):
                pass
        Ok(())

# Serves connections and stops once its one connection and the client have
# both stopped.
pub class Server:
    pub listener: Listener
    pub ended: i64

pub enum Node:
    Serving(Server)
    Echoing(Connection)
    Calling(Client)

    pub def started(self: &mut Node, context: &mut Context[i64, Fault]) -> Result[(), Fault]:
        match self:
            case Node.Serving(server):
                server.listener.start(context)?
                context.spawn(client_tag, 65536)?
            case Node.Echoing(echo):
                echo.start(context)?
            case Node.Calling(client):
                client.started(context)?
        Ok(())

    pub def receive(self: &mut Node, mail: Mail[i64, Fault], context: &mut Context[i64, Fault]) -> Result[(), Fault]:
        match self:
            case Node.Serving(server):
                server.listener.handle(&mail, context)?
                match mail:
                    case Mail.Down(_, _, _):
                        server.ended += 1
                        if server.ended == 2:
                            context.stop()
                    case _:
                        pass
            case Node.Echoing(echo):
                events = echo.handle(&mail, context)?
                if echo.available() > 0:
                    echo.forward(echo.available(), context)?
                if events.ended and echo.available() == 0:
                    echo.shutdown_write(context)?
                if not echo.reading() and echo.pending() == 0:
                    context.stop()
            case Node.Calling(client):
                client.receive(&mail, context)?
        Ok(())

    # Builds each child by its tag: a connection's actor, with its buffers in
    # the child's heap, or the client.
    pub def spawn(self, tag: i64, child: Address, context: &mut Context[i64, Fault]) -> Result[Option[Node], Fault]:
        match self:
            case Node.Serving(server):
                if tag == client_tag:
                    return Ok(Some(Node.Calling(Client(Step.Waiting(server.listener.local_address()?)))))
                match server.listener.accept()?:
                    case Some(echo):
                        Ok(Some(Node.Echoing(echo)))
                    case Nothing:
                        Ok(Nothing)
            case _:
                Ok(Nothing)

def main() -> Result[(), AllocError | IoError | StageError | WaitError | NetError]:
    socket = listen(&loopback_v4(0u16), 16)?
    limits = Limits(4096, 4096, seconds(30), seconds(30))
    server = listener(socket, limits, 100, 65536, connection_tag, Admission.Refuse)?
    mut actors = stage[Node, i64, Fault](8, 4, 8)?
    actors.add_timers(8)?
    actors.start(Node.Serving(Server(server, 0)), 65536).unwrap()
    actors.run(&deadline_after(&monotonic_now(), &seconds(10)).unwrap(), &CancellationToken()?)?
    stats = actors.stats()
    print(stats.started, stats.stopped, heaps_live())?
    Ok(())
reply hello, actors
3 3 0

The repository’s examples/actor_echo.plenty is a longer echo server: a connection whose first byte is X traps, and only that connection closes.

Listening

A Listener belongs to one actor, which watches the socket and starts a child actor for each connection. Call start from the actor’s started and pass every mail to handle. On readiness, handle spawns one child with the listener’s tag and heap limit; the actor’s spawn hook then calls accept, which accepts the connection and allocates its buffers in the child’s heap. A Down from a child with the listener’s tag frees its place; handle ignores readiness of other handles and other children’s Down, so the actor can watch other handles and start other children under other tags, as the client in the example above is.

At most capacity connections are live. A connection that arrives at capacity, or when the stage has no free slot or heap, is handled by Admission:

AdmissionAt capacity
PauseStops accepting until a connection ends. Arrivals wait in the kernel’s backlog; once it is full the kernel drops further attempts, so peers wait on their own connect timeouts
RefuseAccepts and closes the connection at once, so the peer sees it end without data. refused() counts them

Pause suits short connections and load bursts; Refuse tells peers at once, which suits clients that retry elsewhere. At the descriptor limit (NetError.HandleLimit), both stop accepting until a connection ends.

Connections

A Connection owns its stream and two buffers sized by Limits. Call start from the actor’s started, and pass every mail to handle; it reads, writes and checks deadlines, and returns what happened as Events:

FieldMeaning
mineThe mail was about this connection
receivedBytes this mail added to the input
endedThe peer finished sending; no more input will arrive
writableOutput space freed after write accepted less than it was offered
drainedAll output has been sent
expiredOption[Expired]: a deadline passed; see Deadlines

handle acts only on Mail.Ready for its own socket, so one actor can hold several connections. Each mail reads at most once and writes at most four times, so a fast connection cannot hold the stage. Readiness is level-triggered, so unread input brings another Mail.Ready.

Reading

Received bytes are input()[input_start():input_stop()], read in place. Call consume(count) once they are handled. When the input buffer is full the connection stops reading, so the kernel’s buffers fill and the peer’s sends block in turn; consume resumes reading. A request larger than Limits.input must be handled in parts or refused.

Writing and back-pressure

write(source, start, stop) copies as much of source[start:stop] as fits into the output buffer, sends what the socket accepts at once, and returns the count accepted. A short count is back-pressure: keep the rest and offer it again when Events.writable is set. forward(count) moves received input to the output, for echoing and relaying. pending() and space() report the output buffer.

Closing

shutdown_write ends output once pending bytes are sent; the peer then reads end of input, and reading continues. close closes the socket at once, discarding unsent output. An actor whose connection is done stops with Context.stop; dropping the connection closes the socket. A reset or other socket failure is returned by handle as an error, and returning it ends the actor with Exit.Failed.

Deadlines

DeadlineSet byEnds when
IdleLimits.idleNo bytes moved either way for this long
StallLimits.stallPending output made no progress for this long, such as a peer that stopped reading
Readread_within(&span)Input did not finish before clear_read_deadline; bounds how long a request may take to arrive

A zero duration turns a Limits deadline off. handle reports a passed deadline once in Events.expired; the actor decides what to do, usually close and stop. Idle and stall then restart from now, and the read deadline is cleared.

Each connection uses one actor timer, re-armed only when it fires or a sooner deadline starts, so moving data does not create or cancel timers. A stage needs add_timers with one record per connection that has a deadline, and one per dial or lookup with a timeout.

Outbound connections and lookup

dial(&address, &timeout, context) starts a nonblocking connect and returns a Dialing; lookup(host, port, &timeout, context) starts a name lookup and returns a Resolving. Pass every mail to their handle, which returns Nothing until the result is ready, then the stream or the addresses, or the error. A zero timeout waits indefinitely; a passed one is NetError.TimedOut. abandon stops waiting. Make a Connection from a dialled stream with connection(stream, &limits).

The lookup runs the system resolver on a helper thread, as std.net.resolve does, so the stage keeps running. Its request and reply are allocated outside the actor’s heap, so a trap in the actor while the lookup runs does not affect the helper thread. At most 32 lookups run at once in a process; another fails with TryAgain. Calling std.net.resolve or connect in a handler also works, but blocks every actor on the stage until it returns.

Memory

A connection actor’s heap holds its node, both buffers and 192 bytes of other state, measured with Context.heap_used() after started: 2240 bytes with 1 KiB buffers and 8384 with 4 KiB buffers. The heap limit passed to listener reserves address space; memory is committed as the heap is used. The stage’s slot, mailbox and timer records are allocated by stage and add_timers. Admission allocates the buffers; nothing else on the connection’s path allocates. lookup allocates its result list in the actor’s heap.

Exports

ExportContract
Limits(input, output, idle, stall)Buffer sizes in bytes (Positive[i64]) and deadlines; see Deadlines
listener(socket, limits, capacity: Positive[i64], heap: Positive[i64], tag, admission)Result[Listener, NetError]; traps for a heap above max_heap_limit
Listener.start(context)Result[(), NetError]
Listener.handle(&mail, context)Result[bool, NetError]: True for mail the listener used
Listener.accept()Result[Option[Connection], NetError | AllocError]: in the spawn hook; Nothing when no connection waits, and the hook should return Nothing
Listener.live(), refused(), tag(), local_address()Admitted connections not yet ended, connections refused, the child tag, the bound address
connection(stream, &limits)Result[Connection, AllocError]: buffers in the current heap
Connection.start(context)Result[(), NetError | TimerError]
Connection.handle(&mail, context)Result[Events, NetError | TimerError]
Connection.input(), input_start(), input_stop(), available()The input buffer and the unconsumed range
Connection.consume(count, context)Result[(), NetError | TimerError]
Connection.write(&source[start:stop], context)Result[i64, NetError | TimerError]: bytes accepted; Stream(StreamError.Io(IoError.Closed)) after shutdown_write or close, and InvalidInput for a range outside source
Connection.forward(count, context)Result[i64, NetError | TimerError]: bytes moved from input to output
Connection.pending(), space()Unsent output bytes; bytes write would accept now
Connection.reserve(wanted: NonNegative[i64]), output(), output_stop(), commit(count, context)Produce output in place: reserve compacts and returns Result[i64, NetError], the room up to wanted; write into output() from output_stop(); commit sends the count bytes written. For layers such as std.actor.tls
Connection.acknowledge_now()Acknowledge received data at once (TCP_QUICKACK), as TcpStream.acknowledge_now in std.net; best effort
Connection.read_within(&span, context), clear_read_deadline()Set or clear the read deadline
Connection.shutdown_write(context), close(context)End output after pending bytes; close at once
Connection.reading(), closed(), stream()Whether input may still arrive; whether closed; the stream
dial(&address, &timeout, context)Result[Dialing, NetError | TimerError]
Dialing.handle(&mail, context)Result[Option[TcpStream], NetError]: Refused, Unreachable, TimedOut and other connect errors
lookup(host, port, &timeout, context)Result[Resolving, NetError | TimerError]
Resolving.handle(&mail, context)Result[Option[list[SocketAddress]], NetError]: NameNotFound, TryAgain, TimedOut
Dialing.abandon(context), Resolving.abandon(context)Result[(), NetError]: stop waiting and cancel the timer
TypeValues
AdmissionPause, Refuse
ExpiredIdle, Read, Write (the stall deadline)
EventsAs in Connections; copyable

Limits

  • TCP only; std.actor.tls adds TLS.
  • The input and output buffers have fixed sizes. Input is read in place, so a message must fit in Limits.input to be handled whole.
  • A trap closes the actor’s sockets without a graceful shutdown; the peer sees a reset or end of input.

std.actor.tls

Plaintext space accounts for TLS record overhead without an overflowing round-up addition, including for i64.MAX free ciphertext bytes. Cursor updates are bounded by the offered slices and allocated plaintext/ciphertext buffers; the existing TLS, network and timer errors are unchanged.

TLS for std.actor.net connections. A TlsConnection runs a std.tls session over a Connection: it moves ciphertext between the session and the connection’s buffers, decrypts into a bounded plaintext buffer, and reports the same Events. Nothing waits, so one stage serves many TLS connections. Certificates, verification and configuration are those of std.tls; cryptography is rustls.

Supported target: x86-64 Linux, as for std.actor.

A server echoes one TLS connection, and a client actor dials it, sends a request once the handshake completes and prints the reply:

from std.actor import Address, Context, Mail, TimerError, heaps_live, StageError, stage
from std.actor.net import Admission, Dialing, Limits, Listener, connection, dial, listener
from std.actor.tls import TlsConnection, client, server
from std.bytes import BytesError
from std.net import NetError, SocketAddress, listen, loopback_v4
from std.ready import WaitError
from std.sync import CancellationToken
from std.time import deadline_after, monotonic_now, seconds
from std.tls import ClientConfig, ServerConfig, TlsError, client_config, roots_from_pem, server_config

type Fault = AllocError | NetError | BytesError | TimerError | TlsError | IoError

# Test-only CA and a localhost leaf it signed; never reuse a published key.
const CA_PEM: str = "-----BEGIN CERTIFICATE-----\nMIIBmjCCAT+gAwIBAgIULOQ2Qxr8CVZY2bG0uRYKKEdMD3swCgYIKoZIzj0EAwIw\nGTEXMBUGA1UEAwwOUGxlbnR5IFRlc3QgQ0EwIBcNMjQwMTAxMDAwMDAwWhgPMjEy\nNDAxMDEwMDAwMDBaMBkxFzAVBgNVBAMMDlBsZW50eSBUZXN0IENBMFkwEwYHKoZI\nzj0CAQYIKoZIzj0DAQcDQgAEkiB3Ogehrrt6YbQghwDPv6tkhE+z9zTCVih8fvtR\nZ1/691Nfh7/J9IgaNgNSTMJjGTolzGnjaCUDqoARlFyM+6NjMGEwHQYDVR0OBBYE\nFOTFBDgWXHj/qmVJDMlAHYCEZFX5MB8GA1UdIwQYMBaAFOTFBDgWXHj/qmVJDMlA\nHYCEZFX5MA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgIEMAoGCCqGSM49\nBAMCA0kAMEYCIQD41ATQnIjKkKMLQAb9AyoyOFmljYZyz348XJEIbqMgKwIhAL3m\nmWd8On5gvkfytspatH5mDWOcSyW+au7vuIC2gHhO\n-----END CERTIFICATE-----\n"
const LEAF_PEM: str = "-----BEGIN CERTIFICATE-----\nMIIBnDCCAUKgAwIBAgIBATAKBggqhkjOPQQDAjAZMRcwFQYDVQQDDA5QbGVudHkg\nVGVzdCBDQTAgFw0yNDAxMDEwMDAwMDBaGA8yMTI0MDEwMTAwMDAwMFowFDESMBAG\nA1UEAwwJbG9jYWxob3N0MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE8THNQeTi\nJ6Eq0tSxtHW5Q4TCsbIqB3MOtM98ERyKgl6kHeHCeDZYKx8B+VRGT2jkWuTeN24Q\nljwUfh7uh7Iaz6N+MHwwGgYDVR0RBBMwEYIJbG9jYWxob3N0hwR/AAABMAkGA1Ud\nEwQCMAAwEwYDVR0lBAwwCgYIKwYBBQUHAwEwHQYDVR0OBBYEFEIFC2Fyd4cfAKTx\nMiNCMdDkqDNbMB8GA1UdIwQYMBaAFOTFBDgWXHj/qmVJDMlAHYCEZFX5MAoGCCqG\nSM49BAMCA0gAMEUCIAUF3HH43lnirgSN1Vr4twZrFmXKBxlbYNQni/h4ZaxhAiEA\n5rD/yE1B4IsbEGCMlyAeX507leXfGY9+wrHaxikU/w0=\n-----END CERTIFICATE-----\n"
const LEAF_KEY: str = "-----BEGIN PRIVATE KEY-----\nMIGHAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBG0wawIBAQQglvSEE3H7v+mInxSF\nAyWFDKYpzpXr20wH8mfFY3bCaB2hRANCAATxMc1B5OInoSrS1LG0dblDhMKxsioH\ncw60z3wRHIqCXqQd4cJ4NlgrHwH5VEZPaORa5N43bhCWPBR+Hu6HshrP\n-----END PRIVATE KEY-----\n"

pub enum Step:
    Waiting(SocketAddress)
    Dialing(Dialing)
    Talking(TlsConnection)

# Connects, sends one request once the handshake completes, and prints the
# reply.
pub class Caller:
    pub config: ClientConfig
    pub step: Step
    pub sent: bool

    pub def started(self: &mut Caller, context: &mut Context[i64, Fault]) -> Result[(), Fault]:
        match &self.step:
            case Step.Waiting(address):
                self.step = Step.Dialing(dial(address, &seconds(5), context)?)
            case _:
                pass
        Ok(())

    pub def receive(self: &mut Caller, mail: &Mail[i64, Fault], context: &mut Context[i64, Fault]) -> Result[(), Fault]:
        match &mut self.step:
            case Step.Dialing(attempt):
                match attempt.handle(mail, context)?:
                    case Some(stream):
                        net = connection(stream, &Limits(4096, 4096, seconds(5), seconds(5)))?
                        # The session is allocated in this actor's heap.
                        mut call = client(net, &self.config, "localhost", 1024)?
                        call.start(context)?
                        self.step = Step.Talking(call)
                    case Nothing:
                        pass
            case Step.Talking(call):
                events = call.handle(mail, context)?
                # `writable` first reports the end of the handshake.
                if events.writable and not self.sent:
                    self.sent = True
                    # The empty output takes all 14 bytes.
                    call.write(b"hello over TLS", context)?
                    call.shutdown_write(context)?
                if events.ended:
                    print("reply", call.input().to_text_range(call.input_start(), call.input_stop())?)?
                    call.close(context)?
                    context.stop()
            case Step.Waiting(_):
                pass
        Ok(())

pub class Server:
    pub listener: Listener
    pub config: ServerConfig

pub enum Node:
    Serving(Server)
    Echoing(TlsConnection)
    Calling(Caller)

    pub def started(self: &mut Node, context: &mut Context[i64, Fault]) -> Result[(), Fault]:
        match self:
            case Node.Serving(serving):
                serving.listener.start(context)?
            case Node.Echoing(echo):
                echo.start(context)?
            case Node.Calling(caller):
                caller.started(context)?
        Ok(())

    pub def receive(self: &mut Node, mail: Mail[i64, Fault], context: &mut Context[i64, Fault]) -> Result[(), Fault]:
        match self:
            case Node.Serving(serving):
                serving.listener.handle(&mail, context)?
                match mail:
                    # This server handles one connection.
                    case Mail.Down(_, _, _):
                        context.stop()
                    case _:
                        pass
            case Node.Echoing(echo):
                echo.handle(&mail, context)?
                # Forwarding frees input room, so more may be decrypted at once.
                while echo.available() > 0:
                    if echo.forward(echo.available(), context)? == 0:
                        break
                if not echo.reading() and echo.available() == 0:
                    echo.shutdown_write(context)?
                    if echo.pending() == 0:
                        context.stop()
            case Node.Calling(caller):
                caller.receive(&mail, context)?
        Ok(())

    # Builds each connection's actor, with its buffers and TLS session in
    # the child's heap.
    pub def spawn(self, tag: i64, child: Address, context: &mut Context[i64, Fault]) -> Result[Option[Node], Fault]:
        match self:
            case Node.Serving(serving):
                match serving.listener.accept()?:
                    case Some(net):
                        Ok(Some(Node.Echoing(server(net, &serving.config, 1024)?)))
                    case Nothing:
                        Ok(Nothing)
            case _:
                Ok(Nothing)

def main() -> Result[(), AllocError | IoError | StageError | WaitError | NetError | TlsError]:
    socket = listen(&loopback_v4(0u16), 16)?
    address = socket.local_address()?
    limits = Limits(4096, 4096, seconds(30), seconds(30))
    # Include each connection's private TLS configuration and full records.
    serving = Server(listener(socket, limits, 100, 262144, 0, Admission.Refuse)?, server_config(&LEAF_PEM, &LEAF_KEY)?)
    caller = Caller(client_config(&roots_from_pem(&CA_PEM)?)?, Step.Waiting(address), False)
    mut actors = stage[Node, i64, Fault](8, 4, 8)?
    actors.add_timers(8)?
    actors.start(Node.Serving(serving), 65536).unwrap()
    actors.start(Node.Calling(caller), 262144).unwrap()
    actors.run(&deadline_after(&monotonic_now(), &seconds(10)).unwrap(), &CancellationToken()?)?
    stats = actors.stats()
    print(stats.started, stats.stopped, heaps_live())?
    Ok(())
reply hello over TLS
3 3 0

Connections

Make a TlsConnection from an admitted or dialled Connection with server(net, &config, input) or client(net, &config, server_name, input), where input is the size of the decrypted input buffer. Call them where the connection’s actor heap is current, such as the spawn hook or a handler, so the session lives in that heap. over(net, session, input) takes a session made with std.tls.client_session or server_session.

Call start from the actor’s started, or once a dialled connection is made; a client then queues its first handshake flight. Pass every mail to handle. Its Events count plaintext:

FieldMeaning
mineThe mail was about this connection
receivedDecrypted bytes this mail added to the input
endedThe peer sent close_notify
writableThe handshake completed, or space freed after write accepted less than it was offered
drainedAll ciphertext has been sent
expiredA deadline passed, as for Connection

A TLS failure, such as an untrusted certificate or a peer’s alert, is returned by handle as a TlsError, after the alert rustls queued is sent if the socket takes it at once. A peer that ends its stream without close_notify is Truncated, as in std.tls. Returning the error ends the actor with Exit.Failed.

Reading

Decrypted bytes are input()[input_start():input_stop()], read in place. consume(count) releases them. Releasing input room decrypts what waited, so available() may grow again without new mail: handle input in a loop until available() is 0, as the example’s echo does. When the plaintext buffer is full the connection stops decrypting, its ciphertext buffer fills, and the connection stops reading the socket, so back-pressure reaches the peer as for std.actor.net.

reading() is false once close_notify or the end of the stream has arrived.

Writing and back-pressure

write(source, start, stop) encrypts as much of source[start:stop] as space() allows into the connection’s output, sends what the socket takes at once, and returns the plaintext count accepted. space() is 0 during the handshake, after shutdown_write, and while the session holds ciphertext that did not fit the output; otherwise it is the free output less 32 bytes of record overhead per 16 KiB. A short count is back-pressure: offer the rest when Events.writable is set. forward(count) encrypts decrypted input back out, for echoing and relaying. pending() is ciphertext not yet sent.

Closing

shutdown_write queues close_notify, then ends the TCP write side once it is sent; reading continues until the peer’s close_notify. close closes the socket at once without close_notify. Stop the actor once reading() is false, input is handled and pending() is 0.

Deadlines

Limits deadlines and read_within work as for std.actor.net and count ciphertext progress. Set read_within in started to bound the handshake and the first request, and clear_read_deadline once established() is true.

Memory

ItemWhereSize
ClientConfig, ServerConfig, trust roots, buildersTheir allocating heap (outside actors when built in main)Owned trust anchors, configuration recipe and parsed configuration
SessionThe connection actor’s heapIts own parsed configuration, keys and rustls state; record buffers grow with record size
Ciphertext buffersThe connection actor’s heapLimits.input and Limits.output
Plaintext inputThe connection actor’s heapThe input argument

Historical measurements before heap-local configuration reconstruction (issue #641), using Context.heap_used() on an echo server: with 1 KiB buffers, an established connection uses 12,944 bytes. After echoing 64 KiB it holds 17,168 bytes when the peer sent 1000-byte records, and 41,744 when it sent full 16 KiB records, which rustls buffers whole. With 4 KiB buffers the figures are 22,160 and 26,512 for 1000-byte records, and 50,960 for full records. A peer controls its record size, so size the heap passed to listener for full records: while a full record arrives rustls grows its record buffer to a 32 KiB allocation, so 64 KiB per connection is not enough. 80 KiB served full records with 1 KiB buffers, and 96 KiB with 4 KiB buffers. These historical figures and heap sizes exclude the configuration graph now owned by each actor session. Measure the current application and its certificate/root set before choosing a heap limit.

The HTTP server TLS comparison records current process RSS and handshake timings after reconstruction, with an ECDSA P-256 server certificate. Those resident-memory measurements do not replace the echo application’s per-actor heap measurements above.

rustls allocates through the Rust allocator, which in a handler is the current actor’s heap. A session that exceeds its heap traps that actor only, and its heap is discarded with the session; other connections continue. Outside actors the same failure stops the program, as described for std.tls.

Configurations keep an owned recipe: certificate chains and encoded private keys, trust anchors, revocation lists, application protocols, version and cipher choices. Opening an actor session reconstructs its complete rustls configuration from that recipe inside the connection actor’s heap. It retains no reference into the parent’s configuration, even during a handshake. The parent may drop or replace its configuration after construction without invalidating the session.

A trap, including allocation failure partway through reconstruction or a handshake, discards the local graph with the heap. Cleanup does not call a destructor on potentially interrupted rustls state. Repeated fresh configurations and trapped sessions therefore do not pin retired configuration generations. Builders, roots and configurations constructed inside an actor also belong to that actor’s heap. An actor node may hold an immutable configuration and lend it to its spawn hook; other std.tls handles cannot cross the actor boundary. The usual Stage.start seed ownership limit still applies: objects built outside the heap remain outside. If the root actor itself traps, its prebuilt configuration can leak just like any other prebuilt owned resource. Construct the root’s owned resources in started when its own trap recovery must reclaim them; connection traps do not retain the root’s configuration.

This isolation costs a configuration reconstruction and private-key parsing for each actor session, and the session’s heap must include its trust anchors, certificates, keys and verification state. Large system root bundles or CRLs can substantially increase that cost. Outside actors, sessions still share the parsed configuration. Configurations themselves retain both a recipe and parsed state; building them validates the key and builds the parsed state. Retained encoded private keys are zeroized on normal destruction; trap cleanup discards actor pages.

Exports

ExportContract
server(net, &config, input: Positive[i64])Result[TlsConnection, TlsError]: a server session in the current heap
client(net, &config, server_name, input: Positive[i64])Result[TlsConnection, TlsError]: verifies the server against config and server_name
over(net, session, input: Positive[i64])Result[TlsConnection, TlsError]
TlsConnection.start(context)Result[(), TlsError | TimerError]
TlsConnection.handle(&mail, context)Result[Events, TlsError | TimerError]
TlsConnection.input(), input_start(), input_stop(), available()The plaintext buffer and the unconsumed range
TlsConnection.consume(count, context)Result[(), TlsError | TimerError]; may decrypt more
TlsConnection.write(&source[start:stop], context)Result[i64, TlsError | TimerError]: plaintext accepted; Closed after shutdown_write or close
TlsConnection.forward(count, context)Result[i64, TlsError | TimerError]: plaintext moved from input to output
TlsConnection.pending(), space()Unsent ciphertext; plaintext write would accept now
TlsConnection.read_within(&span, context), clear_read_deadline()Set or clear the read deadline
TlsConnection.shutdown_write(context), close(context)Send close_notify then end output; close at once
TlsConnection.established(), reading(), closed()Handshake complete; input may still arrive; socket closed
TlsConnection.session(), connection()The Session, for the negotiated protocol, version and peer certificate; the Connection

Limits

  • consume and forward may decrypt more input without new mail; loop until available() is 0.
  • Plaintext compaction uses Bytes.copy_within to move the unconsumed range in place without allocating.
  • No session resumption, as in std.tls.
  • std.actor.http.server builds HTTP/1.1 over these sessions; its qualification is separate.

std.actor.http.server

HTTP/1.1 with one supervised actor per connection, over plain TCP or TLS. The server uses the actor networking, actor TLS and resumable HTTP decoder modules. Application callbacks run on one thread. A contained trap discards the connection’s heap and closes its tracked sockets; admission and unrelated connections continue.

The qualification record separates tested behavior from outstanding production-readiness gates. Qualification currently targets x86-64 GNU/Linux. HTTP/2 and HTTP/3 are outside this implementation.

Choosing a server

Both HTTP servers remain supported, with different execution contracts.

Choicestd.http.serverstd.actor.http.server
Ordinary handlersSequential Exchange operations, shared handlerComplete bounded Request, mutable handler per connection
StreamingSequential reads and writes on workersEvents, partial writes and explicit resumable state
SchedulingBounded worker pool; idle heads wait off workersOne nonblocking stage; callbacks run to completion
Contained failuresReturned errors; runtime traps end the processReturned errors, handler traps and actor heap exhaustion
Existing helpersRouter, uploads, files, WebSockets and SSEHTTP framing and response primitives; application dispatch

Choose using the application’s failure model, I/O needs and measured workload. The comparison harness does not assume that either implementation is faster. The actor server does not make blocking database or HTTP-client operations nonblocking.

Complete requests

Start with the runnable actor HTTP lesson. The simple entry points are:

Entry pointContract
serve(listener, factory, &config, &stop)Serve plain TCP until shutdown; returns Result[Report, ServerError]
serve_tls(listener, tls, factory, &config, &stop)The same lifecycle using an owned std.tls.ServerConfig
config()Default configuration, without starting a server

The server takes ownership of the listener and factory, and of a supplied TLS configuration. A failed setup releases these resources; they are not returned for retry. This differs from a queue’s failed submission: serve owns the server lifecycle, including cleanup after an error. Configuration and the stop token are borrowed for the call.

factory is a named function returning Result[H, FE]. The server calls it inside each connection’s heap. H implements RequestHandler[E]:

def handle(self: &mut H, request: Request, response: &mut Response) -> Result[(), E]

The handler persists across requests on that connection. It is not shared across connections. Factory and handler error types may differ. Its stored fields must obey the actor heap boundary: use owned Bytes for persistent text, not str, shared owners or locks. Closures with captured state do not cross that boundary. Build connection state in the factory instead of allocating it outside the actor heap.

Request owns head: RequestHead, target: RequestTarget, body: Bytes and trailers: list[Header]. These values are temporary callback inputs. The complete request body is buffered up to config.limits.max_body, subject also to the actor’s heap limit. Route using the validated target and method; the worker server’s Exchange and stored Router are not actor state.

Use Response.send_text, send, or respond for a whole bounded response. It must fit in pending_output together with its encoded head and framing reserve. Larger bodies use the streaming interface. Returning without a response produces 500; an unfinished response is finished before the callback completes, which can fail on a length mismatch.

Configuration and bounds

FieldDefaultMeaning
capacity256At most this many connection actors; accepted range 1–1000
heap_limit4 MiBBudget per connection; accepted range 256 KiB–64 MiB
input_buffer, output_buffer4096 eachTCP ciphertext or plaintext transport buffers; at least 64 bytes
pending_output65536Application response queue, including its head; at least 512 bytes
limitsstd.http.limits()Maximum line, head, header count and decoded request body
idle_timeout5 sWaiting for a request
head_timeout10 sReceiving a request head
body_timeout30 sReceiving its body
write_timeout10 sStalled transport output
response_timeout60 sCompleting a response
handshake_timeout10 sEstablishing TLS
drain_timeout5 sGrace period before forced stage shutdown
max_requests1000Maximum requests per connection, a Positive[i64]

Zero phase durations disable those phase timers. A zero drain duration instead requests immediate teardown. The independent heap, buffer and protocol limits all apply: increasing max_body alone does not increase available memory. The runtime has at most 1024 live actor heaps per process, including the server’s acceptor and any other stages. Capacity is not a promise that admission can always reserve another heap or descriptor.

At capacity or when admission cannot reserve a child, the server closes a newly accepted connection. It does not promise an HTTP 503 response for overload. HTTP parse refusals use bounded lingering close so unread request bytes do not immediately turn a completed response into a TCP reset. Its private lingering state records only whether the 16 MiB drain threshold has been reached, not an unbounded byte total; transport consumption still uses the full available input.

Errors, reports and shutdown

ServerError distinguishes invalid configuration and loss of the acceptor, and preserves allocation, byte-buffer, HTTP, network, TLS, timer, selector, actor-start and time-range causes in named variants. Application failures are classified as Handler; application error values are not sent to the peer. Report application-specific diagnostics before returning them when needed, using an operation that respects the nonblocking callback contract.

A handler Err produces a generic 500 when a response has not started. After output has started, an error closes the connection rather than adding a second response. A trap closes the connection without a recoverable HTTP answer. Isolated construction covers failures before the handler starts too. It does not retry a partially handled request.

Report.actors is the underlying actor Stats; it includes the acceptor and admitted construction attempts. requests counts completed responses, rejected counts admission refusals, and protocol_errors counts reported parse failures. These observation-only counters saturate at u64.MAX; completion credit, admission and shutdown still progress normally at that limit. A selector or setup failure returns Err instead of a report.

A connection submits one completion at a time and waits for admission to acknowledge it before starting another request. A full admission mailbox uses a private 1 ms retry timer instead of runnable retries on every scheduling round. Live application messages and timers continue to dispatch while it waits; retirement retains pending completion and shutdown control. If the shared timer pool is exhausted, runnable retries preserve control delivery. Admission retains at most one pending acknowledgement per connection and retries it when the child’s mailbox is full. This bounds the added queue by capacity; the stage reserves one additional timer record per connection. Phase deadlines and the external forced-drain deadline continue to apply.

Notice.CompletedBy(Address) submits completion credit and Notice.Completed returns it. These variants, Resume, Rejected and Drain are reserved for server control; applications send Notice.Message(value). The additional enum variant requires updating exhaustive matches over Notice.

Cancel the external stop token, or call Response.shutdown_server() from an authorized application route, to start draining. Admission stops, idle connections close, and active responses get the configured grace period. The external driver then calls stage shutdown, even if application mailboxes or timers are full. Early error returns also drop the stage and reclaim its live heaps. Use std.signal.cancel_on outside the stage to turn process signals into cancellation.

Reliability boundary

Callbacks and destructors are not preempted. Blocking I/O or an infinite loop can stall the whole stage, including its shutdown checks. Stack overflow and internal runtime panics remain process-fatal. Traps do not roll back external effects or run user destructors; foreign-library allocations and process-wide locks have the actor runtime’s limits.

TLS sessions reconstruct their configuration inside the connection actor’s heap; traps reclaim it without retaining retired configurations. Configuration rotation and repeated fresh server lifecycles can therefore release old TLS configurations. Size connection heaps for the local certificate, key, trust-anchor and CRL graph; see TLS memory ownership. WebSocket upgrades, dedicated actor SSE helpers, blocking-work offload and multithreaded stages are not provided by this module.

Scheduler adapter

Node[F, H] is an opaque actor-dispatch adapter with private state. Its public started, receive and spawn methods implement the lifecycle expected by std.actor, using Context[Notice[M], ServerError] and returning ServerError. spawn returns an optional child node. These methods advance the server’s admission or connection state; applications use the serve entry points instead of constructing or driving nodes directly. The type and lifecycle methods are public because the existing generic actor dispatcher checks their visibility across module boundaries. This is a visibility exception, not a new raw runtime hook.

Streaming actor HTTP

std.actor.http.server offers serve_events and serve_tls_events for requests or responses that should progress in bounded parts. They take the same listener, factory, configuration and stop token as the complete-request entry points, and return Result[Report, ServerError]. The TLS form additionally owns a std.tls.ServerConfig.

The factory creates one mutable Handler[M, E] per connection. Its callbacks and helper functions retain their own arithmetic contracts: include escaping integer arithmetic errors in their declared result errors, independently of the server’s ServerError contract.

def receive(self: &mut H, event: Event[M], response: &mut Response,
            context: &mut Context[Notice[M], ServerError]) -> Result[(), E]

Events and stored state

EventMeaning
Head(head, target)Validated request head and target
Body(bytes)Owned decoded body bytes, bounded by the decoder’s per-feed budget
Trailers(fields)Validated chunked-body trailer fields
EndThe complete request body has arrived
WritableResponse output progressed; offer previously unaccepted bytes again
Message(value)A user Notice.Message(value) sent through the actor context
Actor(mail)Unconsumed actor readiness, user timer, request/reply or child-exit mail
StoppingOne notification when this connection starts draining; finish pending work promptly

Keep owned byte buffers, numeric state and actor-safe connection objects in the handler. Text in a head or trailer event is for that callback; copy needed text into Bytes before returning. Events are owned, so the handler decides what body bytes to retain, consume or drop. Retained application data counts against the connection’s heap budget.

Context uses Notice[M] as its message type and ServerError as its actor error type. Wrap user messages in Notice.Message. Other Notice variants belong to server lifecycle/accounting and are not application commands. Timer tags below zero are reserved by the server; use nonnegative tags for application timers. Event.Actor preserves the original Mail, so existing std.actor.net and std.actor.tls state machines can consume readiness and timers without reconstructing them.

For an outbound operation, retain a Dialing or Connection in the handler, start it from a request callback, and pass its Event.Actor(mail) values to the networking object’s handle method with the supplied context. Watch the relevant handle and keep application transitions bounded. A blocking std.http.client.fetch inside the callback still blocks every actor. Custom child spawning from a connection handler is not supported; the server’s spawn hook is for connection admission. Application timers share a bounded pool and can return TimerError.Capacity.

Writing responses

OperationContract
start(head)Validate and queue an owned response head; Result[(), ServerError]
write(source: &[u8])Queue part of source; returns accepted byte count or ServerError
finish()Complete framing; a declared length must match; Result[(), ServerError]
respond(head, &body)Queue and finish one whole response; it must fit
send(status, &content_type, &body)Whole response with an automatically encoded head
send_text(status, &text)Whole UTF-8 plain-text response
pending()Bytes waiting in the application queue, excluding transport/TLS buffers
space()Payload space presently available, after framing reserve
is_started(), is_done(), is_stopping()Whether framing began, finished, or shutdown was requested
close_after()Disable reuse after this response
shutdown_server()Start server-wide draining; this response gets the shutdown grace period

All whole-response operations return Result[(), ServerError]. start and respond consume their heads; a failed call does not return the head for retry. Byte inputs are borrowed and accepted bytes are copied into the bounded queue.

A successful write returns 0 <= count <= len(source). A short successful write is back-pressure. Retain the suffix and resume it on Writable; do not call finish until every intended byte was accepted. For an output offset with 0 <= offset <= body.len(), adding the accepted count from body[offset:] stays within that same bound. Update the offset only after a successful write. Do not spin on a zero-byte write. Transport progress drives subsequent events, and request decoding pauses when response capacity is exhausted. Application state needs its own bound too: a callback must not keep appending to a separate buffer while the response is blocked.

start accepts a Content-Length header, or selects chunked output for HTTP/1.1 when the length is absent. HTTP/1.0 uses close-delimited output in that case. Do not supply Transfer-Encoding yourself. respond supplies the content length and rejects caller-supplied framing headers. Bodyless statuses and HEAD obey their framing rules; HEAD consumes offered body bytes without sending them. Invalid lengths and sequencing return HTTP errors.

A response can begin while request bytes are still arriving. The server keeps delivering body events under back-pressure, but conservatively closes that connection after the response rather than reusing it. Finishing a response before request End abandons the unread body and uses bounded lingering close. For reusable ordinary request/response traffic, finish reading the request before starting the response.

The server bounds decoder advancement per actor turn. A handler must likewise bound its own work; actor heap isolation does not impose a CPU-time budget. Direct 101 upgrades and the worker server’s Exchange-based WebSocket/SSE adapters are not supported by this interface.

Actor HTTP qualification

This record covers std.actor.http.server. Its core HTTP/TLS implementation has passed the checks recorded below. It is not yet labelled production-ready: the platform/sanitizer scope below is deliberately limited. TLS configuration retention after traps (issue #641) is addressed by heap-local reconstruction. The worker server’s qualification is separate evidence and remains in force for that implementation.

Gates

GateCurrent evidence
Shared protocol grammarResumable decoder reuses existing head/target/framing rules; focused decoder and existing HTTP regression suites passed during development
Admission isolationActor regression tests cover constructor error/trap/empty result, restart-constructor failure, full parent mailbox, and heap/timer/descriptor recovery
Bounded teardownActor tests cover full-mailbox shutdown, idempotence, reuse, and repeated live-stage drop with trapping node destructors
Real HTTP/TLS connectionstest_std_actor_http_server passed with runtime/allocation checks, including TLS close notification, keep-alive and nonblocking outbound TCP
Corpus and deterministic mutationstest_std_actor_http_corpus passed: 80 shared requests sent in 1-byte and 64-byte writes, plus 600 mutations
Coverage-guided fuzzingIncremental decoder: 1,688,343 libFuzzer executions in 61 seconds, seed 592601, zero findings; coverage instrumentation, not ASan/TSan. Whole actor lifecycle is covered by socket tests rather than this target
Slow clients and output back-pressureSocket tests passed for partial heads/bodies, handshakes and slow readers
Shutdown under load and signalsForced drain with a busy mailbox passed; SIGTERM completes an in-flight streamed response; HTTP/TLS application shutdown is tested, including a TLS peer omitting close notification
Repeated failure/resource stabilityRepeated handler traps, heap exhaustion, factory failures and 5,120 connections at concurrency 16 pass; fresh-configuration trap/normal-drop/construction-OOM regressions check actor heaps and process allocation balance
TLS and reverse proxiescurl 8.18.0, nginx 1.30.6 and Caddy 2.11.7 passed streamed/sized/Expect uploads, chunked replies, keep-alive, repeated 413 refusals and proxy TLS verification
Installed/relocated toolchainCopied compiler alone into an empty environment, compiled HTTP/TLS applications, removed compiler and source, then ran both native binaries through body/echo/trap/recovery/shutdown probes successfully
Comparative measurementsOriginal HTTP/TLS comparison: 540,000 successful requests; refreshed TLS comparison: 270,000 successful requests and 9,000 verified handshakes after issues #641/#644; actor mixed-fault checks passed

This table records evidence rather than inheriting a pass from a shared component. The B85 backlog entry owns remaining work. The design records the two-server goal and acceptance rationale. The remaining platform scope is x86-64 GNU/Linux; no macOS, ARM, ASan or TSan result is implied by these checks.

The actor-specific suites are in tests/test_std_actor_http_server.rs and tests/test_std_actor_http_corpus.rs; decoder tests are in tests/test_std_http_decoder.rs. Run the integrated suites with cargo test --locked --all-features --test integration test_std_actor_http_ and the decoder with the test_std_http_decoder:: filter. These results do not establish behavior on untested platforms.

Performance comparison

examples/http_server_bench.py selects the worker or actor implementation, with corresponding plain and TLS fixtures. It uses the same small response, 16 KiB response, POST echo, keep-alive and new-connection workloads. Compiler hash, revision, source, CPU affinity, settings, failures, throughput, latency, CPU time and memory are recorded with the results. Failed or incomplete requests make the driver fail; they are not a clean throughput result.

For an equal-core comparison, use one worker and one actor stage on the same server CPU, with the client on a separate CPU. Run each at least three times, alternating implementations against fresh processes. Compare the ordinary four-worker configuration separately; it has a different CPU budget. Match idle connection counts explicitly (--idle 984 fits both), and keep load-test concurrency within both admission limits. Use --tls for HTTPS.

Performance under faults is a separate measurement: combine healthy traffic with trapped handlers, heap exhaustion, slow clients and overload, and record healthy-request latency, recovery, lost connections and resource growth. Clean-request throughput alone does not measure the actor server’s purpose. Machine load and client bottlenecks must accompany any reported comparison.

The refreshed TLS comparison records current throughput, handshake latency, idle memory and trap recovery after issues #641 and #644; the original comparison below remains historical evidence.

Recorded comparison

These measurements predate the heap-local TLS configuration reconstruction for issue #641. Their source/compiler provenance and results remain unchanged. They do not establish current TLS setup costs or memory use; actor sessions now own a parsed configuration and rebuild their keys and verification state on construction. Plain HTTP does not use that path.

Measured on 2026-11-01 at source revision 12c021c9, with compiler SHA-256 53e6b1c791bd104ba9795a739b91211025d2b10201ed757ca0e053cfc5ccc8d4. Native fixtures were compiled with --release, on a Ryzen 7 7840HS, Linux 7.2.11 x86-64. One-worker and actor servers used CPU 0; four workers used CPUs 0, 2, 4 and 6 (four physical cores). Clients used separate physical cores 8, 10, 12 and 14. No compilation, qualification suite or fault load ran alongside these measurements. Machine load snapshots are retained in the raw results and reproduction sources.

Each configuration ran three times in alternating worker/actor order against fresh processes: 5,000 requests per workload per run, with 984 connections for the separate idle measurement. These are short loopback measurements, not sustained deployment-capacity claims. ApacheBench measured ordinary workloads; four curl processes measured new TLS connections. The result artifact contains all sample ranges, p50/p99 latency, server CPU time, RSS, failures and settings.

Median requests/second (all measured requests completed successfully):

HTTP

WorkloadOne worker / one coreActor / one coreFour workers / four cores
Small, 1 keep-alive client21,77930,07823,433
Small, 16 keep-alive clients25,54527,70396,802
Small, 64 keep-alive clients24,51513,59782,129
New connection, 16 clients15,16012,21515,035
16 KiB response, 16 clients18,57513,40272,903
1 KiB POST echo, 16 clients24,21525,07198,404

HTTPS

WorkloadOne worker / one coreActor / one coreFour workers / four cores
Small, 1 keep-alive client19,19526,60521,324
Small, 16 keep-alive clients22,56825,43079,376
Small, 64 keep-alive clients20,66311,97665,670
New connection, 16 clients3,7964,1094,285
16 KiB response, 16 clients14,90211,85953,654
1 KiB POST echo, 16 clients20,93923,10481,723

The actor wins several small-response cases on one core, while the worker wins at 64 clients and for 16 KiB responses. Four worker cores provide much higher concurrent throughput. HTTP p99 at 16 small-response clients was 1.013 ms for one worker, 1.625 ms for the actor, and 0.286 ms for four workers; a throughput improvement need not improve tail latency.

Median incremental idle memory per connection was 0.86 KiB (one worker), 21.73 KiB (actor), and 1.13 KiB (four workers) for HTTP. With TLS it was 41.07, 45.86 and 41.35 KiB respectively. These are resident-memory differences for this fixture, not the configured maximum actor heap reservation.

The 64-client actor HTTP samples recorded roughly 617,000 full-mailbox retry attempts for 5,000 completed responses, without lost messages or failed HTTP requests. These are the original measurements before the completion-credit fix for issue #644; the retry comparison records the changed behavior. The raw actor statistics also distinguish abrupt TLS peer closure from an HTTP request failure: curl/ApacheBench can receive the complete response and close without TLS close notification.

Reproduce a single configuration from the repository root; change --implementation actor to --implementation worker --workers 1, or use --implementation worker --workers 4 --server-cpus 0,2,4,6, and add --tls for HTTPS:

python3 examples/http_server_bench.py --compiler target/debug/plenty \
    --implementation actor --server-cpus 0 --client-cpus 8,10,12,14 \
    --requests 5000 --idle 984 --runs 3 --output target/actor-http-comparison

For the recorded samples, each invocation used --runs 1 and the six configurations alternated for three rounds. Adapt CPU affinity to the machine.

Completion retry comparison

Issue #644 replaces repeated runnable completion retries with one outstanding completion per connection, an admission acknowledgement, and a private 1 ms retry after a full admission mailbox. Retrying a full child acknowledgement is bounded by connection capacity; application mail and timers remain live.

The raw comparison freezes the module before the change at 5fad016c, the changed module, the shared fixture, and a replay script. Both snapshots were imported as the same local server module and compiled with the same compiler/runtime and --release; only the fixture’s module imports changed. The compiler SHA-256 is 7da9d5da6d3bbab8a5ae08ea18f5d8fa7ce755e38f6cb0bedf0654cc82875dcf. This is a controlled HTTP comparison on x86-64 GNU/Linux, not a new TLS or worker-server qualification. CPU affinities remain server 0, clients 8,10,12,14. Three alternating before/after runs of all six workloads completed 180,000 measured requests with zero failures or missing responses. Each run’s actor statistics include its additional /stop request and final drain; requests was exactly 5,001.

HTTP workloadBefore median requests/sAfter median requests/s
Keep-alive, 1 client27,52427,664
Keep-alive, 16 clients26,08734,867
Keep-alive, 64 clients12,05431,511
New connection, 16 clients10,81210,449
Keep-alive 16 KiB response, 16 clients12,22813,422
Keep-alive 1 KiB POST echo, 16 clients23,15529,999

At 64 clients, median full-mailbox attempts fell from 616,899 to 6,707 and mail deliveries from 319,895 to 26,840. This improves throughput, but p99 latency rose from 12.464 to 16.854 ms in this workload. The retry delay and credit exchange are a throughput/latency tradeoff; this does not establish an across-the-board latency improvement. Incremental idle RSS with 984 connections was 21.73 KiB before and 21.80 KiB after per connection; setup RSS and raw runs are in the artifact. No task-local builds ran during measurement; unrelated activity on the shared host was not controlled, and the recorded load snapshot was taken after the run. Short local runs do not establish long-term soak behavior or precise performance across environments.

Replay the frozen sources from a checkout with a compiler, ApacheBench and taskset; adapt the script’s affinities on other machines:

python3 - <<'PYREPLAY'
import json
from pathlib import Path
record = json.loads(Path("book/src/assets/actor-http-retry-comparison.json").read_text())
Path("/tmp/actor-http-retry-comparison.py").write_text(record["reproduction_script"])
PYREPLAY
python3 /tmp/actor-http-retry-comparison.py \
    book/src/assets/actor-http-retry-comparison.json target/debug/plenty \
    target/actor-http-retry-comparison

Mixed failures

The separate fault workload placed 32 stalled request heads beside four healthy clients and 100 intentional handler traps in each of four batches. All 4,000 healthy requests and 400 post-batch recovery requests succeeded on the actor server; all 400 trapping connections ended without an HTTP response. RSS stayed at 4,136 KiB and descriptor count at six after every batch. The server then answered its shutdown request and exited successfully, with zero rejected admissions or lost messages. Healthy batch p99 was 1.20–1.28 ms.

Both worker configurations passed their ordinary and stalled-head baselines, then exited on the first intentional runtime trap, as their execution contract specifies. This comparison demonstrates trap containment, not immunity to arbitrary native faults. CPU 6 and client CPU 7 are SMT siblings in this separate probe, so its latency is descriptive failure evidence rather than a throughput comparison. The result artifact embeds the temporary fault harness and its two fixtures, along with compiler and native-binary hashes.

Actor HTTP TLS comparison

Current matched measurements for std.http.server and std.actor.http.server. The actor qualification record retains the original comparison and the platform limits.

Measured on 2026-11-01 at source revision c397a845, after the private TLS configuration reconstruction (issue #641) and completion-credit changes (issue #644). The raw results and replay sources retain all samples, source fixtures, the driver, a reproduction script, an independent validation script, compiler/native hashes and machine load snapshots. Compiler SHA-256: 13c6ed9514305a6d67592417265034555283bb7e8635188dcddc77091c907aa6. The compiler was built with cargo build --locked --bin plenty; native fixtures used --release --compile with PLENTY_CACHE=off.

The Ryzen 7 7840HS host ran Linux 7.2.11 x86-64 GNU/Linux. One worker and the single actor stage used CPU 0; four workers used physical cores 0, 2, 4 and 6. Clients used physical cores 8, 10, 12 and 14. Three rounds alternated worker/actor/four-worker order, reversing it in the middle round. Every workload started a fresh server, with 5,000 requests at the stated concurrency. All 270,000 measured requests completed with zero failures or missing responses. Idle measurements used 984 connections in each configuration. No task-local builds or search indexing ran during measurement; unrelated shared-host activity was uncontrolled. The host’s one-minute load average was 2.88 before and 2.72 after the complete run.

HTTPS throughput and latency

Median requests/second across three samples:

WorkloadOne worker / one coreActor / one coreFour workers / four cores
Small, 1 keep-alive client18,87625,95620,925
Small, 16 keep-alive clients21,87434,37989,255
Small, 64 keep-alive clients19,95528,64164,121
New connection, 16 clients3,7883,7944,465
16 KiB response, 16 clients14,27913,33251,283
1 KiB POST echo, 16 clients21,05130,17184,747

The actor leads the equal-core small-response and echo cases in these samples; one worker leads the 16 KiB case, and four worker cores have substantially higher concurrent throughput. New-connection throughput is nearly equal between the single-core configurations. This is a current comparison of the implementations, not an isolated before/after measurement of either fix.

Median sample p99 latency in milliseconds (new-connection latency includes TLS and the HTTP response):

WorkloadOne workerActorFour workers
Small, 1 keep-alive client0.0920.0640.085
Small, 16 keep-alive clients1.5760.5790.443
Small, 64 keep-alive clients28.13722.58319.484
New connection, 16 clients5.3135.3944.324
16 KiB response, 16 clients1.8171.4410.575
1 KiB POST echo, 16 clients0.9810.6290.367

The 64-client p99 values are high for all configurations. The three p99 samples range from 24.349–32.037 ms for one worker, 22.431–22.710 ms for the actor, and 16.729–24.064 ms for four workers. Short loopback runs on a shared host do not establish a service latency objective or sustained deployment capacity. ApacheBench drives keep-alive workloads; four parallel curl processes drive new connections, without certificate verification, as in the historical comparison.

TLS handshake and idle memory

A separate Python probe made 1,000 new connections at concurrency 16 per sample, three samples per configuration: 9,000 successful verified TLS handshakes and validated HTTP responses. It used one shared client SSL context loaded with the fixture CA. Every session negotiated TLS 1.3 with TLS_AES_256_GCM_SHA384 and reported no session reuse. Timing begins after TCP connect and ends when the client’s TLS handshake returns, before the HTTP request. It includes client verification, scheduling and library overhead; it does not isolate server CPU cost or prove that the server has consumed the client’s Finished message.

MeasurementOne workerActorFour workers
Median sample handshake p50, ms1.7823.6301.383
Median sample handshake p99, ms3.4895.4435.192
Median incremental idle RSS, KiB/connection41.1257.8041.42

The actor fixture uses 16.68 KiB more incremental idle RSS than one worker, including its private per-session configuration, for this ECDSA P-256 certificate fixture. This comparison does not isolate configuration memory from other actor overhead. Idle RSS is a process resident-memory difference with connection capacity raised to 1,000, not an actor heap high-water mark or a general heap-sizing recommendation. The historical 45.86 KiB actor TLS figure predates the ownership change. Different certificate chains, root sets, record sizes and application state need their own measurement.

TLS failures and recovery

A separate descriptive probe combined 32 stalled TLS request heads, four healthy clients and 100 intentional handler traps per batch. Across four actor batches, all 4,000 healthy requests and 400 post-batch recovery requests succeeded; all 400 trap requests ended without a successful response. Healthy-request p99, including TLS setup, ranged from 1.699 to 2.957 ms. After each batch RSS was 5,128 KiB and the descriptor count was six; before fault injection RSS was 5,124 KiB. Shutdown succeeded with exactly 4,501 completed requests (including 100 baseline requests and /stop), 400 trapped actors, zero rejected admissions and zero lost messages. Abrupt client TLS closes also increment actor failure counts; they do not mean those complete HTTP responses failed.

Both worker configurations completed their 100-request baselines, then exited with status 1 on the first handler trap; neither served recovery requests. This demonstrates the actor containment contract for these injected traps. It is one fault run per configuration, with a fixed root TLS configuration, not a configuration-rotation soak, an allocation-failure test, or a new platform qualification. The actor server’s production-readiness limits remain unchanged.

Replay from the recorded source revision with ApacheBench, curl, taskset and a compiler built there. Extract the scripts from this artifact before switching to that checkout, then pass its path and compiler to the replay script:

python3 - <<'PYREPLAY'
import json
from pathlib import Path
record = json.loads(Path("book/src/assets/actor-https-comparison.json").read_text())
Path("/tmp/actor-https-replay.py").write_text(record["reproduction_script"])
Path("/tmp/actor-https-validate.py").write_text(record["validation_script"])
PYREPLAY
TLS_CHECKOUT=/path/to/checkout-of-c397a845
python3 /tmp/actor-https-replay.py "$TLS_CHECKOUT" \
    "$TLS_CHECKOUT/target/debug/plenty" /tmp/actor-https-replayed
python3 /tmp/actor-https-validate.py /tmp/actor-https-replayed/results.json

Adapt CPU affinities to the host. Keep newly recorded results separate from the published artifact; numerical RSS stability and negotiated-cipher assertions in the validation script describe this fixture and platform, and may need adjustment elsewhere.

std.archive

import std.archive reads and writes TAR and ZIP archives over std.io streams, and extracts them into a new directory under a std.fs Root. It has one error type, ArchiveError.

The module is ordinary Plenty over std.bytes, std.compress (DEFLATE and CRC-32) and std.fs; it adds no runtime code. It is available on the supported x86-64 Linux GNU target.

import std.archive
import std.bytes
import std.io
import std.path as path

def main() -> Result[(), IoError | AllocError | std.archive.ArchiveError | std.bytes.BytesError]:
    mut writer = std.archive.tar_writer(std.io.cursor(std.bytes.empty()?))?
    writer.add_dir(&path.from_text("docs")?, 493, 1700000000)?
    readme = std.bytes.from_text("read me first")?
    writer.add_file(&path.from_text("docs/README")?, &readme, 420, 1700000000)?
    writer.finish()?
    mut reader = std.archive.tar_reader(std.io.cursor(writer.inner.data.copy()?))?
    while True:
        match reader.next()?:
            case Some(entry):
                print(entry.name.to_string()?, entry.kind, entry.size)?
            case Nothing:
                break
    Ok(())
docs/ std.archive.EntryKind.Directory 0
docs/README std.archive.EntryKind.File 13

Functions

FunctionResultContract
tar_reader(inner: R)Result[TarReader[R], ArchiveError]A streaming TAR reader over any Reader
tar_writer(inner: W)Result[TarWriter[W], ArchiveError]A streaming TAR writer over any Writer
zip_reader(inner: R, max_entries: NonNegative[i64])Result[ZipReader[R], ArchiveError]Reads the central directory of a seekable inner
zip_writer(inner: W)Result[ZipWriter[W], ArchiveError]A ZIP writer over any Writer; it never seeks
policy()PolicyThe default extraction limits
extract_tar(archive: &mut TarReader[R], parent: &Root, destination: &Path, policy: &Policy, cancel: &CancellationToken)Result[Summary, ArchiveError]Extract every remaining entry into the new directory parent/destination
extract_zip(archive: &mut ZipReader[R], parent: &Root, destination: &Path, policy: &Policy, cancel: &CancellationToken)Result[Summary, ArchiveError]Extract every entry into the new directory parent/destination
stream_error(error: ArchiveError)StreamErrorHow a reader reports error through the Reader protocol

Entries

pub enum EntryKind:
    File
    Directory
    Symlink
    Hardlink
    Other

pub class Entry:
    pub name: Path
    pub kind: EntryKind
    pub size: i64
    pub compressed: i64
    pub mode: i64
    pub mtime: i64
    pub link: Path

name holds the raw bytes stored in the archive; it is not checked or decoded until extraction. size is the length of the content, compressed its length in the archive (equal to size in TAR), mode the Unix permission bits (0 to 4095, or 0 for ZIP entries made on other systems), and mtime the modification time in Unix seconds. link is a TAR link target; ZIP stores a symbolic link’s target as its content. Other covers devices, FIFOs, and unknown TAR types.

TarReader

TarReader[R] reads POSIX ustar, pax, GNU, and V7 archives. inner is public.

MethodResultContract
next()Result[Option[Entry], ArchiveError]The next entry, skipping unread content; Nothing at the end
read(buffer: &mut Bytes, start: i64, stop: i64)Result[i64, StreamError]The current entry’s content; 0 at its end
failure()Option[ArchiveError]The failure that ended the archive, if any

Format rules:

  • Each header’s checksum is verified; old writers’ signed sums are accepted. A mismatch is Checksum.
  • Numbers are octal, or GNU base-256 for values beyond 8 GiB. Any other digit, a negative base-256 value, or an overflow is Corrupt.
  • Long names and link targets come from the ustar prefix, from GNU L and K entries, and from pax path and linkpath records; pax size replaces the header size. Other pax keys and global pax headers are ignored. A pax header or GNU long name larger than 64 KiB is Limit.
  • Only regular files and unknown types have content; links, directories, and devices have none, whatever their size field says.
  • The archive ends with two zero blocks. Input that ends before them is Truncated; one zero block followed by anything else is Corrupt.

A malformed header ends the archive: next and read repeat the error, and failure names it. An error from inner is returned unchanged and leaves the reader usable, so Interrupted may be retried.

TarWriter

TarWriter[W] writes POSIX ustar headers with user and group 0 and no owner names. A name or link target longer than 100 bytes is stored in a pax extended header; a size beyond 8 GiB uses GNU base-256.

MethodResultContract
add_file(name: &Path, data: &[u8], mode: i64, mtime: i64)Result[(), ArchiveError]A regular file holding data
add_from(name: &Path, size: i64, source: &mut S, mode: i64, mtime: i64)Result[(), ArchiveError]A regular file of exactly size bytes read from source
add_dir(name: &Path, mode: i64, mtime: i64)Result[(), ArchiveError]A directory
add_symlink(name: &Path, target: &Path, mtime: i64)Result[(), ArchiveError]A symbolic link; target is stored as given
finish()Result[(), ArchiveError]Write the end blocks and flush inner

Names are normalised: empty and . components are dropped, and directories get a trailing /. A name that is empty after that, absolute, or contains .., NUL, or a backslash is UnsafePath, so a writer cannot produce an archive that extraction would refuse. A mode outside 0 to 4095, a negative size or time, a link target that is empty or contains NUL, or any call after finish is InvalidInput. A source that ends early in add_from is Truncated and leaves the archive unusable.

An error while writing an entry or the end blocks to inner, including a Truncated source, leaves the archive unusable: that call returns the error, and every later add_* or finish call returns InvalidInput without writing.

ZipReader

ZipReader[R] needs a stream with seek, such as FileStream or Cursor. zip_reader locates the end record, reads the whole central directory, and keeps one Entry per member; more than max_entries entries is Limit. inner is public.

MethodResultContract
len()i64The number of entries
entries()&list[Entry]Every entry, in central-directory order
open(index: i64)Result[(), ArchiveError]Select entry index for read
read(buffer: &mut Bytes, start: i64, stop: i64)Result[i64, StreamError]The selected entry’s content; 0 at its end
failure()Option[ArchiveError]The failure that ended the selected entry, if any

Format rules:

  • The end record must be the last one whose comment reaches the end of the input exactly. ZIP64 end records are read when a count, size, or offset is saturated, and ZIP64 extra fields supply each entry’s sizes and offset.
  • Spanned (multi-disk) archives are Unsupported.
  • Entries may not overlap: each entry’s local header, name, and data must fit before the next entry’s local header or the central directory. Overlapping entries, the basis of some ZIP bombs, are Corrupt.
  • The local header must repeat the central name. Only methods 0 (stored) and 8 (DEFLATE) are read; others and encrypted entries are Unsupported.
  • The content must be exactly the declared size, its CRC-32 must match (Checksum), and a DEFLATE stream must end exactly at the declared compressed size (Corrupt).
  • An entry is a directory when its name ends in / or its attributes say so; Unix attributes (host 3) also give mode, symbolic links, and special files. Times are DOS times in local time, read as UTC.

open clears an earlier failure. A content failure ends the entry: read repeats it and failure names it. CRC and size are checked when the content ends, so treat bytes as unverified until read returns 0. Errors from inner are returned unchanged, as in TarReader.

ZipWriter

ZipWriter[W] writes each entry’s header, then its data, and the central directory at finish. It never seeks, so it can write to a pipe.

MethodResultContract
add_file(name: &Path, data: &[u8], method: Method, mode: i64, mtime: i64)Result[(), ArchiveError]A regular file; Method.Deflate compresses at level 6, Method.Stored does not
add_dir(name: &Path, mode: i64, mtime: i64)Result[(), ArchiveError]A directory
add_symlink(name: &Path, target: &Path, mtime: i64)Result[(), ArchiveError]A symbolic link
finish()Result[(), ArchiveError]Write the central directory and end records, and flush inner

Names and link targets follow the TarWriter rules. A name with non-ASCII bytes that is valid UTF-8 sets the UTF-8 flag. Attributes are Unix (host 3) with mode. Times are stored as DOS times, clamped to 1980 through 2107, with two-second resolution. ZIP64 fields are written for an entry whose size or offset reaches 2³²−1, and ZIP64 end records when there are 65535 or more entries or the central directory lies beyond 2³²−1.

An error while writing an entry or the directory to inner leaves the archive unusable: that call returns the error, and every later add_* or finish call returns InvalidInput without writing.

Offsets and entry counts remain exact i64 values. An entry or final directory whose complete output extent cannot be represented returns Stream(StreamError.OutOfRange) before new I/O, directory mutation, or the failed-writer latch. This preflight failure leaves the writer usable and preserves previously written output; it does not wrap or saturate progress.

Extraction

extract_tar and extract_zip create destination, a single name, under parent. Nothing appears at destination until every entry has been extracted:

  1. A staging directory named .plenty-extract- and 32 random hex digits is created under parent.
  2. For each entry, in archive order: check cancel, the entry count, and the name; refuse or skip links and special files; check the size limits; then create the directory or write the file inside the staging directory. Files are written through std.fs.copy_from, so each is staged anonymously and published with a no-replace link.
  3. The staging directory is renamed to destination without replacing anything. An existing destination is Exists.

On any error the staging tree is removed and the first error returned. Removal allocates; if it fails too, the staging directory keeps its .plenty-extract- name for the caller to remove. Resource bookkeeping is exact and includes newly created intermediate directories, not just archive entries. Before creating a directory or copying a file, extraction checks that its creation count and the two cleanup traversal slots remain representable; exhaustion is Limit before that creation/copy. Cleanup still preserves the original extraction error.

Every path is resolved through std.fs.Root handles that refuse symbolic links, so an entry cannot reach outside the staging directory. Name rules:

  • An absolute name, a .. component, NUL, or a backslash is UnsafePath.
  • Empty and . components are dropped; a file whose name is then empty is UnsafePath, and such a directory is ignored.
  • A name longer than max_name bytes, or with more than max_depth components, is Limit.
  • Two entries with the same file name, or a file and a directory with the same name, are Duplicate. Repeated directory entries are allowed.
  • Names are byte strings; they are not decoded or normalised.

Files are created with mode 0600 and directories with 0700. Stored modes, times, and owners are not applied.

Policy

pub class Policy:
    pub max_entries: i64
    pub max_entry_size: i64
    pub max_total_size: i64
    pub max_ratio: i64
    pub max_depth: i64
    pub max_name: i64
    pub skip_links: bool
    pub skip_special: bool
FieldDefaultMeaning
max_entries10000Entries of any kind, including skipped ones
max_entry_size1 GiBBytes in one file
max_total_size4 GiBBytes in all files
max_ratio100Largest size / compressed for ZIP entries over 1 MiB
max_depth32Components in a name, 1 to 62
max_name4096Bytes in a name
skip_linksFalseSkip symbolic and hard links instead of failing with Link
skip_specialFalseSkip devices, FIFOs, and unknown types instead of failing with Special

Sizes are checked against the declared size before any byte is written, and a file that turns out longer or shorter than declared fails. A field out of range is InvalidInput. The ratio check applies to ZIP entries only: a compressed TAR stream is bounded by its std.compress decompressor’s limit.

Summary reports entries (all entries), files, bytes (file content written), and skipped.

Errors

pub enum ArchiveError:
    Stream(StreamError)
    Io(IoError)
    Fs(FsError)
    Allocation(AllocError)
    Corrupt
    Checksum
    Truncated
    Unsupported
    UnsafePath
    Link
    Special
    Duplicate
    Exists
    Limit
    Cancelled
    InvalidInput
VariantMeaning
Stream(cause)inner or a source stream failed
Io(cause)A directory operation failed
Fs(cause)A std.fs failure with no closer variant
Allocation(cause)A buffer, name, or list could not be allocated
CorruptMalformed archive structure
ChecksumA TAR header checksum or ZIP CRC-32 mismatch
TruncatedThe input ended inside the archive
UnsupportedEncryption, a compression method other than stored or DEFLATE, or a spanned ZIP
UnsafePathA name that could escape the destination
LinkA link entry, when skip_links is false
SpecialA device, FIFO, or unknown entry, when skip_special is false
DuplicateA name already extracted
Existsdestination already exists
LimitA policy or format limit was exceeded
Cancelledcancel was set
InvalidInputA bad argument, policy, or destination, or a writer call after finish or a failed write

stream_error maps Stream(cause) to cause, Truncated to UnexpectedEof, Allocation(cause) to Allocation(cause), Io(cause) to Io(cause), and everything else to Io(IoError.InvalidInput).

Ownership and threads

Readers and writers own inner and their buffers; dropping one releases them and does not flush or close inner. None can be copied. The module has no global state, passes the ordinary worker-effect checks, and may be used from any thread.

Costs

ValueAllocates
TarReader, TarWriterOne 512-byte block
ZipReaderUp to 64 KiB to find the end record, one Entry and record per member, a 16 KiB input buffer, and an Inflater (about 62 KiB)
ZipReader.openA fresh Inflater for a DEFLATE entry
ZipWriterThe central directory, about 50 bytes plus the name per entry
ZipWriter.add_fileThe compressed copy of data for Method.Deflate
ExtractionEach entry’s name components, one directory handle per level, and an 8 KiB copy buffer per file

Reading content allocates nothing; time is linear in the archive size. A TAR archive is read once, front to back; a ZIP archive is read once for the central directory and once per opened entry.

Limits

  • No link creation: links are refused or skipped.
  • No restoring of modes, times, or owners.
  • No appending to an existing archive or rewriting entries.
  • No streaming ZIP writes: each add_file holds the whole entry in memory, and data descriptors (sizes after the data) are not written.
  • No ZIP encryption, and no methods other than stored and DEFLATE.
  • No sparse TAR files, and no TAR archives split across volumes.
  • TAR reader content lengths are at most 2⁶² bytes, keeping block padding representable. PAX decimal fields accept nonnegative values through i64.MAX (9223372036854775807); larger values and invalid digits are Corrupt. This numeric range does not raise the content-length limit. Octal accumulation subtracts ASCII bias before adding a digit; fixed-block checksums do not wrap.
  • An unrepresentable PAX output record length is Allocation(AllocError.CapacityOverflow) before appending the record.
  • One target, x86-64 Linux GNU.

Acknowledgements

std.atomic

Inline atomic integers and booleans that scoped workers update through shared references. Exports Atomic[T], Ordering, and fence. They are not prelude names: each use needs an import, through any item or module alias. Supported target: x86_64 Linux GNU.

from std.atomic import Atomic, Ordering

def work(hits: &Atomic[u64]) -> ():
    for _ in range(1000):
        hits.fetch_add(1, Ordering.Relaxed)

def main() -> Result[(), SpawnError[()] | IoError]:
    hits = Atomic[u64](0)
    with spawn(work, &hits)?, spawn(work, &hits)?:
        pass
    print(hits.load(Ordering.Acquire))?
    Ok(())
2000

Atomic[T]

T is bool, i8, i16, i32, i64, u8, u16, u32, or u64; any other element type is a compile-time error. Atomic[T](value) returns an Atomic[T] directly. Atomic(value) takes T from the expected type, such as an annotation or a field, or else from the value.

An atomic is stored inline wherever its owner stores values: a local, a class field, a tuple or enum payload, or a collection element. Construction and every operation allocate nothing and call no runtime function.

An atomic is affine: it moves and cannot be copied. It has no equality, no ordering, and no formatting; read its value with load. *r cannot read it and *r = ... cannot replace it. Moving or reassigning an unborrowed atomic is ordinary.

Methods take the receiver by shared borrow, so a &Atomic[T] given to several workers can update it. The receiver must be a place: a name, field, element, or * dereference. An operation on a temporary is rejected, except into_inner.

MethodResultNotes
load(order)T
store(value, order)()
swap(value, order)Tprevious value
compare_exchange(current, new, success, failure)Result[T, T]Ok(previous) when it equaled current and was replaced, else Err(actual)
fetch_add(value, order), fetch_subTinteger only; overflow traps before storing
fetch_wrapping_add(value, order), fetch_wrapping_subTinteger only; wraps
fetch_and, fetch_or, fetch_xor, fetch_nandT
fetch_max(value, order), fetch_minTinteger only; compares by T’s signedness
fetch_not(order)boolbool only
into_inner()Tconsumes an owned atomic

Every fetch_ method returns the previous value. fetch_add and fetch_sub use the ordinary integer overflow trap at the call’s position. They compute the checked result from the value last read and store it with a compare-exchange, retrying if another thread changed the value, so no thread observes a wrapped value. They trap on the value they read even if another thread changes it afterwards: that is a valid ordering of the operations, and it keeps the guarantee that no wrapped value is stored, unlike Rust’s wrapping fetch_add. The other methods lower to one atomic instruction, or a compare-exchange loop where the target has none. No operation blocks or takes a lock; a compare-exchange loop retries while other threads keep changing the value, so it is lock-free but not wait-free.

Ordering

Ordering is an enum: Relaxed, Release, Acquire, AcqRel, SeqCst. Each ordering argument is written as Ordering.<variant>, through any alias. A variable, call, or other expression is a compile-time error, as is an ordering the operation does not allow:

OperationAllowed orderings
loadRelaxed, Acquire, SeqCst
storeRelaxed, Release, SeqCst
Read-modify-write methodsall five
compare_exchange successall five
compare_exchange failureRelaxed, Acquire, SeqCst
fenceRelease, Acquire, AcqRel, SeqCst

The orderings mean what they mean in C++ and Rust. Every operation currently compiles to a sequentially consistent instruction, which is at least as strong as any requested ordering, so a program cannot rely on a weaker ordering being weaker. Ordering values can also be stored and matched like any enum value.

fence

fence(order) -> () orders memory operations around it without an atomic location. It compiles to a full sequentially consistent fence.

Sharing

Scoped threads borrow an atomic, or a class holding atomics, for the task’s scope. The parent may keep calling methods on it while the workers run. Executor jobs own their inputs, so they share an atomic through a Shared[Atomic[T]] handle: each job receives its own handle.share() and calls methods on it directly.

Not provided: compare_exchange_weak (identical to compare_exchange on the supported target), fetch_update (write a compare_exchange loop), get_mut, as_ptr, compiler-only fences, and pointer atomics.

For a lesson see count with atomics; the architecture page describes the lowering.

Acknowledgements

std.bytes

import std.bytes provides Bytes, an owning contiguous buffer that stores one byte per element, and BytesError. Use Bytes for checked byte-range operations and byte-oriented I/O. A list[u8] also stores one byte per element and provides ordinary list operations and element references; its capacity is rounded to allocation units as described under list storage costs. Bytes are never text: converting to str is an explicit, validated operation.

The module is embedded Plenty code over a private native storage owner. It is available on the supported x86-64 Linux GNU target.

Constructing

FunctionResultContract
std.bytes.empty()Result[Bytes, AllocError]Empty buffer with no reserved capacity
std.bytes.with_capacity(capacity)Result[Bytes, AllocError]Empty buffer with exactly capacity bytes reserved; negative is CapacityOverflow
std.bytes.from_list(values: &list[u8])Result[Bytes, AllocError]Independent copy of the list’s bytes
std.bytes.from_text(text: &str)Result[Bytes, AllocError]Independent copy of the exact UTF-8 bytes
std.bytes.from_bytes(source: &[u8])Result[Bytes, AllocError]Independent copy of a byte view: a literal, Bytes, list[u8], or part of one

Bytes has no public constructor and its storage field is private. Every constructor allocates and can report Allocation.

A byte literal such as b"\r\n" has the separate, immutable prelude type bytes, which lives in read-only data and never allocates. Operations that only read another byte sequence take a &[u8] view, so they accept a literal, a buffer, a list, or part of any of them without copying.

len(buffer) calls its shared len method and returns the number of bytes, just as len does for a byte literal or byte slice view. Membership operators are unsupported on Bytes; use the explicit search methods below.

Operations

start, stop, index, offset, count, and sizes are i64; elements are u8.

MethodReceiverResultContract
len()sharedi64Number of bytes
is_empty()sharedboolWhether the length is zero
capacity()sharedi64Bytes storable without reallocating
get(index)sharedu8Read one byte
find(value, start, stop)sharedOption[i64]First index of value within the range
find_in(set, start, stop)sharedOption[i64]First index holding any byte of set
find_not_in(set, start, stop)sharedOption[i64]First index holding no byte of set: the end of a run
find_bytes(needle, start, stop)sharedOption[i64]First index where all of needle occurs inside the range
count(value, start, stop)sharedi64Occurrences of value within the range
matches(offset, source)sharedboolWhether source occurs at offset
matches_text(offset, text: &str)sharedboolWhether the UTF-8 bytes of text occur at offset
ascii_run(start, stop)sharedi64Index after the run of bytes below 128 that begins at start
utf8_run(start, stop)sharedi64Index after the longest valid UTF-8 that begins at start
slice(start, stop)sharedResult[Bytes, AllocError]Independent copy of a range, with exact capacity
copy()sharedResult[Bytes, AllocError]Independent copy of the whole buffer
to_list()sharedResult[list[u8], AllocError]Independent list[u8] for the transport builtins
to_text()sharedResult[str, BytesError]Strictly validated UTF-8 copy; otherwise InvalidUtf8
to_text_range(start, stop)sharedResult[str, BytesError]to_text of one range, validated and copied once with no intermediate Bytes; a range that splits a UTF-8 sequence is InvalidUtf8
set(index, value)exclusive()Replace one byte
write_float(value: f64)exclusivei64Write shortest scientific ASCII into existing bytes at zero; requires len() >= 32 and traps otherwise; returns bytes written, never allocates or changes length
write_float_spec(value: f64, style: i64, precision: i64, offset: i64)exclusivei64Write a Python format-spec spelling at offset for std.fmt: style 1 fixed, 2 exponent, 3 general, 4 general keeping .0, 5 f64 and 6 f32 print spelling, plus 16 for #; a negative value starts with -. Returns bytes written, or the negated total length needed when shorter, writing nothing. Other styles, precision outside 0..=4096, or an offset past the length trap. Never allocates
fill(start, stop, value)exclusive()Replace every byte of a range
overwrite(offset, source)exclusive()Copy source over existing bytes at offset; never grows
copy_within(start, stop, to)exclusive()Copy self[start:stop] to to in the same buffer, like memmove; the ranges may overlap; never grows
push(value)exclusiveResult[(), AllocError]Append one byte
extend(source)exclusiveResult[(), AllocError]Append source; pass &other[a:b] to append part of a buffer
reserve(additional)exclusiveResult[(), AllocError]Ensure room for additional more bytes
resize(count, fill)exclusiveResult[(), AllocError]Set the length, appending fill or dropping the tail
truncate(count)exclusive()Shorten to count; a negative count or one longer than the length traps
pop()exclusiveOption[u8]Remove and return the last byte, or Nothing
clear()exclusive()Set the length to zero, keeping capacity

source, set, and needle parameters are &[u8] views. == and != compare contents. A buffer prints as a byte literal.

Bytes is a resource: assignment and argument passing move it, and the builtin copy rejects it. Duplication is always the fallible copy() or slice() method. It has no hashing. A buffer can be stored in collections, sent through channels, and moved into worker threads.

Printing

print, str.repr, and f-string fields spell a Bytes buffer, a bytes literal, and a &[u8] view the same way: as a byte literal that reads back as the same bytes. Printable ASCII appears as itself, except that a backslash, double quote, LF, CR, and TAB are \\, \", \n, \r, and \t; every other byte is \xhh with lowercase digits. This matches Python’s bytes repr, except that the quotes are always double. The private storage field never appears, also inside a class, list, or Option.

import std.bytes
import std.encoding
from std.bytes import BytesError
from std.encoding import EncodingError

def main() -> Result[(), AllocError | IoError | BytesError | EncodingError]:
    frame = std.bytes.from_bytes(b"GET\t\"/\"\r\n\x00\xff")?
    print(frame)?
    print(&frame[0:3], [std.bytes.from_text("ok")?]?)?
    print(std.encoding.hex_text(&frame)?)?
    Ok(())
b"GET\t\"/\"\r\n\x00\xff"
b"GET" [b"ok"]
47455409222f220d0a00ff

The literal spelling suits data that is mostly ASCII, such as protocol text. For binary data such as digests, keys, or packed records, write the hex form with std.encoding.hex_text, two lowercase digits per byte, or format each byte explicitly.

Byte views

An operation that only reads bytes takes &[u8], a slice view: a pointer and a length. &buffer, &buffer[a:b], &list, a b"..." literal, and a view held by the caller all convert to it with no copy and no allocation. The range is part of the view, so these operations take no start and stop and report no range error: view bounds clamp, and a negative bound counts from the end.

import std.bytes

def ends_line(line: &[u8]) -> bool:
    count = len(line)
    count >= 2 and line[-2] == 13u8 and line[-1] == 10u8

def main() -> Result[(), AllocError | std.bytes.BytesError | IoError]:
    mut head = std.bytes.from_text("Host: example")?
    head.extend(b"\r\n")?
    mut name = std.bytes.empty()?
    name.extend(&head[:4])?
    print(head.find_bytes(b": ", 0, head.len()), ends_line(&head), ends_line(b"no"), name)?
    Ok(())
Option[i64].Some(4) True False b"Host"

The operations taking &[u8] are extend, overwrite, find_in, find_not_in, find_bytes, and matches here, the stream writes in std.io, std.net, and std.tls, and the read-only inputs of std.crypto. Each is one compiled body for every byte owner; native operations read the view’s bytes in place. A view has len(), indexing, slicing, ==, and iteration, but no methods. The receiver of a Bytes method is always a buffer: copy a literal or view with from_bytes to scan it with find or count, or to change it.

std.bytes.ByteSource, the protocol these operations took before, is removed; naming it is a compile error that points to &[u8].

Index syntax and iteration

buffer[i] reads one byte as u8 and buffer[i] = value replaces one. They follow the list index rules, not the method range rule: a negative index counts from the end, and a missing index traps with exit status one and index out of bounds: index I, length N at the source position. Use get and set when an out-of-range index is an expected outcome.

Inside a plain unchecked: suite under --unchecked assume, index syntax requires 0 <= i < len(buffer) and omits bounds checks and negative-index normalization. Violations are undefined behavior. Default builds, unchecked trap: and buffer methods retain their checks; see unchecked blocks.

Writes need a mut binding, a mutable field, or &mut Bytes, and evaluate as indexed assignment does: the right side, then the index, then the exclusive loan. buffer[i] += 1u8 evaluates the index once. Indexing works on named buffers, fields, references, and temporary values; it never grows the buffer and allocates nothing.

for b in buffer consumes the buffer and yields each u8. for b in &buffer and for b in &mut buffer keep it and loop over a slice view of the whole buffer, as &buffer[:] does: each &u8 or &mut u8 points into the buffer, so *b += 1u8 changes it. reversed(&buffer) walks from the end. The loop holds the view’s loan, so its body cannot grow, shrink, or move the buffer, and the body of a shared loop cannot write to it.

Slice syntax buffer[a:b:c]? copies the selected bytes into a new Bytes with Python’s clamping and returns Result[Bytes, AllocError]; it also slices bytes literals. The slice(start, stop) method keeps the range rule: a range outside the buffer traps instead of clamping. A slice view, &buffer[a:b], borrows the bytes without a copy.

Scanning

find, find_in, find_not_in, find_bytes, count, matches, matches_text, ascii_run, and utf8_run each make one native call over their range. A parser uses them to step from one significant byte to the next without a get call per byte.

  • A found index is absolute, never relative to start. Nothing means the range holds no match.
  • A set is a byte view whose elements are the members; order and repeats do not matter. An empty set matches no byte, so find_in returns Nothing and find_not_in returns start for a nonempty range. Build a set once and reuse it. The set, needle, or source may be the buffer being scanned.
  • find_bytes reports a match only when the whole needle lies inside the range. An empty needle is found at start. It takes time linear in the range and needle lengths, even on repetitive input, and does not allocate.
  • matches and matches_text require 0 <= offset <= len. When fewer bytes remain after offset than the comparison needs, the result is False, not an error. An empty comparison is True.
  • ascii_run and utf8_run return an index from start to stop; equal to start when the first byte does not qualify and to stop when every byte does. utf8_run accepts exactly what to_text accepts and excludes a sequence that stop cuts short.

The range rule

Internal index arithmetic uses small explicit unchecked: regions only where validated ranges or native scan bounds prove the result fits in i64. Growable sizes remain native allocation results, and invalid ranges still trap before native access. This does not make caller arithmetic unchecked: index updates such as buffer[i] += 1u8 follow the caller’s ordinary arithmetic contract.

Every range must satisfy 0 <= start <= stop <= len, and every index passed to a method 0 <= index < len. Methods have no negative indexing and no clamping, unlike list slicing and index syntax. A violation is a caller bug: it traps with a message naming the method and argument, such as Bytes.get: index is outside 0..len(), and the report’s note names the calling line. Check a range that comes from input before passing it. A zero-length range is valid anywhere from 0 to len inclusive. overwrite additionally requires offset + len(source) <= len of the destination, and copy_within requires 0 <= to and to + (stop - start) <= len. matches and matches_text take an offset from 0 to len inclusive and report a short remainder as False. The same rule applies to the range arguments of read_file and read_stdin and of to_text_range.

Caller buffer ranges in stream reads, encoding offsets, UUID writes, crypto output offsets, and CSV field positions also trap; see the invalid-argument convention.

Errors

pub enum BytesError:
    InvalidUtf8
    Allocation(AllocError)

The constructors and the methods that can fail only to allocate (push, extend, reserve, resize, copy, slice, to_list) return AllocError; a negative reserve or resize count is CapacityOverflow. to_text and to_text_range return BytesError for InvalidUtf8. A range outside the buffer traps. The only other result shape is pop, whose Option[u8] reports an empty buffer. Allocation carries the storage AllocError: OutOfMemory when the allocator refuses, CapacityOverflow when the requested total cannot be a size. A failed operation leaves the buffer’s length and contents unchanged and releases anything it allocated.

Views

A view of a buffer is an ordinary reference plus explicit offsets: &Bytes is a shared view and &mut Bytes an exclusive one. A function that works on part of a buffer takes the reference and a start/stop pair and uses the checked operations above.

The existing loan rules therefore define what may happen while a view exists:

  • while a &Bytes is live, the owner cannot be mutated, grown, cleared, moved, or exclusively borrowed;
  • while a &mut Bytes is live, the owner cannot be read, borrowed, or moved;
  • a buffer cannot be extended or overwritten from itself, because the exclusive receiver conflicts with the shared source. Use copy_within to move a range inside one buffer, or copy the range first.

These are compile-time conflicting borrow errors. Growth may relocate the storage, but only a live loan can hold an address into it, and growth conflicts with every live loan, so relocation cannot invalidate anything a program holds.

A slice view, &buffer[a:b] or &mut buffer[a:b], carries its own bounds: it is a &[u8] or &mut [u8] of the selected bytes, and holds the same whole-buffer loan as &Bytes or &mut Bytes. It supports len, index reads and writes, loops, iter pipelines, narrower views, and ==, and prints as a bytes literal, b"bc". A &[u8] parameter also accepts views of list[u8] and bytes literals. It has no Bytes methods: a scan’s receiver is the buffer and its range, while the operations that read another byte sequence take the view itself. A view cannot be stored or returned yet; that needs stored references (backlog B11).

Costs

Storage is one byte per element plus one fixed-size owner. with_capacity, slice, copy, and the from_ constructors reserve exactly what they need. Growth through push, extend, and resize is amortized and may over-reserve; reserve requests an exact total.

AllocatesDoes not allocate
Constructors, slice, copy, to_list, to_text, to_text_rangelen, is_empty, capacity, get, index syntax, iteration, every scan, ==, transport
push, extend, resize, reserve beyond capacityThe same operations within capacity
set, fill, overwrite, copy_within, truncate, pop, clear

get, set, index syntax, push within capacity, pop, truncate, and clear take constant time. Range operations take time linear in the range; extend, overwrite, copy_within, slice, copy, resize, and fill are single native bulk operations.

The scans run natively. find and find_in with a set of at most four bytes examine eight bytes per step. find_in with a larger set and find_not_in build a 256-entry table on each call, then examine one byte per step. count, ascii_run, utf8_run, matches, and matches_text are linear in the bytes examined. find_bytes searches for the needle’s first byte and compares at each candidate: linear for ordinary data, and at most the range length times the needle length when the needle’s first byte repeats throughout the range.

Measured with a release compiler on x86-64 Linux, 8,388,608 elements, pinned to one core:

ProgramByteslist[u8]
Build and copy once: peak resident memory18.9 MB526.8 MB
Build and copy once: wall time0.01 s (resize + copy)1.4 s (append loop + copy)
Build, then read every element: peak resident memory10.8 MB264.6 MB
Build, then read every element: wall time0.7 s (resize + get loop)1.6 s (append loop + index loop)
Append one element at a time0.6 s (push loop)included above

An empty program peaks at 2.4 MB. A single-element call costs roughly 70 to 85 ns here, so prefer the bulk operations; the saving is storage and copying.

A scan call costs about the same as a get call, however long its range. Finding the only comma at the end of a 67,108,864-byte buffer, same machine, pinned to one core:

ProgramWall time
find(44u8, 0, len)4 ms
find_in with a four-byte set18 ms
find_not_in with a 94-byte set15 ms
count(44u8, 0, len)17 ms
A get loop over the same range2.3 s

Index syntax and iteration read the storage through a borrowed reference rather than calling a std.bytes method. Reading every element of an 8,388,608-byte buffer, same machine, pinned to one core:

LoopWall time
buffer.get(i)? over range(n)0.56 s
buffer[i] over range(n)0.43 s
for b in &buffer0.15 s

fill over 67,108,864 bytes takes about 5 ms.

Transport

The read methods fill a range of a buffer from a descriptor, and the module functions write a byte view to one, without an intermediate list. Each makes at most one native call, returns the bytes moved as Result[i64, IoError], and never changes a buffer’s length or capacity.

OperationContract
buffer.read_file(file: &mut File, start, stop)Read into buffer[start:stop]; zero for a nonempty range is end of input
buffer.read_stdin(start, stop)Read standard input into buffer[start:stop]
std.bytes.write_file(file: &mut File, source: &[u8])Write a prefix of source

A count below the requested length is normal. A read range outside the buffer traps under the range rule; a write takes a view, which carries its own range. A closed file is Closed, and the open mode is enforced with NotReadable and NotWritable. An interrupted call is IoError.System(4). File transfers are raw bytes with no newline translation. write_file is a module function, not a method, so a b"..." literal or a subview needs no owned buffer. For the process streams, the prelude’s write_stdout_bytes and write_stderr_bytes take the same &[u8] view and write all of it.

Programs normally use these through std.io, which adds retry, end of input, seeking, and bounded buffering on top.

Bytes has no native back-end methods. std.secrets, std.crypto, std.compress, std.db.sqlite, std.net, std.ready and std.tls call compiler intrinsics (__crypto_*, __compress_*, __tls_*, __sqlite_*, __net_*, __ready_*, __io_entropy_read). They read byte inputs as &[u8] views, passed as a pointer and a length with no copy. Output buffers are &mut Bytes operands whose ranges Plenty checks first; native handles are held in byte storage. These intrinsics are reserved for the standard library: user code calling one is rejected with `__crypto_update` is reserved for the standard library. Import the module that owns them instead.

The older binary I/O builtins and std.path still take and return list[u8]. Convert explicitly with from_list and to_list; each conversion allocates and copies.

Acknowledgements

  • find_bytes uses the two-way string-matching algorithm by Maxime Crochemore and Dominique Perrin (1991), as Rust’s core string search does.
  • Word-at-a-time scans use the zero-byte test from Sean Eron Anderson’s Bit Twiddling Hacks, which credits a 1987 newsgroup post by Alan Mycroft.

std.cache.lru

from std.cache.lru import LruCache provides a fixed-capacity cache that evicts its least recently used entry. It is ordinary embedded Plenty code over a private dictionary and lists. Keys follow the dictionary key rules: integers, bool, str, bytes, or tuples of those. Values may be any type a list can store, including owners such as lists and classes with destructors.

from std.cache.lru import LruCache

def main() -> Result[(), AllocError | IoError]:
    mut names = LruCache[i64, str](2)?
    drop(names.put(1, "Ada")?)
    drop(names.put(2, "Grace")?)
    print(names.get(&1), names.peek(&4))?
    match names.put(3, "Edsger")?:
        case Some(entry):
            key, name = entry
            print("evicted", key, name)?
        case Nothing:
            pass
    print(names.contains(&2), names.len())?
    Ok(())
Option[str].Some("Ada") Option[str].Nothing
evicted 2 Grace
False 2

Operations

OperationResultRecencyContract
LruCache[K, V](capacity)Result[LruCache[K, V], AllocError]Reserve all storage for capacity entries
cache.capacity()i64The construction capacity
cache.len()i64Live entry count
cache.is_empty()boolWhether the count is zero
cache.contains(&key)boolUnchangedMembership test
cache.get(&key)Option[V]PromotesCopy the value; copyable V only
cache.peek(&key)Option[V]UnchangedCopy the value; copyable V only
cache.get_ref(&key)Option[&V]PromotesBorrow the value
cache.get_mut(&key)Option[&mut V]PromotesBorrow the value mutably
cache.peek_ref(&key)Option[&V]UnchangedBorrow the value
cache.peek_mut(&key)Option[&mut V]UnchangedBorrow the value mutably
cache.promote(&key)boolPromotesMark used; False if absent
cache.put(key, value)Result[Option[(K, V)], AllocError]PromotesInsert or replace, reporting the displaced entry
cache.remove(&key)Option[V]Remove and transfer the value
cache.pop_lru()Option[(K, V)]Remove and transfer the least recent entry
cache.clear()()Drop entries from least to most recent

Mutation, promotion included, requires a mutable owner or exclusive reference.

put with a new key makes it the most recent entry. If the cache was full, it removes the least recent entry and returns it as Some((key, value)). With an existing key, put keeps the stored key, replaces the value, promotes the entry and returns Some((key, old_value)). Otherwise it returns Nothing. This follows push in the Rust lru crate. Compare the returned key with your own to tell eviction from replacement.

Copying and borrowed access

get and peek copy the value, like dictionary.get; see copying and borrowed lookup. They accept only copyable values, such as integers, strings and tuples of those. A cache may still store lists, classes and other affine values: each method of a generic class is checked when it is first called, so only a call to get or peek with such a value type is rejected. The error points at the call and names the rejected instance, such as LruCache[i64, list[i64]].get, with a note that its body cannot move out of indexed storage.

For affine values, borrow instead. get_ref and peek_ref return Option[&V]; get_mut and peek_mut return Option[&mut V]. All four return Nothing for an absent key. A borrowed result follows the optional reference rules: it borrows the whole cache until its last use, so the cache cannot change meanwhile. Match it, or use ? or unwrap(), to reach the value:

from std.cache.lru import LruCache

def main() -> Result[(), AllocError | IoError]:
    mut pages = LruCache[str, list[str]](2)?
    drop(pages.put("home", ["intro"]?)?)
    match pages.get_mut("home"):
        case Some(lines):
            lines.append("news")?
        case Nothing:
            pass
    match pages.peek_ref("home"):
        case Some(lines):
            print(len(lines), lines)?
        case Nothing:
            print("missing")?
    print(pages.peek_ref("about").is_none())?
    Ok(())
2 ["intro", "news"]
True

Capacity

Capacity is fixed. Zero is valid and stores nothing: put returns its own entry as Some((key, value)) at once, matching functools.lru_cache with maxsize=0. A negative capacity, or i64.MAX, returns AllocError.CapacityOverflow. Construction reserves storage for capacity + 1 entries, so a large capacity costs memory before any entry is stored.

Errors and costs

AllocError is the only error. Construction allocates everything the cache will use; it returns OutOfMemory or CapacityOverflow and releases partial storage on failure. After construction, no operation allocates. put still returns a Result, because insertion into the reserved collections remains fallible. On failure the cache is unchanged and the incoming key and value are dropped.

Expected cost is constant for lookup, promotion, insertion, eviction and removal, apart from hashing and value cleanup. Adversarial key collisions can make dictionary lookup linear. clear and dropping the cache are linear.

The cache stores each key twice: in its private dictionary and in its recency slots. Keys are always copyable, so this needs no explicit copy.

Limits

There is no iteration in recency order, resizing, statistics, expiry or thread-safety contract. A memoizing decorator such as functools.lru_cache is out of scope: Plenty has no decorators. Wrap a function in a cache explicitly with get and put.

Acknowledgements

  • Behavior modelled on Python’s functools.lru_cache, including no caching at maxsize=0.
  • API modelled on the Rust lru crate: put reporting the displaced entry follows its push, along with peek, promote and pop_lru.

std.channel

Explicit imports for bounded channels and timed waits and selection. Supported target: x86_64 Linux GNU. Imports are allocation-free aliases of the existing prelude facilities, with identical generic type identity.

ExportContract
channel[T](capacity: Positive[i64])Result[tuple[Sender[T], Receiver[T]], AllocError]
Sender[T], Receiver[T]Owned endpoints; explicit share() retains the queue
select_recv(&first, &second)Result[Selected[A, B], SelectError]
select_recv_nowait(&first, &second)Same, without waiting for messages
select_recv_timeout(&first, &second, timeout_ms: u64)Same, with a millisecond budget
SendError[T]Full(value), Disconnected(value)
RecvErrorEmpty, Disconnected
SendTimeoutError[T]TimedOut(value), Disconnected(value)
RecvTimeoutErrorTimedOut, Disconnected
Selected[A, B]First(value), Second(value)
SelectErrorEmpty, TimedOut, Disconnected

A worker sums messages until the sender’s scope drops it:

from std.channel import channel, Receiver, SendError
from std.thread import spawn, SpawnError

def total(inbox: Receiver[i64]) -> Result[i64, OverflowError[i64]]:
    mut sum = 0
    while True:
        match inbox.recv():
            case Ok(value):
                sum += value
            case Err(_):
                break
    Ok(sum)

def main() -> Result[(), AllocError | SendError[i64] | SpawnError[()] | IoError | OverflowError[i64]]:
    sender, inbox = channel[i64](2)?
    with spawn(total, inbox).drop_job()? as task:
        with sender as outbox:
            for n in range(1, 6):
                outbox.send(n * 10)?
        print(task.join()?)?
    Ok(())
150

The linked facility pages specify every endpoint method. Capacity must be a Positive[i64] and bounds queued messages. Computed capacities use Positive.try_from(count)?; invalid literals are compile errors. Construction returns only AllocError for capacity overflow or exhaustion. Construction makes one fallible allocation; send, receive, share, error transport and selection allocate nothing. Messages move without deep copying; rejected sends return the original value. Queue operations have constant bookkeeping plus value transfer, with unbounded lock and scheduling latency; destroying queued values costs their cleanup. Last receiver drop destroys queued values and wakes blocked senders; last sender drop allows buffered messages to drain before disconnection.

Messages must be wholly owned and worker-eligible. References, uncertified foreign handles, and recursive receiver retention are still rejected. Imports do not weaken the checks for live opposite endpoints across joins or future waits. Timeouts are budgets, not hard deadlines. Use std.time to compute remaining milliseconds; tokens do not interrupt an ordinary channel wait. No unbounded channel or arbitrary selector is added.

Acknowledgements

std.cli

For terminal progress bars and spinners, see std.cli.progress.

std.cli parses an explicit schema into typed owned values. Its policy is checked Plenty; it uses the existing byte arguments, strict UTF-8 decoder, integer parser, and lossless Path. No reflection, native parser, package download, or additional link dependency is involved. Supported target: x86_64 Linux GNU.

from std.cli import CliError, Kind, Spec, Value, parse

def main() -> Result[(), AllocError | IoError | CliError]:
    schema = [
        Spec("--count", Kind.Integer, False, False, False, "times to greet"),
        Spec("--loud", Kind.Flag, False, False, False, "shout"),
        Spec("NAME", Kind.Text, True, True, False, "who to greet")
    ]?
    # A real tool passes args_bytes()? with start 1 to skip the program name.
    raw = [encode_utf8("--count")?, encode_utf8("3")?, encode_utf8("Ada")?]?
    parsed = parse(&schema, &raw, 0)?
    for entry in &parsed.entries:
        print(schema[entry.spec].name, entry.value)?
    Ok(())
--count std.cli.Value.Integer(3)
NAME std.cli.Value.Text("Ada")

Schema and results

Construct Spec(name: str, kind: Kind, positional: bool, required: bool, repeatable: bool, help: str). All fields are public. Kind is Flag, Integer, Text, or Path; there are no implicit defaults or conversions.

Option names include their leading --; positional names are display labels. Names contain ASCII letters, digits, _, and interior -. Empty/duplicate names, the reserved --help, positional flags, required positionals after optional ones, and positionals after a repeatable positional are invalid. Only the final positional can repeat. Options can appear between positionals in the schema.

validate(schema: &list[Spec]) -> Result[(), CliError] checks these rules. Both parsing and help generation validate first, including when input requests help.

parse(schema: &list[Spec], arguments: &list[list[u8]], start: i64) -> Result[Parsed, CliError] borrows both inputs. Pass the unmodified args_bytes() snapshot with start=1 to skip the invocation name, or fixture tokens with start=0. A start outside 0..=len(arguments) is UnexpectedArgument(start). Indices in errors are absolute indices in these inputs, never relative to start.

Parsed has public help_requested: bool and entries: list[Entry] fields. Each Entry has spec: i64 (the schema index) and value: Value. Entries preserve argument encounter order, including interleaved repeated options. Match Value:

VariantMeaning
FlagThe option was present
Integer(i64)Checked decimal integer; invalid or overflowing input fails
Text(str)Owned, strictly decoded UTF-8, including an empty string
Path(Path)Owned exact native bytes, including invalid UTF-8

Absent optional specs have no entry; applications supply their own defaults. Neither inputs nor returned data borrow each other. Raw access through args() and args_bytes() remains available.

Token policy

  • Only exact long options are supported: --count 12, not -c, abbreviations, combined short flags, or --count=12.
  • A non-flag option consumes exactly its next token, even --, --help, or another option’s name. End of input is MissingValue; a noninteger value for an integer option is InvalidInteger. This makes negative values unambiguous.
  • An unconsumed -- ends option recognition. Every later token is positional, including --help. A lone -, an empty argument, and tokens beginning with a minus followed by a digit are also positional before termination. Other dash-prefixed tokens are options; use -- before a path like -file.
  • --help returns immediately with help_requested=True and any preceding entries. It skips requiredness checks and later tokens, but does not override earlier errors. As an option value or after --, it is ordinary data.
  • Nonrepeatable duplicate options fail. Required specs need at least one entry. Positionals fill schema order; a repeatable final positional takes the rest.

Help, errors, and exit codes

help(schema: &list[Spec], program: &str) -> Result[str, CliError] generates a usage heading and schema-ordered descriptions, with types and required/repeatable markers plus --help. It borrows its inputs and returns owned text.

CliError distinguishes Schema(schema_index), UnknownOption(argument_index), UnexpectedArgument(argument_index), MissingValue(schema_index), Duplicate(schema_index), Missing(schema_index), InvalidInteger(argument_index), InvalidUtf8(argument_index), and Allocation(AllocError). Paths never go through text decoding. Integer/text values and option names do; invalid UTF-8 is rejected without replacement decoding.

describe(error: &CliError) -> Result[str, AllocError] provides a readable diagnostic with the relevant index. A UI may match the typed error to also show schema[index].name or an escaped path from arguments[index]. describe_for(error: &CliError, schema: &list[Spec]) -> Result[str, AllocError] uses schema names for schema-index errors, such as missing required argument FILE; it safely falls back to the numeric index when it is outside the schema. exit_status(error: &CliError) -> i32 returns 2 for usage errors and 1 for schema or allocation failures. Neither function writes or exits. Applications return 0 for help/success, report errors to stderr, and return the selected status. See std.error for nonallocating fallback reporting.

Costs and limits

Schema validation uses quadratic name comparisons and allocates temporary name bytes. For S schema entries and A arguments, parsing takes O(S² + A·S) comparisons plus decoding/copying input bytes. It allocates S counters, returned entries, and owned text/path payloads. Integer/flag payloads need no additional owned storage. Partial output is released on failure; all these allocation failures are typed. Argument indices and per-spec counts are bounded by the supplied argument list length; counting never wraps or saturates. The existing CliError contract therefore needs no arithmetic error variant. Help repeatedly concatenates, with worst-case quadratic copying in output size. describe may allocate; the allocation-error message itself is static.

There are no globals or OS handles, and input is only observed. Values follow ordinary Plenty ownership and worker-send rules; imports receive the same checks as application code. There are no subcommands, short aliases, float/bool-string conversion, automatic environment defaults, or process-exit side effects.

The typed arguments lesson shows a reusable data-tool schema and explicit error handling. The repository’s examples/countbytes.plenty is a complete byte-counting tool: compile it with plenty --compile examples/countbytes.plenty -o countbytes, then run ./countbytes --total FILE.... It reads each file fully and handles parsing, help, path context, and exit statuses explicitly.

Acknowledgements

  • Schema design draws on Python’s argparse and the Rust clap crate.

std.cli.fileinput

Read text lines from successive files or standard input without loading whole files. The module is implemented in Plenty over std.io, with owned fallible generators, not global state.

lines() snapshots native command-line arguments after the executable name. With no arguments it reads stdin. Use line? inside the loop: the outer ? handles argument construction, while each item can report opening, reading, decoding, or allocation failure.

import std.cli.fileinput as fileinput
from std.cli.fileinput import FileInputError

def main() -> Result[(), IoError | AllocError | FileInputError]:
    for line in fileinput.lines()?:
        write_stdout(line?)?
    flush_stdout()?
    Ok(())

Compile this filter and run ./filter first.txt second.txt, or pipe input into ./filter. write_stdout does not add a second newline. print would.

Explicit paths and options

APIResultBehavior
lines()Result[Generator[Result[str, FileInputError]], IoError]Raw argv after the executable name; no option parsing. Failure to allocate the path list is IoError.Data(DataError.Allocation(...)), as for print
files(paths: list[Path])Generator[Result[str, FileInputError]]Consumes explicit native paths in order; an empty list selects stdin

Applications with flags should parse them with std.cli, collect the selected Path values, and call files. The module never treats application flags specially: lines() attempts to open even --help as a filename. Path preserves native bytes, including non-UTF-8 filenames. Only an exact one-byte - selects stdin; use ./- for a file literally named -.

import std.cli.fileinput as fileinput
import std.path as paths
from std.cli.fileinput import FileInputError

def main() -> Result[(), IoError | AllocError | FileInputError]:
    write_text("first.txt", "one\ntwo")?
    write_text("second.txt", "three\n")?
    names = [paths.from_text("first.txt")?, paths.from_text("second.txt")?]?
    for line in fileinput.files(names):
        print(str.repr(line?)?)?
    Ok(())
"one\n"
"two"
"three\n"

Line, buffering, and ownership contracts

  • Lines end at LF, retaining that LF and any preceding CR. Lone CR is ordinary text. The last nonempty, unterminated line is returned without adding LF. No line is synthesized at EOF, and empty files yield nothing.
  • Decoding is strict UTF-8, including stdin. A completed line is validated as a whole, so multibyte characters may cross read-buffer boundaries. Invalid input is an error, not replacement text. There is no encoding option.
  • Files open lazily, one at a time, using ordinary path and symlink semantics. Each file and each stdin occurrence has an 8192-byte BufReader. Unread bytes survive yields. Delimiter scanning advances through each chunk once; line accumulation uses amortized buffer growth, then one text conversion. Storage is the owned path list, a fixed read buffer, and the current line (including temporary bytes during conversion), not the entire file. A single very long line can still exhaust memory; there is no line-length limit.
  • Each - reads the same process stream from its current position. After EOF, another - normally yields nothing; it can read new data if the underlying stream has subsequently been reset or an interactive source resumes.
  • Exhaustion explicitly closes each owned file and reports a close error. break, return, propagation, or dropping the generator closes any remaining file through ordinary ownership cleanup; close errors during destruction cannot be reported. Stdin is never closed.
  • Buffering reads ahead. Do not mix an active iterator with other stdin readers. Dropping it early discards its unread buffered bytes; a fresh iterator cannot recover those bytes. This applies even though descriptor 0 stays open.

Errors

FileInputError has Open(Option[Path], StreamError), Read(Option[Path], StreamError), Text(Option[Path], BytesError), and Close(Option[Path], StreamError) variants. The path is the exact owned native path supplied by the caller (Some even for -); Nothing identifies implicit stdin. Open includes allocating the read buffer. Text includes allocating line storage and strict UTF-8 conversion; line growth during reading can also return Read(..., StreamError.Allocation(...)).

Underlying errors are preserved, including OS codes and allocation causes. An error yields one Err and terminates the whole sequence, not just that file. Owned file resources are released before yielding a read or text error. Already returned lines remain valid; failed partial lines are not returned and consumed input cannot be rolled back. Handle or propagate each item’s Result; a loop does not propagate it automatically.

This is core input only: no rewriting, backups, open hooks, filename/line-number tracking, skip-file operation, or shared global iterator. Unlike Python’s default text mode, it does not perform universal-newline translation.

Acknowledgements

  • Python fileinput models the ordered-files, stdin fallback, dash sentinel, and retained-line-ending API. This is an original Plenty implementation, not copied Python code.

std.cli.progress

Bars and spinners for long-running command-line work, implemented in Plenty over std.io, std.time and std.report. The only new native operation asks whether a process stream is a terminal.

import std.cli.progress as progress
from std.cli.progress import ProgressError

def main() -> Result[(), ProgressError]:
    mut bar = progress.bar(100u64)
    for item in range(100):
        # Process one item here, then count it.
        bar.advance()?
    bar.finish()?
    Ok(())

Construction and updates

bar(total: u64) and spinner() return an owned Progress, initially at zero. A bar has a known total; a spinner counts work without predicting a total. Construction does not draw. Neither creates a worker or timer thread.

MethodContract
advance(count: u64 = 1u64)Add completed work; checked for overflow and the bar’s total
set_position(position: u64)Set an absolute count; it must not decrease or exceed the total
tick()Request a redraw without changing the count; useful for a spinner
finish()Draw the final count and a newline, ignoring the throttle; repeated calls do nothing
position() / total()Exact u64 count / Option[u64] total
is_finished()Whether finished or stopped by a display error

Updates and finish return Result[(), ProgressError]. Finishing early preserves the actual count: it does not claim remaining work was done. A zero-total bar is already full, but still needs finish() if it was drawn. Updates after finishing return Finished; invalid positions/overflow return InvalidPosition and leave the count unchanged. Output failure preserves the accepted count, stops further updates, and returns Output(std.fmt.FormatError) with its original cause. Counts never wrap or saturate: an advance beyond u64.MAX is InvalidPosition, even for a spinner. This check happens before changing the count or drawing; a finished progress object still returns Finished first.

Display and metrics

Output goes only to stderr, and only if stderr is a terminal both at construction and before each draw. Redirected runs emit nothing, including on finish; stdout stays available for application results. Closed/undetectable streams count as nonterminals. The first update draws immediately. Subsequent updates draw at most every 100 ms, using a monotonic clock; skipped updates still count completed work. The spinner advances its frame on actual redraws, not every update call. If a later query finds redirected stderr, that progress display stays hidden.

The default line uses a ten-cell ASCII bar or a four-frame spinner, counts, average whole items/second and estimated whole seconds remaining. Counts and metrics use truncated decimal suffixes k, M, G, T, P, E to keep lines short, including at u64.MAX. The exact counts remain available through getters. The line is under 80 columns. Interactive rendering assumes an ANSI-compatible terminal at least 80 columns wide: carriage return and erase-line replace the previous frame. Cursor visibility and terminal modes are never changed.

rate(now: &Instant) -> Option[f64] is the average count per second since construction. eta(now: &Instant) -> Option[f64] is remaining seconds at that average. Zero elapsed time yields no rate; an unknown total or zero rate yields no ETA, displayed as --. Completed known totals have ETA zero. Estimates use floating-point arithmetic; displayed values truncate and saturate to u64. Queries use the supplied time even after finishing; they do not freeze metrics.

For tests or an application-owned clock, bar_at(total, now), spinner_at(now), set_position_at(position, now) and finish_at(now) take a borrowed Instant. Use one timeline consistently. Backwards elapsed time saturates at zero; backwards redraw timestamps cannot bypass the throttle. render[W](now, output: &mut Formatter[W]) writes the plain line without newline or cursor controls and leaves redraw state unchanged. A string formatter allows snapshot testing; formatter errors propagate unchanged.

Costs and limits

Progress state is fixed-size. Normal drawing uses the bounded reporting formatter, without constructing temporary strings. Hidden and throttled updates skip formatting. Stderr writes may fail or block; a frame consists of multiple small writes and is not atomic with concurrent output. Keep one progress owner on one thread, and finish it before writing other diagnostics to stderr. Dropping it does not write: explicitly finish before normal early returns if a clean line is needed. An output failure may leave a partial line.

This initial API has no automatic ticking, multiple-bar coordination, templates, labels, colors, terminal resizing or iterator adapters. It follows the manually updated, terminal-only stderr model of indicatif, without embedding it. See Show progress for a small example.

Acknowledgements

  • Design follows indicatif 0.18.6; no indicatif code is used.

std.collections

from std.collections import Heap, HeapBy, drain, drain_by provides owned binary heaps (priority queues). They are ordinary embedded Plenty code over a private list[T]; no native code or runtime primitive is involved.

TypeFirst elementElement types
Heap[T]Least by <Integers, floats, str, and tuples of those
HeapBy[T, F]The one the comparator puts before all othersAny storable type

Heap[T] uses the element type’s own <. Classes, ordinary enums, collections, and tuples containing unordered fields are rejected at compile time with “operator < does not accept” the type. A scalar NaN element compares false against everything and leaves the removal order unspecified. Tuples use the lexicographic total sorting order, including signed zeros and NaNs, so Heap[(i64, str)] orders by priority then text.

HeapBy(before) stores a comparator F: Callable[[&T, &T], bool]: a named function or an owned closure. before(a, b) returns whether a must leave ahead of b; it must be a consistent strict order and must return False for equal elements. Reverse the test for a greatest-first queue. The comparator is inferred from its signature, as in HeapBy(urgent)?. A closure with borrowed captures cannot be stored.

from std.collections import Heap, drain

def main() -> Result[(), AllocError | IoError]:
    mut tasks = Heap[(i64, str)]()?
    tasks.push((3, "write docs"))?
    tasks.push((1, "fix crash"))?
    tasks.push((2, "review"))?
    print(tasks.peek())?
    for priority, name in drain(tasks):
        print(priority, name)?
    Ok(())
Option[tuple[i64, str]].Some((1, "fix crash"))
1 fix crash
2 review
3 write docs

Operations

Both types share one interface. heap is Heap[T] or HeapBy[T, F].

OperationResultContract
Heap[T]()Result[Heap[T], AllocError]Construct an empty heap
HeapBy(before)Result[HeapBy[T, F], AllocError]Construct an empty heap owning its comparator
heap.len()i64Observe the element count
heap.is_empty()boolObserve whether the count is zero
heap.reserve(additional)Result[(), AllocError]Make room for at least additional: i64 more elements
heap.push(value)Result[(), AllocError]Consume a value and insert it
heap.peek()Option[T]Copy the first element; copyable T only
heap.peek_ref()Option[&T]Borrow the first element
heap.pop()Option[T]Remove and transfer the first element
heap.clear()()Destroy all elements, retaining capacity
drain(heap)Generator[T]Consume a Heap[T], yielding elements in removal order
drain_by(heap)Generator[T]The same for a HeapBy[T, F]

Mutation requires a mutable owner or exclusive reference. On an empty heap, pop, peek and peek_ref return Nothing. peek follows the copying lookup rule: it type-checks only for a copyable T, and is checked where it is called, so a heap of class instances uses peek_ref. The reference from peek_ref borrows the heap, so the heap cannot change while it is live. There is no peek_mut: changing the first element in place could break the heap order.

Elements that compare equal leave in an unspecified order. For first-in, first-out ties, store an increasing sequence number in the element and compare it after the priority.

Ownership and cleanup

A heap owns its elements and moves like any other owner; duplicate it only with an explicit copy. push takes ownership, and pop transfers it back. Reordering exchanges rows in place: no element is copied, retained, or dropped while it is in the heap. References, generators, and unit cannot be elements.

Dropping or clearing a heap destroys the remaining elements once each, in storage order, not priority order. A drain frame owns the heap: abandoning the frame early destroys the elements not yet yielded.

A heap holds no OS resource and no shared state. It has no thread-safety contract of its own; it is an ordinary owned value of its element type.

Errors and costs

AllocError is the only error. push allocates only when the backing list must grow and reserve only when the requested room is missing. On failure both leave the existing contents and order unchanged, and push drops the incoming value. Every other operation, including drain, allocates nothing.

OperationTime for n elements
len, is_empty, peek, peek_refConstant
pushAt most floor(log2 n) comparisons; a growing push also moves the backing list
popAt most 2 * floor(log2 n) comparisons
clear, dropLinear, plus element cleanup
Draining all elementsO(n log n) comparisons

Supported target: x86_64-unknown-linux-gnu. See the runnable priority queue lesson.

Limits

There is no construction from an existing list, bulk insertion, merging, indexed access, borrowed iteration, removal of an arbitrary element, or priority update. A comparator cannot fail; one that reports inconsistent answers gives an unspecified order but cannot break memory safety.

Acknowledgements

  • Modelled on Python’s heapq, which also puts the least element first, and Rust’s BinaryHeap.

std.complex

from std.complex import Complex provides complex numbers with f64 real and imaginary parts. The module is ordinary embedded Plenty: arithmetic is written with operator hooks and the functions use the f64 numeric methods. Nothing in it allocates or fails.

from std.complex import Complex, from_polar

def main() -> Result[(), IoError | AllocError | FormatError]:
    a = Complex(1.5, 2.0)
    b = Complex(-0.5, 4.0)
    print(f"{a + b} {a - b} {a * b} {-a}"?)?
    print(f"{a / b}"?)?
    print(f"{b.conj()} {Complex(3.0, 4.0).abs()} {Complex(3.0, 4.0).norm()}"?)?
    print(a == Complex(1.5, 2.0), a != b, a.re, a.im)?
    print(a)?
    Ok(())
1.0+6.0i 2.0-2.0i -8.75+5.0i -1.5-2.0i
0.4461538461538462-0.4307692307692308i
-0.5-4.0i 5.0 25.0
True True 1.5 2.0
1.5+2.0i

Complex(re, im) is the generated constructor, and re and im are public fields. Complex is a copy class: assignment, by-value argument passing, return, and collection insertion copy its two f64 parts and leave the source usable. Copying never allocates. Operators and methods borrow their operands, which reads the parts in place instead of copying them into the call. There are no in-place hooks; accumulate with total = total + term on a mut binding.

from std.complex import Complex

def double(z: Complex) -> Complex:
    z + z

def main() -> Result[(), AllocError | IoError]:
    a = Complex(1.0, 2.0)
    mut b = a
    b.im = -2.0
    zs = [a, b, double(a)]?
    print(a == zs[0], b == zs[1], zs[2].re, len(zs))?
    Ok(())
True True 2.0 3

Operations

OperationResultContract
a + b, a - b, a * b, a / bComplexArithmetic on two Complex operands
-aComplexNegate both parts
a == b, a != bboolStructural: both parts compare as f64
z.scale(factor: f64)ComplexMultiply both parts by a real factor
z.conj()ComplexComplex conjugate
z.abs()f64Magnitude, scaled like hypot
z.norm()f64Squared magnitude re * re + im * im
z.arg()f64Phase angle im.atan2(re), in [-pi, pi]
z.to_polar()(f64, f64)(z.abs(), z.arg())
from_polar(r: f64, theta: f64)ComplexComplex(r * cos(theta), r * sin(theta))
z.exp()ComplexComplex exponential
z.log()ComplexPrincipal natural logarithm: abs().log() + arg() i
z.sqrt()ComplexPrincipal square root: nonnegative real part
z.pow(exponent: &Complex)Complexz.powi(n) for a real integral exponent n with abs(n) <= 100; otherwise the principal value of exp(exponent * log(z))
z.powi(n: i64)Complexz multiplied by itself n times by repeated squaring; a negative n gives 1 / z.powi(-n)

Both operands of an operator must be Complex; there are no mixed operations with f64. Operator hooks dispatch on the left operand’s type and cannot be overloaded, so z * 2.0 cannot share __mul__ with z * w. Use z.scale(2.0) for a real factor, or Complex(2.0, 0.0) as an operand. scale is also the more exact choice for non-finite values: `Complex(inf, 1.0)

  • Complex(2.0, 0.0)computesinf * 0.0for its imaginary part and gives NaN, whilescale(2.0)givesinf+2.0i`.

Multiplication uses the textbook formula. Division uses Smith’s algorithm, which scales by the larger divisor part so that c * c + d * d never overflows or underflows. abs scales the same way, so Complex(1e300, 1e300).abs() is finite. norm does not scale and overflows first.

from std.complex import Complex, from_polar
import std.math as math

def main() -> Result[(), IoError | AllocError | FormatError | math.MathError]:
    z = from_polar(2.0, 0.5)
    r, theta = z.to_polar()
    print(math.isclose(r, 2.0, 1e-15, 0.0)?, math.isclose(theta, 0.5, 1e-15, 0.0)?)?
    print(f"{Complex(-4.0, 0.0).sqrt()} {Complex(3.0, 4.0).sqrt()}"?)?
    print(f"{Complex(-1.0, 0.0).log()}"?)?
    i = Complex(0.0, 1.0)
    print(f"{i.pow(i)} {Complex(0.0, 0.0).pow(Complex(2.0, 0.0))}"?)?
    print(f"{Complex(1.0, 1.0).pow(Complex(2.0, 0.0))} {Complex(1.0, 1.0).powi(-2)}"?)?
    print(f"{Complex(0.0, f64.PI).exp()}"?)?
    Ok(())
True True
0.0+2.0i 2.0+1.0i
0.0+3.141592653589793i
0.20787957635076193+0.0i 0.0+0.0i
0.0+2.0i 0.0-0.5i
-1.0+1.2246467991473532e-16i

The functions are computed from f64 primitives, so results carry their rounding: exp(pi i) has a tiny imaginary part. Compare results with a tolerance, for example math.isclose on each part.

pow uses the method of Python’s complex.__pow__. A real integral exponent n with abs(n) <= 100 uses powi, which multiplies by repeated squaring, so small powers of exactly representable values are exact: (1+1i).pow(2) is 0+2i. Each squaring can roughly double the relative error, so larger or non-integral exponents use exp(exponent * log(z)) and inherit the rounding of exp and log. A zero exponent gives 1+0i, including for a NaN base, as f64.pow does. A zero base gives 0+0i for a real positive exponent; otherwise it follows f64 division and log, so 0.pow(-2) is inf+NaNi rather than an error. Call powi directly for an i64 exponent of any size, including i64.MIN. Its negative-exponent magnitude is formed without signed overflow; the result remains an infallible Complex under checked defaults.

Floating-point behavior

Complex arithmetic follows f64 arithmetic: it never traps and never returns Result. Overflow gives infinity and invalid operations give NaN, which then propagates. This matches the / operator on f64 and the numeric methods, and lets operators compose. It differs from the std.math functions, which report domain and overflow errors; check parts with math.isfinite when a result must be finite.

Dividing by Complex(0.0, 0.0) divides each part of the numerator by the real zero, exactly as f64 division does: (1+2i) / 0 is inf+infi, and 0 / 0 is NaN+NaNi. Python raises ZeroDivisionError instead.

Signed zero is preserved where the formulas define it. log of zero is -inf plus the zero’s arg, and sqrt of a signed zero keeps the sign of the imaginary part. Complex(-1.0, 0.0).arg() is pi and Complex(-1.0, -0.0).arg() is -pi, so the branch cut of log, sqrt and pow along the negative real axis follows the sign of the imaginary zero. abs returns infinity when either part is infinite, even if the other is NaN.

from std.complex import Complex

def main() -> Result[(), IoError | AllocError | FormatError]:
    print(f"{Complex(1.0, 2.0) / Complex(0.0, 0.0)} {Complex(0.0, 0.0) / Complex(0.0, 0.0)}"?)?
    print(f"{Complex(-4.0, 0.0).sqrt()} {Complex(-4.0, -0.0).sqrt()}"?)?
    print(f"{Complex(0.0, 0.0).log()} {Complex(f64.INF, f64.NAN).abs()}"?)?
    print(Complex(0.0, 0.0) == Complex(-0.0, 0.0), Complex(f64.NAN, 0.0) == Complex(f64.NAN, 0.0))?
    Ok(())
inf+infi NaN+NaNi
0.0+2.0i 0.0-2.0i
-inf+0.0i inf
True False

Equality is the language’s structural class equality, not a hook: 0.0 equals -0.0, and a NaN part makes values unequal, as for f64.

Formatting

Complex has a public display method, so print, f-strings and std.fmt.to_string write re+imi or re-imi, each part in the f64 print spelling: 1.5+2.0i, 1.0-0.5i, NaN+infi. A negative zero imaginary part is written -0.0i. It has no debug method, so str.repr, !r and list elements use the structural spelling std.complex.Complex(re=1.5, im=2.0). A format spec applies to the whole text; format the parts directly, as in f"{z.re:.3f}{z.im:+.3f}i", to control precision.

Acknowledgements

std.compress

import std.compress compresses and decompresses DEFLATE (RFC 1951), zlib (RFC 1950), gzip (RFC 1952) and Brotli (RFC 7932) data, whole std.bytes buffers at once or as std.io streams. It has one error type, CompressError.

DEFLATE codecs and CRC-32 use zlib-rs; Brotli uses the reference codec with fallible allocation callbacks, bundled into the runtime. The Plenty module owns buffering, the output limit, and the policy for input that follows a stream. The module is available on the supported x86-64 Linux GNU target.

import std.bytes
import std.compress
from std.compress import Format

def main() -> Result[(), IoError | AllocError | std.bytes.BytesError | std.compress.CompressError]:
    mut log = std.bytes.empty()?
    for _ in range(50):
        log.extend(&std.bytes.from_text("GET /index.html 200\n")?)?
    packed = std.compress.compress(Format.Gzip, &log, 6)?
    print(log.len(), packed.len() < log.len())?
    restored = std.compress.decompress(Format.Gzip, &packed, 4096)?
    print(restored == log)?
    Ok(())
1000 True
True

Functions

FunctionResultContract
compress(format: Format, data: &[u8], level: i64)Result[Bytes, CompressError]All of data, compressed
decompress(format: Format, data: &[u8], limit: NonNegative[i64])Result[Bytes, CompressError]The contents of data, at most limit bytes
compressor(format: Format, inner: W, level: i64)Result[Compressor[W], CompressError]A writer that compresses into inner
decompressor(format: Format, inner: R, limit: NonNegative[i64])Result[Decompressor[R], CompressError]A reader of the decompressed contents of inner
inflater(format: Format, limit: NonNegative[i64])Result[Inflater, CompressError]A step-by-step decoder; see Inflater
crc32(seed: i64, data: &[u8])Result[i64, CompressError]The CRC-32 of data, continuing from seed
stream_error(error: CompressError)StreamErrorHow a Decompressor reports error

Format is Deflate (raw, no header or checksum), Zlib (Adler-32), Gzip (CRC-32 and length), or Brotli (no checksum). For DEFLATE formats, level runs from 0, which stores the data uncompressed in blocks, to 9, which is smallest and slowest; 6 is a usual choice. Brotli takes qualities 0–11; 5 is a useful starting point, while 10–11 spend substantially more time and memory. A level outside its format’s range is OutOfRange.

crc32 is the gzip and zip checksum; start a fresh sum with seed 0. The check value of the nine bytes 123456789 is 3421780262. seed must lie in 0 to 2³²−1. CRC-32 retains its native 32-bit polynomial checksum semantics; checked integer defaults do not change the checksum algorithm.

Input policy

A decoder accepts exactly one stream, with these rules:

  • Truncation. Input that ends inside a stream is Truncated.
  • Concatenated gzip members. When a gzip member ends and the next byte is 31 (the first byte of the gzip magic), the next member is decoded and its output appended, as gzip -d does for cat a.gz b.gz. Each member’s CRC and length are checked.
  • Trailing data. Any other byte after the end of a stream is TrailingData, including the zero padding some tools append to gzip files. Raw DEFLATE, zlib and Brotli streams never continue.
  • Integrity. A gzip CRC-32 or length mismatch, a zlib Adler-32 mismatch, and a gzip header CRC mismatch are Checksum. Raw DEFLATE and Brotli have no checksum. Any other malformed input, a zlib stream that needs a preset dictionary, or a stream of another format is Corrupt.
  • Output limit. Producing more than limit bytes in total, counted across gzip members, is LimitExceeded. Exactly limit bytes succeeds. Choose the limit from what the program can hold, not from the input size: a few kilobytes of DEFLATE can expand by a factor of 1000. Decoded progress stays exact: exceeding the i64 representation also reports LimitExceeded, rather than wrapping or saturating the counter.

Gzip header fields (name, comment, modification time, extra data) are validated and discarded; the encoder writes a minimal header with no name and time 0, so equal input gives equal output.

Compressor

Compressor[W] has the Writer methods, so write_all, copy_all, and anything else generic over Writer accept it. inner is public.

MethodResultContract
write(source: &[u8])Result[i64, StreamError]Compress part of source, a view of a Bytes, a list or a b"..." literal; the count is the input consumed
flush()Result[(), StreamError]Deliver everything written so far as complete blocks, then flush inner
finish()Result[(), StreamError]Write the stream’s end and checksum, then flush inner

Output reaches inner in order, on later write calls and in flush and finish. A write first delivers pending output from the previous one; if that fails, the error is returned and none of source is consumed. A failure keeps the undelivered bytes, so calling the method again resumes where it stopped. Interrupted inner writes are retried; a zero-byte inner write is WriteZero.

Call finish before using the result. Dropping a compressor without it leaves inner holding an incomplete stream, which a decoder reports as Truncated. After finish, write is StreamError.Io(IoError.Closed) and finish and flush only deliver what is left. Each flush ends the current block, which costs some compression ratio; flush at message boundaries, not after every write.

Decompressor

Decompressor[R] has the Reader method read(buffer, start, stop), which returns at least one decoded byte, or 0 once the stream and the input have both ended. inner is public.

MethodResultContract
read(buffer: &mut Bytes, start: i64, stop: i64)Result[i64, StreamError]Decode into buffer[start:stop]
failure()Option[CompressError]The decoding failure that ended the stream, if any

An error from inner is returned unchanged and leaves the reader usable, so Interrupted and other transient failures may be retried. A decoding failure ends the stream: read returns it through the Reader protocol, and every later call repeats it. failure gives the precise cause, which stream_error maps as follows:

CompressErrorStreamError
TruncatedUnexpectedEof
Allocation(cause)Allocation(cause)
OutOfRangeOutOfRange
Corrupt, Checksum, TrailingData, LimitExceededIo(IoError.InvalidInput)

gzip and zlib store their checksum after the data, so a reader may already have returned bytes when it finds a Checksum failure. Treat decoded output as unverified until read has returned 0. Bytes decoded in the step that fails are discarded. decompress returns nothing on any error.

Inflater

Inflater is the decoder inside Decompressor, for a reader that owns its input and cannot hand over an inner stream, as std.archive does for ZIP entries. It applies the same input policy and limit.

MethodResultContract
step(input: &[u8])Result[i64, CompressError]Decode from input; returns the count consumed
output()&BytesThe output of the last step, at most 16384 bytes
clear()()Empty the pending output
complete()boolWhether the input so far is a complete stream

Empty the output with clear before each step. After consuming the input, call step(b"") with an empty view until complete() or a step produces no output: Brotli can have pending decoded bytes after consuming its final input byte. A step that returns 0 with no output needs more input unless complete. Discard the Inflater after any error; do not rely on retries of codec errors.

Errors

pub enum CompressError:
    OutOfRange
    Corrupt
    Checksum
    Truncated
    TrailingData
    LimitExceeded
    Allocation(AllocError)
VariantMeaning
OutOfRangeA level or seed out of bounds
CorruptMalformed compressed data
ChecksumWell-formed data whose checksum or length does not match
TruncatedThe input ended inside a stream
TrailingDataBytes after the end of the stream; see Input policy
LimitExceededMore than limit bytes of output
Allocation(cause)A codec state, buffer, or output could not be allocated

Ownership and threads

A Compressor or Decompressor owns its inner stream, its codec state, and its buffers; dropping it releases them all, finished or not, and does not flush or close inner. Neither can be copied. The module has no global state, passes the ordinary worker-effect checks, and may be used from any thread; moving a stream to a worker moves its ownership.

The codec state is held in a Bytes whose contents are the pending output. The __compress_* intrinsics that operate on it are reserved for the standard library. Given a buffer not made by std.compress, they stop the program with operand is not a std.compress state. Both native engines report a successful consumed count within the offered input view and keep pending output within the reserved capacity. The Plenty adapters rely on these bounds for their buffer cursors; these native wrapper contracts are unchanged by checked arithmetic defaults. Inner stream progress is still validated separately: InvalidProgress does not advance the cursor or discard the pending tail.

Costs

For the DEFLATE formats, memory is fixed when a stream is made and does not grow with the data. Brotli’s different allocation contract is described separately:

ValueAllocates
Compressor371 KiB of zlib-rs state at every level (window, hash chains, symbol buffer) and a 16 KiB output buffer
Decompressor46 KiB of zlib-rs state (including the 32 KiB window), a 16 KiB output buffer, and a 16 KiB input buffer
compress, decompressOne codec state and buffer as above, and the result, which grows by doubling
OperationAllocatesTime
Compressor.write, flush, finishNothingLinear in the input; higher levels are slower
Decompressor.readNothingLinear in the output
crc32NothingLinear in the view

Inputs are read directly from the caller’s buffer; output is copied once, from the codec’s buffer to inner or the caller’s buffer. Allocation failures are reported as Allocation; dropping a failed or unfinished codec releases its state and buffers.

Supply

The runtime builds zlib-rs from an unmodified copy under plenty-runtime/vendor, so building the compiler needs no network and an installed compiler needs neither the sources nor a Rust toolchain:

CrateVersionLicenceMaintainer
zlib-rs0.6.8ZlibTrifecta Tech Foundation, as part of zlib-rs / libz-rs-sys

zlib-rs has no dependencies. It is a Rust port of zlib with zlib-ng’s optimisations, and reports allocation failure as an error rather than aborting, which Allocation relies on. The runtime calls its zlib-style interface (deflate::init, inflate::inflate, and so on), enabled by the crate’s __internal-api feature.

The licence file is kept with the sources. The version is pinned in Cargo.lock, and a test fails if the two disagree. Updating it is the runtime maintainers’ job, with each release that fixes a defect: replace the directory with the new release, run cargo update -p zlib-rs --precise VERSION, and rerun tests/test_std_compress.rs, whose fixtures come from CPython’s zlib and gzip and from the system gzip tool. The __internal-api interface is not covered by semver, so read the release’s changes to deflate.rs and inflate.rs.

Executables are linked with --gc-sections, and the codecs are reached only through their own runtime entry symbols, so a program that does not use std.compress contains none of them.

Limits

  • No bzip2, xz, zstd, or lz4. ZIP and TAR are in std.archive.
  • No preset dictionaries, compression strategies, or window and memory settings; DEFLATE encoders use a 32 KiB window and zlib’s default memory level, and Brotli uses the standard 4 MiB encoder window.
  • No access to gzip header fields.
  • Trailing padding after gzip members is an error, not ignored.
  • No seeking within compressed data.
  • One target, x86-64 Linux GNU.

Acknowledgements

  • Wraps zlib-rs 0.6.8, vendored in the runtime (Zlib).

Brotli streams

Use std.compress.Format.Brotli with the same whole-buffer functions, Compressor, Decompressor and Inflater as the other compression formats. The format is RFC 7932, also used by HTTP’s Content-Encoding: br.

import std.bytes
import std.compress
from std.compress import Format, CompressError
from std.bytes import BytesError

def main() -> Result[(), AllocError | BytesError | CompressError | IoError]:
    original = std.bytes.from_text("A Brotli message")?
    encoded = std.compress.compress(Format.Brotli, &original, 5)?
    decoded = std.compress.decompress(Format.Brotli, &encoded, 1024)?
    print(decoded.to_text()?)?
    Ok(())
A Brotli message

Quality ranges from 0 (fastest) to 11 (most effort). It is a compression-effort setting, not a guarantee that each higher setting produces a smaller result. Start with 5 for general use; choose 10–11 when extra compression work is acceptable, such as preparing static assets. Quality 0 still compresses; it does not mean the stored-block mode of DEFLATE level 0. The encoder uses the standard 4 MiB window. The decoder accepts standard windows up to 16 MiB; the nonstandard large-window extension and custom dictionaries are disabled.

flush() makes all written input decodable without ending the stream. finish() writes its final marker; repeated successful finishes are harmless. Short/interrupted inner I/O and retrying an inner write failure preserve pending output. Codec failures, including allocation failures, require discarding that codec. A whole-buffer call returns no partial value on any error.

The decoder rejects trailing data, including a second Brotli stream. Only gzip has the concatenated-member policy. Truncated input is Truncated; malformed input is Corrupt. Brotli has no checksum: a changed stream can still be valid and decode to changed data. Use transport/application integrity when required.

Memory and failures

The Plenty input/output staging buffers remain 16 KiB. Unlike the DEFLATE implementation, Brotli allocates codec working memory during write, read, flush and finish. Working memory depends on quality, stream window and block structure; a decoder may allocate its history before hitting the output limit. The output limit is not a total-memory quota. It applies to all produced output, including bytes buffered internally after the last input byte has been consumed.

The native callbacks use Plenty’s fallible allocator and matching deallocator, so failure is CompressError.Allocation(AllocError.OutOfMemory) (or its StreamError.Allocation projection), not process termination. Failed and unfinished states release every native allocation on drop. There is no allocation-free-after-setup guarantee for Brotli.

The low-level Inflater caller must drain output even when all input has been consumed: clear its output, call step(b""), and repeat until complete or no output is produced. decompress and Decompressor do this automatically. An incomplete decoder with no more input/output is truncated.

HTTP

The HTTP client continues to expose the encoded entity body; it does not negotiate or decode content automatically. If your request sends Accept-Encoding: br, check the actual response’s Content-Encoding and use decompress(Format.Brotli, &fetched.body, decoded_limit) for a single br coding. Bound the encoded HTTP body and the decoded output independently. Multiple content codings must be decoded in reverse application order; do not assume every response is Brotli merely because the request accepted it.

Servers can compress response bytes with the same function and send Content-Encoding: br. The content length describes the compressed bytes. Choose a representation from the request’s accepted encodings, and send Vary: Accept-Encoding when the representation varies. Streaming compression uses the existing Writer API; no new HTTP framing or automatic negotiation is introduced here.

Supply and update policy

The pinned, unmodified reference C codec 1.2.1 is in plenty-runtime/native/brotli-1.2.1, with its MIT license and source archive digest. It includes the decoder block-switch validation fix for CVE-2026-111497; the vendored source must not be replaced with an unpatched older version. Release third-party notices include its license.

The shared build helper compiles the same C sources for runtime unit tests and both compiler-embedded runtime archives. The explicit BROTLI_ENCODER_CLEANUP_ON_OOM flag is mandatory: without it upstream selects exit-on-allocation-failure mode. Allocation-failure tests check this behavior. An installed Plenty compiler needs neither a system Brotli library nor the source tree. Unused compression is removed by native section garbage collection.

This complex codec is wrapped rather than rewritten in Plenty. The Rust brotli 9.1.0 allocator interface was inspected, but its encoder accesses allocation results without a failure check (for example RingBufferInitBuffer). Returning empty memory can panic; unwinding is not an allocation-error strategy for Plenty’s abort-on-panic runtime. The C interface has explicit nullable callbacks and recoverable allocation handling.

Updates require source/license/digest and security review, plus all codec, external-interop, malformed-input, allocation-failure, cleanup, executable-size and installed-compiler checks. Output bytes may change with a codec update; the interoperability and error contracts are the stable API.

Acknowledgements

  • Wraps the Brotli reference C codec 1.2.1, vendored in the runtime (MIT).

std.concurrent

Explicit imports for executors, futures, parallel map, and tree reduction. Exports: ThreadPoolExecutor, Future[T], PoolError, SubmitError[F], FutureError, PoolMapError, and ParallelError[E]. Imports preserve prelude identity and add no allocation or wrapper. Supported target: x86_64 Linux GNU.

from std.concurrent import ThreadPoolExecutor, PoolError, FutureError, SubmitError

def square(n: i64) -> Result[i64, OverflowError[i64]]:
    Ok(n * n)

def main() -> Result[(), AllocError | IoError | PoolError | FutureError | SubmitError[()] | OverflowError[i64]]:
    with ThreadPoolExecutor(2, 4)? as pool:
        first = pool.submit(square, 7).drop_job()?
        second = pool.submit(square, 8).drop_job()?
        print(first.result()??, second.result()??)?
    Ok(())
49 64

ThreadPoolExecutor(workers: Positive[i64], queue_capacity: Positive[i64]) returns Result[ThreadPoolExecutor, PoolError]; both arguments are positive domain values. Computed counts use Positive.try_from(count)?; invalid literals are compile errors. PoolError has Allocation(AllocError) and Thread(ThreadError). The linked pages specify submission, mapping, reduction and shutdown signatures. Accepted jobs and results are wholly owned; borrowing belongs in scoped threads. Pools and futures cannot enter workers. All reachable code still passes worker effect checking.

Construction makes one allocation and starts the requested OS workers. Each accepted job needs one fallible cell; a full queue bounds outstanding queued jobs and can block submit. submit_nowait reports SubmitError.Full(job); other variants are Shutdown(job), OutOfMemory(job), and CapacityOverflow(job). Recovery retains ownership without a hidden clone. submit(...).drop_job() drops a rejected job at the call and reports the job-free SubmitError[()]. Waiting, result transfer, status and cancellation allocate nothing. Queue bookkeeping is constant apart from job/result transfer and cleanup; waiting and shutdown have no bounded latency. Map allocates outputs and uses a bounded work window; reduction costs and ordering are specified in its facility page.

Future[T].result() consumes the future and returns Result[T, FutureError]. When a worker returns a Result, that is the success payload: the example’s ?? handles the future error and then the worker’s overflow error separately. Cancellation before execution yields FutureError.Cancelled; it never forcibly stops running code. done(), cancel(), and wait_timeout_ms(timeout_ms) observe the same state as the prelude methods. Use std.sync tokens for cooperative stopping. Pool exit drains and joins; dropping a future does not cancel its job. Live-receiver shutdown and result-wait diagnostics also apply through imports. There is no second future or cancellation model.

Acknowledgements

std.crypto

Cryptography here is upstream code, used as-is. Plenty implements no cryptographic algorithm. Every hash, MAC, cipher, signature, key agreement, key derivation and constant-time comparison is the upstream Rust implementation, vendored unmodified and compiled into the runtime: the RustCrypto crates, subtle, the BLAKE3 team’s blake3, and ring. This is deliberate. Constant-time code, side-channel resistance and algorithm correctness stay entirely with those projects and their review and testing. The Plenty side validates arguments, owns buffers, parses key encodings and reports errors. Versions and licences are under Supply and Acknowledgements.

import std.crypto computes BLAKE3, SHA-256, SHA-512 and legacy SHA-1 digests and HMAC tags over std.bytes buffers, and compares byte buffers in constant time. The digest and HMAC operations use CryptoError.

The module also provides authenticated encryption, Ed25519 signatures, X25519 key agreement, and HKDF and RSA and ECDSA signatures with key import, with AdvancedError.

The module is available on the supported x86-64 Linux GNU target.

import std.bytes
import std.crypto
import std.encoding
from std.bytes import BytesError
from std.crypto import Algorithm, CryptoError
from std.encoding import EncodingError

def main() -> Result[(), IoError | AllocError | BytesError | CryptoError | EncodingError]:
    message = std.bytes.from_text("hello")?
    digest = std.crypto.sha256(&message)?
    print(std.encoding.hex_text(&digest)?)?

    key = std.bytes.from_text("server key")?
    tag = std.crypto.hmac_sha256(&key, &message)?
    # A receiver recomputes the tag and compares in constant time.
    mut check = std.crypto.hmac(Algorithm.Sha256, &key)?
    check.update_text("hello")?
    print(tag.len(), check.verify(&tag)?)?
    Ok(())
2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824
32 True

Functions

See BLAKE3 hashing and key derivation for its one-shot, incremental keyed-hash and derivation functions.

FunctionResultContract
sha256(data: &[u8])Result[Bytes, CryptoError]The 32-byte SHA-256 digest of data
sha512(data: &[u8])Result[Bytes, CryptoError]The 64-byte SHA-512 digest of data
sha1(data: &[u8])Result[Bytes, CryptoError]The 20-byte SHA-1 digest of data; legacy, see below
hmac_sha256(key: &[u8], data: &[u8])Result[Bytes, CryptoError]The 32-byte HMAC-SHA-256 tag
hmac_sha512(key: &[u8], data: &[u8])Result[Bytes, CryptoError]The 64-byte HMAC-SHA-512 tag
digest(algorithm: Algorithm)Result[Digest, CryptoError]A new incremental digest
hmac(algorithm: Algorithm, key: &[u8])Result[Mac, CryptoError]A new incremental HMAC under key
equal(a: &[u8], b: &[u8])boolWhether a and b hold the same bytes, in constant time
output_size(algorithm: Algorithm)i6432 for Sha256 and Blake3, 64 for Sha512, 20 for Sha1

Algorithm is Sha256, Sha512, Sha1 or Blake3. HMAC supports the SHA algorithms; BLAKE3 has its own keyed mode. HMAC follows RFC 2104 and accepts a key of any length, including empty; a key longer than the hash block (64 bytes for SHA-256 and SHA-1, 128 for SHA-512) is hashed first. Choose a key of at least the output size from std.secrets.

SHA-1 is broken for collision resistance: an attacker can make two inputs with the same digest. It is here only for protocols that fix it, such as the WebSocket handshake and Git object names. Do not use it for signatures, integrity checks against an adversary, or new designs.

Incremental states

Digest and Mac have the same methods. Feeding input in any number of pieces gives the same output as one call over the joined input.

MethodResultContract
size()i64The output size in bytes
update(data: &[u8])Result[(), CryptoError]Absorb data; pass &buffer[a:b] for part of a buffer
update_text(text: &str)Result[(), CryptoError]Absorb the UTF-8 bytes of text
finish()Result[Bytes, CryptoError]The output in a new buffer
finish_into(target: &mut Bytes, offset: i64)Result[(), CryptoError]Write the output at target[offset:offset + size()]
Mac.verify(expected: &[u8])Result[bool, CryptoError]Finish, and compare the tag with expected in constant time

update and verify read a &[u8] view, whose bounds clamp; finish_into takes its offset under the range rule. Each state finishes once: finish, finish_into, and verify end it, and every later call is Finished. When finish cannot allocate its result, the state is left unfinished and the call may be retried. A tag whose length differs from size(), such as a truncated tag, does not verify; verify does not accept truncation.

The fields are private, so a state can only be made by the module’s constructors. Both are redacted: printing one shows std.crypto.Digest(@redacted) or std.crypto.Mac(@redacted).

Digest.finish_into and Mac.finish_into require the complete digest to fit at offset. Invalid offsets trap with finish_into: offset outside target buffer before touching the buffer or finalizing the state. A valid output range on an already-finished state still returns CryptoError.Finished.

Errors

pub enum CryptoError:
    Finished
    InvalidKeyLength
    UnsupportedAlgorithm
    Allocation(AllocError)
VariantMeaning
FinishedThe state has already produced its output
InvalidKeyLengthA BLAKE3 key does not contain exactly 32 bytes
UnsupportedAlgorithmhmac was called with Algorithm.Blake3; use native keyed BLAKE3
Allocation(cause)A state or output buffer could not be allocated

Constant-time comparison

equal and Mac.verify use subtle::ConstantTimeEq (through Mac::verify_slice for HMAC). Their running time depends on the lengths of the operands, not on their contents or where they first differ. Lengths are not secret: comparing buffers of different lengths returns False at once. Digest has no verify; compare a finished digest with equal.

== on Bytes stops at the first difference. Never use it, or a loop of get calls, to check a tag, token, or password hash: its timing tells an attacker how many leading bytes of a guess are right.

The guarantee is that of subtle 2.6: comparisons combine bytes without branches and pass the result through an optimisation barrier. It is a best-effort property of compiled code, not a hardware guarantee, and it covers only the comparison, not how the program reacts to the answer.

Key material

A Mac holds its key’s padded forms or BLAKE3 key state. Finishing it and dropping it overwrite that state with volatile writes before the memory is released. Plenty does not scrub the rest:

  • The key Bytes passed to hmac belongs to the caller, who must protect it; Plenty does not zero memory on drop.
  • Outputs from finish are ordinary buffers.
  • Copies the hashing code makes on the stack while it runs are not cleared.
  • update_text reads a str, which is never scrubbed.

Ownership and threads

A state owns one runtime object and nothing else; it holds no descriptor and needs no closing. Inputs are shared loans and finish_into’s target is an exclusive loan. States cannot be copied. The module has no global state, passes the ordinary worker-effect checks, and may be used from any thread; moving a state to a worker moves its ownership.

update and finish_into take buffers, not states, as operands. The __crypto_* intrinsics behind them are reserved for the standard library. Given a buffer not made by std.crypto, they stop the program with operand is not a std.crypto state.

Costs

OperationAllocatesTime
SHA digest, hmacOne state object of 368 bytes plus the allocation headerConstant, plus one hash of the key when it exceeds a block
update, update_textNothingLinear in the input
finish_into, verifyNothingFinal blocks; BLAKE3 also folds its bounded tree stack
finishOne output bufferAs finish_into
sha256 and the other one-shotsA state and the outputLinear in the input
equalNothingLinear in the length; buffers of different lengths return at once

Input is never copied: update hashes directly from the buffer’s storage. For BLAKE3 state and finalization costs, see its allocation contract. SHA-256 uses the x86-64 SHA extensions and SHA-512 uses AVX2 when the processor reports them at run time, and portable code otherwise; the result is the same.

Supply

The runtime builds these crates from unmodified copies under plenty-runtime/vendor, so building the compiler needs no network and an installed compiler needs neither the sources nor a Rust toolchain:

CrateVersionLicence
sha20.10.9MIT or Apache-2.0
sha10.10.6MIT or Apache-2.0
hmac0.12.1MIT or Apache-2.0
subtle2.6.1BSD-3-Clause
digest, block-buffer, crypto-common0.10.7, 0.10.4, 0.1.7MIT or Apache-2.0
generic-array, typenum0.14.7, 1.20.1MIT; MIT or Apache-2.0
cpufeatures, cfg-if0.2.17, 1.0.4MIT or Apache-2.0

Each crate’s licence file is kept with its sources. The same versions are pinned in Cargo.lock, and a test fails if the two disagree. To update one, replace its directory with the new release, run cargo update -p CRATE --precise VERSION, and rerun the known-answer tests in tests/test_std_crypto.rs.

Executables are linked with --gc-sections, and the hashing code is reached only through its own runtime entry symbols, so a program that does not use std.crypto contains none of it. The link-args.txt written by --emit-runtime includes that flag.

Limits

  • No password hashing or password KDFs (PBKDF2, Argon2).
  • No MD5, SHA-3, BLAKE2, or truncated SHA-512 variants.
  • No zero-on-drop buffer type; see Key material.
  • One target, x86-64 Linux GNU.

Arithmetic and output bounds

Finalization validates the caller offset before subtracting it from the target length or touching the digest state. Only that bounds-proven arithmetic opts out of recoverable integer errors; invalid offsets still trap, and allocation and finished-state errors keep their existing contracts.

Acknowledgements

BLAKE3 hashing and key derivation

std.crypto provides BLAKE3’s three native modes. Each produces 32 bytes and accepts std.bytes.Bytes input. The incremental forms use the same Digest and Mac methods and range rules as SHA and HMAC.

Upstream code, used as-is. Every mode is the BLAKE3 team’s blake3 crate, vendored unmodified; Plenty implements none of the hash. See std.crypto for this policy.

FunctionResultContract
blake3(data: &[u8])Result[Bytes, CryptoError]Unkeyed hash
digest(Algorithm.Blake3)Result[Digest, CryptoError]Incremental unkeyed hash
blake3_keyed(key: &[u8], data: &[u8])Result[Bytes, CryptoError]Native keyed hash; exactly 32 key bytes
blake3_keyed_hasher(key: &[u8])Result[Mac, CryptoError]Incremental keyed hash, including constant-time verify
blake3_derive_key(context: &str, key_material: &[u8])Result[Bytes, CryptoError]Derive a key from high-entropy material under a context
blake3_derive_hasher(context: &str)Result[Digest, CryptoError]Incremental key derivation

output_size(Algorithm.Blake3) is 32. A keyed constructor rejects any other key length with CryptoError.InvalidKeyLength. Generate keys with std.secrets.token_bytes(32). This keyed mode is BLAKE3’s own construction; hmac(Algorithm.Blake3, key) returns CryptoError.UnsupportedAlgorithm.

For derivation, choose a fixed, globally unique, application-specific context that identifies the purpose, such as "example.org messenger 2026-10-27 session key". The context is public UTF-8 text, not a salt, password or secret key. Different purposes should use different contexts. The key material must already have sufficient entropy: this API is not password hashing or password stretching.

State, allocation and implementation

Each constructor makes one fallible state allocation. BLAKE3 has a larger fixed-size inline tree stack than SHA; it uses a separate object layout, so adding this algorithm does not increase SHA/HMAC state allocations. Context hashing in the derivation constructor needs no extra allocation.

update, update_text, finish_into and Mac.verify allocate nothing, including when input crosses tree-chunk boundaries. finish allocates an output buffer; one-shot functions allocate a state and output buffer. An output allocation failure leaves the state unfinished and retryable. Invalid ranges also leave it unchanged. A successful finish or a verification (including a mismatch) ends the state.

The runtime wraps the BLAKE3 team’s blake3 crate 1.8.7 rather than implementing a cryptographic primitive. Both runtime builds use its pure feature without std or Rayon: Rust SIMD implementations select SSE2, SSE4.1 or AVX2 at runtime on supported processors, with a portable fallback. There are no worker threads, assembly or AVX-512 paths in this configuration. Only the standard 32-byte output is exposed; extendable output is not exposed.

Finishing or dropping a BLAKE3 state overwrites the stored state using volatile writes. This includes keyed and derivation modes. Caller-owned keys, input buffers, result buffers and temporary upstream stack copies are not scrubbed; see key material.

The unmodified sources and licenses are vendored alongside arrayvec 0.7.8, constant_time_eq 0.4.2 and cpufeatures 0.3.1. Release notices include these Cargo dependencies. Tests use the upstream 1.8.7 known-answer vectors for all three modes and allocation-failure instrumentation for the Plenty API.

Acknowledgements

  • Wraps the BLAKE3 team’s blake3 1.8.7 implementation and uses its published test vectors (CC0-1.0 OR Apache-2.0 OR Apache-2.0 WITH LLVM-exception); code and licenses are vendored in the runtime.
  • BLAKE3’s vendored dependencies are arrayvec 0.7.8 for its inline tree stack (MIT OR Apache-2.0), constant_time_eq 0.4.2 for comparison support (CC0-1.0 OR MIT-0 OR Apache-2.0), and RustCrypto’s cpufeatures 0.3.1 for processor detection (MIT OR Apache-2.0).

Authenticated encryption, signatures, and key agreement

std.crypto provides AES-256-GCM, ChaCha20-Poly1305, Ed25519, ephemeral X25519, and HKDF-SHA256 over Bytes.

Upstream code, used as-is. These are unmodified ring primitives. Plenty implements argument validation, buffer ownership and error handling, and no cryptography: the algorithms and their side-channel resistance are deliberately left to ring. See std.crypto for this policy.

All functions return Result[_, AdvancedError]. This error is separate from CryptoError, which describes the digest/HMAC API. See the signature index for complete declarations and the encryption tutorial for runnable examples.

Authenticated encryption

AeadAlgorithm has Aes256Gcm and ChaCha20Poly1305. Both use a 32-byte secret key, a 12-byte nonce, and a 16-byte authentication tag.

seal_in_place(algorithm, key: &[u8], nonce: &[u8], aad: &[u8],
              data: &mut Bytes) -> Result[(), AdvancedError]
open_in_place(algorithm, key: &[u8], nonce: &[u8], aad: &[u8],
              data: &mut Bytes) -> Result[(), AdvancedError]

seal_in_place replaces plaintext with ciphertext followed by its tag. It reserves/grows the buffer before encrypting; reserve 16 additional bytes in advance to avoid allocation. open_in_place authenticates and decrypts that format, removing the tag on success. Associated data (aad) is authenticated but never encrypted or appended; the recipient must supply identical bytes. Empty plaintext and empty associated data are supported. These APIs cap plaintext at 68,719,476,704 bytes for both algorithms.

Never reuse a nonce with the same key. The API does not remember nonces or manage counters, persistent state, key rotation, replay protection, or message framing. Use a protocol that specifies these. Keys should come from std.secrets or an appropriate key derivation scheme.

Invalid lengths leave data unchanged. If growth fails, plaintext remains unchanged. Authentication failure zeros the entire ciphertext/tag buffer, retains its original length, and returns AuthenticationFailed. It never returns unauthenticated plaintext. Successful opening allocates nothing. The other inputs and output must be separate buffers under the loan rules.

Ed25519

FunctionOutput
ed25519_public_key(seed)A new 32-byte public key
ed25519_public_key_into(seed, target)Writes an exactly 32-byte buffer
ed25519_sign(seed, message)A new 64-byte signature
ed25519_sign_into(seed, message, target)Writes an exactly 64-byte buffer
ed25519_verify(public_key, message, signature)True for a valid signature; False for a mismatch; SmallOrderKey for a small-order key

Every input is a &[u8] view; target is &mut Bytes. A seed must be exactly 32 cryptographically random bytes. Generate one with std.secrets.token_bytes(32); its public key may be shared, but its seed must remain secret. Signing is deterministic, and a signing seed can sign multiple messages. Verification requires exactly 32 public-key bytes and 64 signature bytes; other lengths return InvalidLength, whereas correctly sized malformed keys/signatures return False. This is plain Ed25519, without a prehash or context mode. Public keys must be authenticated by the application’s protocol.

Small-order public keys

ed25519_verify and SPKI import reject a small-order public key with SmallOrderKey. Such a key is a point whose order divides 8; no private key stands behind it, so anyone can make signatures that verify under it. Under the identity point, the signature R = B, S = 1 verifies for every message. The rule covers all 14 encodings of the eight points, including the non-canonical ones (y = p and y = p + 1) and both sign bits. Verification checks lengths first, so a small-order key with a wrong-length signature is still InvalidLength. The result is an error rather than False because no answer about the signature is meaningful for such a key.

import std.bytes
import std.crypto
import std.fmt
import std.encoding
import std.report

def main() -> Result[(), IoError | AllocError | std.bytes.BytesError | std.encoding.EncodingError | std.fmt.FormatError]:
    identity = std.encoding.hex_from_text("0100000000000000000000000000000000000000000000000000000000000000")?
    forged = std.encoding.hex_from_text("58666666666666666666666666666666666666666666666666666666666666660100000000000000000000000000000000000000000000000000000000000000")?
    message = std.bytes.from_text("any message")?
    match std.crypto.ed25519_verify(&identity, &message, &forged):
        case Ok(valid):
            print(valid)?
        case Err(error):
            print(std.report.to_string(&error, 256, 4)?)?
    Ok(())
Ed25519 public key has small order: signatures can verify under it without any private key, so it is rejected

RFC 8032 permits these keys and ring accepts them, so a peer that publishes one interoperates with RFC 8032 verifiers but not with Plenty. libsodium rejects the same set.

Mixed-order keys, a prime-order point plus a small-order one, are accepted. Signing under one still needs the private key of its prime-order part, so they do not allow forgery. They do let a key holder produce signatures that verifiers with and without the cofactor in their equation judge differently. Rejecting them needs a scalar multiplication by the group order, which ring does not expose; libsodium’s verification accepts them too. The Wycheproof vectors pin both rules.

X25519

x25519() -> Result[X25519Key, AdvancedError] generates an ephemeral private key with ring’s operating-system random source. The private key has no export/import operation. An X25519Key has these methods:

MethodContract
public_key()Returns a new 32-byte buffer
public_key_into(target: &mut Bytes)Writes an exactly 32-byte buffer
agree(peer_public_key: &[u8])Consumes the key’s agreement capability and returns 32 shared-secret bytes
agree_into(peer_public_key: &[u8], target: &mut Bytes)Same, into an exactly 32-byte buffer

Both agreement methods require a mutable receiver. An agreement attempt consumes the private key even when the peer is rejected. A peer producing an all-zero shared secret is InvalidPeer; agree_into zeros its output on that failure. Later agreement or public-key requests return Finished. Invalid lengths and output allocation failure leave the key usable and existing output unchanged. Exchange public keys before calling agree. X25519Key is redacted: printing one shows std.crypto.X25519Key(@redacted).

X25519 alone does not authenticate the peer. Authenticate the exchange through a suitable protocol and pass the shared secret through HKDF with protocol-specific salt and context before using it as a symmetric key.

HKDF-SHA256

hkdf_sha256(salt: &[u8], key_material: &[u8], info: &[u8], size: NonNegative[i64])
    -> Result[Bytes, AdvancedError]
hkdf_sha256_into(salt: &[u8], key_material: &[u8], info: &[u8],
                 target: &mut Bytes) -> Result[(), AdvancedError]

These perform HKDF extract followed by expand. Salt and context (info) can be empty. Output length is 0 through 8160 bytes inclusive. hkdf_sha256 traps for a larger size; the _into form uses the existing target length and returns InvalidLength for one over 8160, leaving the output unchanged. Use context to separate keys with different purposes. HKDF is not password hashing and does not make a weak password into a strong key.

Errors, allocation, and key material

AdvancedErrorMeaning
InvalidLengthIncorrect key/nonce/tag/output length or output beyond the supported bound
AuthenticationFailedAEAD authentication failed
InvalidPeerX25519 rejected the peer key
FinishedX25519 key already used for an agreement attempt
RandomFailureOS entropy or X25519 key/public-key generation failed
InvalidKey, WeakKeyRaised only by RSA, ECDSA and key import
SmallOrderKeyAn Ed25519 public key has small order
Allocation(AllocError)A fallible buffer or private-state allocation failed

AdvancedError satisfies std.fmt.Formattable: each variant formats as a fixed sentence stating its cause, without key material. An entrypoint report uses it when main returns a bare AdvancedError; an error sum uses it only when every member is formattable.

The _into functions, verification, and opening allocate nothing. Sealing allocates only if growth exceeds capacity. Functions returning Bytes allocate their result. X25519 creation fallibly allocates its state and a short-lived 32-byte public-key buffer to validate generation before returning. Failure at any of these allocation points is reported as Allocation.

The chosen ring paths use fixed-size stack key objects, signatures, tags, and HKDF state. They do not use its allocating PKCS#8 or append-tag APIs. Tests enforce the allocation-free paths with the runtime’s allocation guard. The X25519 state’s private-key slot is overwritten with volatile zero writes when consumed or dropped. This does not scrub copies in registers or on the stack, caller-owned seeds/keys/shared-secret buffers, or allocator caches. Bytes does not zero its storage on drop; see key material.

There are no password KDFs, streaming AEAD, or persistent private-key stores in this API. Ed25519 keys can be read from PKCS #8 and SubjectPublicKeyInfo with the key import functions. It supports the project’s x86-64 Linux GNU target. Runtime entry symbols keep unused primitive paths eligible for linker garbage collection; ring is also used by std.tls.

Arithmetic bounds

AEAD validates protocol lengths before adding or removing its 16-byte tag. These bounded operations cannot overflow; existing invalid-length, allocation, and authentication errors remain unchanged, including validation and allocation before encryption mutates plaintext.

Acknowledgements

  • Wraps ring 0.17.14 (Apache-2.0 AND ISC), already vendored for TLS, for all primitives on this page. Its complete license text and third-party notices are included by the release notice generator through the Cargo dependency graph.
  • RFC 8032 supplies the Ed25519 known-answer test.
  • RFC 5869 supplies the HKDF-SHA256 known-answer test.
  • RFC 8439 supplies the ChaCha20-Poly1305 known-answer test.
  • The published GCM test cases supply the AES-256-GCM known-answer test.

RSA and ECDSA signatures and key import

std.crypto signs and verifies with RSA (PKCS #1 v1.5 and PSS) and ECDSA over P-256, and imports public and private keys from the standard DER encodings, JWK components, and (through std.encoding) PEM. These are the signature schemes of JOSE and JWT: RS256, PS256 and ES256, with the SHA-384 and SHA-512 RSA variants. Ed25519 keys import here too and sign with the Ed25519 functions.

Upstream code, used as-is. Signing, verification and private-key parsing are unmodified ring primitives; Plenty implements no cryptographic arithmetic. Plenty parses public-key encodings and checks RSA key sizes itself, in checked code, so a public key reaches ring only after its structure is validated. See std.crypto for this policy. Every function returns Result[_, AdvancedError]. See the signature index for complete declarations.

import std.bytes
import std.crypto
import std.encoding
from std.bytes import BytesError
from std.crypto import AdvancedError
from std.encoding import EncodingError

def main() -> Result[(), IoError | AllocError | BytesError | AdvancedError | EncodingError]:
    # The ES256 example from RFC 7515 appendix A.3: a JWK's x and y, the
    # signing input, and its fixed 64-byte signature.
    x = std.encoding.base64url_from_text("f83OJ3D2xF1Bg8vub9tLe1gHMzV76e8Tus9uPHvRVEU")?
    y = std.encoding.base64url_from_text("x_FEzRu9m36HLN_tue659LNpXW6pCyStikYjKIWI5a0")?
    key = std.crypto.p256_public_key_from_coordinates(&x, &y)?
    mut message = std.bytes.from_text("eyJhbGciOiJFUzI1NiJ9.eyJpc3MiOiJqb2UiLA0KICJleHAiOjEzMDA4MTkzODAsDQogImh0dHA6Ly9leGFtcGxlLmNvbS9pc19yb290Ijp0cnVlfQ")?
    signature = std.encoding.base64url_from_text("DtEhU3ljbEg8L38VWAfUAqOyKAM6-Xx-F4GawxaepmXFCgfTjDxw5djxLa8ISlSApmWQxfKTUJqPP3-Kg6NU1Q")?
    print(key.verify(&message, &signature)?)?

    message.set(0, 0u8)
    print(key.verify(&message, &signature)?)?
    Ok(())
True
False

Key types

Each key type is an opaque class for one algorithm. There is no conversion between them, no way to read a private key’s material back out, and no way to use one as an HMAC key or Ed25519 seed. The constructors are private: a key exists only after an import function validated it. Keys cannot be copied. Private keys are redacted: printing one shows std.crypto.RsaPrivateKey(@redacted) or std.crypto.P256PrivateKey(@redacted).

TypeHoldsMethods
RsaPublicKeyA 2048- to 8192-bit modulus and its exponentsignature_size(), verify(scheme, message, signature)
RsaPrivateKeyA 2048- to 4096-bit two-prime key, held by the runtimesignature_size(), public_key(), sign(scheme, message), sign_into(scheme, message, target)
P256PublicKeyA point on P-256verify(message, signature)
P256PrivateKeyA P-256 scalar and its public point, held by the runtimepublic_key(), sign(message), sign_into(message, target)

RsaScheme selects the RSA padding and hash for each call:

RsaSchemeJOSE algPadding
Pkcs1Sha256, Pkcs1Sha384, Pkcs1Sha512RS256, RS384, RS512PKCS #1 v1.5 (RFC 8017 section 8.2)
PssSha256, PssSha384, PssSha512PS256, PS384, PS512PSS with MGF1 over the same hash and a salt of the hash length (RFC 7518 section 3.5)

P-256 keys use ECDSA with SHA-256 only (JOSE ES256). Choose the scheme from the key’s configured algorithm, never from an unauthenticated message header.

Import

FunctionInputResult
rsa_public_key_from_spki(der)SubjectPublicKeyInfo with rsaEncryption and NULL parametersRsaPublicKey
rsa_public_key_from_components(n, e)Big-endian modulus and exponent, as a JWK’s decoded n and eRsaPublicKey
rsa_private_key_from_pkcs8(der)Unencrypted PKCS #8 v1 holding an RSAPrivateKeyRsaPrivateKey
p256_public_key_from_spki(der)SubjectPublicKeyInfo with id-ecPublicKey and the named curve prime256v1P256PublicKey
p256_public_key_from_point(point)A 65-byte uncompressed SEC1 point: 0x04, x, yP256PublicKey
p256_public_key_from_coordinates(x, y)32-byte big-endian coordinates, as a JWK’s decoded x and yP256PublicKey
p256_private_key_from_pkcs8(der)Unencrypted PKCS #8 v1 holding an ECPrivateKey with its public keyP256PrivateKey
ed25519_public_key_from_spki(der)SubjectPublicKeyInfo with id-Ed25519; a small-order key is SmallOrderKeyThe 32-byte public key, as Bytes
ed25519_seed_from_pkcs8(der)Unencrypted PKCS #8 v1 or v2 holding an Ed25519 private keyThe 32-byte seed, as Bytes

Every input is &Bytes. These are the forms openssl genpkey and openssl pkey -pubout write. For PEM, decode with std.encoding.pem_decode(source, b"PUBLIC KEY") or b"PRIVATE KEY" first; the label must match. Encrypted PKCS #8, the PKCS #1 RSA PRIVATE KEY and SEC1 EC PRIVATE KEY forms, and X.509 certificates are not accepted.

DER is parsed strictly: lengths must be definite and minimal, integers minimal and positive, and nothing may follow the outer element. The algorithm identifier must be exactly the expected encoding, so another algorithm, another curve, explicit curve parameters, or RSA-PSS-restricted keys (id-RSASSA-PSS) are rejected. Public points must be uncompressed and on the curve. RSA public keys must have an odd modulus of 2048 to 8192 bits without a leading zero byte and an odd exponent from 3 to 233 - 1; private keys are limited by ring to 2048 to 4096 bits, an exponent of at least 65537, and two primes, and their components must be consistent. A P-256 PKCS #8 key must include its public key, which must match the scalar. An Ed25519 v2 key may include its public key, which must match the seed.

Validating a public point runs one ephemeral P-256 key agreement, because ring has no separate point check; it needs operating-system randomness and can fail with RandomFailure.

Signing and verification

verify returns True for a valid signature and False for any signature that does not verify, including malformed values of the right length. Only misuse is an error: a signature whose length is wrong for the key is InvalidLength. An RSA signature is exactly signature_size() bytes, the modulus length. An ECDSA signature is the fixed 64-byte r || s form that JOSE uses (RFC 7518 section 3.4), so a DER-encoded ECDSA signature never verifies; it is InvalidLength.

sign returns a new buffer; sign_into writes a target of exactly the signature size and otherwise returns InvalidLength without writing. PKCS #1 v1.5 signatures are deterministic. PSS draws a random salt and ECDSA a random nonce (hedged by ring with the key and message), so repeated signatures differ, and both can fail with RandomFailure. A PSS failure zeros the target.

Errors

AdvancedErrorMeaning here
InvalidKeyA key encoding is malformed, uses another algorithm or curve, or breaks a size or consistency rule other than the minimum
WeakKeyAn RSA modulus is shorter than 2048 bits
SmallOrderKeyAn Ed25519 public key has small order
InvalidLengthA signature or signing target has the wrong length for the key
RandomFailureOperating-system randomness failed during point validation, PSS or ECDSA signing, or private-key import
Allocation(AllocError)A fallible buffer or key-state allocation failed, or memory for an RSA operation could not be reserved

These extend the existing AdvancedError rather than adding a type, because the operations are the same family as Ed25519 and share its length, entropy and allocation failures; one error type keeps a JWT verifier’s error set small. InvalidKey and WeakKey carry no payload: no offset, field name or value, so an error message cannot reveal anything about a private key.

Allocation and key material

Each import allocates its result fallibly: a public key’s validated DER or point, a private key’s runtime state object, or a returned Ed25519 key or seed. rsa_public_key_from_components builds the DER from n and e. P-256 signing into an existing target and P-256 verification allocate nothing; tests enforce this with the runtime’s allocation guard, as they do for ring’s P-256 private-key parsing and point validation.

ring’s RSA operations allocate big-number buffers through the Rust global allocator, which cannot report failure. Before each one the runtime reserves a fixed block, then runs ring with that thread’s allocations taken from it:

OperationReservedWhenReleased
rsa_private_key_from_pkcs812 KiBbefore parsingwhen the key is dropped; at once if parsing fails
RsaPrivateKey.sign, sign_into8 KiBeach callwhen the call returns
RsaPublicKey.verify12 KiBeach callwhen the call returns

signature_size and public_key reserve nothing; public_key allocates only its result. Each size is at least twice the most that ring 0.17.14 uses for the largest key it accepts: a 4096-bit private modulus or an 8192-bit public one. ring makes a fixed number of buffers per operation, each at most 1 KiB, so the use does not depend on the message or the key’s value. A test measures it and fails if a ring update outgrows a block.

If the block cannot be reserved, or ring asks for more than it holds, the call returns Allocation. A failure inside ring abandons its frames, which hold only the block and the stack, so nothing else needs repair; the block is then released. The key, the target and the program stay usable; a failed signature leaves the target zeroed. The blocks come from the current heap: in an actor handler they count against the actor’s heap limit.

A private key’s runtime state is overwritten with volatile zero writes when it is dropped. For P-256 that covers the scalar and nonce key, which ring keeps inline. For RSA, the key’s reserved block, which holds the primes, exponents and coefficients, is zeroed before it is freed, as is each call’s block. A key may be used, and dropped, on any thread. Not scrubbed:

  • the caller’s DER or PEM input buffers, and the intermediate base64 and decoded buffers that std.encoding.pem_decode releases;
  • the seed returned by ed25519_seed_from_pkcs8, which belongs to the caller (on rejection, the function zeros its own copy);
  • copies ring makes in registers or on the stack while it runs.

Bytes does not zero its storage on drop; see key material.

Test vectors

Every valid and invalid case in these Wycheproof files gives the expected result, with the key imported through each available form (SPKI, JWK members, and the raw point or key):

FileCasesResult
rsa_signature_2048_sha256, _2048_sha384, _2048_sha512, _3072_sha256, _4096_sha512258 to 259 eachAll pass
rsa_pss_2048_sha256_mgf1_32, _2048_sha384_mgf1_48, _4096_sha512_mgf1_64108, 141, 179All pass
ecdsa_secp256r1_sha256_p1363262All pass
ecdsa_secp256r1_sha256 (DER signatures)484None verifies: each is InvalidLength
ed25519210All pass

Cases marked acceptable are pinned to the observed behavior:

  • The one PKCS #1 v1.5 case per file whose DigestInfo omits the NULL parameters (MissingNull) is rejected.
  • The 8 Ed25519 cases with small-order public keys (tcId 152 to 159) are SmallOrderKey, from SPKI import and from verification. The 21 with mixed-order keys (160 to 180) verify. See small-order public keys.

The extracts and their generator, with the upstream commit and file digests, are in tests/fixtures/crypto/wycheproof.

Limits

  • No key generation for RSA or P-256, and no private-key export.
  • No P-384, P-521, secp256k1, or ECDSA with DER signatures.
  • No encrypted PKCS #8, PKCS #1 or SEC1 private-key forms, or certificates.
  • No JWK parsing: decode the JSON and base64url members and pass the bytes.
  • No RSA encryption.
  • One target, x86-64 Linux GNU.

Arithmetic bounds

DER cursors stay within validated ordered buffer spans; length encodings use at most three bytes. RSA import rejects moduli longer than 1024 bytes before bit counting, and bounds exponents to five bytes. Consequently bit counts and DER output extents cannot overflow. Small arithmetic opt-outs preserve InvalidKey and WeakKey rather than exposing machine-arithmetic errors for malformed keys.

Acknowledgements

  • Wraps ring 0.17.14 (Apache-2.0 AND ISC), already vendored for TLS, for RSA and ECDSA signing, verification, and private-key parsing.
  • Project Wycheproof (C2SP/wycheproof, Apache-2.0) supplies the RSA, ECDSA and Ed25519 test vectors.
  • RFC 7515 appendices A.2 and A.3 supply the RS256 and ES256 known-answer tests, and RFC 8037 appendix A the Ed25519 one.
  • Encodings follow RFC 5280 (SubjectPublicKeyInfo), RFC 5208 and RFC 5958 (PKCS #8), RFC 5480 and RFC 5915 (elliptic-curve keys), RFC 8410 (Ed25519 keys), and RFC 8017 (RSA).

std.csv

import std.csv reads and writes delimited records over std.io byte streams. A Reader fills one reusable Record per call from a fixed buffer. A Writer encodes one record at a time. Neither loads a whole file, and every size is bounded by the Dialect.

The module is checked Plenty over std.bytes and std.io, with no native calls of its own. It is available on the supported x86-64 Linux GNU target.

import std.bytes
import std.csv
import std.io
from std.csv import Newline

def main() -> Result[(), AllocError | IoError | ParseError | std.bytes.BytesError | std.csv.CsvError | OverflowError[i64]]:
    text = "item,price\r\npen,3\r\n\"book, large\",12\r\n"
    mut input = std.csv.reader(std.io.cursor(std.bytes.from_text(text)?), &std.csv.rfc4180(), 4096)?
    mut record = std.csv.record()?
    input.read(&mut record)?
    mut total = 0
    while input.read(&mut record)?:
        total += i64.parse(&record.text(1)?)?
    mut dialect = std.csv.rfc4180()
    dialect.newline = Newline.Lf
    mut output = std.csv.writer(std.io.stdout(), &dialect)?
    output.texts(&["rows", "total"]?)?
    output.texts(&[str.from(input.records() - 1)?, str.from(total)?]?)?
    output.flush()?
    Ok(())
rows,total
2,15

Format

The grammar is RFC 4180 with a configurable delimiter and quote byte. There is one accepted form; nothing is trimmed, skipped, or repaired.

  • A record ends at LF, CRLF, or CR. The last record may end at end of input.
  • Fields are separated by the delimiter. a, has two fields, the second empty.
  • An empty line is one record with one empty field. Empty input has no records.
  • A field that starts with the quote byte is quoted and ends at the next unpaired quote. Inside it, the delimiter, CR, and LF are data, and two quotes are one quote byte.
  • A closing quote must be followed by the delimiter, a record end, or end of input.
  • A quote byte anywhere else in an unquoted field is an error.
  • Spaces are data, including those next to a delimiter or quote.

There are no headers, comments, or escape characters other than the doubled quote. The first record is an ordinary record.

Dialect

Dialect is a class with public fields. std.csv.rfc4180() returns the values in the third column; change fields before creating a reader or writer.

FieldTyperfc4180()Meaning
delimiteru844u8 (,)Field separator
quoteu834u8 (")Quote byte
newlineNewlineCrLfRecord end a Writer emits: Lf or CrLf
quotingQuotingNeededNeeded quotes a field that contains the delimiter, the quote, CR, or LF; Always quotes every field
widthWidthFirstRow-width policy, below
skip_bomboolFalseA Reader discards a leading UTF-8 byte-order mark
max_fieldi6465536Most bytes in one unescaped field
max_recordi641048576Most bytes in the unescaped fields of one record
max_fieldsi644096Most fields in one record

delimiter and quote must be different ASCII bytes other than CR and LF, each limit at least one, and Width.Exactly(count) at least one. Otherwise reader and writer return InvalidDialect. A reader accepts all three record ends whatever newline says, and ignores quoting.

WidthPolicy
AnyEach record keeps its own field count
FirstEvery record has the field count of the first
Exactly(count)Every record has count fields

Fields are bytes. The module never decodes them on its own: Record.text validates one field as UTF-8 when asked, and a writer emits the bytes it is given. Other encodings are read with Record.bytes and converted by the program.

Errors

CsvErrorMeaning
Invalid(Fault, Position)A reader found malformed or oversized input
Rejected(Fault, record, field)A writer refused a field that exceeds a limit
Width(record, expected, found)A record’s field count breaks the width policy
EmptyRecord(record)A writer was asked to end a record with no fields
InvalidUtf8(record, field)Record.text found bytes that are not UTF-8
InvalidDialectThe dialect breaks the rules above
StoppedThe reader already failed
Stream(StreamError)The inner stream failed
Allocation(AllocError)A record or buffer could not grow
FaultReaderWriter
BareQuoteA quote inside an unquoted field
AfterQuoteData between a closing quote and the next delimiter or record end
UnterminatedQuoteInput ended inside a quoted field
FieldTooLongA field exceeds max_fieldThe same
RecordTooLongA record exceeds max_recordThe same
TooManyFieldsA record exceeds max_fieldsThe same

Position locates the byte a reader stopped at:

FieldCounts fromMeaning
record1The record being read; matches Record.number()
field0The field being read; matches the index Record.text takes
line1The physical line, counting record ends and line breaks inside quotes; CRLF counts once
offset0The byte offset in the stream; for UnterminatedQuote, the input length

A limit fault points at the first byte that does not fit. Record numbers and field indexes in the other variants use the same bases.

Reading

FunctionResult
reader(inner: R, dialect: &Dialect, capacity: Positive[i64])Result[Reader[R], CsvError]
record()Result[Record, CsvError]

reader takes ownership of inner, copies the dialect, and allocates one buffer of exactly capacity bytes, which never grows. capacity is Positive[i64], so 0 is a compile error. inner needs the Reader method of std.io. The reader does its own buffering, so inner needs no BufReader.

Reader[R]ResultContract
read(record: &mut Record)Result[bool, CsvError]Replace record with the next record; False is end of input
records()i64Records returned so far
line()i64The current line, from one
offset()i64Bytes consumed from the stream

read retries Interrupted and continues across short reads; a record may span any number of buffer fills. After False, the record is empty and every later call returns False without reading the stream again.

Record, line and byte positions are semantic i64 counters, not saturating diagnostics. If the next position is unrepresentable, reading returns Stream(StreamError.OutOfRange), empties the record and stops the reader. Record number i64.MAX is still processable when the preceding count is i64.MAX - 1. A failing line increment leaves the parser cursor before that newline; CRLF still counts as one line, inside and outside quotes. The inner stream may already have read ahead: parser consumption is distinct from transport progress. Buffer fills are capped by the remaining representable byte span; at zero room, a reader can refuse before an EOF probe.

A Width error is reported after the whole record has been read: record holds it, and the next read continues with the following record. Every other error empties record and stops the reader; later calls return Stopped. A stopped reader cannot resynchronize, because a malformed quote leaves no reliable record boundary.

inner is a public field, for calls such as input.inner.close(). Reading it directly skips bytes the reader already holds.

Records

A Record stores the unescaped fields end to end in one buffer and one list of field ends. A reader overwrites it, keeping its capacity.

RecordResultContract
len()i64Number of fields
number()i64The record’s number from one; zero before the first read
line()i64The line on which the record started
text(index)Result[str, CsvError]An owned copy of one field, validated as UTF-8
texts()Result[list[str], CsvError]An owned list of every field as text
bytes(index)Result[Bytes, CsvError]An owned copy of one field
append_to(index, out: &mut Bytes)Result[(), CsvError]Append one field to the caller’s buffer
data()&BytesEvery field’s bytes, borrowed
start(index), stop(index)i64Field index is data()[start:stop]
clear()()Remove the fields; number() and line() are kept

An index outside 0 <= index < len() traps before reading or appending. The message names Record.start or Record.stop and index outside record; bytes, text, and append_to validate through those position methods.

data() is the borrowed view. It and the offsets describe the current contents only. Reader.read needs &mut Record, so the compiler rejects a program that still holds the view across the next read. text, texts, bytes, and append_to return or fill storage the caller owns, which survives any number of reads.

Writing

FunctionResult
writer(inner: W, dialect: &Dialect)Result[Writer[W], CsvError]

writer takes ownership of inner, which needs the Writer methods of std.io, and copies the dialect.

Writer[W]ResultContract
field(source: &[u8])Result[(), CsvError]Add source as the next field
text(value: &str)Result[(), CsvError]Add the UTF-8 bytes of value as the next field
end_record()Result[(), CsvError]Finish the record in progress and deliver it
record(record: &Record)Result[(), CsvError]Add every field of record, then end the record
texts(fields: &list[str])Result[(), CsvError]Add every text, then end the record
discard()()Drop the record in progress
flush()Result[(), CsvError]Deliver held bytes, then flush the inner stream
records()i64Records finished so far
pending()i64Bytes of finished records the stream has not accepted

A record in progress lives in the writer’s buffer and reaches the stream only at end_record, as one run of writes that retries Interrupted and short counts. A record with one empty field is written as two quote bytes, so that it is not an empty line. end_record with no fields is EmptyRecord.

Failures leave a defined state:

  • Rejected and Allocation from field or text add nothing; the record in progress keeps its earlier fields.
  • Width from end_record discards the record in progress. Nothing of it was written. With Width.First, the first finished record sets the count.
  • A transport Stream error from end_record means the record is finished and counted, and its undelivered bytes are held. The next field, end_record, or flush delivers them first and fails again if the stream still refuses.
  • record and texts discard the record in progress on any error. A Stream error still leaves the finished record held.

Position exhaustion is distinct from that ordinary transport retry: after record i64.MAX is finished, field, text and end_record reject an unrepresentable next record as Stream(StreamError.OutOfRange) before encoding or implicitly draining any pending output. Already-finished pending bytes and their count remain intact; explicit flush still delivers them. No counter wraps, saturates, or adds a public arithmetic error type. Internal field/content and pending-buffer arithmetic uses validated remaining budgets and held-buffer bounds; scan endpoints clamp room before adding it to a cursor.

Nothing is written when a Writer is dropped. End the last record and call flush. The writer does not close inner; it is the public field inner.

Ownership and threads

A reader or writer owns its stream and has one user at a time: every operation that moves data takes &mut self. A Record is an ordinary owned value, independent of the reader that filled it, and can be moved to another thread. The module holds no global state and adds no synchronization.

Costs

Reading examines each input byte once and copies field bytes into the record in runs. Writing examines each field byte once to decide quoting, and once more when quoting to double quote bytes. Both are linear in the bytes processed. Field access by index is constant time.

The reader and writer find the next delimiter, quote, or line break with the std.bytes scans, so the work in Plenty code is per field and per quote, not per byte. Measured with a release compiler on x86-64 Linux, pinned to one core, reading 64 MB from standard input into one reused Record:

InputBefore the scansNow
3,234,608 fields of about 19 bytes, some quoted4.73 s (14 MB/s)2.00 s (32 MB/s)
376,760 fields of about 180 bytes, some quoted3.37 s (19 MB/s)0.30 s (213 MB/s)

Short fields are bounded by the cost of the calls made per field, about 600 ns, not by scanning.

AllocatesDoes not allocate
reader: one buffer; record, writer: empty storageReader.read once the record’s capacity covers the record
Reader.read, when a record outgrows the RecordRecord.append_to within the capacity of out
Record.text and bytes: one copy; texts: one list and a copy per fieldRecord.data, start, stop, len, clear
Writer.text: one temporary copy of the textWriter.field, record, end_record, flush within the writer’s capacity
Writer.field, when a record outgrows the writer’s bufferWriter.discard, records, pending

A Record holds at most max_record bytes of content and max_fields offsets. A reader consumes at most 2 * max_record + 3 * max_fields + 2 bytes for one record before returning it or a limit fault. A writer holds at most one finished record that the stream refused and one record in progress, each at most that many bytes. Storage grows geometrically, so capacity can be up to twice the content.

Limits

  • Only x86-64 Linux GNU is supported.
  • There is no lenient mode: stray quotes, and data after a closing quote, are errors, and a reader does not continue after them.
  • There are no header mapping, typed columns, comment lines, or whitespace trimming. Convert fields with the type’s parse method.
  • Only the UTF-8 byte-order mark is recognized, and only with skip_bom. A writer never emits one. Other encodings need conversion by the program.
  • delimiter and quote are single ASCII bytes.
  • A stream error that could be retried, such as EAGAIN on a nonblocking descriptor, stops the reader like any other.
  • End of input is final: a reader does not look for data appended later.
  • Writer.text copies the text once to reach its bytes. Use field with a reused Bytes to avoid the copy.

std.dataframe

Polars-backed analytical dataframes, expression plans and CSV/Parquet files. The optional, separately compiled engine is reused by every Plenty program. See installation and deployment and the tutorial. The daily sales report application combines ingestion, checked catalogue joins, row quarantine and typed exports.

Use clist[T] for a lightweight mutable columnar collection with a compile-time class schema and Plenty arithmetic. Use std.dataframe for analytical queries and file-backed datasets with runtime schemas. Neither replaces ordinary lists.

import std.dataframe as df

def main() -> Result[(), df.DataFrameError | AllocError | IoError]:
    data = df.frame([
        df.column_str("station", ["north", "south", "north", "south"]?)?,
        df.column_f64("reading", [12.0, 30.0, 18.0, 24.0]?)?
    ]?)?
    recent = df.col("reading")?.gt(df.lit(15.0)?)?
    keys = [df.col("station")?]?
    averages = [df.col("reading")?.mean()?.alias("average")?]?
    result = data.lazy()?.filter(recent)?.group_by(keys, averages)?.sort(
        [df.SortKey(df.col("station")?)]?
    )?.collect()?
    print(result.values_str("station")?)?
    print(result.values_f64("average")?)?
    Ok(())
["north", "south"]
[18.0, 27.0]

Construction and extraction

column_i64(name: &str, values: &list[i64]) and column_f64(name: &str, values: &list[f64]) return Result[Column, DataFrameError]. Each makes one native call and copies the borrowed numeric values into a native contiguous column. Names are copied; inputs remain usable. No input pointer survives the call.

frame(columns: &list[Column]) -> Result[DataFrame, DataFrameError] shares the columns’ native buffers. Duplicate names and unequal lengths fail. An empty column list creates a zero-row, zero-column frame. Zero-length typed columns retain their types. This engine supports at most 2**32 - 1 rows.

DataFrame.height() returns Result[i64, DataFrameError]. get_i64(name, row) and get_f64(name, row) return Result[Option[i64], DataFrameError] and Result[Option[f64], DataFrameError]. Bounds, missing names and mismatched types fail; a valid null cell returns Ok(Nothing).

values_i64(name) and values_f64(name) return a Plenty list in a Result. They allocate initialized Plenty storage, then copy the whole column in one native call. A column containing any nulls fails with DataFrameError.Null; use values_i64_nullable or values_f64_nullable to preserve missing values, or fill_null when replacement is appropriate. Strings, booleans, binary payloads, nullable input/output and schema inspection are described in Column types and schemas. No implicit casts or zero-copy borrowing occur at the Plenty boundary. Logical Date and Datetime columns add checked temporal interchange, parsing, formatting and calendar expressions.

Expression plans

Every operation below returns a Result[..., DataFrameError]. The signature index gives exact declarations.

OperationResult value
col(name), lit(f64), lit_i64(i64), lit_str(text), lit_bool(value), lit_binary(payload: &[u8]), null_f64()Expr
when(predicate, then_value, otherwise), expr.cast(dtype, strict=True)Expr
expr.str_lower/str_upper/str_len(), str_contains/str_starts_with/str_ends_with(pattern)Expr
expr.n_unique(), expr.over(partition_by)Expr
expr.add/sub/mul/div/gt/eq/lt/ge/le(&other)Expr
expr.and_(&other), or_(&other), fill_null(&other)Expr
expr.sum/count/mean/min/max/is_null()Expr
expr.alias(name)Expr
frame.lazy(), scan_parquet(path)Query
query.filter(&predicate)Query
query.with_columns(&expressions), select(&expressions)Query
query.group_by(&keys, &aggregates)Query
query.join(&other, &left_keys, &right_keys, how=JoinKind.Inner, suffix="_right", nulls_equal=False)Query
query.join_asof(&other, "key", "key", strategy=AsOfStrategy.Backward, tolerance=Nothing, allow_exact_matches=True, suffix="_right")Query
query.sort(&keys), slice(offset, length), limit(count)Query
query.collect(), collect_streaming()DataFrame
scan_csv(path, ...)Query
frame.write_csv(path, ...), query.sink_parquet(path)()
frame.write_parquet(path)()

See expression contracts for conditional evaluation, explicit casts, string operations, distinct counts and partitioned windows.

Expressions describe native Polars operations; they are not opaque Plenty callbacks. Polars can optimize a complete plan, push projections/filters into Parquet scans, and use its worker pool. A scan is lazy; errors can arise during construction or collection. with_columns replaces matching names or adds new columns. Expressions in one call see the input schema; chain calls when a later expression depends on a newly derived column. Selection projects or aggregates. Grouping supports multiple keys and aggregates, but does not promise group order. Add sort to the plan when an application needs a particular order.

Owners and plans are immutable through this public API. Methods borrow their receivers; resulting plans clone native handles and share buffers. Dropping the original columns, frame or expressions does not invalidate a derived query. Each handle is freed by the engine that allocated it. There are no public raw pointers or user-created native handles.

Joins, ordering and row windows

For sorted integer keys, as-of joins attach the previous, next or nearest right row with an optional distance bound. Unlike equality joins, they produce at most one match for each left row.

These operations return new immutable query plans; neither input is consumed or mutated. They execute inside Polars, with no Plenty row callbacks or intermediate conversion into Plenty lists. Input owners, key expressions and sort specifications may be dropped after constructing the resulting plan.

JoinKind offers Inner, Left, Full, Semi and Anti equality joins. Inner/left/full retain both sets of key columns; semi/anti are membership queries returning only left columns. See equality join contracts for duplicate/null matching, key retention, suffix collisions and validation.

SortKey(expression: Expr, descending: bool = False, nulls_last: bool = True) owns its expression; constructing the wrapper itself is infallible. query.sort(keys: &list[SortKey]) orders lexicographically, with the first key most significant. Every key has independent direction and null placement. Null placement is independent of direction: nulls_last=True puts nulls last even in descending order. NaN follows Polars value ordering. Equal keys retain their input order; that does not make ties from an unordered join/group deterministic. Include an additional identifying key when that matters. An empty sort list fails with Schema.

slice(offset: i64, length: i64) selects a row window. The second argument is a count, unlike Plenty list slice(start, stop). Negative offsets count from the end. The requested interval is intersected with the available rows: slice(-2, 10) keeps the last two rows, while slice(-100, 2) on a three-row frame is empty. Offsets beyond the end also produce empty frames. length must be in 0..4294967295 inclusive; negative or larger lengths fail immediately with DataFrameError.Bounds. limit(count) is slice(0, count), with the same length checks. Sort first when a page or top-N result needs a defined ordering.

See CSV and streaming I/O for file options, direct sinks, create-new output behavior and buffering limits.

Semantics and limits

Polars 0.55.2 semantics apply inside expressions, not Plenty scalar semantics. Schemas are checked at runtime. Input/extraction supports all Plenty numeric widths (i8/i16/i32/i64, u8/u16/u32/u64, f32/f64), str, bool, Binary (std.bytes.Bytes), Date and naive/UTC Datetime, including nullable columns. Lists and structs have column-oriented construction and extraction, native access expressions and structured schema inspection. Other unexposed native types may pass through Parquet queries and appear in schema inspection, but cannot be extracted through this API. Paths are UTF-8 strings. There is no automatic lambda translation, mutation API, cloud credential configuration or stable serialization of plans.

Null is missing data, distinct from floating-point NaN. Comparisons with null yield null, which filtering drops. Polars orders NaN above finite numbers: NaN > 50.0 may keep a row that a Plenty scalar comparison would drop. Aggregations generally skip nulls; count counts non-null values and returns i64. Floating-point sums may differ with execution order and parallelism. Group order is unspecified. Native integer arithmetic and reductions are not Plenty’s checked arithmetic: overflow must not be assumed to return a recoverable error. Validate input ranges where needed.

DataFrameError retains native messages in Schema, Query, Io, NativePanic, Null, and Bounds. Polars errors, including some file errors, use Query; direct filesystem errors use Io.

Offset-table lengths and packed string byte totals are checked on the Plenty side; an unrepresentable extent returns DataFrameError.Allocation(AllocError.CapacityOverflow) before allocating the affected table or advancing the total. This does not change the native engine’s row limit or its allocation-failure policy. Copying a native error message can itself fail with Allocation(AllocError) or Message(IoError). Plenty-side list allocation is recoverable. Native Rust/Polars allocation failure can abort the process. Caught Rust panics become errors where unwinding is available; this does not make arbitrary native failures recoverable.

POLARS_MAX_THREADS configures the engine’s global pool before first use. It is independent of Plenty executors. Avoid oversubscribing both pools when running many analytical queries concurrently.

Acknowledgements

  • Wraps Polars 0.55.2, built into the optional dataframe engine library (MIT).

Dataframe column types and schemas

The optional dataframe engine supports these boundary types. Construction and extraction return Result[..., DataFrameError]; names and dtypes are checked at runtime. No implicit cast is performed.

import std.dataframe as df

def main() -> Result[(), df.DataFrameError | AllocError | IoError]:
    labels: list[Option[str]] = [Some("low"), Nothing]?
    data = df.frame([
        df.column_u8("level", [3u8, 200u8]?)?,
        df.column_str_nullable("label", labels)?
    ]?)?
    for field in data.schema()?:
        print(field.name, field.dtype)?
    print(data.values_u8("level")?)?
    print(data.values_str_nullable("label")?)?
    print(data.get_str("label", 1)?)?
    Ok(())
level std.dataframe.DType.U8
label std.dataframe.DType.String
[3, 200]
[Option[str].Some("low"), Option[str].Nothing]
Option[str].Nothing
Column dtypeNon-nullable constructorNon-nullable extraction
i8, i16, i32column_i8/i16/i32(name, values)frame.values_i8/i16/i32(name)
i64column_i64(name, values)frame.values_i64(name)
u8, u16, u32, u64column_u8/u16/u32/u64(name, values)frame.values_u8/u16/u32/u64(name)
f32column_f32(name, values)frame.values_f32(name)
f64column_f64(name, values)frame.values_f64(name)
strcolumn_str(name, values)frame.values_str(name)
boolcolumn_bool(name, values)frame.values_bool(name)
Binary (std.bytes.Bytes)column_binary(name, values)frame.values_binary(name)

Logical Date and Datetime columns have separate checked constructors/getters with explicit epoch-day or timestamp-unit interchange.

Constructors borrow list[T]; extraction returns an owned list[T]. All scalar types also have column_TYPE_nullable(name, &list[Option[T]]) and frame.values_TYPE_nullable(name) -> Result[list[Option[T]], DataFrameError]. Use these when nulls are meaningful. Non-nullable extraction fails with DataFrameError.Null if any cell is null, even if the column was originally constructed without nulls and a later join introduced them.

get_TYPE (for each numeric type, str, bool and binary) accepts a column name and zero-based row index and returns Result[Option[T], DataFrameError]. A null cell is Ok(Nothing); an unknown name, wrong dtype or invalid index is an error. Empty and all-null typed columns retain their declared dtype. Native columns support at most 2**32 - 1 rows. String input currently has the slightly lower limit 2**32 - 2: its offset buffer needs one entry beyond the number of rows. Binary input has the same offset limit and currently accepts at most 2**32 - 1 packed payload bytes per call.

Numeric input, extraction and Parquet round trips preserve signedness and width. In particular, u64.MAX never passes through an i64 or a float. Extraction requires the exact dtype: use an explicit expression cast when conversion is intended. Native operations can change a result dtype (for example, a reduction may widen a narrow integer); inspect the result schema and choose its matching getter. Polars arithmetic and overflow semantics still apply inside queries.

lit_i8/i16/i32/i64, lit_u8/u16/u32/u64 and lit_f32 create expressions with those exact types; lit(value) remains the f64 literal constructor.

Text, booleans and missing values

Strings are UTF-8, preserving Unicode scalars and embedded NUL bytes. There is no normalization or locale conversion. Empty text is Some(""), not Nothing. Booleans remain booleans; they are not implicitly converted to integers. Floating-point NaN and infinity remain present floating-point values. In particular, Some(f64.NAN) does not become null.

Both inputs and outputs own their data independently of the other side. Constructing a column never retains pointers into a Plenty list. A query may outlive its original frame, and extracted lists may outlive the native frame or query. Native owners are released by their allocating engine.

Existing non-nullable numeric paths transfer a whole column in one native call. Nullable numeric and boolean paths pack scalar values and validity bytes, then transfer whole buffers. String input joins UTF-8 text and constructs byte offsets and validity; output takes an owned native snapshot, copies packed bytes/offsets/validity in bulk and decodes each byte range into an ordinary Plenty string. This is copying, with temporary buffers and additional per-string allocations, not Arrow zero-copy interchange. It avoids per-cell foreign calls, but conversion can dominate small queries. All these allocations remain subject to the native OOM limitation.

Binary payloads

Binary columns contain arbitrary byte sequences, including embedded NUL and invalid UTF-8. Their Plenty values are owned std.bytes.Bytes buffers. Some(empty_buffer) is a present zero-byte payload; Nothing is missing data. Constructors borrow buffers and copy their contents. Extraction returns independent mutable buffers; changing them cannot change the frame.

lit_binary(value: &[u8]) copies a constant payload into a native expression. It accepts literals, buffers, byte lists and subviews without a Plenty-side allocation; the native expression owns its copy after the call. Use it with equality, filtering, or fill_null. cast(DType.Binary) explicitly converts strings to UTF-8 bytes. Binary-to-string casts require valid UTF-8; the pinned engine rejects invalid sequences even with strict=False. Binary columns round-trip through Parquet, but are not supported by the CSV interface.

Transfers pack bytes, offsets, and validity into whole-column native calls. Construction passes the packed Bytes as a borrowed byte view, without a second payload copy into list[u8]. Extraction copies through a packed Bytes before allocating each result buffer. These are copying APIs with temporary memory overhead, not zero-copy views; select/filter in Polars before extracting large payloads.

See the binary-column lesson.

Inspect a schema before collecting

DataFrame.schema() and Query.schema() return Result[list[Field], DataFrameError], in column order. Each Field has public name: str and dtype: DType fields.

DType has I8, I16, I32, I64, U8, U16, U32, U64, F32, F64, Bool, String, Binary, Date, Datetime(TimeUnit, Option[str]), List, Struct and Other(str) variants. Datetime metadata identifies its unit and optional timezone label; construction and extraction currently support naive and UTC values only. Other(description) preserves the engine’s native dtype description, including unexposed numeric or fixed-size array types. It does not imply that a type can be extracted or that the description is a stable serialization format. Do not parse that text to manufacture a cast.

Query schema inspection resolves the native plan without collecting all rows. A Parquet scan may read file metadata, and invalid paths, missing columns or incompatible expressions may fail at this stage. It is not a promise that later collection will succeed: data and I/O failures can still occur. Inspection returns independent metadata and does not mutate or consume the query.

See the runnable lesson and the signature index.

Nested columns add List and Struct dtype variants, column-oriented construction/extraction and structured schema_tree() inspection.

Nested dataframe columns

Lists and structs compose from existing native columns. They do not require a dynamic Plenty object for every row. Constructors borrow their inputs; returned columns, extracted parts and query plans remain usable after those inputs drop. All operations below return Result[..., DataFrameError].

import std.dataframe as df

def main() -> Result[(), df.DataFrameError | AllocError | IoError]:
    items = df.column_i64("item", [1, 2, 3, 4, 5]?)?
    # Offsets split the items into rows [1, 2, 3], [] and [4, 5].
    baskets = df.column_list("basket", items, [0, 3, 3, 5]?)?
    report = baskets.to_frame()?.lazy()?.select([
        df.col("basket")?.list_len()?.alias("count")?,
        df.col("basket")?.list_get(0)?.alias("first")?
    ]?)?.collect()?
    print(report.values_i64("count")?)?
    print(report.values_i64_nullable("first")?)?
    Ok(())
[3, 0, 2]
[Option[i64].Some(1), Option[i64].Nothing, Option[i64].Some(4)]

Construct lists and structs

column_list(name, values, offsets) groups a flattened child Column into variable-length lists. For n rows, offsets contains n + 1 integers: it starts at zero, never decreases, and ends at the child column length. Row i contains child values in [offsets[i], offsets[i + 1]). Equal offsets make an empty list. The child dtype, including nested types, is preserved even when the child is empty or entirely null. Its column name is not part of the list element schema.

column_list_nullable(name, values, offsets, valid) adds a list[bool] with one entry per parent row. A false entry means a null list, not an empty list. Null rows must have equal consecutive offsets; hidden children are rejected. Child nulls are independent: a present list can contain null elements. The non-nullable constructor only promises present parent lists. The extra offset entry limits list construction to 2**32 - 2 parent rows; the flattened child retains the engine’s 2**32 - 1 row limit.

column_struct(name, fields) takes a child DataFrame. Its column names, order and dtypes become the struct fields, and its height becomes the parent row count. At least one field is required. Zero-row typed structs work. column_struct_nullable(name, fields, valid) takes one boolean per parent row. Null parents mask their fields; values underneath null parents are not retained as meaningful data. A present struct with all-null fields remains distinct from a null struct.

Input lists and validity buffers are copied into native storage. Struct fields can share immutable native buffers. Constructors validate lengths and offsets; malformed input returns an error rather than indexing unchecked memory.

Extract column-oriented parts

frame.column(name) returns an independent Column handle sharing native buffers. column.to_frame() wraps it in a single-column dataframe with its existing name; use that frame’s ordinary typed extraction methods when values must enter Plenty. column.len() returns its row count.

column.list_parts() returns ListParts with:

FieldMeaning
values: ColumnFlattened child values, named item
offsets: list[i64]Canonical offsets starting at zero
valid: list[bool]Parent presence, independently of child nulls

Null parent ranges are empty. Extraction may compact/rechunk native child buffers; it is not a zero-copy promise. It transfers offsets and validity in bulk, and leaves the child data native until explicitly extracted. The returned parts can be passed back to column_list_nullable to reconstruct the column.

column.struct_parts() returns StructParts(fields: DataFrame, valid: list[bool]). The child dataframe retains field order and types; null parent rows have null child fields. The mask is necessary to distinguish a null parent from a present struct whose fields are all null. These parts can be passed to column_struct_nullable. Either method rejects the wrong column dtype.

Query nested values

ExpressionBehavior
expr.list_len()i64 element count; null list stays null, empty list yields zero
expr.list_get(index)Element at a fixed i64 index; negative indexes count from the end; out of bounds yields null
expr.struct_field(name)Named child field; null parent yields null; missing field is an error

These are native Polars expressions and compose with filtering, projection, aggregation and other expressions. An extracted element can itself be a list or struct. Operations validate their input schema during native plan resolution or collection; creating an expression does not prove that a later input fits it. Nested casts, list-element callbacks and row-exploding operations are not exposed by this initial interface.

Structured schema inspection

Existing schema() reports DType.List and DType.Struct for nested columns. DataFrame.schema_tree() and Query.schema_tree() return a preorder list[SchemaNode], where each node has parent: i64, name: str, and dtype: DType. Root columns have parent -1; other parents index an earlier node. Struct children retain field names/order; a list has one child named item. This synthetic name cannot collide with a struct field because its parent identifies its role.

The flat representation avoids recursive DType ownership: existing scalar dtype values stay copyable, printable and comparable. It supports arbitrary combinations of lists and structs without parsing DType.Other descriptions. Other native dtypes remain Other leaves. Schema inspection does not collect rows, although a file scan can read metadata and fail.

Parquet compatibility

Lists and structs preserve their semantic types, values and parent/child nulls through the pinned Polars Parquet reader/writer. Physical encoding and unused values hidden beneath null parents need not survive. Fixed-size arrays, arbitrary map semantics and other unexposed dtypes are not promised here.

Polars 0.55.2 has no native Map dtype. Its Parquet schema conversion reads maps as list/struct data; its Arrow metadata conversion explicitly maps Map to a large list of key/value structs (polars-parquet, src/arrow/read/schema/{convert,metadata}.rs). Use the schema tree and ordinary list/struct operations for that representation. No dictionary uniqueness, key ordering, or preservation of the Parquet MAP annotation on write is promised. This is a supported Polars representation, not a new Plenty Map API.

See the runnable lesson and column types. The existing native allocation-failure and 32-bit row-count limits also apply to flattened child columns.

Dataframe expressions

These operations extend the native expression plans. Every function and method below returns Result[Expr, DataFrameError]. They build immutable Polars expressions, not per-row Plenty callbacks. Receivers and expression arguments are borrowed; returned plans remain valid after those inputs are dropped. Invalid column names and incompatible dtypes can fail during schema resolution or collection.

import std.dataframe as df

def main() -> Result[(), df.DataFrameError | AllocError | IoError]:
    data = df.frame([
        df.column_str("name", ["Ada", "lin"]?)?,
        df.column_i64("score", [92, 61]?)?
    ]?)?
    result = df.when(
        df.col("score")?.ge(df.lit_i64(70)?)?,
        df.lit_str("pass")?,
        df.lit_str("retry")?
    )?.alias("result")?
    report = data.lazy()?.with_columns([df.col("name")?.str_upper()?, result]?)?.collect()?
    print(report.values_str("name")?)?
    print(report.values_str("result")?)?
    Ok(())
["ADA", "LIN"]
["pass", "retry"]

Literals and conditions

lit_str(value: &str) copies a UTF-8 literal into the native plan; lit_bool(value: bool) creates a boolean literal. Alongside lit(f64) and lit_i64(i64), these distinguish literal values from col(name). lit_binary(value: &[u8]) copies an arbitrary byte payload into the native plan without decoding it as text. Pass a b"..." literal, &buffer, &buffer[a:b], or &list_of_u8. The view reaches the native copy directly; the Plenty facade allocates no temporary payload.

when(predicate, then_value, otherwise) accepts three expressions. For each row, a true boolean predicate selects then_value; false or null selects otherwise. Polars determines a common output dtype for the two branches, so branches can undergo native coercion. An incompatible predicate or branch type produces a query error.

This is not short-circuit evaluation. Both branches must be valid independently: a condition cannot guard an invalid cast, missing column or other failing expression. Use an appropriate non-strict cast or validate the data separately. Nested when expressions can represent several choices, but this API does not translate Plenty if statements automatically.

Explicit casts

expr.cast(dtype: DType, strict: bool = True) supports all numeric DType variants (I8/I16/I32/I64, U8/U16/U32/U64, F32/F64), Bool, String and Binary. Temporal targets and DType.Other(...) fail immediately with DataFrameError.Schema; native dtype descriptions are not executable cast specifications. Use the explicit temporal parsing and calendar methods to construct dates/datetimes in a query.

The default strict mode reports invalid conversions as a query error. With strict=False, failed value conversions become null when Polars supports that conversion. It does not make every source/target pair legal; an unsupported conversion can still fail. Existing nulls remain null.

Strict does not mean lossless. For example, casting a finite fractional float to an integer truncates toward zero, and integer-to-float conversion can lose precision. Numeric-to-boolean converts zero to false and nonzero values to true. String parsing and boolean formatting follow Polars, not Plenty’s scalar parsing/formatting rules. Use the typed nullable getters when a non-strict cast can introduce missing values. String-to-binary casts preserve the UTF-8 bytes. Binary-to-string casts validate UTF-8: the pinned engine rejects invalid sequences even with strict=False. Extraction itself never performs these conversions.

String expressions

MethodResult dtypeMeaning
str_lower(), str_upper()StringUnicode lowercase or uppercase
str_len()I64Number of Unicode scalar values
str_contains(pattern: &Expr)BoolContains a literal substring
str_starts_with(prefix: &Expr)BoolStarts with the literal text
str_ends_with(suffix: &Expr)BoolEnds with the literal text

Use df.lit_str("text")? for a constant pattern, or df.col("pattern")? for row-wise patterns. Null text or a null pattern produces null. A null predicate is dropped by filter. The receiver must be a string expression. Use string patterns for predictable text semantics: Polars can coerce a primitive pattern to text, but does not similarly accept a numeric receiver. This native expression coercion does not change the strict dtype checks on Plenty’s column construction and extraction boundary.

Contains is literal: ".", "*" and "[" have no regex meaning. The engine enables Polars’ regex crate feature because its literal contains implementation is behind that feature; no regex API is exposed here.

Character counts are neither UTF-8 byte counts nor grapheme counts. Combining marks count separately. Casing can change the number of scalars (for example uppercase ß becomes SS); no normalization or locale-specific collation is promised. Embedded NUL bytes remain text.

Distinct counts and partition windows

expr.n_unique() computes an exact distinct count and returns I64. Unlike count(), it counts null as one distinct value. Empty input has zero distinct values; all-null nonempty input has one. It can be used in select, group_by aggregates or a window expression.

expr.over(partition_by: &list[Expr]) builds a native group-to-rows window. For example, df.col("amount")?.sum()?.over([df.col("team")?]?)? computes each team’s total and broadcasts it back to that team’s rows, without collapsing the frame or converting it into Plenty objects. Multiple key expressions form composite partitions; null keys group together.

The key list must be nonempty. Empty keys fail immediately with Schema; missing keys and invalid expressions fail when resolving or executing. Scalar aggregates map to every row in the partition. Non-scalar expressions must satisfy Polars’ group-to-rows length requirements. Output maps back to the input row positions; it does not introduce a sort order.

This increment has no window ordering, frame bounds, rolling operations, cumulative sums or ranking methods. Sorting a query does not turn sum().over into a running sum: it still sees the entire partition. Floating reductions retain the existing native rounding/order caveats.

See the runnable lesson and exact signatures.

Dataframe equality joins

Query.join(other, left_keys, right_keys, how=JoinKind.Inner, suffix="_right", nulls_equal=False) returns a new immutable query. It borrows both plans and expression lists; the resulting query remains valid after those owners are dropped.

Key lists must be nonempty and equally long. Multiple expressions form composite keys, matching every component. Key dtypes must be compatible; there is no implicit Plenty-side cast. Empty/unequal lists fail immediately with DataFrameError.Schema; missing columns, incompatible dtypes and other native plan errors can fail during schema resolution or collection.

KindRows keptOutput columns
InnerEvery matching left/right pairBoth sides
LeftMatching pairs, plus each unmatched left rowBoth sides; unmatched right cells are null
FullMatching pairs, plus each unmatched row from either sideBoth sides; absent-side cells are null
SemiEach left row with at least one right matchLeft side only
AntiEach left row with no right matchLeft side only
import std.dataframe as df

def main() -> Result[(), df.DataFrameError | AllocError | IoError]:
    orders = df.frame([df.column_i64("product", [3, 1, 2, 1]?)?, df.column_i64("units", [5, 2, 1, 3]?)?]?)?
    prices = df.frame([df.column_i64("product", [1, 2]?)?, df.column_f64("price", [10.0, 25.0]?)?]?)?
    keys = [df.col("product")?]?
    order = [df.SortKey(df.col("product")?), df.SortKey(df.col("units")?)]?
    priced = orders.lazy()?.join(prices.lazy()?, keys, keys)?.sort(order)?.collect()?
    print(priced.values_i64("product")?, priced.values_f64("price")?)?
    unpriced = orders.lazy()?.join(prices.lazy()?, keys, keys, how=df.JoinKind.Anti)?.collect()?
    print(unpriced.values_i64("product")?)?
    Ok(())
[1, 1, 2] [10.0, 10.0, 25.0]
[3]

Multiplicity and validation

Inner, left and full joins are many-to-many. Two equal left keys and three equal right keys yield six matching pairs. No uniqueness check is performed; these are not validated one-to-one or many-to-one lookups.

Semi and anti joins test membership. Duplicate right matches do not multiply the output. Duplicate left rows remain distinct input rows and are retained or removed independently. For example, two matching left rows survive a semi join as two rows even when the right side contains three copies of the key. These joins are not deduplication operations.

An empty left input produces no semi/anti rows. An empty right input makes a semi join empty and an anti join retain all left rows. A full join with one empty side retains the other side, padding the absent columns with nulls.

Nulls and order

Null keys do not match by default. Thus a null left key is excluded by semi, retained by anti, and remains unmatched in a full join. With nulls_equal=True, nulls in corresponding key positions compare equal, including inside composite keys. Pair-producing joins still multiply duplicate null-key matches; membership joins still retain each left row at most once.

NaN remains a native floating-point value governed by Polars equality, not a null. No join kind promises output order. Apply explicit sorting when an application needs a reproducible presentation, including a tie-breaker when keys alone are insufficient.

Keys, names and suffixes

Inner, left and full joins retain both sides’ key columns without automatic coalescing. A right column whose name is also present on the left gets the suffix, including keys. If this still leaves duplicate output names, the query fails with DataFrameError.Query; it does not overwrite a column.

For an unmatched right row in a full join, the left key is null while the right key keeps its value. To create a common display key, derive it explicitly: df.col("id")?.fill_null(df.col("id_right")?)?.alias("joined_id")?. Keep the originals when their provenance matters. This is a normal expression step, not a different join coalescing policy.

Semi and anti joins emit only left columns, so they neither append right keys nor suffix right names. The suffix argument has no effect for them, including an empty suffix. Right-side name collisions are therefore harmless for these membership joins.

The same expression-key and ownership rules apply to all five kinds. Native query execution supports ordinary and explicit streaming collection; streaming is not a promise of bounded join memory. For nearby-key matching, use integer as-of joins. Temporal/grouped as-of joins and uniqueness-validation options are not exposed.

See membership and full-join examples, joining and ranking, and the signature index.

Integer as-of joins

Query.join_asof(other, left_on, right_on, strategy=AsOfStrategy.Backward, tolerance=Nothing, allow_exact_matches=True, suffix="_right") matches each left row to at most one nearby right key. It returns a new immutable native query and borrows both inputs. It remains usable after their owners are dropped.

left_on and right_on are column names. Both columns must have dtype I64; there is no implicit cast. Derive a named key with with_columns when needed. Floating-point, date/time, grouped and composite as-of keys are not supported. Use equality joins for exact multi-column matching.

import std.dataframe as df

def main() -> Result[(), df.DataFrameError | AllocError | IoError]:
    trades = df.frame([df.column_i64("second", [3, 7]?)?]?)?
    quotes = df.frame([
        df.column_i64("second", [1, 5, 6]?)?,
        df.column_f64("bid", [9.5, 10.0, 10.5]?)?
    ]?)?
    result = trades.lazy()?.join_asof(quotes.lazy()?, "second", "second")?.collect()?
    print(result.values_f64("bid")?)?
    Ok(())
[9.5, 10.5]

Ordering and matching

Both inputs must be sorted ascending by their key, with nulls first. Sorting is not automatic: use sort([df.SortKey(df.col("key")?, nulls_last=False)]?) when the source is not already ordered. The native execution checks sortedness and rejects an unsorted input with DataFrameError.Query. The requirement applies to the inputs of the join, after any preceding filters or key transformations.

StrategyRight key chosen
BackwardGreatest key less than or equal to the left key
ForwardSmallest key greater than or equal to the left key
NearestKey with the smallest absolute integer distance

Set allow_exact_matches=False to exclude equal keys. tolerance=Some(n) adds an inclusive distance bound of n key units; Nothing means no bound. The bound must be nonnegative and fit in i64. Some(0) permits only exact matches, so disabling exact matches with zero tolerance yields no matches. Integer distance comparisons handle the full signed i64 range without wrapping at its endpoints.

Backward selects the last right row at the chosen key; forward selects the first. Nearest breaks equal-distance ties toward the greater key and selects the last duplicate at its chosen key. These rules refer to right input order: when duplicate selection matters, explicitly sort by the key and a suitable secondary column before joining.

Null left keys are unmatched; null right keys are skipped. Null keys never match one another. Nulls-first placement is validated consistently, including for inputs whose native metadata already marks them sorted. Every left row survives once, including duplicates and unmatched rows. An empty right input supplies null right fields; an empty left input yields no rows.

Output, failures and costs

Both key columns remain in the output. Right column names that collide with left names receive the suffix. An unresolved collision is a query error; columns are not silently overwritten. Unmatched right-side cells are null, so use nullable extraction unless matching is guaranteed.

Negative tolerances and unsupported key dtypes fail with DataFrameError.Schema. Missing columns, invalid native plans, suffix collisions and unsorted data are errors rather than unmatched rows. Some errors arise while the plan is built; data-dependent failures arise when it is collected or written. Native allocation/panic limits are the same as for other dataframe operations.

The join remains a Polars query-plan node; no Plenty per-row callback or intermediate Plenty collection is used. A native lazy validation step before each input scans its entire key column at execution time and returns the frame unchanged. These steps materialize both inputs and block optimizations from moving operations across validation. Put intended filters, projections and sorting before join_asof to reduce that work.

The validation steps compensate for a pinned Polars streaming path that can accept an unsorted right input despite its sortedness-check option. Ordinary and explicit streaming collection therefore have the same checked contract, but as-of joins are not bounded-memory streaming operations. Plan creation resolves schemas and does not eagerly collect the datasets.

Rebuild the optional engine to obtain its asof_join feature. Ordinary programs still do not link Polars.

See the sensor lookup example and signature index.

Dates and datetimes

Temporal columns have logical types, not merely integer labels. Their construction and extraction use explicit physical units at the Plenty boundary; native parsing and calendar expressions handle ordinary analytical workflows. All operations return Result[..., DataFrameError].

import std.dataframe as df

def main() -> Result[(), df.DataFrameError | AllocError | IoError]:
    source = df.frame([df.column_str("text", ["2024-02-29", "1970-01-02"]?)?]?)?
    day = df.col("day")?
    result = source.lazy()?.select([
        df.col("text")?.str_to_date()?.alias("day")?
    ]?)?.with_columns([
        day.dt_year()?.alias("year")?,
        day.dt_format("%d/%m")?.alias("label")?
    ]?)?.collect()?
    print(result.values_date_days("day")?)?
    print(result.values_i64("year")?)?
    print(result.values_str("label")?)?
    Ok(())
[19782, 1]
[2024, 1970]
["29/02", "02/01"]

Units, timezones and the boundary

Logical typeConstructionExtraction
Datecolumn_date(name, days: &list[i32])values_date_days(name) / get_date_days(name, row)
Datetimecolumn_datetime(name, ticks: &list[i64], unit, utc=False)values_datetime_ticks(name, unit, utc=False) / get_datetime_ticks(name, row, unit, utc=False)

Dates count proleptic Gregorian calendar days since 1970-01-01. Datetimes count ticks from 1970-01-01 00:00:00. TimeUnit.Milliseconds, Microseconds and Nanoseconds mean 1,000, 1,000,000 and 1,000,000,000 ticks per second. Negative values precede the epoch. No unit is inferred or converted by a getter. Date extraction returns i32; datetime extraction returns i64.

utc=False describes a timezone-free civil datetime. utc=True describes UTC; it does not convert a local wall-clock value into UTC. No host timezone is consulted. Named-zone localization, timezone conversion, daylight-saving ambiguities and leap seconds are not supported in this foundation. std.time.Timestamp uses seconds plus nanoseconds and is not implicitly converted to these column representations.

Each constructor also has a _nullable form accepting list[Option[T]]. Each bulk getter has a _nullable form returning list[Option[T]]; the plain getter rejects any null with DataFrameError.Null. Scalar getters return Option[T] inside Result. Empty and all-null columns retain their dtype. Transfers copy whole buffers; nullable values use a separate validity buffer.

Construction and typed extraction validate the engine’s representable calendar range. Not every i32 day or i64 millisecond/microsecond value is a valid calendar date. The full i64 nanosecond range is representable. Invalid values produce errors rather than wrapping; null payload bits are ignored. Wrong dtype, unit or UTC metadata is an error, not an implicit conversion.

Schemas report DType.Date or DType.Datetime(unit, timezone: Option[str]). Nothing is naive and Some("UTC") is UTC. Other timezone labels from files are preserved honestly, but the typed temporal getters and calendar operations reject them. Schema inspection is not a promise that every reported type or value can be extracted. Generic get_i64 does not discard datetime metadata.

Parse strings and extract calendar fields

Expression methodOutput
str_to_date(format="%Y-%m-%d", strict=True)Date
str_to_datetime(format, unit, utc=False, strict=True)Datetime with the specified unit/zone
dt_year(), dt_month(), dt_day()I64 calendar component
dt_hour(), dt_minute(), dt_second()I64 time component of a Datetime
dt_date()Date portion of a Datetime
dt_format(format)String

Formats use Chrono directives such as %Y-%m-%d, %H:%M:%S and %.f (optional fractional seconds). Parsing consumes the whole string. Omitted clock fields default to zero. Invalid values and out-of-range conversions fail with Query in strict mode or become null with strict=False. Existing nulls remain null. Parsing leap-second :60 is rejected.

Fractional precision beyond the selected unit is discarded toward the earlier instant: -1 nanosecond becomes -1 millisecond, not zero. Choose nanoseconds when that precision matters; their calendar range is much smaller than that of milliseconds or microseconds. Nanosecond overflow is checked explicitly.

Invalid formats and timezone directives (%z, %Z, %+, including modified forms) are rejected. A literal Z is just text; use it only when the data is explicitly UTC. Date-only values cannot supply time-of-day formatting fields. Unicode literals in formats are preserved. Calendar components use the stored naive/UTC interpretation and return null for null inputs.

Parsing and range checks execute as checked native column expressions inside Polars plans, with no Plenty callback or per-cell FFI. These validation/parsing nodes are opaque to the optimizer; calendar kernels remain native Polars operations. The checked parser avoids overflow and leap-second normalization in the pinned native parser. Imported invalid calendar values fail explicitly before calendar extraction or formatting.

Parquet preserves temporal dtypes and metadata. CSV does not: format columns explicitly before writing and parse the resulting strings after reading. CSV schema overrides and Expr.cast accept numeric, boolean and string target types; they do not construct temporal types. As-of joins remain integer-key operations. See the runnable lesson and signature index.

Acknowledgements

  • Date parsing, calendar validation and format directives use Chrono 0.4.45, built into the dataframe engine (MIT OR Apache-2.0).

CSV and streaming dataframe I/O

These operations use the optional Polars engine. They return Result[..., DataFrameError], retain native error messages, and leave query/frame owners usable after each call.

import std.dataframe as df

def main() -> Result[(), df.DataFrameError | AllocError | IoError]:
    data = df.frame([
        df.column_str("city", ["Oslo", "Lima", "Pune"]?)?,
        df.column_i64("population", [709, 10883, 7764]?)?
    ]?)?
    data.write_csv("cities.csv")?
    query = df.scan_csv("cities.csv", schema_overrides=Some(data.schema()?))?
    large = query.filter(df.col("population")?.gt(df.lit_i64(1000)?)?)?.collect()?
    print(large.values_str("city")?)?
    Ok(())
["Lima", "Pune"]

Scan a local CSV file

scan_csv(path, has_header=True, separator=",", infer_rows=100, null_value=Nothing, schema_overrides=Nothing) returns a Query. The path names one file; wildcard characters are literal, not a glob.

OptionContract
has_header: boolFirst record supplies names; false uses column_1, column_2, …
separator: &strOne ASCII byte, excluding NUL, double quote, CR and LF; tab is allowed
infer_rows: i64Positive: sample that many rows; zero: infer strings; -1: inspect the whole file
null_value: Option[str]One optional exact token treated as null in every column
schema_overrides: Option[list[Field]]Named overrides using any numeric DType, Bool or String; other columns retain inference

Overrides reuse schema fields. The optional list is consumed, so schema_overrides=Some(frame.schema()?) is convenient for CSV round-trips that must preserve these scalar override types. Duplicate or unknown override names, temporal targets and DType.Other fail with Schema. For temporal columns, explicitly format to strings before CSV output and parse those strings after input; Parquet preserves their metadata. Overrides select types by name; they are not a full schema or a column rename/reorder facility. Headerless empty input has no width/names to infer.

CSV carries text, not a durable typed schema. Quoting does not guarantee that an inferred column stays a string; specify a String override for identifiers such as "001". A sample can infer an integer and then fail on a later non-integer value. Use overrides, increase the sample, or read strings and cast explicitly. Whole-file inference costs a full scan before query execution.

Input must be valid UTF-8. Double quotes delimit quoted fields and doubled quotes escape quote characters. Quoted separators and newlines are supported. Malformed records, invalid typed values and invalid UTF-8 produce errors; the facade does not enable lossy decoding, ignored parse errors or ragged-row truncation. A scan may inspect schema/sample data immediately; errors can occur during creation, schema inspection or later execution. The file must remain available while the query executes.

Empty strings and null tokens

By default, an unquoted empty field is null; a quoted empty string field is present empty text. This preserves Nothing versus Some("") in typed string round-trips. Floating NaN remains a present floating value.

A custom null token also matches quoted fields: with null_value=Some("NA"), both NA and "NA" become null. Choose a token that does not occur as a real value. CSV quoting cannot disambiguate this collision. Empty/missing fields still follow the native missing-is-null policy.

Write a collected frame as CSV

frame.write_csv(path, has_header=True, separator=",", null_value="") returns Result[(), DataFrameError]. It writes a new local UTF-8 file, with double-quote escaping and LF record endings. This starts from an already collected dataframe; it is not a query sink.

The output null token is written verbatim. It cannot contain the separator, double quote, CR or LF; invalid options fail before creating a file. The empty default writes null as an unquoted empty field and preserves present empty strings through quoting. Read a custom token back by passing the same token to scan_csv. CSV does not preserve types by itself.

Execute with the streaming engine

query.collect_streaming() returns a DataFrame and explicitly selects Polars’ streaming engine. It still materializes the entire final result. It is useful when streaming intermediate work helps, not when the final result itself cannot fit in memory. Ordinary collect() retains its native automatic execution policy.

query.sink_parquet(path) executes a native Parquet file-sink plan using the streaming engine and returns (). The wrapper does not collect a complete dataframe and then call a writer. The opened output file is passed directly to Polars, and intermediate rows remain inside the native plan. For example, a CSV scan can filter/project and write Parquet without converting rows into Plenty values or retaining the whole output frame. Default native Parquet compression and row-group settings apply.

Neither method guarantees constant or bounded memory for arbitrary plans. Sorts, joins, high-cardinality groups, distinct counts and partition windows can buffer substantial data, and the streaming planner can use in-memory stages for some operations. A lazy frame built from Plenty columns already holds its source data in memory. Unsupported operations or resource failures can still error, and native OOM can abort. No batch iterator, memory budget, spill-directory control or cloud configuration is exposed by this increment.

The sink preserves the plan’s order where defined; it does not make unordered group/join output deterministic. Sort explicitly when output order matters, accepting the associated buffering cost.

Output ownership and failures

The new write_csv and sink_parquet methods create a new file. An existing path, including a symlink or an alias of a lazy input, fails with Io without truncation. Parent directories must already exist. The sink keeps the opened file descriptor through execution instead of reopening its path. Neither method mutates or consumes the input owner.

After creation, an I/O or query error may leave a partial new file. These methods do not provide atomic publication, automatic cleanup or an fsync durability guarantee. Choose a fresh output name; explicitly manage cleanup or publication in the application when necessary. The older DataFrame.write_parquet retains its documented overwrite behavior.

Native Polars failures use DataFrameError.Query; direct file creation failures use Io, and invalid facade options use Schema. Other native failure and allocation contracts are unchanged.

See the runnable lesson. For row-oriented parsing over Plenty streams without Polars, use std.csv.

Install the dataframe engine

The ordinary compiler build does not compile Polars. Importing std.dataframe can be type-checked without its engine. Linking needs the engine only when retained native code reaches a dataframe operation, including calls through dependencies. An unused import or unused local helper does not add a Polars dynamic dependency.

Use the release engine

Linux release archives include bin/libplenty_dataframe_v1.so and its bin/dataframe-notices/. The compiler finds the library beside itself, so no separate installation is needed. Set PLENTY_DATAFRAME_ENGINE only to select a different library. The notices directory holds engine.json, license files and dependency notices for redistribution. The manifest records the Plenty version, source commit, target, ABI generation, library checksum and minimum glibc. It is descriptive metadata, not a loader configuration or a compiler-enforced version check.

The prebuilt engine supports x86-64 and ARM64 GNU/Linux with glibc 2.28 or later and libgcc_s.so.1, with no AVX2 requirement. Rust, Cargo and a source checkout are unnecessary. The experimental Windows x86-64 release also includes bin/plenty_dataframe_v1.dll and bin/dataframe-notices/, requiring Windows 10 or Windows Server 2016 or later and the MSVC runtime. Its manifest records minimum_windows instead of minimum_glibc. The experimental macOS ARM64 release includes bin/libplenty_dataframe_v1.dylib and the same notices layout, requiring macOS 11 or later. Its manifest records minimum_macos. musl/Alpine and other native targets remain unsupported. The compiler still needs its linking prerequisites. See shipping dataframe applications for runtime library placement, notices and compatibility checks.

Build from source

For the library plus local --dist metadata/licenses, use just engine dataframe. just build builds the compiler and both engines; see the local build recipes. The lower-level command below installs only the library.

From a Plenty source checkout, with a stable Rust toolchain:

python3 scripts/build_dataframe_engine.py --out target/release

Build the compiler separately with cargo build --release --locked. The script pins Polars 0.55.2 with lazy, parquet, csv, strings, regex, temporal, dtype-date, dtype-datetime, semi_anti_join and asof_join, uses three Cargo jobs by default, and installs libplenty_dataframe_v1.so. Use --target-dir PATH to reuse engine build artifacts and --jobs N to control build concurrency. It forces baseline x86-64 code generation and the system C linker, overriding ambient CPU flags; AVX2 is not required. The script supports native x86-64/ARM64 GNU/Linux builds, Linux cross targets, and native x86_64-pc-windows-msvc and aarch64-apple-darwin builds. On Windows it uses MSVC, installs plenty_dataframe_v1.dll, and avoids ELF linker flags. On macOS it builds for Apple M1 with MACOSX_DEPLOYMENT_TARGET=11.0, installs libplenty_dataframe_v1.dylib with identity @rpath/libplenty_dataframe_v1.dylib, and uses the native C linker. Build on a system with a compatible glibc version for the deployment host. This source-build command does not impose the release’s glibc 2.28 baseline; the release workflow builds and audits the engine inside its pinned container.

The extra string features support the native expression API. Polars 0.55.2’s string dispatch imports its optional time module even without string-to-date operations, so this feature combination also needs temporal to compile. The date/datetime dtype features support temporal columns and expressions. Checked parsing and calendar validation also use the pinned Chrono dependency. Ordinary compiler/runtime builds still do not build Polars. The parquet feature already enables Polars’ streaming engine; CSV support adds only csv, not a second execution engine. The semi_anti_join feature supports native membership joins; asof_join supports checked integer-key as-of matching. Temporal and grouped as-of APIs are not exposed.

Discovery and compatibility

The dataframe engine follows the general native engine contract. The compiler looks beside its own executable, or at the file named by PLENTY_DATAFRAME_ENGINE. An explicit environment path takes precedence. The Rust compilation API also offers CompileOptions.dataframe_engine. Missing engines produce an installation hint at link time.

export PLENTY_DATAFRAME_ENGINE=/opt/plenty/libplenty_dataframe_v1.so
plenty --compile analysis.plenty -o analysis

The resulting executable contains a versioned shared-library dependency, not a copy of the entire engine. For shipping, prefer distribution output: --compile analysis.plenty --dist my-app copies the selected engine and its packaged notices, with only $ORIGIN lookup. The ordinary -o output above has RUNPATH searching its own directory and the engine’s build-time directory. Deploy the executable beside the matching libplenty_dataframe_v1.so, or install that library in the loader’s search path. Keep compiler bindings and engine from the same Plenty release; the v1 name denotes the C ABI generation, not compatibility with arbitrary future Polars builds. Directly emitted objects also require this engine when externally linked. Engine directories containing a colon are rejected because ELF RUNPATH uses that character as a separator.

Rebuild/reinstall the engine when updating the compiler’s dataframe API. New query operations add version-1 symbols; an older engine cannot satisfy their native links even though its filename is unchanged. The system loader’s LD_LIBRARY_PATH and LD_PRELOAD can override RUNPATH; keep stale engine copies out of those paths when switching installations.

Normal programs need neither this library nor Rust. They do not gain a DT_NEEDED entry for Polars. A project fingerprint includes the selected engine path and metadata, so replacing or relocating it triggers relinking. Already-linked executables still need their dynamic dependencies at run time.

macOS application linking

The macOS compiler uses the same option/environment/beside-compiler discovery precedence and links the selected ARM64 dylib directly. Ordinary compilation copies it beside the executable, as described in engine copies beside applications, which preserves old file mappings and cached signatures. Invalid selected engines report the build script needed to produce a compatible dylib. --dist adds the validated manifest, licenses and notices, and checks the copied dylib identity and checksum. The application uses @loader_path to resolve the library’s @rpath identity from its own directory, without a fallback to the original build directory.

Release packaging checks ARM64 Mach-O headers, a minimum macOS version no newer than 11, system-only dependencies, the versioned dylib identity, facade exports and the code signature. Apple tools verify the linker-generated ad-hoc signature; the release remains unnotarized. Keep the dylib unmodified and beside the executable when moving the application. Native CI tests Parquet queries and application-only relocation.

Windows application linking

The Windows compiler selects the DLL through the same option/environment/beside- compiler precedence. It derives an MSVC import library from that DLL’s exported C symbols using lib.exe, supplied by Visual Studio Build Tools. Ordinary compilation copies the DLL beside the output executable; --dist includes the DLL and validated notices. Programs use Windows DLL lookup, with the application directory searched by the loader. Keep the DLL beside the executable, and use trusted application directories and PATH entries. Type-checking needs no DLL.

Engine copies beside applications

On Windows and macOS, ordinary compilation links the selected engine and, after the link succeeds, places a copy beside the executable. A failed link leaves an existing application and its engine unchanged. When the existing copy already has the same contents, it is not rewritten. Otherwise the new engine is copied to a fresh file in the output directory and renamed over the old one, so programs that have the old engine open keep it. Windows refuses to replace a DLL that a running program has loaded; the error names the file, and closing the program and compiling again replaces it. plenty FILE runs the program from a temporary directory and hard-links the engine there when the file system allows, instead of copying it.

Development checks

To exercise the optional component after building it:

export PLENTY_DATAFRAME_ENGINE="$PWD/target/release/libplenty_dataframe_v1.so"
export PLENTY_DATAFRAME_TESTS=1
cargo test --locked --test integration test_std_dataframe::
cargo test --locked --test integration test_tutorial::dataframe_examples_with_optional_engine

Regular tutorial checks type-check dataframe examples without the engine. The separate dataframe CI job builds it and executes those examples and the real-engine integration tests. These tests cover bulk transfer, query plans, nulls, errors and Parquet, as well as absence of the dynamic dependency from programs that do not use it.

Ship a dataframe application

A compiled program that uses std.dataframe needs libplenty_dataframe_v1.so on Linux, libplenty_dataframe_v1.dylib on macOS, or plenty_dataframe_v1.dll on Windows at run time. Ship the same engine you compiled against, from the matching Plenty release, beside your executable. End users need the host C libraries, but no Plenty compiler, Rust, Cargo or cc. Programs without reachable dataframe operations need no engine.

Bundle the executable and engine

With a release compiler, the engine and its notices are already in bin/:

plenty --compile analysis.plenty --dist my-app
./my-app/analysis

my-app must not exist. Plenty stages and publishes the executable, the exact engine used for linking, and its packaged notices together. No reachable engine calls means no engine files; reachable linear algebra calls also add the linalg engine and its notices. See application distributions for metadata checks, supported input layouts, local-build packaging and CLI restrictions. A bare installed library without notices is not enough for --dist.

The resulting directory can be archived, installed elsewhere and renamed:

my-app/
  analysis
  libplenty_dataframe_v1.so
  dataframe-notices/
    engine.json
    LICENSE-MIT
    LICENSE-APACHE
    THIRD-PARTY-NOTICES.md
    licenses/rust/...

Keep the engine notices with distributions of the library, alongside any notices your own application needs. The release archive’s README contains the same distribution command for users who have no source checkout.

Test on the oldest supported system

The prebuilt engine targets x86-64 and ARM64 Linux, glibc 2.28 or later and libgcc_s.so.1. The experimental Windows x86-64 engine requires Windows 10 or Windows Server 2016 or later and the MSVC runtime. The experimental macOS ARM64 engine requires macOS 11 or later. musl/Alpine remains unsupported. Your executable links against the build machine’s C libraries and can require a newer glibc. Compile and test the application on the oldest distribution you support; a baseline-compatible engine alone does not make your executable portable. No particular memory bound is implied by the shared-library format.

Test the bundle in a clean machine or container without your original Plenty installation. Run it from a different working directory, then move the bundle and run it again. Test a missing-library failure too. The release smoke tests exercise app-only relocation and a native Parquet query on each supported test distribution after removing the compiler and original engine location.

How library lookup works

This applies to every native engine. On macOS ARM64, --dist produces analysis, libplenty_dataframe_v1.dylib and dataframe-notices/. Ordinary compilation also copies the dylib beside the executable. The dylib identifies itself as @rpath/libplenty_dataframe_v1.dylib; the executable searches @loader_path (its own directory). No original build directory is retained. Keep the linker-signed dylib unmodified. Loader environment variables such as DYLD_LIBRARY_PATH can affect lookup; remove them when testing relocation.

On Windows, --dist produces analysis.exe, plenty_dataframe_v1.dll and dataframe-notices/. Ordinary compilation also copies the DLL beside its executable. Windows loader search rules apply; the application directory supplies the bundled DLL. Test with the compiler directory removed from PATH.

On Linux, distribution output embeds an ELF RUNPATH of only the executable’s own directory ($ORIGIN), so a missing bundled library cannot fall back to the original engine directory. A sibling lib/ is not searched automatically.

For manual deployment with ordinary --compile analysis.plenty -o analysis, copy the exact selected library beside the executable and all the archive’s notices into dataframe-notices/, as in the layout above. Ordinary output still searches $ORIGIN then the build-time engine directory; that fallback can mask a missing bundled library on the development machine. Prefer --dist for shipping applications.

PLENTY_DATAFRAME_ENGINE chooses an engine at compile time; setting it when running an already-built application does not affect library lookup. LD_LIBRARY_PATH and LD_PRELOAD can override RUNPATH, so remove stale overrides when testing or switching releases. Keep the application directory and library search paths trusted. System-wide library installation is also possible through the operating system’s loader configuration.

The v1 filename denotes the C ABI generation, not a promise that every release exports every operation. New operations can add symbols without changing this name. Keep releases matched and do not substitute a differently built Polars library. engine.json records the engine’s origin; --dist checks release/target/ABI metadata and the copied library checksum. Ordinary compilation does not validate its version; matching metadata is not proof of same-source identity for local builds.

FailureWhat to check
cannot open shared object file naming libplenty_dataframe_v1.soPut the matching library beside the executable
undefined reference or symbol lookup error naming pldf_v1_Check for an old engine or a loader environment override
GLIBC_2.xx not foundCheck both the application and engine’s platform requirements; rebuild on the oldest supported system

Directly emitted object files need an appropriate external link command and loader configuration; the automatic RUNPATH described here applies to executables linked by Plenty.

std.db.sqlite

import std.db.sqlite stores and queries data in SQLite databases through prepared statements with typed parameters and typed column reads. It has one error type, DbError.

For named byte-key maps without application SQL, use std.db.kv, which shares this engine. Use this module for relational schemas, SQL queries, or reusable prepared statements. See the storage comparison.

SQLite is vendored C code compiled into the runtime. The Plenty module owns the error model, row state, value limits, and transaction ownership. It is not an ORM: SQL is written by the program, and values are bound, never spliced into SQL text. The module is available on the supported x86-64 Linux GNU target.

import std.db.sqlite as sqlite

def main() -> Result[(), AllocError | IoError | sqlite.DbError]:
    mut db = sqlite.connect("shop.db", sqlite.Mode.Create, 65536)?
    db.execute("CREATE TABLE items(name TEXT NOT NULL, price INTEGER NOT NULL)")?
    mut insert = db.prepare("INSERT INTO items VALUES (?, ?)")?
    for name, price in [("pen", 3), ("book", 12), ("lamp", 25)]?:
        insert.bind_text(1, name)?
        insert.bind_integer(2, price)?
        insert.run()?
    mut query = db.prepare("SELECT name, price FROM items WHERE price > ? ORDER BY price")?
    query.bind_integer(1, 5)?
    while query.step()?:
        print(query.text(0)?, query.integer(1)?)?
    Ok(())
book 12
lamp 25

Opening

FunctionResultContract
connect(path: &str, mode: Mode, max_length: i64)Result[Connection, DbError]The database file at path
memory(max_length: i64)Result[Connection, DbError]A new private database in memory, gone when the connection closes

Mode is ReadOnly, ReadWrite (the file must exist), or Create. A missing file without Create is SQLite’s cant_open (14). path is a file name, not a URI; "", ":memory:", and paths containing a NUL byte are InvalidPath.

max_length is the largest text, blob, or row, in bytes, that the connection stores or reads. It must lie in min_length_limit (1024) to max_length_limit (1,000,000,000, SQLite’s own maximum); otherwise OutOfRange. Binding, computing, or reading a larger value is TooLarge. Choose it from the memory the program can spend: every text or blob read is copied into a new str or Bytes.

Connection

MethodResultContract
execute(sql: &str)Result[(), DbError]Run every statement in sql to completion, discarding rows
prepare(sql: &str)Result[Statement, DbError]Compile exactly one statement
begin(mode: Begin)Result[Transaction, DbError]Start a transaction
changes()i64Rows changed by the most recent INSERT, UPDATE, or DELETE
last_insert_rowid()i64The rowid of the most recent successful INSERT
set_busy_timeout(budget: &Duration)Result[(), DbError]How long to retry a locked database; see Busy databases
load_extension(path: &str, entrypoint: &str = "")Result[(), DbError]Load a trusted native SQLite extension into this connection
close()Result[(), DbError]Close now

execute takes no parameters and is meant for schema and fixed SQL. prepare rejects SQL with no statement (EmptySql) or more than one (TrailingSql); a trailing ;, whitespace, or comment is allowed.

close is InUse while a statement or transaction made from the connection is alive, and leaves it open. After a successful close, every method that reaches SQLite is Closed, changes and last_insert_rowid return 0, and a second close succeeds. Dropping a connection without close closes it once its last statement and transaction are gone.

load_extension loads a native shared library for the supported target. Only load libraries you trust: their initialization and SQL functions execute native code in your process. The extension must use SQLite’s extension API table; it does not need a separately linked SQLite library. Use an explicit filesystem path to select the library. SQLite may try platform suffixes if it cannot open that path directly.

An empty entrypoint asks SQLite to discover the initialization symbol; otherwise it names the exact exported symbol (for example, sqlite3_vec_init). Empty or NUL-containing paths are InvalidPath; a NUL in the entrypoint is InvalidExtensionEntry. Loader failures return Sqlite with SQLite’s loader message. Closed is returned if the connection has been closed.

The native loading API is enabled only during the call, under the connection mutex, and disabled again on success or failure. SQL’s load_extension() stays disabled. Extensions belong to the connection and must be loaded again when reopening a database that uses them. There is no public enable/disable toggle.

FTS5 is built in: CREATE VIRTUAL TABLE ... USING fts5(...), MATCH, and bm25(...) work without loading an extension. Vector extensions such as sqlite-vec are installed separately; Plenty does not download them.

Statement

Parameters are numbered from 1 and columns from 0.

MethodResultContract
parameter_count()i64The largest parameter number
parameter_index(name: &str)Result[i64, DbError]The number of :name, @name, or $name, prefix included; unknown is OutOfRange
bind_null(index)Result[(), DbError]
bind_integer(index, value: i64)Result[(), DbError]
bind_real(index, value: f64)Result[(), DbError]NaN is OutOfRange
bind_text(index, value: &str)Result[(), DbError]Copied
bind_blob(index, value: &[u8])Result[(), DbError]Copied
clear_bindings()Result[(), DbError]Every parameter back to NULL
step()Result[bool, DbError]Advance: True with a row current, False once finished
reset()()Rewind, keeping the bindings
run()Result[(), DbError]Reset, step to the end discarding rows, reset again
column_count()i64Columns in each row
column_name(column)Result[str, DbError]The column’s name or AS alias
kind(column)Result[ValueType, DbError]The storage class of the value in the current row
is_null(column)Result[bool, DbError]
integer(column), real(column), text(column), blob(column)Result[i64 / f64 / str / Bytes, DbError]The value, which must have that storage class
optional_integer(column) and the other optional_ readsResult[Option[...], DbError]Nothing for NULL, otherwise as the strict read

An index outside the statement’s parameters or columns is OutOfRange. Binding after a step rewinds the statement first, so a statement is reused by binding new values and stepping again. A statement that has finished keeps returning False from step until reset or a new binding. A failed step rewinds it.

Reads require a current row (NoRow otherwise) and never convert: SQLite’s storage class must match the read, or the error is TypeMismatch(found), where found is the stored class. ValueType is Integer, Real, Text, Blob, or Null. A column’s declared type does not decide its storage class; SQLite’s type affinity does, so an INTEGER column given the text '7' stores the integer 7, but a TEXT column given the integer 7 stores the text '7'. STRICT tables still allow lossless conversion, including '7' into an INTEGER column; they reject values that cannot be losslessly converted to the declared type. Text that is not valid UTF-8 is InvalidUtf8.

Every text and blob read is copied out, so values stay valid after the next step, reset, or the statement’s drop. There are no borrowed row views.

Transactions

Begin is Deferred (locks taken on first use), Immediate (write lock at once), or Exclusive.

MethodResultContract
commit()Result[(), DbError]Make the changes durable
rollback()Result[(), DbError]Discard the changes

A connection has at most one transaction: begin while one is open is TransactionActive. Dropping a transaction that was neither committed nor rolled back rolls it back, so ? out of a function undoes its work. A finished transaction, or one ended by SQL such as COMMIT run through execute, is TransactionFinished on commit and rollback. A failed commit that SQLite leaves open, such as one that is busy, may be retried or rolled back.

Savepoints are available through execute("SAVEPOINT name") inside a transaction.

Busy databases

A write needs a lock on the database file. When another connection holds it, the default is to fail at once with SQLite’s busy (5) code. set_busy_timeout(&budget) takes a std.time.Duration. Zero restores the default; positive fractional milliseconds round up. A rounded budget above 2,147,483,647 milliseconds is OutOfRange. Use Begin.Immediate for transactions that write, so a conflict is found at begin rather than partway through.

Errors

pub enum DbError:
    Sqlite(SqliteError)
    Closed
    InUse
    EmptySql
    TrailingSql
    TransactionActive
    TransactionFinished
    InvalidPath
    InvalidExtensionEntry
    OutOfRange
    NoRow
    TypeMismatch(ValueType)
    TooLarge
    InvalidUtf8
    Allocation(AllocError)
VariantMeaning
Sqlite(error)A failure reported by SQLite: error.code is its extended result code, error.message its message
ClosedThe connection was closed
InUseclose with statements or a transaction alive
EmptySql, TrailingSqlprepare given no statement or more than one
TransactionActivebegin with a transaction already open
TransactionFinishedcommit or rollback on a finished transaction
InvalidPathAn empty, ":memory:", or NUL-containing path
InvalidExtensionEntryAn extension entrypoint containing NUL
OutOfRangeA parameter, column, limit, or timeout out of range, a NaN, or an unknown parameter name
NoRowA read with no current row
TypeMismatch(found)A read of the wrong storage class
TooLargeA value or row over the connection’s max_length
InvalidUtf8Stored text that is not UTF-8
Allocation(cause)The runtime or SQLite could not allocate

SqliteError.primary() is the primary result code, the extended code modulo 256, using Plenty modulo with a positive divisor (for example, a manually constructed SqliteError with code -1 has primary 255). Internal negative native sentinel codes are decoded into their existing DbError variants before primary-code classification. The module names the common ones as constants: error (1), busy (5), locked (6), read_only (8), cant_open (14), constraint (19), and mismatch (20). For example, a NOT NULL violation has code 1299 and primary code 19. SQLite’s own nomem, toobig, and range codes become Allocation, TooLarge, and OutOfRange.

Ownership and threads

A Connection, Statement, or Transaction is an owned value; none can be copied. A statement or transaction keeps its connection’s SQLite handle alive, so a statement may outlive the Connection value it came from. Dropping a statement finalizes it.

SQLite is compiled thread-safe and every connection is opened in serialized mode, so a connection, statement, or transaction may be moved to a worker thread and used there. Opening and loading an extension are path effects; other operations act on a handle the caller owns. Extension code must itself support the threads on which its functions are used. SQLite’s messages are copied per thread, so concurrent errors on different threads do not overwrite each other.

The handles are ByteStorage values. The __sqlite_* primitives are reserved for the standard library and exist only for this module. Given a handle not made by it, they stop the program with operand is not a std.db.sqlite connection (or statement, or transaction).

Costs

OperationAllocates
connect, memoryA copy of the path, the connection owner, and SQLite’s connection state and page cache
load_extensionCopies of the path and entrypoint, plus loader and extension allocations
prepare, beginThe statement or transaction owner and SQLite’s compiled program
bind_*, step, reset, kind, integer, realNothing in the runtime; SQLite copies bound text and blobs
text, blob, column_nameThe returned value

SQLite allocates through the C allocator, outside Plenty’s allocation accounting and failure injection; its own allocation failures are reported as Allocation.

Supply

The runtime compiles the SQLite amalgamation from an unmodified copy under plenty-runtime/native/sqlite-3.54.0, so building the compiler needs a C compiler (cc, or CC) but no network, and an installed compiler needs neither the sources nor a system SQLite:

LibraryVersionLicenceSource
SQLite3.54.0Public domainsqlite-amalgamation-3540000.zip from sqlite.org, SHA3-256 7b670a62fdfbd672b75fef004cb703c8a3e87d3a5cc7d675b4a08337004a2d93

It is built with SQLITE_THREADSAFE=1, SQLITE_DEFAULT_MEMSTATUS=0, SQLITE_DQS=0 (double-quoted strings are identifiers only), SQLITE_LIKE_DOESNT_MATCH_BLOBS, SQLITE_OMIT_DEPRECATED, SQLITE_ENABLE_FTS5, and SQLITE_OMIT_SHARED_CACHE. Native extension loading is available through Connection.load_extension; SQL loading stays disabled.

Updating it is the runtime maintainers’ job, with each SQLite release that fixes a defect: replace the directory with the new amalgamation, verify its published SHA3-256, update the directory name in build.rs and the table above, and rerun tests/test_std_sqlite.rs, which checks that the vendored header, this page, and sqlite_version() agree.

Executables are linked with --gc-sections, SQLite is compiled with function and data sections, and it is reached only through its own runtime entry symbols, so a program that does not use std.db.sqlite contains none of it. A program that uses SQLite includes the database engine and its enabled features.

Limits

  • No connection pool, ORM, or query builder.
  • No user-defined SQL functions, collations, hooks, or virtual tables.
  • No incremental blob I/O, backup API, or online serialization.
  • No URI file names.
  • No borrowed row views: every text and blob read is a copy.
  • SQLite’s allocations are outside Plenty’s allocation accounting.
  • One target, x86-64 Linux GNU.

Acknowledgements

  • Wraps SQLite 3.54.0, vendored as the C amalgamation and compiled into the runtime (public domain).

std.db.kv

import std.db.kv provides persistent, ordered byte-key storage without SQL in application code. The small public API is implemented in Plenty over the existing std.db.sqlite engine. It adds no native dependency. See the engine decision for the SQLite/redb comparison.

import std.bytes as buffers
import std.db.kv as kv
from std.bytes import BytesError

def main() -> Result[(), IoError | AllocError | BytesError | kv.KvError]:
    mut store = kv.connect("settings.db")?
    store.put("config", &buffers.from_text("theme")?, &buffers.from_text("dark")?)?
    store.close()?

    mut reopened = kv.connect("settings.db", mode=kv.Mode.ReadOnly)?
    match reopened.get("config", &buffers.from_text("theme")?)?:
        case Some(value):
            print(value.to_text()?)?
        case Nothing:
            print("unset")?
    print(reopened.len("config")?)?
    reopened.close()?
    Ok(())
dark
1

Stores and tables

connect(path, mode=Mode.Create, max_length=1048576) returns Result[Store, KvError]. Modes are Create, ReadWrite (existing file) and ReadOnly. memory(max_length=1048576) creates a private in-memory store. Paths are UTF-8 strings with the same validation as SQLite paths.

Logical table names are nonempty strings, bound as data rather than interpolated into SQL. Each table is an independent namespace. It appears on first put; an empty table and a nonexistent table behave identically. Keys and values are byte strings, including empty ones, NULs and non-UTF-8 bytes: inputs are &[u8] views and results are owned std.bytes.Bytes. Keys sort lexicographically by unsigned bytes, shorter equal prefixes first. Integer encodings must be chosen deliberately if numerical ordering matters.

Store methodResultBehavior
put(table, key, value)Result[(), KvError]Insert or replace one key
get(table, key)Result[Option[Bytes], KvError]Copy out the value, or Nothing
contains(table, key)Result[bool, KvError]Test presence without copying the value
remove(table, key)Result[bool, KvError]True if an entry was removed
clear(table)Result[i64, KvError]Remove all entries; return their count
len(table)Result[i64, KvError]Count entries (a table scan, not a cached counter)
set_busy_timeout(budget: &Duration)Result[(), KvError]Retry lock conflicts for this duration (rounded up to milliseconds); default zero
close()Result[(), KvError]Close, or Database(InUse) while transactions/scans still own handles

All methods take &mut Store. Key/value arguments are borrowed and copied into SQLite; returned values are owned and survive the next operation. Ordinary methods prepare a statement per call. Use transactions to amortize commit I/O.

Transactions

store.begin(immediate=True) returns a Transaction. Its commit() and rollback() return Result[(), KvError]. Dropping an unfinished transaction rolls back, including when ? exits a function. Every operation on that Store participates until completion. Outside a transaction each write auto-commits.

Only one transaction may be active per Store. Immediate mode takes the write lock at the beginning; immediate=False defers locks until first use and is appropriate for a read transaction. Deferred does not prohibit writes; use Mode.ReadOnly for a read-only connection. A busy commit can be retried or rolled back according to SQLite’s transaction contract.

Different Store connections follow SQLite isolation and locking. This module uses SQLite’s default rollback journal rather than enabling WAL automatically. Readers may delay a writer’s commit. No background thread, automatic retry loop or connection pool is introduced.

Bounded scans

scan(table, start, stop, max_rows: NonNegative[i64], after=False, reverse=False) returns a Scan over [start, stop); after=True excludes start too, allowing an ascending scan to resume after its last returned key. reverse=True changes the order without changing the bounds. Reversed/equal bounds yield no rows. scan_all(table, max_rows, reverse=False) has no key bounds, so it can include every byte key, including keys composed entirely of 0xff bytes.

The row limit is required and is a NonNegative[i64]; zero yields nothing. A limit truncates the scan, not an error; it does not promise that no more matching rows exist. Scan.next() returns Result[Option[Entry], KvError]; each Entry owns public key and value buffers. Only one row is copied per call. A copy allocation failure preserves the pending row for retry; a SQLite step failure terminates that scan, avoiding silent replay of earlier rows. Repeated calls after exhaustion return Nothing.

The snapshot starts at the first next, not construction. Finish/drop scans before writing through the same Store or ending its transaction: SQLite does not promise a stable scan when its own connection changes the rows being read. Separate Store connections supply normal SQLite snapshot isolation. A Scan keeps its underlying connection alive even if the Store is dropped. Drop the Scan to release its statement and any read lock promptly.

Limits, errors and allocation

max_length is SQLite’s per-text/blob/encoded-row limit, between 1024 and 1,000,000,000 bytes. It includes row encoding overhead; the maximum value payload is therefore smaller. It is not a total database size, page-cache memory cap or execution deadline. Callers choose scan row limits independently.

KvError includes InvalidName, ForeignDatabase, UnsupportedVersion(version), Allocation(AllocError) and Database(std.db.sqlite.DbError). SQLite allocation errors are flattened to KvError.Allocation; other database failures retain their structured cause, including busy, read-only, I/O, closed handles and overlarge values.

Plenty-owned handle/row allocations participate in allocation accounting and failure injection. SQLite’s C allocations are outside those hooks, but its SQLITE_NOMEM failures are returned through the same allocation error.

File compatibility

Files use SQLite’s documented format and a versioned Plenty schema:

PRAGMA application_id = 1347177302; -- ASCII PLKV
PRAGMA user_version = 1;
CREATE TABLE plenty_kv(
    table_name TEXT NOT NULL,
    key BLOB NOT NULL,
    value BLOB NOT NULL,
    PRIMARY KEY(table_name,key)
) STRICT, WITHOUT ROWID;

Mode.Create initializes only an empty, untagged SQLite database, in one transaction. It does not repurpose an unrelated database. Opening a known application ID with an unsupported version fails without migrating it. Version 1 remains readable across Plenty releases; future incompatible changes require an explicit migration/export path rather than silently rewriting old files. SQLite tools can inspect a store directly; use SQLite 3.37+ for STRICT tables. Manual schema/tag changes are unsupported, and the application ID is a format tag, not authentication of an untrusted database.

Durability, recovery and journals remain SQLite’s responsibility. Do not copy an active database by copying just its main file: its journal may be needed for recovery. This API does not yet supply online backup or compaction methods. See SQLite file format and atomic commit.

The tutorial shows a batch transaction and bounded scan. Direct SQL, alternate engines, prefix helpers, zero-copy values and automatic typed serialization are outside this API.

Choosing key-value storage

std.db.kv is a convenience interface over SQLite, not a new database engine. The implementation keeps byte-key policy in Plenty and reuses SQLite for storage, transactions, locking and crash recovery.

ChoiceBest fitTradeoffs
std.db.kvA dedicated file of named byte-key maps, ordered scans and atomic batchesSmall API; copied byte values; prepares statements per call; only the documented KV schema
std.db.sqliteSQL queries, relational tables, joins, indexes, shared application schemas, or tuned prepared-statement reuseMore application SQL, but access to the same mature engine and tools
redb (not shipped by Plenty)An application wanting a specialized embedded ordered KV engineRust-native typed/zero-copy access and MVCC; another file format and dependency; its allocation contract currently prevents a thin Plenty wrapper

SQLite implements ordered-key storage through WITHOUT ROWID. Our composite primary key is (table_name,key), so scans can seek to a namespace and byte range. This avoids a separate rowid table plus primary-key index. SQL parsing, binding and copied-out values still cost work. Direct SQLite can reuse prepared statements for a hot loop; the initial KV convenience API does not cache them. Batch transactions often matter more than individual call overhead for writes. No benchmark here establishes that SQLite is faster or slower than redb for your workload; durability settings, batches, cache and key sizes must match.

Why not wrap redb now?

The evaluated crate is redb 4.4.0, downloaded for source inspection only; it is not a build dependency. Its published design uses copy-on-write B+trees with concurrent snapshot readers and a writer. Its README describes a stable file format with reasonable efforts toward future upgrades. Those are useful properties for a dedicated KV engine.

However, its ordinary database creation and page-cache paths contain infallible Box::new, Arc::new, vector allocation and collection into Arc<[u8]>. Concrete examples are Builder::new/database setup in src/db.rs and zero_filled_arc/read_direct in src/tree_store/page_store/cached_file.rs. There is no fallible-allocation boundary around those paths that a small Plenty wrapper can use. Returning Result for database I/O does not make those allocations fallible, and catching a panic does not recover from allocator aborts.

See the published crate source and crate description. Meeting Plenty’s requirement would need a substantial fallible-allocation audit/change to the engine, not just converting its public error enum. SQLite already reports allocation failures via SQLITE_NOMEM, which Plenty’s existing adapter maps to AllocError. Reusing it is a much smaller change than building and validating another durable storage engine in Plenty.

This choice gives Plenty users the requested KV ergonomics now. It does not promise redb-compatible files, redb-level zero-copy reads, or backend-neutral file interchange. The KV reference publishes the actual SQLite schema and compatibility rules.

std.deque

from std.deque import Deque provides an owned generic ring buffer. It is ordinary embedded Plenty code with private list[Option[T]] storage; it does not change list indexing or list.pop(0) costs.

OperationResultContract
Deque[T]()Result[Deque[T], AllocError]Construct an empty deque
len(queue) or queue.len()i64Observe the live element count
queue.is_empty()boolObserve whether the count is zero
queue.append(value)Result[(), AllocError]Consume a value and append on the right
queue.appendleft(value)Result[(), AllocError]Consume a value and append on the left
queue.pop()Option[T]Remove and transfer the rightmost element
queue.popleft()Option[T]Remove and transfer the leftmost element
queue.clear()()Destroy elements from right to left, retaining capacity
from std.deque import Deque

def main() -> Result[(), AllocError | IoError]:
    mut queue = Deque[str]()?
    queue.append("b")?
    queue.append("c")?
    queue.appendleft("a")?
    print(len(queue), queue.pop())?
    while True:
        match queue.popleft():
            case Some(item):
                print(item)?
            case Nothing:
                break
    Ok(())
3 Option[str].Some("c")
a
b

Mutation requires a mutable owner or exclusive reference. Empty removals return Nothing. Stored elements follow ordinary collection storage restrictions, including affine owners; references, generators, and unit elements are unsupported.

End removal takes constant time apart from payload cleanup and allocates nothing. Appending is amortized constant time; growth takes linear time and doubles capacity, starting at eight slots. It reserves and initializes the full new buffer before transferring any existing element. Allocation or capacity failure preserves existing contents and drops the incoming value. Appending within capacity, removing, and clearing allocate no runtime storage.

The deque moves and requires explicit copy for independent duplication. It has no indexed access, membership operator, iteration protocol, bounded capacity, rotation, or bulk extension API. To consume it, repeatedly pop an end. See the runnable queue lesson.

Acknowledgements

std.diff

import std.diff compares two sequences and returns the edit operations that turn the old one into the new one. It diffs generic lists, bytes, and text split into lines, words, or Unicode scalars, and formats text differences as unified diffs. It is ordinary embedded Plenty code: every allocation failure is reported as AllocError. Unrepresentable frontier storage sizes return AllocError.CapacityOverflow before frontier allocation.

OperationResultContract
std.diff.diff(&old, &new, algorithm, max_cost)Result[list[Op], AllocError]Edit operations between two list[T]; T must be usable as a dictionary key
std.diff.diff_bytes(&old, &new, algorithm, max_cost)Result[list[Op], AllocError]Edit operations between the bytes of two Bytes values
std.diff.lines(old, new, algorithm, max_cost)Result[TextDiff, AllocError]Compare &str texts line by line
std.diff.words(old, new, algorithm, max_cost)Result[TextDiff, AllocError]Compare runs of whitespace and runs of other scalars
std.diff.chars(old, new, algorithm, max_cost)Result[TextDiff, AllocError]Compare Unicode scalar by scalar
text_diff.ops()&list[Op]The operations, indexing the tokens
text_diff.old_tokens(), text_diff.new_tokens()&list[str]Each side’s tokens
text_diff.has_changes()boolFalse when the texts are equal
text_diff.unified(context, old_name, new_name)Result[str, AllocError]Unified diff text, or "" for equal texts

algorithm is optional and defaults to Algorithm.Myers. max_cost is an optional Option[i64] that defaults to Nothing, no limit; see Cost limit. Import Algorithm, Kind, Op, and TextDiff from std.diff to name them.

import std.diff

def main() -> Result[(), AllocError | IoError]:
    old = "apples\nbread\nmilk\n"
    new = "apples\nbutter\nmilk\neggs\n"
    changes = std.diff.lines(old, new)?
    print(changes.has_changes(), len(changes.ops()))?
    print(changes.unified(1, "a/list.txt", "b/list.txt")?.rstrip()?)?
    Ok(())
True 5
--- a/list.txt
+++ b/list.txt
@@ -1,3 +1,4 @@
 apples
-bread
+butter
 milk
+eggs

Operations

An Op has public kind, old_start, old_end, new_start, and new_end fields. It says that old[old_start:old_end] becomes new[new_start:new_end]. op.old_len() and op.new_len() return Result[i64, OverflowError[i64]]. They subtract the public endpoints as given, including representable negative lengths; an unrepresentable difference returns the precise wrapped overflow value. op.is_equal() returns bool. Operations produced by the diff algorithms have nonnegative, representable lengths.

kindOld rangeNew range
Kind.EqualNonemptyEqual elements, same length
Kind.DeleteNonempty, removedEmpty, at the position of the removal
Kind.InsertEmpty, at the insertion positionNonempty, added

Operations are in order and cover both sequences without gaps: each starts where the previous one ended. No operation is empty, two adjacent operations never have the same kind, and a deletion comes before an adjacent insertion. There is no separate replace kind; a replacement is a deletion followed by an insertion. Equal inputs give one Equal operation, or none when both are empty.

Algorithms

Algorithm.Myers finds a shortest edit script: the number of equal elements is the length of a longest common subsequence. It uses the linear-space divide-and-conquer form of Myers’ algorithm, after removing the common prefix and suffix. It takes O((N+M)·D) time and O(N+M) space, where D is the number of deleted and inserted elements. Small changes to large inputs are fast; two large inputs with nothing in common take time quadratic in their size unless max_cost limits the search.

Algorithm.Patience matches the elements that occur exactly once in each sequence, keeps the longest increasing run of those matches as anchors, and runs Myers between anchors. The script is valid but not always shortest. It often reads better for source code, where repeated lines such as braces and blank lines otherwise pull unrelated blocks into alignment.

Both algorithms first intern elements to integer ids through a dictionary, so comparisons during the search are integer comparisons. Text tokens are str copies that share storage with each other but not with the input text.

Cost limit

max_cost=Some(c) bounds the work for inputs that differ a lot, such as large untrusted texts. The result is still a valid script, but may not be shortest.

  • Each step of the divide-and-conquer search looks for the middle of a shortest path through one range. With a limit, that search stops once its forward and backward paths together reach about c deletions and insertions.
  • When the search stops, the range is split at the end of the search path that covers the most elements, as git does without --minimal. The part that path covers is within the limit and is diffed exactly; the rest is diffed again with the same limit. Elements on opposite sides of the split are never matched.
  • If a shortest script for the whole input has at most c deletions and insertions, the limit never fires and the result is the same as without it.
  • Time is roughly proportional to (N+M)·c instead of (N+M)·D. Space is unchanged.
  • A negative c counts as zero. Values up to 2 behave alike: each search stops after one edit in each direction.

Algorithm.Patience applies the limit both to matching the unique elements and to each Myers search between anchors. The limit does not change which elements are compared or how text is split into tokens.

For two texts of 3000 lines with no line in common, lines takes about 300 ms without a limit, 15 ms with max_cost=Some(256), and grows linearly with the input under the limit. Git stops each search after at least 256 edits in each direction, which corresponds to max_cost=Some(512).

Tokens

Each side’s tokens concatenate to its original text.

  • lines splits after each LF. A line keeps its LF, and a CR before it, so a final line without an LF differs from the same line with one.
  • words alternates runs of whitespace (str.isspace) and runs of other scalars.
  • chars makes one token per Unicode scalar, not per grapheme cluster.

Unified format

unified follows the output of GNU diff -U:

  • The text starts with --- old_name and +++ new_name lines. No timestamp is added.
  • Each hunk header is @@ -start,count +start,count @@, with 1-based starts. A count of 1 is omitted, and an empty range gives the line before it as its start, as in @@ -0,0 +1 @@.
  • Each token is written as one line prefixed by a space, -, or +. A token that does not end in LF is followed by an LF and the line \ No newline at end of file.
  • context unchanged tokens surround each change. Changes separated by at most 2 * context unchanged tokens share a hunk. A negative context counts as zero. Even i64.MAX context is accepted without overflowing the hunk comparison.

unified is meant for lines. For word or scalar diffs, every token still becomes its own output line.

There is no inline (word-level) rendering, patch parsing or application, or three-way merge. See the runnable diff lesson.

Acknowledgements

std.encoding

import std.encoding provides hex, base32, base64, PEM and custom codecs and fixed-width integer and IEEE 754 float packing over std.bytes buffers, with one error type, EncodingError.

Byte order, range checks, growth and byte manipulation are checked Plenty code. Four std-only compiler intrinsics reinterpret same-width integer/float bits with a native bitcast; they add no runtime library or dependency. Bytes are accessed through validated buffer operations. Availability matches integer packing on the supported targets; the float tests are qualified on x86-64 Linux GNU.

These are binary encodings. They do not normalize, transcode, or validate text; turning decoded bytes into a str is the separate, strict Bytes.to_text conversion.

import std.bytes
import std.encoding
from std.encoding import EncodingError, Endian

def main() -> Result[(), AllocError | IoError | std.bytes.BytesError | EncodingError]:
    data = std.bytes.from_text("hi!")?
    print(std.encoding.hex_text(&data)?)?
    print(std.encoding.base64_text(&data)?)?
    print(std.encoding.base64_from_text("aGkh")?.to_text()?)?
    mut header = std.bytes.with_capacity(4)?
    std.encoding.append_u32(&mut header, 258u32, Endian.Big)?
    print(std.encoding.hex_text(&header)?)?
    print(std.encoding.unpack_u32(&header, 0, Endian.Big))?
    Ok(())
686921
aGkh
hi!
00000102
258

Codecs

source is a byte view &[u8]: pass &buffer for a whole Bytes, &buffer[start:stop] for part of it, or a b"..." literal.

FunctionResultContract
hex_encode(source)Result[Bytes, EncodingError]Two lowercase digits per byte
hex_decode(source)Result[Bytes, EncodingError]Digits of either case; even length
base64_encode(source)Result[Bytes, EncodingError]RFC 4648 section 4, padded
base64_decode(source)Result[Bytes, EncodingError]The padded form only
base64url_encode(source)Result[Bytes, EncodingError]RFC 4648 section 5, unpadded
base64url_decode(source)Result[Bytes, EncodingError]The unpadded form only
hex_text, base64_text, base64url_text (source)Result[str, EncodingError]The same encoding as a str
hex_from_text, base64_from_text, base64url_from_text (text: &str)Result[Bytes, EncodingError]Decode the whole text

Each result is a new buffer whose capacity equals its length. Encoders produce ASCII only.

For base32/base32hex, MIME line wrapping, PEM, and reusable custom alphabets, see additional codecs. They use the same views, result types, and byte indexes as the functions here.

Accepted forms

The codecs in this table accept one form. They never skip, trim, or repair input: whitespace, line breaks, and any other byte outside the alphabet are errors.

CodecAlphabetLengthPadding
hex0-9, a-f, A-F; encoder writes lowercaseEvenNone
base64A-Z, a-z, 0-9, +, /Multiple of 4= required to complete the last group
base64urlA-Z, a-z, 0-9, -, _Any except 1 more than a multiple of 4= rejected

Base64 text must be canonical: bits of the last symbol that encode no byte must be zero, so each byte sequence has exactly one accepted text per codec. The empty range encodes and decodes to an empty buffer.

Decoder checks, in order:

  1. The range, then the length.
  2. For base64, up to two trailing = are set aside as padding.
  3. Bytes are examined in order. The first one outside the alphabet is InvalidPadding if it is = and InvalidByte otherwise.
  4. The trailing bits of a final partial group.

Integer packing

value has the type in the function name; offset is i64; order is Endian.Little or Endian.Big. There is no default or native order.

FunctionResultContract
pack_u16, pack_u32, pack_u64, pack_i16, pack_i32, pack_i64 (target: &mut Bytes, offset, value, order)()Overwrite the bytes at offset; never grows
append_u16, append_u32, append_u64, append_i16, append_i32, append_i64 (target: &mut Bytes, value, order)Result[(), EncodingError]Grow by exactly the width
unpack_u16, unpack_u32, unpack_u64, unpack_i16, unpack_i32, unpack_i64 (source: &[u8], offset, order)u16, and so onRead the bytes at offset

Endian.Big puts the most significant byte at the lowest offset. Signed values use two’s complement. pack_ and unpack_ require 0 <= offset <= len - width; a short buffer or invalid offset traps before anything is read or written. Bytes are accessed one at a time, so no offset has an alignment requirement. A single byte needs no order: use Bytes.get, set, and push.

Floating-point packing

FunctionResultContract
pack_f32, pack_f64 (target: &mut Bytes, offset: i64, value, order: Endian)()Overwrite exactly four/eight existing bytes; never grows
append_f32, append_f64 (target: &mut Bytes, value, order: Endian)Result[(), EncodingError]Append exactly four/eight bytes
unpack_f32, unpack_f64 (source: &[u8], offset: i64, order: Endian)f32, f64Read exactly four/eight bytes

The value type is the width in the name: f32 is IEEE 754 binary32 and f64 is binary64. Endianness is explicit, with no native-order default, alignment, padding, or object-layout serialization. Sources use the same byte views as integer unpacking: a whole buffer, a borrowed range, or a byte literal.

These operations reinterpret bits without numeric conversion, promotion, arithmetic, or text formatting. They preserve every representation, including positive and negative zero, infinities, subnormals, and quiet/signaling NaNs with their signs and payloads. Every input bit pattern is accepted. Subsequent arithmetic on a decoded float can change a NaN representation and lies outside this codec’s guarantee. Compare re-encoded bytes when testing representations; float equality cannot distinguish signed zeros or verify NaNs.

The integer bounds and failure guarantees apply unchanged. pack_ and unpack_ require 0 <= offset <= len - width, including at unaligned offsets inside a larger record. Negative, excessively large, and truncated ranges trap before reading or modifying bytes. The messages are pack: offset outside target buffer and unpack: offset outside source buffer. Packing leaves length, capacity, and surrounding bytes unchanged. Appending keeps the existing prefix; when growth fails, contents and length remain unchanged and the error is Allocation(cause). Range checks avoid overflowing offset arithmetic.

Errors

pub enum EncodingError:
    InvalidLength
    InvalidByte(i64)
    InvalidPadding(i64)
    NonCanonical(i64)
    InvalidAlphabet
    InvalidPem(i64)
    PemLabel
    Allocation(AllocError)
VariantMeaning
InvalidLengthThe encoded length is impossible for the codec
InvalidByte(index)The byte at index is outside the alphabet
InvalidPadding(index)A configured padding byte at index where the codec does not accept it
NonCanonical(index)The symbol at index has nonzero bits that encode no byte
InvalidAlphabetA custom alphabet or padding symbol violates its construction rules
InvalidPem(index)The PEM framing is malformed at index
PemLabelThe PEM label is not the expected one, or the expected label is invalid
Allocation(cause)Storage for the result could not be allocated, including an unrepresentable expanded size (CapacityOverflow)

An index is a position in the source view; for the _from_text forms it is a UTF-8 byte offset into the text. Every fallible operation returns Result[T, EncodingError]. A failed operation leaves its target unchanged and releases anything it allocated. Wrap a result with std.error.context to name the operation.

Hex, base64, MIME and PEM expansion sizes are checked before allocating output. They return Allocation(CapacityOverflow) rather than trapping on integer overflow. Bounded decoder cursors, fixed-width packing and alphabet arithmetic use explicit unchecked: regions whose source comments state their bounds. PEM decoding records the validated body endpoint instead of reconstructing it with a potentially overflowing size formula. Invalid input keeps its existing codec error and source offset; callers’ arithmetic modes do not change these contracts.

Ownership and threads

Sources are shared loans and targets exclusive loans, so the view rules apply: a buffer cannot be packed into while a view of it is live, and cannot be encoded into itself. The module holds no global state. Its functions pass the ordinary worker-effect checks and may run in any thread.

Costs

This table covers the original hex/base64 codecs and integer/float packing. See additional codec costs for custom table construction and MIME’s temporary buffers.

OperationAllocatesTime
_encode, _decodeOne buffer of exactly the output size, after the range and length checksLinear in the range
_textThe encoded buffer, then a str copy of itLinear
_from_textA Bytes copy of the text, then the decoded bufferLinear
append_Only when capacity is exceeded, as Bytes.resizeConstant, amortized
pack_, unpack_NothingConstant

Output sizes for n input bytes: hex 2n; base64 4 * ceil(n / 3); base64url ceil(4n / 3). A decoder allocates its output before it examines the alphabet, so rejected input pays for that buffer, which is released before returning.

Every byte goes through a checked single-element Bytes call. Measured with a release compiler on x86-64 Linux, pinned to one core, on an 8,388,608-byte input:

OperationWall time
hex_encode1.9 s
hex_decode of the 16,777,216 digits2.5 s
base64_encode1.3 s
base64_decode of the 11,184,812 symbols1.3 s
1,048,576 pack_u64 and 1,048,576 unpack_u642.9 s

That is a few megabytes per second. Bulk native primitives would be faster and are not implemented.

Limits

  • No streaming or incremental codecs: input and output are whole buffers.
  • No arbitrary line widths, automatic case folding, or permissive recovery from damaged encodings.
  • No variable-length integers, 128-bit integers, or additional float formats.
  • No record-layout description: pack fields one call at a time.

Acknowledgements

  • RFC 4648 specifies the base16, base32 and base64 formats implemented by the codecs.
  • RFC 7468 specifies the textual encapsulation and label syntax used by PEM encoding.
  • IEEE 754 defines the binary32 and binary64 representations used by float packing; no standard text or external implementation is shipped.

Base32, MIME, PEM and custom alphabets

These operations extend std.encoding. They are implemented in Plenty over Bytes, without another runtime primitive or native dependency. Inputs are whole buffers; there is no streaming state.

import std.bytes
import std.encoding
from std.encoding import EncodingError

def main() -> Result[(), AllocError | IoError | std.bytes.BytesError | EncodingError]:
    data = std.bytes.from_text("hello")?
    print(std.encoding.base32_text(&data)?)?
    print(std.encoding.base32_from_text("NBSWY3DP")?.to_text()?)?
    lower = std.encoding.custom("abcdefghijklmnopqrstuvwxyz234567", Nothing)?
    text = lower.text(&data)?
    print(text, lower.from_text(&text)?.to_text()?)?
    Ok(())
NBSWY3DP
hello
nbswy3dp hello

Base32

base32 uses the uppercase alphabet ABCDEFGHIJKLMNOPQRSTUVWXYZ234567 from RFC 4648 section 6. base32hex uses 0123456789ABCDEFGHIJKLMNOPQRSTUV from section 7. Both require = padding to complete eight-symbol groups. They are case-sensitive and do not accept whitespace, 0/1 substitutions, or nonzero unused trailing bits.

Function suffix, for both base32 and base32hexResult
_encode(source: &[u8])Result[Bytes, EncodingError]
_decode(source: &[u8])Result[Bytes, EncodingError]
_text(source: &[u8])Result[str, EncodingError]
_from_text(text: &str)Result[Bytes, EncodingError]

For example, f encodes as MY====== or CO======. Unpadded forms are available through custom below; a decoder never guesses the padding policy. The empty input encodes as empty, not a group of padding characters.

Range validation comes first. Padded lengths must be a multiple of eight. After stripping trailing padding, the number of symbols must describe whole bytes with fewer unused bits than a symbol. Padding must fill exactly one partial group. Symbols and trailing bits are then checked in input order. Impossible lengths produce InvalidLength; excessive or misplaced padding produces InvalidPadding(index) when the preceding length checks pass.

MIME line wrapping

base64mime_encode, base64mime_decode, base64mime_text and base64mime_from_text have the corresponding base64 signatures.

The encoder produces padded standard base64 with CRLF between lines of at most 76 symbols, following the line width of RFC 2045 section 6.8. It adds no final CRLF and emits nothing for empty input. Add the surrounding message headers and any final line termination in the enclosing protocol.

The decoder skips only ASCII space, tab, CR and LF. It then applies ordinary strict base64 length, alphabet, padding and trailing-bit validation. It accepts unwrapped text and does not enforce a line width. This is deliberately narrower than RFC 2045’s permissive recovery from arbitrary non-alphabet bytes: punctuation, non-ASCII whitespace, and damaged padding remain errors. Error indexes still refer to the original buffer, including skipped whitespace; _from_text indexes are UTF-8 byte offsets. It does not parse MIME headers or multipart messages.

PEM

pem_encode(source: &[u8], label: &[u8])
    -> Result[Bytes, EncodingError]
pem_text(source: &[u8], label: &[u8])
    -> Result[str, EncodingError]
pem_decode(source: &[u8], label: &[u8])
    -> Result[Bytes, EncodingError]
pem_from_text(text: &str, label: &[u8]) -> Result[Bytes, EncodingError]

These read and write one block in the strict form of RFC 7468 (section 3, stricttextualmsg), the textual wrapping of DER keys and certificates. label is the text between BEGIN and the dashes, such as b"PUBLIC KEY" or b"PRIVATE KEY"; a bytes literal costs no allocation. The decoder takes the label it expects, so a private key cannot be read where a public key was meant.

import std.bytes
import std.encoding
from std.encoding import EncodingError

def main() -> Result[(), AllocError | IoError | std.bytes.BytesError | EncodingError]:
    pem = "-----BEGIN PUBLIC KEY-----\nMCowBQYDK2VwAyEA11qYAYKxCrfVS/7TyWQHOg7hcvPapiMlrwIaaPcHURo=\n-----END PUBLIC KEY-----\n"
    der = std.encoding.pem_from_text(pem, b"PUBLIC KEY")?
    print(der.len(), std.encoding.pem_text(&der, b"PUBLIC KEY")? == pem)?
    print(str.repr(std.encoding.pem_from_text(pem, b"PRIVATE KEY"))?)?
    Ok(())
44 True
Result[std.bytes.Bytes, std.encoding.EncodingError].Err(std.encoding.EncodingError.PemLabel)

The decoder accepts exactly:

  • -----BEGIN LABEL----- at the start of the view, with the expected label;
  • one line terminator, LF or CRLF, used for every line that follows;
  • padded standard base64 in lines of exactly 64 symbols, except a final line of 1 to 64 symbols, with at least one symbol in all;
  • -----END LABEL----- with the same label;
  • at most one final line terminator, then the end of the view.

Explanatory text around the block, RFC 1421 headers such as Proc-Type, blank lines, spaces, tabs, and other line widths are errors. A label that is not an RFC 7468 label (printable ASCII except -, with single spaces or hyphens only between other characters) or a BEGIN label other than label is PemLabel. Other structural errors are InvalidPem(index); base64 errors give the same variants as base64_decode. Every index is a position in source. The decoder does not interpret the DER it returns; the key import functions do.

The encoder writes that form with LF terminators, including a final one. An empty range has no strict PEM form and is InvalidLength.

Reusable custom encoding

custom(symbols: &str, padding: Option[u8] = Some(61u8))
    -> Result[Encoding, EncodingError]

symbols lists digits in increasing value order. It must contain 2, 4, 8, 16, 32 or 64 distinct printable ASCII bytes (! through ~). A padding byte must also be printable ASCII and must not occur in the alphabet. Invalid settings return InvalidAlphabet. Some(61u8) means =; Nothing forbids padding. With no configured padding, = may itself be an alphabet symbol.

Encoding owns its validated tables and has four methods:

MethodResult
encode(source: &[u8])Result[Bytes, EncodingError]
decode(source: &[u8])Result[Bytes, EncodingError]
text(source: &[u8])Result[str, EncodingError]
from_text(text: &str)Result[Bytes, EncodingError]

Bits are read most significant first. Padding completes the smallest group of symbols representing whole bytes. Decoding accepts only the selected alphabet and policy, with zero unused trailing bits; it does not fold case, translate lookalike characters, ignore whitespace, or automatically detect a radix. If no padding is configured, a byte outside the alphabet is InvalidByte, including =. If configured, misplaced occurrences of that padding byte are InvalidPadding.

For a known protocol, prefer the named codec. Construct a custom encoding once and reuse it when the protocol requires a different alphabet or padding. The tutorial demonstrates this.

Allocation and limits

Base32 and custom encode/decode allocate one result buffer of exactly its final length. Fixed base32 alphabets need no temporary table. custom allocates its alphabet and a 128-byte decoding table once; its shared methods do not mutate them. Construction and all codec allocations report EncodingError.Allocation. The source and existing encoding remain unchanged on failure.

MIME encoding holds ordinary base64 plus its wrapped result; MIME decoding holds a whitespace-free copy plus its decoded result. These are whole-buffer copies, not zero-copy adapters. _text additionally copies output into a str; _from_text additionally copies input into Bytes, as with existing codecs. PEM encoding holds ordinary base64 plus the framed result; PEM decoding holds the base64 body without line terminators plus its decoded result. A decoded private key is therefore left in freed buffers that are not zeroed; see key material.

All operations take linear time in input length. Base32 output is 8 * ceil(n / 5) bytes. Unpadded custom output is ceil(8n / bits_per_symbol); padded output rounds that to a whole symbol group. New radix encoders reject unrepresentable size arithmetic with InvalidLength. No custom bit order, case folding, ignored-character configuration, or arbitrary wrapping is exposed.

std.env

import std.env reads the process environment and the working directory. Names, values, and directories are operating-system byte strings; text is a checked view of them.

Encoding and validation are checked Plenty; four native primitives read a variable, snapshot the environment, and read or change the working directory. The module is available on the supported x86-64 Linux GNU target; elsewhere every operation reports IoError.Unsupported.

import std.env

def main() -> Result[(), IoError]:
    mode = std.env.get("PLENTY_EXAMPLE_MODE")?.unwrap_or("development")
    print("mode:", mode)?
    cwd = std.env.current_dir()?
    print(cwd.is_absolute())?
    Ok(())
mode: development
True

Variables

FunctionResultContract
get(name: &str)Result[Option[str], IoError]The value as text; Nothing when unset
get_bytes(name: &list[u8])Result[Option[list[u8]], IoError]The value unchanged
vars()Result[list[Variable], IoError]A snapshot of every variable
valid_name(name: &list[u8])boolNonempty, with no = and no NUL

Variable has two public list[u8] fields, name and value.

  • An unset variable is Ok(Nothing). A variable set to the empty string is Ok(Some("")).
  • A name that is not valid_name is IoError.InvalidInput: no variable can have it.
  • get reports a value that is not UTF-8 as IoError.Data(DataError.InvalidUtf8). It never replaces or drops bytes; read such a value with get_bytes.
  • vars returns each variable once, in the order the process stores them. The snapshot is detached: it does not follow later changes.
  • Each result is a new allocation; failure is IoError.Data(DataError.Allocation(...)).

There is no operation that sets or removes a variable of the running program. The environment is process-wide state that other threads and native code read without synchronization. Give a child its variables through Command.env instead. Native code embedding Plenty must likewise avoid concurrent mutation of the process environment while runtime calls read it. Both single-variable lookup and snapshots copy directly into fallible Plenty storage, without intermediate host strings or snapshots.

Command-line arguments are the prelude’s args and args_bytes, and typed options are std.cli.

Working directory

FunctionResultContract
current_dir()Result[Path, IoError]The absolute working directory
set_current_dir(path: &Path)Result[(), IoError]Change it for the whole process

Both use std.path.Path, so a directory name that is not UTF-8 round-trips. current_dir fails with the system error when the directory has been removed. set_current_dir reports a missing directory as IoError.System(2) and leaves the working directory unchanged. current_dir() grows a fallible native buffer and reports allocation failure as IoError.Data(DataError.Allocation(...)).

The working directory is shared by every thread, and relative paths resolve against it at the moment of each call. Prefer absolute paths, or Command.current_dir for a child, over changing it in a program with more than one thread. A worker thread may not call set_current_dir at all; the compiler rejects any worker that can reach it (see native effects).

Limits

  • Linux x86-64 GNU only.
  • No mutation of the program’s own environment.
  • No executable-path lookup. The home and XDG base directories are std.fs.dirs; temporary-directory selection is std.tempfile.default_dir.

Acknowledgements

std.error — explicit mapping and contextual errors

Import std.error. A Context[E] records which operation failed and, when there is one, the path it acted on, around an unchanged typed cause E. The module is ordinary embedded Plenty code over Result and std.path; it adds no exceptions, implicit conversions, or runtime support.

pub enum Subject:
    Absent
    At(Path)
    Unavailable(AllocError)

pub class Context[E]:
    pub operation: str
    pub subject: Subject
    pub cause: E
OperationResultContract
map_error(result, mapper)Result[T, G]Consumes Result[T, E]; invokes an owned OnceCallable[[E], G] only on Err; same as the result.map_error(mapper) method
Context(operation, subject, cause)Context[E]Takes ownership of all three; allocates nothing
context(result, operation)Result[T, Context[E]]Wraps an Err with Subject.Absent; allocates nothing
path_context(result, operation, &path)Result[T, Context[E]]Wraps an Err, copying the path only then
error.describe()Result[str, AllocError]One-line text ending in str.repr of the cause
error.describe_with(&cause_text)Result[str, AllocError]The same text with a caller-supplied cause
error.format(&mut output)Result[(), FormatError]Borrowed std.report formatting when E satisfies Formattable
error.report()()Writes error: and the description to standard error
error.report_with(description)()The same for a description Result the caller built
native_code(error)Option[i32]The OS code of IoError.System(code); otherwise Nothing

context and path_context consume the Result and return Ok payloads unchanged, including () from operations such as flush_stdout() and File.close(). Conversion is always this explicit call: ? neither wraps an error in a Context nor unwraps one, and the identical-error rule applies to Context[E] as to any error type. A Context moves, because it can own a path. Its fields are public: read error.cause or match it to recover the typed cause, at any nesting depth. ? in a function returning Result[T, Failure], the last-resort erased error, discards a Context like any other error; list Context[E] in the sum to keep it.

from std.error import Context, path_context
from std.path import Path, from_text

def read_settings(path: &Path) -> Result[list[u8], Context[IoError]]:
    path_context(read_bytes(path.as_bytes()), "read settings", path)

def main() -> Result[(), AllocError | IoError]:
    path = from_text("settings.toml")?
    match read_settings(&path):
        case Ok(bytes):
            print(len(bytes), "bytes of settings")?
        case Err(error):
            # A missing settings file is expected: report it and use defaults.
            print(error.describe()?)?
            print(error.cause == IoError.System(2i32))?
            print("using default settings")?
    Ok(())
read settings "settings.toml": IoError.System(2)
True
using default settings

Explicit error mapping

result.map_error(mapper) consumes Result[T, E] and returns Result[T, G], where mapper is an owned OnceCallable[[E], G]. A one-payload enum variant is the usual mapper; use a named classifier when several error variants need different policies, or a closure to add other payloads or captured context:

i64.parse(text).map_error(ConfigError.Parse)?
print(line).map_error(def [path](cause: IoError) -> AppError: AppError.Write(path, cause))?

The library function map_error[T, E, G, F: OnceCallable[[E], G]](result: Result[T, E], mapper: F) has the same behavior; the method is a compiler-provided Result method, not an import. A class or enum that defines its own map_error keeps it.

The receiver and mapper expressions run once, receiver first, even on success. The mapper runs exactly once on Err, taking ownership of the original error; on Ok, the success payload moves through unchanged and the unused mapper is dropped. Named functions, reusable closures (including mutable environments), and consuming closures are accepted. Captures and affine payloads follow normal cleanup rules, including early return while constructing the mapper.

Mapping adds no allocation or boxing. Callback construction, execution, and destruction retain their ordinary effects. If the callback returns Result[G, X], the output is Result[T, Result[G, X]]: nothing is flattened and no cause-preservation policy is invented. Use context or path_context when their explicit context-allocation policy fits.

? still requires identical error types (except legacy Failure erasure); the conversion is visible at the map_error call. The mapper must be owned and take exactly the error type: a reference, a variant with several payloads, or a callable returning unit is rejected. .map_err(...) is not implemented. See the lesson.

Worker-reachable calls accept thin named or anonymous function values, including variant constructors such as ConfigError.Parse, when every function address of that callable signature in the program passes worker-effect certification. The compiler checks every candidate body and its stored types; one safe target never excuses an unsafe one. A main-thread-only address with the same signature can conservatively prevent certification. A concrete closure retains its own body and capture evidence; it cannot bypass the checks either.

Allocation failure

Attaching context never replaces the cause. path_context copies the borrowed path only on Err; if that copy fails, the context holds Subject.Unavailable(alloc_error) with the original operation and cause. To avoid the copy, move an owned path into Context(operation, Subject.At(path), cause). The Ok path of both helpers, construction, and field access allocate nothing.

describe allocates the cause’s representation, the escaped path, and intermediate strings, and returns AllocError if any step fails. The context is unchanged either way.

Description format

operation: cause
operation "escaped path": cause
operation <path unavailable>: cause

The path uses Path.display: printable ASCII is kept, backslash and double quote are escaped, and every other byte is \xhh, so a description is one line of ASCII apart from the operation and cause. describe renders the cause as str.repr does; a native code appears as IoError.System(2).

A nested context is a cause like any other, so describe shows its structural representation. Describe the inner context first and pass the text outward:

outer.describe_with(&outer.cause.describe()?)

That gives load settings: read_bytes "missing/settings": IoError.System(2). An application error enum can match its variants and call the method that fits each payload. std.fmt.Formattable provides borrowed formatting for application types. Context[E] supports this protocol when E does: it emits the operation and subject, then calls render on the typed cause exactly once. Nested contexts therefore produce nested operation labels without duplicating the cause. Each nested render counts toward the depth limit.

This conditional method is compiler-supported for the embedded std.error type; it does not introduce conditional methods for arbitrary generic classes. An unformattable cause keeps the existing description APIs and structural entrypoint reporting. describe and describe_with retain their existing behavior even when the cause has a format method.

Formatting an absent or unavailable subject adds no allocation. A present path uses the allocating Path.display; failure is returned as FormatError.Allocation(...), preserving the cause and any already-written prefix. Operation and cause text are emitted unchanged.

Reports

report flushes standard output, then writes error: , the description, and a newline to standard error in one write. It returns unit: flush and write failures are dropped, because standard error is the last channel. The caller keeps the error and chooses the exit status, normally by returning i32 from main.

Neither fallback allocates:

ConditionStandard error
The description cannot be allocatederror: operation: no memory to describe the error
The description exists but the whole line cannot be allocatedThe normal line, written in three pieces

report_with(description) applies the same rules to a description the caller built, such as the nested one above; pass the Result unhandled.

Returning Err(context) from main produces the entrypoint report. It selects the typed formatting method when the cause is formattable, otherwise renders the context structurally and shows a path as its byte list. The explicit report and report_with APIs keep their description-based behavior.

Conventions for standard modules

  1. An operation returns its precise error type: IoError, AllocError, or the module’s own enum. It does not attach context, format text, or allocate to describe a failure.
  2. A module enum that can fail from allocation has a variant carrying the AllocError, as WalkError.Allocation and DataError.Allocation do. Allocation failure is never folded into Other or Failure.
  3. A native code is kept as IoError.System(code) and never translated to text in the error value.
  4. A composite operation that must say which step or path failed returns Context[E], or an enum whose variants hold contexts. The operation label is a string literal naming the public operation, such as read_dir.
  5. Failing to allocate context reports the original cause, with Subject.Unavailable.
  6. Only an application decides to print. Library code returns errors; report belongs at the boundary that owns the exit status.

WalkError.Io predates this module and keeps its own (path, operation, cause) payload.

Limits

  • describe and report require a cause type str.repr accepts. A cause holding recursive data or a generator is rejected where one of them is first called for that Context[E].
  • No OS description text is added to a native code.
  • The legacy report and report_with methods use write_stderr: a pipe whose reader has gone terminates the process with SIGPIPE. The std.report formatter and entrypoint reporter suppress it and preserve normal cleanup.
  • A piecewise fallback report is not atomic against concurrent writers.
  • Subject has no network address or other non-path variant.

A Context[E] can cross threads when E can. Every operation takes time linear in the text it produces. Supported target: x86_64 Linux GNU. The lesson is Say which operation failed.

Acknowledgements

  • context and path_context are modelled on the Context trait of the Rust anyhow crate.

std.fmt

std.fmt separates user-defined display from debug output and streams UTF-8 to any static std.io.Writer. It leaves builtin print unchanged. Formatting policy is checked Plenty; the native primitives only write floating-point spellings into existing byte storage. There is no runtime reflection, format-string interpreter, or dynamic writer object. f-strings compile to calls on this module, and out.format(f"...") writes one to a formatter out without an intermediate str.

The same formatter serves std.report: its writer-generic Formattable.format protocol shares these methods and errors. stderr_formatter(max_bytes, max_depth) creates a Formatter[Stderr] without allocating; text, boolean and integer output on that path also needs no heap storage. Advanced numeric formatting acquires scratch lazily.

import std.fmt
from std.fmt import Formatter

pub class Point:
    x: i64
    y: i64

    pub def display[W](self, out: &mut Formatter[W]) -> Result[(), FormatError]:
        out.text("(")?
        out.integer(self.x, 0)?
        out.text(", ")?
        out.integer(self.y, 0)?
        out.text(")")

def main() -> Result[(), AllocError | IoError | FormatError]:
    print(std.fmt.to_string(&Point(3, -4), 80, 8)?)?
    mut out = std.fmt.string_formatter(80, 8)?
    out.text("total:")?
    out.float_spec(1234.5, &std.fmt.spec(width=10, grouping=",", precision=2, kind="f"))?
    print(std.fmt.finish(&out)?)?
    Ok(())
(3, -4)
total:  1,234.50

Writers and strings

formatter(writer, scratch, max_bytes, max_depth) consumes a writer and a Bytes scratch buffer and returns Result[Formatter[W], FormatError]. Scratch must already have length at least 32, not just capacity. Both limits are NonNegative[i64], so a negative literal is a compile error. Depth is at most 64, since nested display calls recurse on the native stack. A shorter scratch or a larger depth traps with std.fmt.formatter: scratch is shorter than 32 bytes or std.fmt.formatter: max_depth is above 64; stderr_formatter and string_formatter trap the same way for depth. Construction consumes both arguments even on failure. The formatter owns them; its public writer field lets the caller flush, close, or inspect the underlying stream. Formatting never flushes or closes it.

string_formatter(max_bytes, max_depth) creates a growable in-memory cursor and scratch. Text emissions on this collector stage their complete UTF-8 spelling in reusable scratch before one cursor write, preserving the output on failed allocation. finish(&output) validates and copies its current bytes into a str; the formatter remains usable. Both are fallible. to_string(&value, max_bytes, max_depth) and to_debug(...) construct a formatter, invoke the corresponding user contract below, and return a string. They return no partial string on error. Use a formatter directly to inspect partial progress or reuse storage.

All formatting methods use an exclusive receiver and return Result[(), FormatError]:

MethodForm
text(&str)Exact UTF-8, including embedded NUL
debug_text(&str)Double quotes; escapes ", \, LF, CR, TAB; other C0/C1 controls use lowercase \u{hex}; other Unicode is preserved
ascii_text(&str)The text with each non-ASCII scalar as lowercase \u{hex}; ASCII is unchanged
substr(&substr), debug_substr(&substr)text and debug_text for a substring view, without copying it
boolean(bool)True or False
integer(i64, width: NonNegative[i64] = 0)Signed decimal, including i64.MIN
unsigned(u64, base, width: NonNegative[i64] = 0)Base 2, 8, 10, or 16, lowercase digits, no prefix
floating(f64, width: NonNegative[i64])Shortest round-trip scientific decimal, e.g. 1.25e0, -0e0, inf, -inf, NaN

Smaller integer/float types can be explicitly converted. Width is a minimum byte count, padded on the left with ASCII spaces, including any sign. It never truncates. An unsupported base returns InvalidSpec before output; convert a computed width with NonNegative.try_from(width)?. These numeric spellings are ASCII, so byte and scalar widths coincide; no grapheme or terminal-cell width is promised. Text has no implicit padding. floating has no precision or locale controls; promoting f32 to f64 spells the promoted value. The spec methods below add Python’s options. There is no format-spec parser: a Spec is built from its fields, and f-strings parse their specs while compiling.

Format specs

Spec holds one field’s options as public fields, in this order: fill (one scalar), align ("", "<", ">", "^", "="), sign ("", "+", "-", " "), coerce_zero (z), alternate (#), zero (0), width, grouping ("", ",", "_"), precision (-1 when absent), and kind (the type letter or ""). spec(...) builds one with keyword defaults, for example spec(width=8, precision=2, kind="f"). Width and precision are at most 4096.

MethodOptions
text_spec(&str, &Spec)Fill, alignment (default left), width; precision truncates. Widths count scalars
signed_spec(i64, &Spec), unsigned_spec(u64, &Spec)Types d, x, X, o, b; # prefixes; sign; , (decimal) or _ grouping; 0 grouped zero padding; default right
signed_spec, unsigned_spec with kind nAs d, without grouping
char_spec(i64, &Spec)Kind c: the scalar with that code point; fill, alignment (default right), width, 0 padding; InvalidSpec for a value that is not a Unicode scalar
float_spec(f64, &Spec), float32_spec(f32, &Spec)Types e, E, f, F, g, G, %, n (as g, without grouping), or none; precision (default 6 when typed); #; sign; coerce_zero drops the sign of a result whose digits are all zero; grouping; 0 padding

Their spellings match Python’s format, except that a float with neither type nor precision uses print’s spelling. A combination the method’s type does not accept, such as a sign on text or a precision on an integer, returns InvalidSpec before output. f-strings reject these at compile time.

Spec numbers are spelled in scratch at offset 8. A spelling longer than the scratch, such as a 64-digit binary integer or 1e300 with f, first grows the scratch fallibly (Allocation(...)); preallocate scratch to keep formatting allocation-free. represent(&value, &mut out) writes the str.repr spelling of any value str.repr accepts, which allocates a temporary string. A Bytes buffer, bytes literal, or &[u8] view spells as a byte literal (printing bytes); write hex with std.encoding.hex_text. repr_string(&value), debug_string(&text), and ascii_string(&text) return the str.repr, debug_text, and ascii_text spellings as fallible strings. A &T parameter cannot take a slice view, so represent_view(v, &mut out) and repr_view_string(v) do the same for a &[T]; f-strings call them for view fields. For a &substr, f-strings call substr, substr_spec(v, &spec) (the text_spec options), and repr_substr_string(v), the debug_substr spelling as a fallible string. When a field’s str.repr spelling runs a debug method, the f-string calls str.repr at its own site and passes the result to repr_text(spelled, &mut out) or repr_result(spelled), which turn AllocError | FormatError into a FormatError. An f-string builds its text as Result[str, FormatError] and converts the result at its site: allocation_only(built) gives Result[str, AllocError] when no field runs a user method or has a computed spec, and allocation_or_format(built) gives Result[str, AllocError | FormatError] otherwise. Both report Allocation(cause) as cause. allocation_only traps on any other FormatError, which the compiler’s choice rules out.

f-strings check the values of a spec’s nested fields with three functions, each returning InvalidSpec for a value the spec cannot take: spec_align(&align) accepts "", <, >, ^, and =; spec_fill(&fill, &align) turns "" into the default space and otherwise requires one scalar and a non-empty alignment; spec_count(n) accepts a width or precision in 0..=4096.

Static user contracts and aggregates

A public class or enum opts in by providing either or both public methods:

pub def display[W](self, out: &mut Formatter[W]) -> Result[(), FormatError]:
    ...
pub def debug[W](self, out: &mut Formatter[W]) -> Result[(), FormatError]:
    ...

std.fmt.display(&value, &mut out) and debug(...) check the selected method at each concrete generic instantiation. These helpers use ordinary structural calls rather than named protocol bounds. Missing/private methods, wrong signatures, moves, conflicting loans and worker effects get ordinary compiler diagnostics. to_string/to_debug use these user methods; primitive scalars use the explicit methods in the table instead of acquiring hidden methods. An enum method can match its borrowed receiver to render the active payload. print, str.repr and f-strings call the same methods, as user formatting methods describe; each call there starts a fresh depth of 32. Entrypoint error reports do not call them, and anonymous sums do not automatically forward them to members.

list_with(&values, &mut out, render) writes [a, b] using a statically typed callback (&T, &mut Formatter[W]) -> Result[(), FormatError] for each borrowed element. pair_with(&pair, &mut out, first, second) similarly writes (a, b) with separate callbacks for a heterogeneous pair. Callbacks choose display or debug and may recursively use these helpers. Other tuple arities, maps, enums and classes compose these primitives explicitly; there is no automatic derive. See the formatting lesson.

Display, debug, list and pair entrypoints each enter one depth level and restore it on success or error. Depth zero permits scalars but rejects those entrypoints. Custom recursive code must recurse through these entrypoints or pair enter()/leave() itself; ordinary direct method recursion cannot be policed. A custom renderer can still loop or deliberately emit unbounded work.

Errors and partial output

FormatError has InvalidSpec, LimitExceeded, RecursionLimit, Allocation(AllocError), Output(StreamError), and OutputClosed variants. String, scratch and sink allocation failures are Allocation(...). Terminal stderr errors set has_failed(); later emissions on that formatter return OutputClosed without writing. Other destinations retain their writer’s recovery policy: a writer error does not make the formatter sticky. Allocation and byte/depth limit failures do not set has_failed(). completed() is the exact count accepted by the writer since construction, including a successful prefix before any error.

The writer’s positive short counts are retried, Interrupted is retried, zero is WriteZero, and negative/oversized counts are InvalidProgress. An Err must mean that call accepted no bytes, as required by std.io. There is no rollback or implicit retry of an entire failed formatting operation. Retrying it can duplicate its successful prefix. A partial OS write can split UTF-8; finish then reports Bytes(InvalidUtf8) until valid text is restored.

The budget for text is checked for the whole UTF-8 input before output; other emissions check each scalar, ASCII byte, or float spelling. Limit errors never exceed it; previous output remains. Budgets count UTF-8 bytes, escapes, delimiters and padding, not input characters. Direct writes through the public writer field are outside the counter/budget contract. Callers must not alter its position or contents during a formatting operation.

Formatting internals use explicit unchecked: regions for arithmetic already bounded by validated output budgets, scratch positions, radix digit counts, format widths/precision or recursion limits. Unrepresentable user input does not acquire a hidden arithmetic trap: existing InvalidSpec, LimitExceeded, allocation and output errors retain their contracts. User display/debug methods still have their own declared arithmetic boundaries, independent of formatter implementation modes.

Cost and availability

After construction, formatting scalars performs no allocation when the writer does not allocate. The caller can preallocate a Cursor buffer to make this true for in-memory streams. Caller-provided scratch is reused for scalar streaming; string formatters may grow scratch to stage a whole text emission. String output grows explicitly and finish copies the complete bytes. A custom renderer/callback can allocate, and its ordinary effects are not suppressed.

Text is linear in input bytes plus output bytes, integer work is bounded by 64 digits plus padding, and f64 conversion has fixed bounded scratch work plus padding. A spec float costs time linear in its precision and magnitude digits. A padded float may be converted twice because padding reuses scratch. List work is linear in element count plus callback work. At most one Unicode scalar (four bytes), one ASCII byte, or one float spelling (at most 32 bytes) is offered per write on a caller-provided writer; wrap a syscall writer in BufWriter to batch these calls. A string formatter submits a whole text emission to its cursor. Direct stderr text submits the unconsumed UTF-8 suffix. Memory is caller scratch plus writer state and bounded recursive call frames.

Formatters are affine because they own Bytes; copying is rejected. Loans and worker transfers follow the underlying writer’s normal ownership/effect rules. The embedded module works offline with a relocated matching toolchain on the supported x86-64 Linux GNU target and adds no external library dependency.

Acknowledgements

std.fs — directory enumeration

Import std.fs and pass a borrowed Path. Metadata and links covers native-byte symlink creation and canonicalize resolution as well as file observations. For owned temporary resources, copy/move/remove and bounded glob filtering, see rooted helpers and file/tree copying. The home and XDG base directories are in std.fs.dirs. For change notifications, see std.fs.watch. For cooperating-process locking, see std.fs.lock. read_dir(&path) -> Result[ReadDir, IoError] opens one directory immediately. ReadDir is affine and owns a lazy native stream. next() requires a mutable owner and returns Result[Option[DirEntry], IoError]: Some is one entry, Nothing is EOF or an already closed stream, and Err is an advancement failure after opening. Any advancement error, including allocation failure, terminates and closes the stream. Exhaustion also closes it before returning EOF.

import std.fs as fs
import std.path as path

def main() -> Result[(), AllocError | IoError]:
    write_text("a.txt", "alpha")?
    write_text("b.txt", "beta")?
    root = path.from_text(".")?
    mut names: list[str] = []?
    for result in fs.entries(fs.read_dir(&root)?):
        entry = result?
        name = entry.file_name()?.to_text()?
        if name.endswith(".txt"):
            names.append(name)?
    names.sort()
    print(names)?
    Ok(())
["a.txt", "b.txt"]

close() -> Result[(), IoError] explicitly relinquishes the stream; repeated close is harmless, including after a close error. closed() -> bool observes the state. Drop closes any remaining stream without allocating or reporting a close error. Scope exit, return, ?, and dropping a suspended consuming generator release their owned stream normally.

entries(read_dir(&root)?) consumes the owner and returns Generator[Result[DirEntry, IoError]]. Handle each item’s Result explicitly. The existing generator/iteration system supplies deterministic cleanup on an early break; no separate fallible for protocol is introduced. Breaking out of a manual while leaves its named ReadDir alive until close or scope exit.

Only structural . and .. are excluded. Hidden names remain visible, order is unspecified native order, and enumeration is not a snapshot. Sorting and ignore rules belong to callers. Additions/removals during enumeration can affect which entries appear.

DirEntry operationResult
file_name()Fallible detached Path containing the exact native name
path()Fallible detached Path joined to the supplied root
file_type()Result[FileType, IoError], without following a link
metadata()Fresh Result[Metadata, IoError], without following a link

Entries own copied path bytes and cached native type information; they never borrow libc’s reused readdir buffer or retain a directory descriptor. Keeping entries in a list survives advancing, closing, or dropping the stream. Name/path copies can fail without invalidating the entry. Relative roots remain relative, and the joined path is not canonicalized.

Type-only queries use d_type when available. An unknown native type triggers a non-following metadata lookup and propagates any lookup error. FileType is File, Directory, Symlink, or Other; sockets, FIFOs and devices are Other, never regular files. Cached native types can become stale; entry metadata is fresh on every call. See metadata and links.

Python’s os.scandir and Rust’s read_dir/DirEntry provide the API baseline. The runtime follows their Unix errno-reset/EOF distinction but copies detached entry data fallibly instead of adopting Rust DirEntry’s descriptor retention. Native enumeration currently supports 64-bit Linux; unsupported hosts return IoError.Unsupported. See walking for tree traversal.

Acknowledgements

  • Directory API modelled on Python’s os.scandir and Rust’s std::fs::read_dir.
  • Uses the Rust libc crate for target-specific Linux filesystem syscalls and flags (MIT OR Apache-2.0).

Metadata and links

In std.fs, metadata(&path) follows symbolic links, while symlink_metadata(&path) inspects the link itself. Both return Result[Metadata, IoError]; missing paths, inaccessible paths and broken link targets preserve their OS error. DirEntry.metadata() follows the latter rule. No primitive converts an error to False. Broken links remain enumerable and inspectable with non-following operations.

import std.fs as fs
import std.path as path

def main() -> Result[(), AllocError | IoError]:
    write_text("notes.txt", "hello")?
    notes = path.from_text("notes.txt")?
    info = fs.metadata(&notes)?
    print(info.is_file(), info.is_dir(), info.size())?
    current = path.from_text(".")?
    print(fs.metadata(&current)?.file_type())?
    missing = path.from_text("missing.txt")?
    match fs.metadata(&missing):
        case Ok(_):
            print("found")?
        case Err(error):
            print(error)?
    Ok(())
True False 5
std.fs.FileType.Directory
IoError.System(2)

Metadata is an affine, detached observation, not an open file or a promise about a later operation. No metadata cache is kept on an entry. Request another path-level observation to refresh it; replacing or changing a file can make earlier data stale.

file_metadata(&stream) -> Result[Metadata, IoError] observes an open FileStream itself (statx with AT_EMPTY_PATH), not whatever its path names now. A file opened through a Root keeps that confinement: renaming or replacing the path afterwards cannot redirect the observation.

Metadata operationValue
file_type()FileType.File, .Directory, .Symlink, or .Other
is_file(), is_dir(), is_symlink()Predicates on that observation
size()u64 size in bytes, also for links and other native types
permissions()Permissions(readonly: bool, unix_mode: u32)
modified(), accessed(), created()Option[Timestamp]
unix_changed()Unix status-change Timestamp, not creation time
unix_uid(), unix_gid()u32 Unix identity fields
unix_device(), unix_inode()u64 Unix directory-identity fields

Timestamp is std.time.Timestamp; import the name from std.time. It has signed i64 seconds since the Unix epoch and u32 nanoseconds in [0, 1_000_000_000). Negative times use floor seconds: half a second before the epoch is (-1, 500_000_000). Nothing means the filesystem did not supply that timestamp. Unsupported creation time is never filled with Unix change time. Metadata getters do not allocate. readonly means no Unix write mode bits; it does not account for ACLs, privileges or mount policy and does not establish that a later operation will succeed.

read_link(&path) -> Result[Path, IoError] returns the stored target exactly, including a relative target or invalid UTF-8. It does not resolve it, make it absolute or require that the target exists. Buffer growth and copying are fallible; a full native buffer is retried rather than silently truncated.

symlink(target: &Path, link: &Path) -> Result[(), IoError] creates a new symbolic link containing the target’s exact bytes. A relative target is interpreted relative to the link’s parent when followed, not relative to the caller’s working directory. The target need not exist. An existing destination, including a dangling link, causes an error without replacing it. Parent components of link follow ordinary filesystem resolution; this is not a rooted no-follow operation. Use Root and TempFile for their separate confinement and publication contracts.

canonicalize(path: &Path) -> Result[Path, IoError] returns a new absolute native-byte path after resolving symbolic links, . and ... Components are resolved in filesystem order: link/.. refers to the target’s parent. Every component must exist; dangling links and missing components return the native error, a non-directory intermediate component or a trailing slash on a file returns ENOTDIR, and loops or excessive link traversal return ELOOP. Empty paths fail. No text decoding or lossy replacement occurs.

Both functions borrow their inputs, are usable on worker threads, and resolve relative input paths against the process working directory. Embedded NUL bytes return IoError.InvalidInput. On the current 64-bit Linux backend each input and the canonical output must fit in PATH_MAX including the terminating NUL (4095 content bytes); excess returns IoError.System(ENAMETOOLONG). Input lengths are checked before copying. Native path buffers are bounded stack storage; symlink needs no Plenty heap allocation. Canonical output allocation is fallible and reports IoError.Data(DataError.Allocation(...)); OS/C-library failures retain IoError.System(errno). Unsupported hosts return IoError.Unsupported.

Canonicalization is a transient observation, not a security boundary or a snapshot. Concurrent rename, removal or replacement can change resolution or invalidate the returned name immediately. A later open need not access the same object; use held handles and descriptor-rooted operations when identity matters. The narrow native wrappers use symlink and caller-buffer realpath so libc implements platform link traversal, directory checks and error behavior instead of duplicating that resolver in Plenty. No realpath(NULL) allocation escapes into the language.

The baseline is Rust’s explicit metadata/symlink_metadata split and non-following DirEntry behavior. This differs from Python’s default followed DirEntry predicates/stat; those defaults are not mixed. The native backend uses Linux’s stable 256-byte statx ABI, also used by Rust’s Unix fs backend, to avoid std’s infallible long-path CString allocation. It requires 64-bit Linux and libc/kernel statx support; absent kernel support or required basic fields returns IoError.Unsupported. Native layouts and descriptors are private.

Acknowledgements

To update ordinary permission bits, use the held-handle operations Root.set_unix_mode and set_file_unix_mode. Changing a Permissions observation’s fields does not change the filesystem.

Walking a tree

std.fs.walk(&root, top_down) -> Result[Walk, AllocError] constructs an explicit Plenty walker over directory enumeration. It copies the root and allocates its pending stack, but opens no directory until advancement. walker.next() -> Result[bool, WalkError] updates walker.current when it returns True. False means exhaustion or explicit close. walker.close() clears pending work and current entry lists without allocation and is repeatable.

import std.fs as fs
import std.path as path

def main() -> Result[(), AllocError | IoError | fs.WalkError]:
    current = path.from_text(".")?
    root = fs.root(&current)?
    root.create_dir(&path.from_text("site")?)?
    root.open_dir(&path.from_text("site")?)?.create_dir(&path.from_text("posts")?)?
    write_text("site/index.html", "home")?
    write_text("site/posts/first.html", "hello")?
    top = path.from_text("site")?
    mut walker = fs.walk(&top, False)?
    while walker.next()?:
        print(walker.current.path.to_string()?, len(walker.current.files))?
    Ok(())
site/posts 1
site 1

current is a WalkDir with path: Path, directories: list[Path], and files: list[DirEntry]. Directories contains only non-followed directory paths; files contains every other native type, including directory links, dangling links, FIFOs and sockets. Non-directory does not mean regular file.

In top-down mode, next yields the parent before any child is opened. Remove or reorder current.directories before advancing again; the walker descends only into the remaining paths, in their resulting order. It consumes that list on the next call. Thus pruning happens before even attempting access to an unreadable child. In bottom-up mode, children have already been visited when the parent is yielded, so editing its directories cannot prune prior descent. Native sibling order remains unspecified; there is no implicit sorting.

The walker never follows discovered directory links. An explicitly supplied root can resolve through a link, just as ordinary path lookup can. This initial walker has no follow-links option or visited-identity set; an ancestor link therefore does not cause recursive descent. Relative roots retain their spelling and the walker never changes the working directory.

WalkError.Io(path, operation, cause) retains the attempted Path, a fixed operation label (read_dir, next, file_type, or entry.path), and concrete IoError. The failing frame has already been removed. Handle Err and call next again to skip that subtree and continue pending siblings; propagate with ? to stop and drop the walker. A partially scanned directory produces no partial row. WalkError.Allocation(error) reports fallible stack/list/path storage; no completeness guarantee applies after an allocation failure. If allocating the error’s entry path also fails, the context reports the containing root and entry.path instead.

Traversal uses a Plenty list of frames and no recursive native call stack. At most one directory stream is open during a scan, and none is retained between next calls. A row allocates one directory’s lists. Top-down storage includes pending sibling paths along the active traversal depth and the current row; bottom-up additionally retains parent rows along that depth. It streams the tree rather than collecting the entire tree in advance; a very wide directory still needs fallible per-directory storage. Pending-stack reservation counts only the frames about to be appended, not frames already held. An unrepresentable additional frame count returns WalkError.Allocation(AllocError.CapacityOverflow) before appending.

Python os.walk supplies top-down/bottom-up and pruning precedent. Grouping directory symlinks with files follows pathlib.Path.walk with link following disabled. The explicit next/error continuation is Plenty’s adaptation of an error callback; failures are visible by default. The implementation is embedded Plenty source in stdlib/std/fs.plenty, not a separate native traversal engine.

Acknowledgements

Rooted filesystem helpers

std.fs mutation helpers operate on an owned Root, opened by root(&path) -> Result[Root, IoError]. The handle anchors operations to that directory even if its pathname changes. Opening rejects a final symlink, including a path with trailing slashes. Ancestor components of the initial root path are resolved normally: selecting that root is the caller’s trust decision, not a sandbox or a canonicalization operation.

import std.fs as fs
import std.path as path

def main() -> Result[(), AllocError | IoError]:
    current = path.from_text(".")?
    root = fs.root(&current)?
    drafts = path.from_text("drafts")?
    final = path.from_text("final")?
    root.create_dir(&drafts)?
    root.move(&drafts, &final)?
    reports = root.open_dir(&final)?
    reports.create_dir(&path.from_text("2024")?)?
    mut names = reports.names()?
    while True:
        match names.next()?:
            case Some(name):
                print(name.to_string()?)?
            case Nothing:
                break
    Ok(())
2024

Every child name is a lossless Path containing one nonempty component. Slash, NUL, . and .. are rejected. Child directory/file opens use O_NOFOLLOW; symlinks and symlink cycles cannot escape through these opens. Mounted directories are not a boundary. A directory moved after opening stays accessible through its handle. These rules do not provide isolation from a process with permission to rename entries in the same tree.

Root methodBehavior
open_dir(&name)New independently owned Root; final symlink refused
read_file(&name)Read-only std.io.FileStream; regular files only
create_dir(&name, unix_mode=0o700u32)Exclusive directory creation; mode restricted by umask
create_file(&name, unix_mode=0o600u32)Exclusive regular-file creation; owned read/write FileStream, mode restricted by umask
set_unix_mode(unix_mode)Change ordinary permission bits on this held directory
remove_file(&name)Unlink one non-directory, including a symlink itself
remove_dir(&name)Remove one empty directory; no link following
move(&source, &destination)Same-root atomic renameat2(RENAME_NOREPLACE)
names()Owned lazy Names; next() yields Result[Option[Path], IoError]

Move refuses every existing destination, including a dangling link. It moves a source symlink itself; there is no replacement or cross-root/cross-device copy-and-delete fallback. Removal has ordinary unlink/rmdir permissions and race semantics. Names.close() is fallible and repeatable. EOF and errors close the stream; drop closes it as well. Names are detached, native-order byte values, including hidden names. No metadata observation through an unanchored joined path is exposed by this interface.

Root, names, streams and temporary owners are affine. An owned Root or FileStream can move to a worker; creation and permission operations are handle effects permitted there. Sharing a handle by loan still follows the worker resource rules. Shared Root loans allow pathname operations within a thread, since they do not mutate a cursor. Filesystem state is external mutable state; this does not synchronize concurrent processes. Every syscall retains its native IoError; buffer and owner allocation is fallible. Paths require O(name length) temporary storage. Native primitives make narrow OS requests; traversal, budgets, retries and temporary-resource lifecycle are ordinary checked Plenty code.

Current target: x86_64 Linux GNU. No new dependency or linker requirement is introduced. The matching installed compiler embeds the module and runtime; imports work offline after relocation. See temporary resources and bounded copying, deletion and globbing.

Private creation and permission changes

Use Root.create_file for a named file that must not already exist. It returns Result[FileStream, IoError], opened for reading and writing; close it through FileStream.close() or let its owner drop. Allocation for the path and owner happens before creation. Neither allocation failure nor an existing entry leaves a new file. Every existing entry, including a dangling symlink, causes an error without truncation, replacement or permission changes. A successful creation remains named after close, drop or a later write failure. Use anonymous staging when partial contents must stay unnamed.

create_dir returns Result[(), IoError] and creates only the named directory, not its parents. Existing directories and symlinks fail too. Open the result with open_dir when a held directory handle is needed. Creation and this subsequent open are not atomic; the parent must be trusted against concurrent entry replacement, as for the other rooted helpers.

Both creation methods accept unix_mode: u32. The default modes (0600 for a file, 0700 for a directory) prohibit group/other access from the instant of creation. The mode goes directly to openat/mkdirat, with no create-then-chmod window. Ordinary Unix umask/default-ACL rules apply: requested bits are an upper bound, and the process umask can remove them when no default ACL applies. The library never reads or changes the process-global umask. Directory setgid inheritance and existing filesystem ACL policy are controlled by the OS.

To change permissions later, call Root.set_unix_mode(unix_mode) on a held directory or fs.set_file_unix_mode(&stream, unix_mode) on a held FileStream. Both return Result[(), IoError], perform no input-sized allocation and call fchmod on the descriptor. Umask does not restrict these updates. They neither reopen a pathname nor follow a newly substituted symlink. The operation changes the inode shared by all of its hard links, even after renaming or unlinking. It does not revoke already-open descriptors, change ownership, or promise an ACL-free filesystem. The OS enforces ownership/capability checks and ACL effects.

Modes outside 0o000u32..0o777u32 return IoError.InvalidInput before any filesystem change: file-type bits, sticky and set-ID bits are deliberately unsupported. Modes are not silently masked. OS failures retain their native IoError.System code; closed streams return IoError.Closed. Creation can also return an allocation error through IoError.Data(DataError.Allocation(...)). These APIs have the same 64-bit Linux scope as the rooted helpers; unsupported hosts return IoError.Unsupported. Native code is limited to descriptor creation and fchmod, because these operations require OS syscalls; mode validation and public policy remain Plenty code.

Temporary resources

For ordinary scratch storage with with support, prefer std.tempfile. These lower-level owners provide rooted staging and explicit publication.

temporary_file(&root) -> Result[TempFile, IoError] creates an unnamed O_TMPFILE inode in the root’s filesystem, mode 0600 restricted by umask. Its public stream: std.io.FileStream is readable and writable; seek back to read what was written. publish(&name) adds a name with atomic no-replace linkat; existing files, directories and dangling links are refused. A successful publication can occur once. It leaves the stream open; close() reports close errors, and ordinary drop closes it. Closing/dropping before publication releases the unnamed inode without an unlink-by-name race. After publication the name persists.

import std.fs as fs
import std.io
import std.path as path

def main() -> Result[(), AllocError | IoError | std.io.StreamError | std.io.Partial]:
    current = path.from_text(".")?
    root = fs.root(&current)?
    mut staged = fs.temporary_file(&root)?
    std.io.write_all(&mut staged.stream, &b"complete contents")?
    staged.stream.sync()?
    staged.publish(&path.from_text("config.txt")?)?
    staged.close()?
    print(read_text("config.txt")?)?
    Ok(())
complete contents

remove_published(&name) -> Result[(), FsError] explicitly removes a published name after comparing it with the still-open stream’s device and inode. A missing name succeeds; a different regular file returns FsError.Replaced, and symlinks are refused. The stream remains open. Publication is still allowed only once. As with directory cleanup, identity checking and removal are not atomic against hostile concurrent renames. This operation supports std.tempfile ownership.

Publication uses /proc/self/fd so unprivileged users need no special capability. The filesystem must support O_TMPFILE, and procfs must be available for publication. Unsupported filesystems and unavailable procfs produce IoError (normally a native System code); there is no named-file fallback. A successful publish is not a durability promise; call stream.sync() before publishing if needed. Directory durability is not provided here. Temporary creation itself copies no metadata. copy_file and copy_tree can preserve ordinary permission bits and access/modification times before completing their copy operation.

temporary_dir_named(&root, &name) exclusively creates a 0700 directory and returns Result[TempDir, FsError]. A collision is reported and never removed. temporary_dir(&root, &prefix, attempts: Positive[i64]) appends 32 lowercase hex digits from 16 OS secret bytes, retrying only EEXIST up to attempts. An attempts above 1024 traps. Prefixes are at most 200 bytes without slash/NUL; no UTF-8 conversion is imposed. Entropy errors are reported without weakening the name source.

TempDir.name() and root() lend its name and directory handle. Explicit cleanup() removes only an empty directory and reports errors. It is idempotent after success. keep() disarms cleanup; the directory remains. The destructor attempts the same empty-only cleanup and ignores errors. This low-level destructor never recursively deletes contents. Fatal traps/process termination do not unwind; explicit cleanup is required when its success matters. Cleanup’s native name conversion can allocate: if allocation remains exhausted during drop, even an empty directory can remain. File/directory handles still close without allocation. Creation reserves names and owner storage before mkdir, so allocation failure during construction leaves no directory behind.

cleanup_tree(max_entries: Positive[i64], max_depth: Positive[i64], &cancel) -> Result[(), FsError] is an explicit recursive alternative, using the bounded deletion budgets. It checks identity before traversing the held directory and again before removing its entry. Successful cleanup disarms the owner. Failure can leave partial cleanup and leaves it armed. std.tempfile.TemporaryDirectory uses this operation for its best-effort destructor.

Temporary directories require a parent whose entries other actors do not rename while the owner is live. Cleanup compares the held directory’s device and inode to the named directory and reports FsError.Replaced for a changed identity. A missing name or symlink is an I/O error. This catches replacements already visible at the check; Linux has no compare-inode-and-unlink syscall, so the check and rmdir are not atomic against a concurrent rename. Empty replacement directories can still be removed by a racing actor. Nonempty replacements are never traversed by the empty-only cleanup. A child-open failure after mkdir attempts empty-only rollback under the same parent-trust contract.

Temporary names/owners and copies are allocated fallibly. A temporary file holds its file descriptor and a duplicate root descriptor. A temporary directory holds its directory and parent descriptors plus its name bytes. See root rules and bounded operations.

Acknowledgements

  • Design references: the Rust tempfile crate and the directory-rooted operations of cap-std.

Bounded copy, deletion and glob filtering

copy_file(&root, &source, &destination, max_bytes: NonNegative[i64], &cancel, preserve=CopyMetadata.ContentsOnly) copies one regular file to a new name in the same root and returns Result[i64, FsError]. copy_from(&mut reader, &root, &destination, max_bytes: NonNegative[i64], &cancel) accepts any checked std.io.Reader. Both stage into an anonymous temporary file and publish only after EOF, flushing, and a final cancellation observation. They never replace an existing name. copy_file optionally preserves selected metadata as described in copy metadata; copy_from has no filesystem metadata source. Recursive copying uses the same policy with explicit tree budgets and partial-output errors. Limits, cancellation, read/write errors, allocation failures and publication errors leave no partial destination name. Unnamed partial data is released by descriptor destruction, not path unlink.

import std.fs as fs
import std.path as path
from std.sync import CancellationToken

def main() -> Result[(), AllocError | IoError | fs.FsError]:
    write_text("report.txt", "quarterly totals")?
    current = path.from_text(".")?
    root = fs.root(&current)?
    cancel = CancellationToken()?
    source = path.from_text("report.txt")?
    backup = path.from_text("report.bak.txt")?
    print(fs.copy_file(&root, &source, &backup, 4096, &cancel)?)?
    pattern = path.from_text("report*")?
    matches = fs.glob(&root, &pattern, 100, 10, 10000)?
    print(len(matches))?
    Ok(())
16
2

The loop uses one 8192-byte buffer, retries interrupted reads, checks reader progress, and checks cancellation before and after each read and before publication. Writes use std.io.write_all. Cancellation is cooperative, cannot interrupt an in-progress syscall/write-all, and may race a successful publication after the final check. Exceeding the byte bound reports Limit. One read of up to 8192 bytes can consume beyond the bound before rejecting it. A concurrently changing source is not a snapshot. The count is bytes published. Costs are O(bytes) time and O(8192 + path lengths) scratch storage, without a whole-file allocation. A successful result does not promise durable storage. See temporary-file requirements.

remove_tree(&root, &name, max_entries: Positive[i64], max_depth: Positive[i64], &cancel) explicitly removes a child directory tree. A max_depth above 64 traps. The root directory counts as one entry and one level. Each encountered child consumes the shared entry budget. Depth is checked before opening a recursive frame. File/link unlinking and empty-directory removal use the anchored parent handle. Links (including cycles) are unlinked and never descended into; the initial name must itself be a real directory.

Deletion is not transactional: cancellation, limits, permissions, concurrent changes or I/O errors can leave a partially removed tree. It checks cancellation between entries. Descent keeps O(depth) directory descriptors and O(depth * name length) path storage, without materializing the tree. It never accepts root/dot/dot-dot names. The initial root’s ancestors, mount points and rename races have the Root contract; do not use this as an adversarial sandbox. A directory identity check precedes the final rmdir, with the same non-atomic race limitation as temporary-directory cleanup. Other processes must not relocate entries during destructive traversal.

glob(&root, &pattern, max_entries, max_matches, max_steps), each budget a NonNegative[i64], returns Result[list[Path], FsError] for names directly inside the root. * matches any byte sequence, ? one byte, and every other byte is literal (including brackets, backslash and leading dot). Slash/NUL and patterns over 4096 bytes are invalid; there is no recursive **, character class, escaping, sorting or Unicode normalization. Consecutive stars retain their ordinary star meaning. Invalid UTF-8 names remain lossless.

Entry, result and matching-step budgets are independent limits. Each examined entry and each matcher iteration consumes its budget; exceeding one returns Limit, releasing partial results. Greedy star backtracking takes O(pattern length * name length) worst-case time, bounded by max_steps across the entire scan. Storage is O(returned name bytes) plus one streaming entry. Enumeration is native order, includes hidden names, and is not a snapshot.

FsError distinguishes Io, Stream, Allocation, Entropy, InvalidInput, Limit, Cancelled, Replaced, Collisions, UnsupportedType, Overlap and InvalidLimits. Basic Root methods preserve IoError directly. Drop is best-effort cleanup, never proof of completion.

File metadata and recursive copying

All operations here are rooted, component-only, no-follow operations on 64-bit Linux. They retain the Root trust and rename contract: concurrent renames can invalidate identity checks, and these operations are not an adversarial sandbox. Filesystem capabilities, mount topology and permissions remain subject to the operating system. The implementation is in Plenty over native handle observation and metadata syscalls.

Copy metadata

CopyMetadata has two modes:

  • ContentsOnly, the default: file data only; new files use mode 0600 restricted by umask and directories use 0700 restricted by umask.
  • PermissionsAndTimes: strictly attempt the source’s ordinary permission bits (mode & 0o777) and access/modification timestamps. Read the source metadata from its opened handle before reading contents or enumerating a directory. Apply timestamps and then permissions to the destination handle. Directory metadata is applied after children, so copying a read-only source directory does not make its destination unwritable before the children are copied.

The second mode excludes ownership, setuid/setgid/sticky bits, ACLs, extended attributes, creation time and inode change time. It does not claim Python copy2 parity. Destination ACLs/xattrs inherited from its filesystem are not copied from the source or stripped by this API. Timestamp precision and permission semantics depend on the destination filesystem; syscall success is the guarantee, not preservation of precision that filesystem cannot represent. A missing source timestamp or a failed metadata syscall returns an error; there is no silent best-effort downgrade. The observation predates copy reads that may update source atime. Changes to the source after observation are not reflected consistently: this is not a filesystem snapshot.

copy_file(..., &cancel, preserve=CopyMetadata.PermissionsAndTimes) retains its anonymous staging and no-replace contract. Metadata failure leaves no published destination. Existing calls without preserve retain their contents-only behavior. The optional named policy follows cancellation to preserve that call shape; there is no required positional boolean. copy_from accepts an arbitrary reader and therefore has no metadata policy.

Recursive copying

copy_tree(source_parent: &Root, source: &Path,
          destination_parent: &Root, destination: &Path,
          limits: &CopyLimits, cancel: &CancellationToken,
          preserve: CopyMetadata = CopyMetadata.ContentsOnly)
    -> Result[i64, TreeCopyError]

The source must name a real directory. The destination must be a new component; existing files, directories and dangling symlinks are refused. Source and target parents may differ, including different filesystems. An opened source directory cannot contain the destination parent: a bounded descriptor-based ancestor walk rejects that overlap, including the same directory opened through another root. An existing destination that contains the source is refused by exclusive creation. The traversal also compares source directory identities with the original destination parent before descent. This catches a bind mount of a source descendant whose namespace parents hide that ancestry; it reports an Entry overlap error after destination creation, leaving partial output. No destination parent is descended into or recursively copied into itself. The ancestry walk holds at most two transient handles and stops after 4096 levels; a longer or changing ancestry produces an I/O invalid-input error. Mount namespace changes and hostile renames are outside the Root stability contract.

Only real directories and regular files are supported. Symlinks, including relative, dangling and cyclic links, and special files return FsError.UnsupportedType; no link is recreated or followed. Native byte names, hidden entries, empty directories and binary contents remain lossless. A type observation is followed by a no-follow typed open. Concurrent replacements can cause an error or change which same-type inode is copied; no snapshot is promised. Enumeration is streaming and native-order, not a materialized/sorted tree.

copy_limits() returns a CopyLimits with public fields:

FieldDefaultContract
max_entries10000Source root and every encountered child count once; positive
max_depth32Root is level 1; valid range 1–64
max_bytes67108864Total bytes successfully copied across all files; nonnegative
max_open_handles128Conservative operation-owned descriptor budget; at least 11

Invalid limits fail before destination creation. Descent additionally requires 3 * level + 8 <= max_open_handles, covering source/destination directories, streaming enumeration and file staging; caller-owned handles are outside this budget. Budgets are shared across the traversal. Exceeding an entry, depth, handle or byte budget reports FsError.Limit. The returned count is copied file data bytes, with zero for an empty tree. Files are transferred with one shared 8192-byte buffer, so time is O(entries + bytes) and storage is O(8192 + depth * name length), with O(depth) descriptors. Individual names are bounded by the native directory-entry limit. An over-limit file can consume one read beyond its remaining allowance before being rejected, without publishing that file.

Cancellation is checked before creation, between entries, during file transfer, and after traversal. It is cooperative and cannot interrupt a syscall or write_all; completion may race cancellation after the final observation. No durability guarantee is made.

Partial output and recovery

TreeCopyError.Setup(FsError) means the destination was not created by this operation. TreeCopyError.Partial(FsError) means its newly created destination may remain with a partial tree. TreeCopyError.Entry(name, depth, cause) also means partial output; it transfers the immediate failing child name and its 1-based tree depth, preserving the cause without allocating while reporting an error. It does not carry the full relative path. Errors while enumerating or finishing a child directory identify that directory; failures on the source root itself use Partial. Earlier copied files remain complete; the current file is staged anonymously and is absent on failure before publication. Empty or populated directories may remain, and metadata can be partly applied to them. The caller retains both destination root and name and can inspect the tree or explicitly call remove_tree with separate cleanup budgets and a fresh token. There is no automatic recursive cleanup that could hide a second failure, erase progress, or pretend rollback is guaranteed. Concurrent actors must not replace or mutate the destination while copying or cleaning it. An error opening the newly created destination is also Partial.

import std.fs as fs
import std.path as path
from std.sync import CancellationToken

def main() -> Result[(), AllocError | IoError | fs.FsError | fs.TreeCopyError]:
    root = fs.root(&path.from_text(".")?)?
    source = path.from_text("assets")?
    root.create_dir(&source)?
    write_text("assets/icon.bin", "binary data")?
    destination = path.from_text("assets-backup")?
    limits = fs.copy_limits()
    cancel = CancellationToken()?
    print(fs.copy_tree(&root, &source, &root, &destination, &limits, &cancel)?)?
    print(read_text("assets-backup/icon.bin")?)?
    Ok(())
11
binary data

Owned names, metadata observations, buffers, and native owners allocate fallibly. Allocation failure preserves the same file-publication and tree-partial contracts; held descriptors close on ordinary return/drop without allocating. Fatal traps or termination do not unwind. Roots must remain accessible under the selected permissions for any later cleanup; preserving read-only directory permissions can require the caller to restore writable permissions outside this API.

Acknowledgements

  • Python’s shutil.copytree and copy2 provide the API precedent. The Plenty implementation uses explicit budgets, rooted no-follow operations and a narrower declared metadata policy; no Python source code is shipped.

User directories

import std.fs.dirs finds the user’s home directory and the per-user base directories where a program keeps its configuration, caches, data, and state. It is written in Plenty over std.env and returns Path values.

FunctionVariableOtherwise
home_dir()HOMENothing
config_dir()XDG_CONFIG_HOME$HOME/.config
cache_dir()XDG_CACHE_HOME$HOME/.cache
data_dir()XDG_DATA_HOME$HOME/.local/share
state_dir()XDG_STATE_HOME$HOME/.local/state

Each returns Result[Option[Path], IoError]. The rules follow the XDG Base Directory specification and the Rust dirs crate on Linux:

  • An empty variable counts as unset.
  • An XDG_* value is used only when it is absolute. The specification calls a relative value invalid, so it is ignored and the default below HOME is used.
  • HOME is used as set, without requiring an absolute path.
  • With no usable XDG_* value and no HOME, the result is Nothing.
import std.fs.dirs
from std.path import from_text

def main() -> Result[(), IoError]:
    match std.fs.dirs.config_dir()?:
        case Some(config):
            settings = config.join(&from_text("myapp/settings.toml")?)?
            # For example /home/ada/.config/myapp/settings.toml
            print(settings.is_absolute())?
        case Nothing:
            print("no configuration directory")?
    Ok(())
True

Paths keep the variable’s exact bytes, so a directory name that is not UTF-8 round-trips. Results are not cached: each call reads the environment again. Nothing is created or checked; a returned directory may not exist yet. Create it with the rooted helpers when the program first writes there. Allocation failure is IoError.Data(DataError.Allocation(...)).

Limits

  • x86-64 Linux GNU only, like std.env; elsewhere each call reports IoError.Unsupported.
  • No password-database fallback: when HOME is unset, dirs consults getpwuid_r, but home_dir() returns Nothing.
  • No runtime, executable, or system-wide (XDG_CONFIG_DIRS, XDG_DATA_DIRS) directories.

Acknowledgements

std.fs.lock — advisory file locks

std.fs.lock coordinates cooperating processes through Linux flock locks on a regular file. It supports shared readers and an exclusive owner, with nonblocking acquisition or a total deadline and cancellation token. The lock is advisory: other processes can still read or write without acquiring it.

import std.fs.lock as lock
import std.path as path
from std.fs.lock import Mode, LockError

def main() -> Result[(), IoError | LockError]:
    file = path.from_text("job.lock")?
    mut owner = lock.acquire_now(&file, Mode.Exclusive)?
    print(lock.acquire_now(&file, Mode.Shared).is_err())?
    owner.close()?
    another = lock.acquire_now(&file, Mode.Shared)?
    print(another.is_closed())?
    Ok(())
True
False

Acquisition

FunctionResult and behavior
acquire_now(path: &Path, mode: Mode)Result[Lock, LockError]: one nonblocking attempt; contention returns WouldBlock
acquire_before(path: &Path, mode: Mode, deadline: &Deadline, cancel: &CancellationToken)Result[Lock, LockError]: retries contention until acquired, cancelled or timed out

Mode.Shared permits other shared owners. Mode.Exclusive excludes both modes. Each call opens its own file description, so independent acquisitions in the same process contend too. There is no upgrade/downgrade operation or fairness guarantee. Use a consistent acquisition order for multiple resources.

A missing file is created with ordinary permission bits 0600, restricted by the process umask. An existing file is opened read/write without truncating its contents or changing its permissions. No writes are made. The final path component must not be a symlink and the opened object must be a regular file; opening a FIFO cannot block. Ancestor components follow normal OS resolution. Use a trusted parent directory. Do not unlink or replace a live lockfile: locks protect the opened inode, and a replacement would create an independent lock domain. These functions never remove the lockfile, even after failure.

The borrowed native-byte Path must contain 1–4096 bytes and no NUL, otherwise InvalidPath is returned before opening. OS path limits still apply. The implementation allocates bounded path scratch and one owned File; allocation failure returns Allocation(AllocError). The small lock facade adds no buffer or queue. Failures close any descriptor already opened.

Waiting checks cancellation before deadline expiry, and both before opening and before every lock attempt. An already expired deadline returns TimedOut even if the lock would be available. After contention it waits on the token for at most ten milliseconds or the remaining budget, whichever is smaller; cancellation wakes that wait. EINTR from a lock attempt retries after checking the token and deadline; the nonblocking form preserves it as Io(IoError.System). Filesystem opening and metadata syscalls themselves cannot be interrupted by the token or bounded by the deadline, including on remote filesystems. Completion can race cancellation after the final observation.

Ownership and release

Lock is an affine owner with a private descriptor. It can move to a worker or return from one; dropping it in that worker releases its ownership normally.

  • close(self: &mut Lock) -> Result[(), LockError] relinquishes this descriptor. Repeated close is harmless, including after a close error.
  • is_closed(self) -> bool reports whether this owner has been closed.
  • Drop closes the descriptor without allocating or reporting a close error, including on return, ?, and scope exit. It never issues LOCK_UN.

On local Linux filesystems a lock belongs to the open file description. Closing its final descriptor, or process death without surviving inherited copies, releases it. A descriptor inherited through fork retains the lock until that copy is also closed, even if the original owner closes or dies. An explicit LOCK_UN would instead unlock all copies, which is why this API has no separate unlock operation.

Descriptors are always created close-on-exec. Normal std.process children do not retain them across exec. This API does not expose a raw file descriptor or a switch to inherit it; explicit child-fd selection is outside this API’s implemented scope. Applications requiring a lock to outlive a runner through an exec’d child must keep their existing inheritance strategy.

This API is supported on 64-bit Linux; other targets return Io(IoError.Unsupported). Network filesystems can emulate or restrict flock and have different failure/mandatory-I/O semantics; consult the mounted filesystem’s contract. It does not provide a distributed lease or fencing token.

Errors

LockError has WouldBlock, TimedOut, Cancelled, InvalidPath, Io(IoError), and Allocation(AllocError). Native errors retain their OS code, including access denial, unsupported locking, symlinks, and missing parents. Closing only reports Io failures. Cancellation/timeouts consume no caller resource: paths, deadlines, and tokens are borrowed, and no partial owner escapes.

Acknowledgements

  • Linux flock defines the advisory locking and open-file-description lifetime semantics.
  • libc, MIT OR Apache-2.0: the narrow native wrappers use its Linux syscall bindings. Waiting and error policy are Plenty.

std.fs.watch — filesystem changes

import std.fs.watch observes Linux files and directories through inotify. It owns one nonblocking, close-on-exec descriptor, starts no threads, and uses std.ready for deadlines, cancellation, and selector integration. For a runnable introduction, see watch a directory.

import std.fs.watch as watch
import std.path as path

def main() -> Result[(), AllocError | IoError | watch.WatchError]:
    mut watcher = watch.watcher()?
    current = path.from_text(".")?
    watcher.add(&current)?
    write_text("log.txt", "started")?
    count = watcher.poll()?
    for index in range(count):
        event = watcher.event(index)?
        if event.overflowed():
            print("rescan")?
        elif event.created():
            print("created", event.name.to_string()?)?
        elif event.closed_after_write():
            print("written", event.name.to_string()?)?
    watcher.close()?
    Ok(())
created log.txt
written log.txt

Batches and ownership

watcher(batch_bytes=4096) -> Result[Watcher, WatchError] allocates a reusable byte buffer and offset table. batch_bytes must be 272..65536: 272 accommodates the longest Linux filename plus its event header. One read returns at most batch_bytes // 16 events, with fewer when filenames occupy buffer space. The offset table reserves that many ordinary list[i64] slots, in addition to the compact byte buffer. This is a bound on each user-space batch, not on the independent kernel queue or total number of watched paths.

OperationBehavior
add(path: &Path)Result[i32, WatchError]: watch the file or immediate directory children; return its watch ID
remove(watch: i32)Result[(), WatchError]: remove that watch; unknown/already removed IDs are errors
poll()Result[i64, WatchError]: replace the last batch with one nonblocking read; zero means no events or an interrupted read
wait(deadline: &Deadline, cancel: &CancellationToken)Result[i64, WatchError]: poll, then wait until a batch is available, cancelled, or timed out
count()Number of events in the last successful batch
event(index: i64)Result[Event, WatchError]: copy one event and its native filename
register(selector: &mut Selector, key: u64)Result[(), WaitError]: register the descriptor for read readiness
unregister(selector: &mut Selector)Result[(), WaitError]: remove that selector registration
close()Result[(), WatchError]: discard the batch and close all watches; repeated close is harmless

poll and wait reuse their buffers without allocating. Events that do not fit remain in the kernel queue for later calls; a full user-space batch is not overflow. Consume the current batch before the next poll, wait, or close, which invalidates it. event is an O(1) lookup plus filename copying. A failed copy leaves the batch intact for retry, and a successful event owns its name independently of the watcher. Dropping the watcher closes the descriptor and all watches automatically.

Cancellation requested before wait wins over a ready event; otherwise an already queued event wins over an expired deadline. After a selector reports the watcher’s key readable, call poll. Closing the watcher removes its descriptor from selectors automatically.

Event fields

Each Event exposes watch: i32, cookie: u32, and name: Path. The name is the immediate child’s native filename bytes, not an absolute path. It is empty for events about the watched object itself. Use Path.join with the appropriate watched directory when a path is needed; name.to_text() validates UTF-8, while name.to_string() provides display text.

Predicates report the event’s flags:

  • created(), deleted(), modified(), metadata_changed() and closed_after_write() describe ordinary changes.
  • moved_from() and moved_to() identify rename endpoints. Their nonzero cookie values pair endpoints within this watcher; either endpoint may be outside the watched directories or in a later batch. They need not be adjacent.
  • deleted_self() and moved_self() describe the watched object itself.
  • ignored() says the kernel removed the watch, after explicit removal, deletion or unmount. unmounted() reports filesystem unmounting.
  • overflowed() means kernel events were lost. Its watch ID is -1 and its filename is empty. Rescan the watched state before trusting an incremental cache again; overflow is returned as an event, not hidden by a successful read.
  • is_directory() identifies directory events where the kernel provides it.

Adding the same inode again, including through another hard link, returns its existing watch ID. Paths follow symlinks. Watches follow inodes, not replaceable pathnames: to detect a program replacing a file atomically, watch its parent directory. Directory watches are not recursive. Reading/opening files does not generate events in this API; it requests only the change flags above.

Kernel notifications can coalesce, and are not an exact operation count or a transaction log. Changes through some remote filesystems, pseudo-filesystems, or memory mappings may not be reported. Events can be stale by the time they are read. Removal does not erase already queued events, and IDs can eventually be reused; use a fresh watcher after rebuilding state if stale IDs would be ambiguous. There is no automatic rescan, recursive registration, rename pairing, debouncing, or background callback execution. These native semantics follow the Linux inotify contract.

Errors and scope

WatchError distinguishes OutOfRange (buffer size/event index), InvalidData (malformed native record), Io(IoError), Wait(WaitError), and Allocation(AllocError). Native operations preserve IoError including its allocation errors; facade buffer/filename allocations use Allocation. Kernel resource limits, missing paths and denied access remain native errors. Record offsets are bounded by the validated 65536-byte maximum batch and each complete header/payload; malformed lengths remain InvalidData, not arithmetic errors. Native signed watch IDs, including the overflow sentinel -1, are preserved. No wrapped crate introduces infallible heap allocation. Watchers can move into worker tasks; their mutable methods require exclusive borrowing.

The native implementation currently targets x86-64 Linux. Unsupported hosts report IoError.Unsupported; there is no polling fallback. This module shares the existing File, Bytes, path, and readiness runtime facilities, adding only three operations for inotify creation, registration and removal.

Acknowledgements

  • The Rust notify crate served as a design reference; no code is taken from it.

std.graph

from std.graph import DiGraph, UnGraph provides owned directed and undirected graphs that store a value of type N on every node and E on every edge, with traversal, shortest-path, ordering, component and spanning tree algorithms. It is ordinary embedded Plenty code over lists; there is no native dependency.

Graph kinds

TypeEdgesAlgorithms only on this kind
DiGraph[N, E]Run from source to targettoposort, strongly_connected_components, predecessors, incoming
UnGraph[N, E]Join two endpoints symmetricallyconnected_components, minimum_spanning_tree, endpoints, opposite

The kind is part of the type, so calling a directed-only algorithm on an undirected graph is a compile-time error rather than a runtime surprise. Both kinds permit parallel edges and self-loops. N and E follow ordinary collection storage rules: affine owners are allowed, references and generators are not, and unit () is not yet a supported type argument; use a small placeholder such as bool for a graph without data.

Handles

NodeId and EdgeId are public aliases of i64. The n-th node or edge added to a graph has handle n, counting from zero. Handles are plain integers so that they copy; nothing stops passing an EdgeId where a NodeId is expected, or a handle from one graph to another.

Removal never moves or reuses a handle. remove_node and remove_edge leave a vacant slot, so every other handle keeps naming the same node or edge and a removed handle stays invalid for the graph’s lifetime. The alternative, swap-removal as in petgraph’s Graph, silently renames the last node to the removed handle, so a handle saved earlier refers to a different node with no way to detect it. Reusing vacant slots would need a generation counter in each handle to catch stale ones. The cost of this rule is that memory stays proportional to the number of nodes and edges ever added, not the number alive; a graph with heavy churn should be rebuilt.

node_count() and edge_count() count live entries. node_bound() and edge_bound() are one past the largest handle issued, so for node in range(graph.node_bound()) with contains_node(node) visits live nodes without allocating. nodes() and edges() return live handles in increasing order.

Operations that take a handle trap when it is negative, out of range, or removed, as indexing does. The trap is an index-out-of-bounds report inside std.graph with a note: called from line naming the caller. remove_node, remove_edge, contains_node, contains_edge and the get_* getters never trap: removal of an invalid handle returns Nothing, and so does a checked getter, so a possibly stale handle needs one lookup rather than a contains_node test followed by node.

Building and reading

OperationResultContract
DiGraph[N, E](), UnGraph[N, E]()Result[Graph, AllocError]An empty graph
graph.reserve(nodes, edges)Result[(), AllocError]Room for that many more nodes and edges
graph.add_node(data)Result[NodeId, AllocError]Consume data into a new node
graph.add_edge(a, b, data)Result[EdgeId, AllocError]Consume data into a new edge; traps for an invalid endpoint
graph.remove_node(node)Option[N]Remove every incident edge, then the node, transferring its data
graph.remove_edge(edge)Option[E]Remove the edge, transferring its data
graph.node(node), graph.edge(edge)&N, &EBorrow the data
graph.node_mut(node), graph.edge_mut(edge)&mut N, &mut EBorrow the data exclusively
graph.get_node_ref(node), graph.get_edge_ref(edge)Option[&N], Option[&E]Borrow the data; Nothing for an invalid handle
graph.get_node_mut(node), graph.get_edge_mut(edge)Option[&mut N], Option[&mut E]Borrow the data exclusively; Nothing for an invalid handle
graph.find_edge(a, b)Option[EdgeId]The earliest live edge from a to b (either way for UnGraph)
di.source(edge), di.target(edge)NodeIdEndpoints of a directed edge
un.endpoints(edge)(NodeId, NodeId)Endpoints in the order given to add_edge
un.opposite(edge, node)NodeIdThe endpoint that is not node (node itself for a self-loop)
di.successors(node), di.predecessors(node)Result[list[NodeId], AllocError]Targets of outgoing edges, sources of incoming edges
un.neighbors(node)Result[list[NodeId], AllocError]The other endpoint of every incident edge

Neighbor lists follow edge insertion order and repeat a node once per parallel edge. An undirected self-loop appears once in its node’s neighbors; a directed self-loop appears in both successors and predecessors.

Each node heads linked chains threaded through its edges, as in petgraph’s adjacency list. Adding a node or an edge appends one slot and allocates only when that list grows, doubling its capacity. Allocation failure leaves the graph unchanged and drops the consumed data. Lookups, mutation through references, removal and walks allocate nothing. Removing an edge takes time proportional to the degrees of its endpoints; removing a node takes time proportional to the degrees of the node and its neighbors.

copy(graph)? duplicates the whole graph, including vacant slots, so handles remain valid in the copy. Graphs move like other owned collections.

Allocation-free walks

di.outgoing(node), di.incoming(node) and un.incident(node) return a Walk: a small cursor that holds no borrow of the graph. Pass it back with graph.next_edge(&mut walk), which returns Option[EdgeId] in insertion order:

from std.graph import UnGraph

def main() -> Result[(), AllocError | IoError]:
    mut roads = UnGraph[str, i64]()?
    home = roads.add_node("home")?
    shop = roads.add_node("shop")?
    park = roads.add_node("park")?
    roads.add_edge(home, shop, 4)?
    roads.add_edge(park, home, 2)?
    mut walk = roads.incident(home)
    while True:
        match roads.next_edge(&mut walk):
            case Some(edge):
                other = roads.opposite(edge, home)
                print(*roads.node(other), *roads.edge(edge))?
            case Nothing:
                break
    Ok(())
shop 4
park 2

Edges added after the walk began at the end of the chain are seen; removing the edge the walk would visit next ends the walk early.

Traversal and components

OperationResultOrder
graph.bfs(start)Result[list[NodeId], AllocError]Breadth-first, neighbors in insertion order
graph.dfs(start)Result[list[NodeId], AllocError]Recursive depth-first preorder
un.connected_components()Result[list[list[NodeId]], AllocError]Components by smallest handle, each in bfs order
di.strongly_connected_components()Result[list[list[NodeId]], AllocError]Reverse topological: a component precedes those that reach it

Traversal returns every node reachable from start, which comes first. Directed traversal follows outgoing edges. dfs produces the order of a recursive search but uses an explicit stack, so a path of any length cannot overflow the call stack. Strongly connected components use iterative Tarjan; each component lists its nodes in discovery order. Roots are tried in handle order. Each algorithm takes O(V + E) time and O(V) extra space, where V is node_bound().

Topological order

di.toposort() returns Result[list[NodeId], GraphError]. It uses Kahn’s algorithm, starting from source nodes in handle order, so the result is deterministic. If the graph has a cycle, the error is GraphError.Cycle(nodes): a list [a, b, ..., z] such that the graph has edges a -> b, …, z -> a. A self-loop reports [node]. Finding the cycle takes O(V + E) time after the sort fails.

Shortest paths

Weights come from a callback, weight: &F with F: Callable[[&E], f64], so edge data can hold anything and the same graph can be searched by several metrics. Integer weights convert with f64(value); they are exact up to 2^53.

OperationResultWeights
graph.dijkstra(start, weight)Result[Paths, GraphError]Non-negative; +inf acts as no edge
graph.astar(start, goal, weight, estimate)Result[Option[(f64, list[NodeId])], GraphError]As Dijkstra
graph.bellman_ford(start, weight)Result[Paths, GraphError]Any value except NaN and -inf

Paths answers queries for every node:

MethodResult
paths.distance(node)Option[f64]: total weight, or Nothing when unreachable
paths.edge_to(node)Option[EdgeId]: the last edge on the path; Nothing for the start
paths.path(node)Result[list[NodeId], AllocError]: start to node, or empty when unreachable

Undirected edges can be used in either direction. Handles added after the search are reported unreachable.

Dijkstra checks each weight when it first examines that edge and returns GraphError.InvalidWeight(edge) for a negative or NaN weight. It uses a binary heap from std.collections and takes O((V + E) log E) time. astar stops at goal and returns the cost and path, or Nothing when goal is unreachable. estimate: &H with H: Callable[[NodeId], f64] must never exceed the true remaining cost for the result to be shortest; a NaN or negative estimate returns GraphError.InvalidEstimate(node). Nodes may be expanded more than once when an admissible estimate is inconsistent.

Bellman–Ford allows negative weights and takes O(V × E) time. A negative cycle reachable from start returns GraphError.NegativeCycle(nodes) in the same form as Cycle. In an undirected graph, any negative edge reachable from start is such a cycle, reported as its two endpoints.

Minimum spanning tree

un.minimum_spanning_tree(weight) returns Result[list[EdgeId], GraphError]: a minimum spanning forest, with one tree per connected component. It uses Kruskal’s algorithm with a union–find, so edges come out in increasing weight, ties in handle order. Self-loops are never chosen. Negative weights are allowed; a NaN weight returns GraphError.InvalidWeight(edge). It takes O(E log E) time and O(V + E) space.

Errors

GraphError has five variants:

VariantRaised by
Cycle(list[NodeId])toposort
NegativeCycle(list[NodeId])bellman_ford
InvalidWeight(EdgeId)dijkstra, astar, bellman_ford, minimum_spanning_tree
InvalidEstimate(NodeId)astar
Allocation(AllocError)toposort, dijkstra, astar, bellman_ford, minimum_spanning_tree

These algorithms return Allocation(cause) when their working lists cannot grow; the traversals and other operations without a domain error return AllocError. Weights and estimates are calculated by user callbacks, which are called only while the algorithm runs.

Limits

There is no graph-wide iteration protocol, compaction, subgraph, isomorphism, max-flow, matching, or file-format support. Weakly connected components of a directed graph, Floyd–Warshall, Prim, and petgraph’s dominators are candidates that need a use case. Floating-point path costs use ordinary f64 addition, so very long paths accumulate rounding.

Acknowledgements

std.http

import std.http reads and writes HTTP/1.1 messages (RFC 9112) over std.io streams: request and response heads, Content-Length, chunked and close-delimited bodies, and the connection reuse rules. It is the framing layer only. std.http.server builds a server on it and std.http.client a client; their HTTPS adapters use std.tls.

std.http.decoder exposes the same request grammar as a bounded byte-feed state machine without I/O. The actor HTTP server uses it to suspend between readiness events while keeping parsing state in its connection heap.

For body transfers that must preserve a concrete transport error type, the additive std.http.typed readers support fixed-length, chunked and close-delimited framing over std.io.typed. They assume validated heads; the clients and servers above continue to use this module’s framing API.

The module is checked Plenty in stdlib/std/http.plenty; it adds no runtime primitives. The same code runs over a std.net TcpStream, a Cursor in a test, or any other Reader and Writer.

import std.bytes
import std.http
import std.io
from std.http import HttpError

def main() -> Result[(), AllocError | IoError | HttpError | std.io.StreamError | std.bytes.BytesError]:
    wire = "POST /notes HTTP/1.1\r\nHost: example.com\r\nContent-Length: 5\r\n\r\nhello"
    limits = std.http.limits()
    mut conn = std.io.buffered_reader(std.io.cursor(std.bytes.from_text(wire)?), 1024)?
    head = std.http.read_request(&mut conn, &limits)?.ok_or(HttpError.Truncated)?
    framing = std.http.request_framing(&head)?
    mut body = std.http.body_reader(conn, &framing, &limits)?
    mut data = std.bytes.empty()?
    std.http.read_to_end(&mut body, &mut data)?
    print(head.method, head.target, data.to_text()?)?

    mut reply = std.http.response(201, "Created")?
    reply.add("Content-Length", "0")?
    print(str.repr(std.http.encode_response(&reply)?.to_text()?)?)?
    Ok(())
POST /notes hello
"HTTP/1.1 201 Created\r\nContent-Length: 0\r\n\r\n"

Messages

ItemMeaning
Header(name: str, value: str)One field line
RequestHead(method, target, minor: i64, headers: list[Header])minor is the HTTP/1.x minor version, 0 or 1
ResponseHead(minor, status: i64, reason: str, headers)
request(method, target), response(status, reason)Result with a head using HTTP/1.1 and no fields; fails only when allocating the field list fails
head.add(name, value)Result[(), HttpError]: append a field
find(headers: &list[Header], name: &str)Option[i64]: index of the first field named name, ignoring ASCII case
same_name(left: &str, right: &str)bool: whether two names are equal ignoring ASCII case; never allocates, and non-ASCII text never matches

Parsed names keep the case they arrived in. Values have surrounding spaces and tabs removed. Fields stay in arrival order and repeated names stay separate; nothing is combined.

Limits

Limits(max_line, max_head, max_headers, max_body) bounds every read:

FieldBoundslimits()
max_lineOne request, status, field or chunk-size line, including its CRLF8192
max_headAll bytes of a head, blank lines before a request line included; also a trailer section65536
max_headersField lines in a head or trailer section100
max_bodyDecoded body bytes1048576

max_headers and max_body are NonNegative[i64], so a negative count is a type error; read them with .get(). max_line and max_head must be at least 2, or the call fails with InvalidLimit. Memory use is one line buffer of at most max_line bytes plus the parsed fields; the body is never held unless the caller collects it.

Reading heads

FunctionResult
read_request(conn: &mut BufReader[R], limits)Result[Option[RequestHead], HttpError]
read_response(conn: &mut BufReader[R], limits)Result[Option[ResponseHead], HttpError]

Nothing means the input ended before the message’s first byte, as when the peer closes an idle connection; an end inside a head is Truncated. Blank lines before a request line are skipped within max_head, and input that ends after only blank lines is Nothing. A 1xx response is returned like any other; read again for the final response.

Parsing is strict. Every line ends in CRLF; a bare LF is BareLineFeed, and a CR inside a line is invalid. A request line is a token method, one space, a target of visible ASCII, one space, and HTTP/1.0 or HTTP/1.1. A status line is the version, a space, three digits from 100, and an optional space and reason; HTTP/1.1 200 with no reason is accepted. A well-formed other version such as HTTP/2.0 is UnsupportedVersion. A field line is a token name, a colon with no space before it, and a value of visible bytes, spaces and tabs that is valid UTF-8. A line starting with a space or tab (an obsolete line fold) is InvalidHeader.

parse_request_line(line: &[u8]), parse_status_line(line: &[u8]) and parse_field(line: &[u8]) expose the same grammar for custom readers. line excludes CRLF; pass &buffer[:size] for a prefix. These helpers do not read streams or enforce head budgets. For typed transports, prefer std.http.typed.read_request and read_response, which enforce those budgets and retain causes and consumed-byte progress.

Framing

Framing is Empty, Length(i64), Chunked, or UntilClose.

FunctionResult
request_framing(head: &RequestHead)Result[Framing, HttpError]; no framing field is Empty
response_framing(head: &ResponseHead, method: &str)Result[Framing, HttpError]; no framing field is UntilClose
request_keeps_alive(head)HTTP/1.1 unless Connection: close; HTTP/1.0 only with Connection: keep-alive
response_keeps_alive(head, framing: &Framing)As for requests, and never after an UntilClose body

method is the method of the request a response answers. Responses to HEAD, 1xx, 204 and 304 responses, and 2xx responses to CONNECT are Empty whatever their fields say.

Framing that two implementations could read differently is an error, never a guess, because that disagreement is how request smuggling works:

FieldsError
Content-Length and Transfer-Encoding togetherConflictingFraming
Either field twice, even with equal valuesDuplicateFraming
Content-Length other than decimal digits (+5, 1,1, empty), or above i64InvalidContentLength
Transfer-Encoding other than exactly chunked, or in HTTP/1.0UnsupportedTransferEncoding

The framing functions check both fields even where the status decides the framing, so a malformed response is rejected consistently.

Targets and Host

FunctionResult
request_target(head: &RequestHead)Result[RequestTarget, HttpError]: the checked authority, path and query

RequestTarget has authority, path and query, all str. path and query are still percent-encoded; query follows the first ? and is empty without one, and an empty absolute-form path is /.

request_target applies the server rules of RFC 9112 section 3.2. The parser reads a head without them, so a client or proxy can still read any well-formed message; a server calls this before routing, as std.http.server does.

RequestResult
HTTP/1.1 without HostMissingHost
More than one Host field, in any versionDuplicateHost
A Host value other than uri-host [ ":" port ]InvalidHost
Origin-form target (/path?query)authority is the Host value; empty for HTTP/1.0 without one
Absolute-form target (http://authority/path?query, or https, scheme in any case)authority is the target’s, replacing Host
Absolute-form with userinfo (@), an empty host, or another schemeInvalidTarget
Asterisk-form (OPTIONS *) or authority-form (CONNECT host:port)InvalidTarget

Host is checked even when an absolute-form authority replaces it. An empty Host value is valid; a client sends one when the target has no authority. A host is a reg-name of unreserved, sub-delimiter and percent-encoded bytes, or an IP literal in brackets. An IPv6 literal is checked for its characters, not parsed as an address; an IPvFuture literal must match "v" 1*HEXDIG "." 1*(unreserved / sub-delims / ":"). A port is digits.

Decimal Content-Length and hexadecimal chunk-size accumulation use local checked results. Every nonnegative value through i64.MAX is representable; larger values return InvalidContentLength or InvalidChunk, not an arithmetic trap. Body limits are enforced separately, so a representable length can still be BodyTooLarge. Bounded line, header and body cursors use documented bounds-proven arithmetic opt-outs.

Reading bodies

Function or methodResult
body_reader(conn: BufReader[R], framing: &Framing, limits)Result[BodyReader[R], HttpError]; takes the connection
body.read_body(buffer, start, stop)Result[i64, HttpError]: decoded bytes; 0 from a non-empty range is the end of the body, and an empty range returns 0 without reading
body.read(buffer, start, stop)The std.io Reader method
read_to_end(body, out: &mut Bytes)Result[i64, HttpError]: append the rest
discard(body)Result[i64, HttpError]: read and drop the rest
finish(body: &mut BodyReader[R])Result[BufReader[R], HttpError]: the connection, at the next message
release(body)Option[BufReader[R]]: the connection in any state
body.trailersTrailer fields, once a chunked body has ended
body.total(), body.is_done()Bytes delivered; whether the end was reached
body.take_failure()Option[HttpError]: the framing error behind a read failure

The body reader owns the connection, so nothing can read the next message while a body is in progress. finish returns it only after the body’s end, including buffered bytes of pipelined messages; a body with bytes left is Unfinished and a failed one is Broken. release returns the connection regardless, for writing a final error response or closing; its read position is then unknown, so it must not carry another message.

A Length body larger than max_body fails before reading. A chunked body fails as soon as a chunk size would pass max_body, and an UntilClose body as soon as a byte arrives past it. Chunk sizes are hexadecimal without signs or leading spaces. Chunk extensions after ; are checked for control bytes and ignored. Each chunk’s data must be followed by CRLF.

parse_chunk_size(line: &[u8]) exposes that chunk-size grammar for custom framed readers. It reads a &[u8] view of one line excluding CRLF, such as &line[:size], and returns Result[i64, HttpError] without allocating. Extensions are checked and ignored. Invalid syntax or a size exceeding i64 returns InvalidChunk. Callers enforce their own line and body limits.

Through read, a body is an ordinary Reader for std.io.copy_all and similar helpers. Stream and allocation errors keep their StreamError; Truncated becomes UnexpectedEof; any other framing error becomes Io(IoError.InvalidInput) and is kept for take_failure. After any error the body reports only Broken.

read_to_end checks its next output-buffer extent before growing the buffer or reading another chunk. An unrepresentable extent is Allocation(AllocError.CapacityOverflow) and leaves the existing output and connection position unchanged for that iteration. Earlier appended bytes remain.

Writing

FunctionResult
write_request(writer: &mut W, head)Result[(), HttpError]
write_response(writer, head)Result[(), HttpError]
write_chunk(writer, source: &[u8])One chunk; an empty view writes nothing
write_last_chunk(writer, trailers: &list[Header])The zero-size chunk and trailer section
write_body(writer, framing, source: &[u8])A whole body under framing; Length and Empty must match len(source)

Writers validate before writing and write nothing when validation fails: token methods and names, targets of visible ASCII, reasons and values without CR, LF or other control bytes, values without surrounding blanks, status 100–999, and the framing rules above. A value cannot add a field of its own. A stream failure may leave part of a message delivered. Heads are written with one write_all; a chunk with three. The caller flushes.

encode_request(head), encode_response(head), encode_last_chunk(trailers) and encode_chunk_size(length: NonNegative[i64]) return Result[Bytes, HttpError]. They perform the same validation and serialization without writing, sharing implementation with the typed writers. A chunk length of zero encodes 0\r\n without the final trailer-section terminator.

Errors

VariantCause
Stream(StreamError)The underlying stream failed, or a range was out of bounds
Allocation(AllocError)Memory for a line, field, or output ran out
InvalidLimitmax_line or max_head below 2
TruncatedInput ended inside a head, a body, or a chunk
LineTooLong, HeadTooLarge, TooManyHeaders, BodyTooLargeA limit was reached
BareLineFeedA line ended in LF without CR
InvalidRequestLine, InvalidStatusLine, InvalidHeaderSyntax outside the grammar above
UnsupportedVersionNot HTTP/1.0 or HTTP/1.1
InvalidContentLength, DuplicateFraming, ConflictingFraming, UnsupportedTransferEncodingSee Framing
InvalidChunkA chunk-size line or chunk terminator outside the grammar
LengthMismatchwrite_body given a range that does not match the framing
Unfinished, Brokenfinish before the end, or after a failure
InvalidTarget, MissingHost, DuplicateHost, InvalidHostSee Targets and Host

Costs, ownership and threads

  • Reading a head is linear in its bytes. It allocates one line buffer, which grows to the longest line read (at most max_line), plus one str per method, target, reason, name and value, and the field list.
  • A body reader allocates its line buffer and trailer list once. Body bytes are copied once, from the BufReader into the caller’s buffer; nothing is retained. read_to_end grows out 4096 bytes at a time; discard uses one 4096-byte scratch buffer.
  • A writer builds each head or chunk-size line in one Bytes and writes it; chunk data is written from the caller’s buffer without copying.
  • Heads, Limits and Framing are ordinary owned values. A BodyReader owns its connection, so it moves to a worker with it when R can. Nothing is shared or global, and no call starts a thread or waits beyond the underlying stream’s own calls.
  • Supported targets are those of std.io: x86_64 Linux GNU.

Example

The tutorial lesson reads pipelined requests from memory and writes a chunked response. The tests in tests/test_std_http.rs serve several requests on one loopback connection.

Limits

  • HTTP/1.0 and HTTP/1.1 only. No HTTP/2, or 100-continue handling beyond returning the interim response. Upgrades are not handled here: std.http.server hands a 101 connection to std.http.websocket, and detach hands a long-lived response to std.http.sse.
  • No connection pool, redirects, proxies, cookies, or TLS here; the server is std.http.server and the client std.http.client. Deadlines are the caller’s: use std.net’s *_before calls, or read with a deadline-aware Reader.
  • Only the chunked transfer coding. A body with another coding is rejected, not decoded.
  • read_request returns the target as text; request_target splits the forms a server accepts. Use std.url for other URLs.
  • Field values must be UTF-8; a peer that sends Latin-1 bytes is rejected.

Acknowledgements

std.http.decoder

A resumable HTTP/1.x request decoder with no I/O. Feed it whatever bytes are available now, and retain the unconsumed suffix. It shares request-head, target, framing, chunk-size and trailer validation with std.http. Its stored state contains owned bytes rather than shared strings, so it can live inside an actor node.

decoder(&limits) returns Result[Decoder, HttpError]. The limits are copied; max_line or max_head below 2 returns InvalidLimit. Allocation failures are HttpError.Allocation. Header bytes and lines are bounded by those limits; body chunks are at most feed_budget (4096) bytes.

import std.http
import std.http.decoder
from std.http.decoder import Event

def main() -> Result[(), std.http.HttpError | IoError | OverflowError[i64]]:
    mut decoder = std.http.decoder.decoder(&std.http.limits())?
    wire = b"POST /echo HTTP/1.1\r\nHost: localhost\r\nContent-Length: 4\r\n\r\nping"
    mut at = 0
    while not decoder.is_done():
        consumed, event = decoder.feed(&wire[at:])?
        at += consumed
        match event:
            case Some(Event.Head(head, _, _)):
                print(head.method)?
            case Some(Event.Body(body)):
                print(body.len())?
            case Some(Event.End):
                print("complete")?
            case _:
                pass
    decoder.finish()?
    Ok(())
POST
4
complete

This example already owns the complete request, so it can loop until End. A readiness loop must instead yield when no currently available input can advance the decoder. Bound the number of feed calls per scheduler turn too; the per-call byte budget alone does not bound an unbounded caller loop.

Progress and events

feed(source: &[u8]) reads a &[u8] view and returns Result[(consumed, Option[Event]), HttpError]. consumed counts from the start of the view, not from the start of the caller’s buffer. Advance the source cursor by exactly that count. Neither a short count nor Nothing implies EOF.

EventMeaning
Head(RequestHead, RequestTarget, Framing)Validated request metadata and body framing
Body(Bytes)Owned decoded body bytes
Trailers(list[Header])Validated trailer fields after a chunked body
EndRequest complete; no pipelined bytes have been consumed

An empty feed can emit a pending End. The returned head/trailer events own text, but the decoder does not retain that text after the call. Callers that keep events across actor callbacks must respect the actor’s storage boundary.

finish() reports explicit input EOF. It succeeds after a complete body, or before any nonblank request has begun, and reports truncated or malformed input otherwise. If body completion is pending, collect its End with an empty feed. finish does not make an incomplete request reusable.

State and reuse

MethodContract
is_started()Nonblank request input has arrived, even if the line is partial
has_head()A valid head was emitted and the decoder has not failed
is_done()End was emitted
total()Decoded body bytes consumed for this request
reset()Begin another request after End; otherwise Unfinished

Reset only when the application is ready to process pipelined input. Framing and allocation errors poison the decoder; discard it instead of resuming with a suffix whose request boundary is no longer trustworthy. Source ranges follow the ordinary byte-view rules; feed takes an already-formed view, not separate range parameters. The decoder never reads a socket, waits, or owns the caller’s input buffer.

Internal cursor arithmetic uses explicit opt-outs whose bounds come from the 4096-byte feed budget, validated CRLF lines and the copied head/body limits. Numeric length parsing keeps the shared parser’s recoverable errors. Calling from another arithmetic mode does not change the decoder’s HttpError contract.

std.http.body

import std.http.body reads request bodies inside a std.http.server handler: URL-encoded forms, JSON documents and multipart uploads. Every reader checks the request’s Content-Type, works within caller limits, and fails with a BodyError that maps to a response status. The module is checked Plenty in stdlib/std/http/body.plenty and adds no runtime primitives. The lesson Read forms, cookies and forwarded clients uses it.

import std.http.body
from std.http.body import BodyError

def main() -> Result[(), BodyError | IoError]:
    limits = std.http.body.form_limits()
    pairs = std.http.body.parse_form_text("name=Ada+Lovelace&lang=en&lang=fr", &limits)?
    for pair in &pairs:
        print(pair.key, pair.value)?
    match std.http.body.find(&pairs, "name"):
        case Some(index):
            print(pairs[index].value)?
        case Nothing:
            print("no name")?
    match std.http.body.parse_form_text("bad=%zz", &limits):
        case Err(error):
            print(error, std.http.body.status(&error))?
        case Ok(_):
            print("accepted")?
    Ok(())
name Ada Lovelace
lang en
lang fr
Ada Lovelace
std.http.body.BodyError.Malformed 400

Errors

VariantMeaningstatus
UnsupportedMediaTypeContent-Type missing, malformed or not the expected type415
TooLargeA byte limit was passed, or Content-Length declares more than it413
TooManyFieldsMore fields or parts than allowed413
MalformedBad percent-escape, multipart structure, boundary or part header400
InvalidUtf8A decoded name or value, or a multipart field, is not UTF-8400
Http(HttpError)Reading the body failed; BodyTooLarge is 413, Stream (timeout, cancellation, reset) 408413, 408, 400 or 500
Json(JsonError)The document is invalid (400), over max_input (413), or fails on allocation (500)400, 413 or 500
Storage(StreamError)Writing an upload to its temporary file failed500
Allocation(AllocError)Memory ran out, including while writing an upload to its file500

status(&error) -> i64 gives the status, and reject(exchange, &error) -> Result[(), HttpError] answers with it and its reason as plain text. The server’s own fallback only maps a failed body read, so a handler that wants 415 or 400 for a bad form calls reject instead of returning the error.

Forms

ItemMeaning
FormLimits(max_bytes, max_fields)Public NonNegative[i64] fields; form_limits() is 64 KiB and 256 fields
read_form(exchange, &limits)Result[list[Pair], BodyError]: read an application/x-www-form-urlencoded body
parse_form(&bytes, &limits)Decode a form already in memory
parse_form_text(&text, &limits)Decode text, such as exchange.query
find(&pairs, key)Option[i64]: index of the first pair named key

Pair(key, value) is std.url’s. Pairs keep their order and duplicates. + decodes to a space and %hh to a byte; a % without two hex digits is Malformed, and the decoded bytes must be UTF-8. Empty pieces between & are skipped, and a piece without = has an empty value. Parameters such as charset on the media type are accepted and ignored: forms are UTF-8. As in RFC 9110, an empty parameter (a/b; or a/b;;c=d) is allowed; a parameter without = makes the media type malformed.

read_form refuses a declared Content-Length over max_bytes before reading, and stops a chunked body as soon as it passes the limit. Byte totals that cannot be represented as i64 are also TooLarge, not arithmetic errors. Multipart part totals are checked before appending or writing the next piece; whole-body totals are checked after reading the next chunk but before retaining it.

JSON

ItemMeaning
read_json(exchange, &limits)Result[Json, BodyError]: read and parse a JSON body
send_json(exchange, status, &value, &limits)Result[(), BodyError]: encode and send as application/json

limits is a std.json.Limits; max_input bounds the body, and depth, duplicate keys and output size follow it. The request must be application/json or another application/...+json type.

Multipart uploads

ItemMeaning
MultipartLimits(max_parts, max_head, max_field, max_file, max_body)Public NonNegative[i64] fields; multipart_limits() is 64 parts, 4 KiB heads, 64 KiB fields, 8 MiB files, 16 MiB in all
read_multipart(exchange, &limits, storage: &Root)Result[list[Part], BodyError]: read a multipart/form-data body
Partname, filename: Option[str], content_type, size, content: Content
ContentText(str) for a field, File(TempFile) for an upload

max_head bounds one part’s header section, max_field a field held in memory, max_file one upload, and max_body every byte read, including the preamble, delimiters and epilogue. A declared Content-Length over max_body is refused before reading.

A part with a filename parameter is an upload. Its content is streamed into a new unnamed temporary file in storage as it arrives, never held whole in memory, and the file is rewound before it is returned: read it through file.stream after matching Content.File(file), or keep it with file.publish(&name). Because the file has no name until published, a request that fails, a connection that is cancelled, and a part the handler drops all leave nothing on disk. The descriptors are closed when the TempFiles drop. Each upload holds two descriptors until then, so max_parts also bounds descriptor use.

filename is what the client sent. It is not a path and is not cleaned; do not use it to name a file without checking it, and prefer a name the program chooses.

Other parts are fields and must be UTF-8. A part’s Content-Type defaults to text/plain and is reported, not checked. Each part needs a Content-Disposition: form-data field with a name. The boundary must be 1 to 70 characters from RFC 2046’s set; per that RFC, the content of a part cannot contain the delimiter line, so a part whose content holds CRLF--boundary ends there. Content-Transfer-Encoding is ignored, as RFC 7578 deprecates it.

Limits

  • Bodies are read whole for forms and JSON; use exchange.read_body directly for streamed processing.
  • Multipart fields are not streamed; their size is bounded by max_field.
  • No multipart/mixed nesting, and no character sets other than UTF-8.

std.http.client

import std.http.client sends HTTP/1.1 requests over pooled keep-alive connections. Each call has a caller deadline and cancellation token and a timeout for each phase. Redirects are bounded and drop credentials when the origin changes. A proxy is used only when configured, and a request is replayed only when that is safe. Request bodies are in memory or streamed from a std.io Reader. Head parsing, writes and body framing use std.http.typed, with shared std.http validation, and sockets from std.net; std.http.https adds https URLs through std.tls. Both modules are checked Plenty in stdlib/std/http/ and add no runtime primitives. The tutorial lessons Send HTTP requests, Share one HTTP client between workers and Send HTTPS requests show complete programs.

import std.bytes
import std.http
import std.http.client
import std.http.server
import std.net
import std.time
from std.http.client import ClientError
from std.http.server import Exchange
from std.net import TcpStream
from std.sync import CancellationToken

pub enum Route:
    Echo

pub class Echoer:
    pub def handle(self, route: &Route, exchange: &mut Exchange[TcpStream]) -> Result[(), AllocError | std.http.HttpError | std.bytes.BytesError]:
        mut body = std.bytes.empty()?
        exchange.read_all(&mut body)?
        exchange.send(200, "text/plain", &body)

type VisitError = AllocError | IoError | std.bytes.BytesError | ClientError | RangeError

def visit(port: u16, stop: CancellationToken) -> Result[(), VisitError]:
    with stop.cancel_on_exit():
        never = CancellationToken()?
        deadline = std.time.deadline_after(&std.time.monotonic_now(), &std.time.seconds(5u32))?
        client = std.http.client.client(std.http.client.config()?)?
        mut post = std.http.client.request("POST", &f"http://127.0.0.1:{port}/echo"?)?
        post.add("Content-Type", "text/plain")?
        post.body = std.bytes.from_text("ping")?
        reply = client.fetch(post, &deadline, &never)?
        print(reply.status, reply.body.to_text()?)?
    Ok(())

def main() -> Result[(), SpawnError[()] | std.http.server.ServerError | std.net.NetError | VisitError]:
    mut routes = std.http.server.router[Route]()?
    routes.add("POST", "/echo", Route.Echo)?
    listener = std.net.listen(&std.net.loopback_v4(0u16), 16)?
    port = listener.local_address()?.port
    stop = CancellationToken()?
    with spawn(visit, port, stop.share()).drop_job()? as task:
        drop(std.http.server.serve(listener, &routes, &Echoer(), &std.http.server.config(), &stop)?)
        task.join()?
    Ok(())
200 ping

Clients

ItemMeaning
client(config: Config)Result[Client[Plain, TcpConnector], ClientError]: plain http over TCP
connector[T: Transport, C: Connector[T]](opener: C, config: Config)Result[Client[T, C], ClientError]: connections come from opener; give both type arguments
client.send(request, &deadline, &cancel)Result[Response[T], ClientError]: the final response head, after redirects
client.send_from(request, &mut body, &upload, &deadline, &cancel)send with the body read from body, a std.io Reader; see Streamed bodies
client.fetch(request, &deadline, &cancel)Result[Fetched, ClientError]: send, read the whole body, finish
client.get(url: &str, &deadline, &cancel)fetch of a GET
client.finish(response)Pool the connection if the body was read to its end and both sides allow reuse; otherwise close it
client.close_idle()Close the connections pooled when it is called
client.idle()i64: pooled connections
client.stats()Stats

Both constructors fail with InvalidConfig for a config outside the bounds below. Every method borrows the client shared, so one client serves many threads through Shared handles; see Sharing a client between threads.

deadline bounds the whole exchange, including later reads of the body of the Response that send returns. cancel stops it: a wait in progress ends with Net(Cancelled). A request whose token is already cancelled, or whose deadline has passed, fails with Net(Cancelled) or Net(TimedOut) before it takes a connection, so it leaves pooled connections alone.

Requests and responses

ItemMeaning
request(method: &str, url: &str)Result[Request, ClientError] with no fields and an empty body
Request.method, url, headers: list[Header], body: BytesPublic fields
request.add(name, value)Result[(), ClientError]: append a field
Response.status, head: ResponseHead, url, redirectsFinal status and head, the URL that answered, and redirects followed
response.read_body(&mut buffer, start, stop)Result[i64, ClientError]: part of the body; 0 is the end
response.read_all(&mut out)Result[i64, ClientError]: append the rest, within limits.max_body
response.read_all_progress(&mut out)Result[i64, Partial[ClientError]]: append the rest and retain the count appended by this call if it fails
response.read_typed(...)Result[i64, StepError[ClientError]]: read with the complete returned cause
typed_response(response), unwrap_response(stream)Owning TypedResponse[T] adapter for std.io.typed.Reader[ClientError]; unwrap before client.finish
response.total(), response.trailers()Payload bytes successfully read, and borrowed trailer fields (complete after is_done)
response.read(...), take_failure()A std.io Reader; a client failure is Io(InvalidInput) and take_failure returns it, a short body is UnexpectedEof
response.is_done()The whole body has been read
Fetched.status, head, url, redirects, body: BytesA whole response

The client writes Host from the URL, and Content-Length when the body is not empty or the method is POST, PUT or PATCH. An in-memory body is sent with Content-Length. A request that sets Host, Content-Length, Transfer-Encoding, Connection, Keep-Alive, Upgrade, TE, Trailer, Expect, Proxy-Authorization or Proxy-Connection fails with InvalidRequest, as do CONNECT and a method that is not an HTTP token. These checks run before any connection opens.

URLs must be absolute http or https URLs with a host. Credentials in the URL (user:pass@) are InvalidUrl; send an Authorization field instead. The fragment is not sent. The TCP connector refuses an https URL with UnsupportedScheme; use std.http.https.

Interim 1xx responses are skipped, up to 16; 101 Switching Protocols is UnexpectedInterim. A response to HEAD, and a 204 or 304, has no body.

Typed response reads preserve network, TLS, cancellation and framing causes directly. read_all_progress counts bytes appended after the output’s existing prefix. It reserves output capacity before consuming payload; on failure the prefix and appended bytes remain available, the connection closes, and total() retains previously returned payload progress. The existing read_all returns the same cause while omitting the count. send and fetch also retain their established ClientError return types; use the lower-level typed HTTP operations when per-head or per-write partial counts are needed.

The legacy read method still implements std.io.Reader and uses take_failure for causes that StreamError cannot represent. Bridging that method with std.io.typed.reader cannot recover the erased cause; use typed_response instead. Internal head parsing, body decoding and CONNECT tunnelling no longer store transport errors for later retrieval.

Streamed bodies

send_from reads the request body from any std.io Reader, 16 KiB at a time, and never holds more of it. The request’s own body must be empty, or the call fails with InvalidRequest before connecting.

ItemMeaning
sized(length: i64)Upload sent with Content-Length: length
chunked()Upload sent with Transfer-Encoding: chunked and no trailers
Upload.length: Option[i64], expect_continue: boolPublic fields; expect_continue is False from both constructors

With sized, the reader must yield exactly length bytes: more, or an end before length, is WrongBodyLength and the connection is closed. A negative length is InvalidRequest. A reader failure is Body(StreamError), or Allocation. Explicit reader interruption is retried; negative or oversized successful counts are rejected as Body(InvalidProgress).

With expect_continue, the head carries Expect: 100-continue and the client waits up to config.continue_timeout before sending the body:

The serverThe client
Sends 100 ContinueSends the body, then reads the final response
Sends nothing before continue_timeoutSends the body anyway, as RFC 9110 allows
Sends a final response first, such as 417 or 401Returns it without sending the body; the connection is closed, not pooled

A 417 Expectation Failed is returned like any other status; the client does not retry without the expectation, since the reader may not be restartable.

Connections and the pool

After fetch, or finish on a fully read response, the connection waits in the pool for another request to the same scheme, host and port (and proxy). The most recently used connection is taken first. A connection is closed instead of pooled when:

  • either side sent Connection: close, or the response is HTTP/1.0 without keep-alive, or its body ran to the end of the connection;
  • the body was not read to its end, or reading it failed;
  • bytes beyond the response are already buffered;
  • the pool already holds max_idle_per_origin connections for that origin, or max_idle in all (the oldest is closed to make room).

Dropping a Response without finish closes its connection. Before reusing a pooled connection the client checks, without waiting, that nothing has arrived on it: a connection the server closed, or one with unexpected bytes, is closed and another is used. A check that cancellation cuts short puts the connection back. A connection idle longer than idle_timeout is closed when the pool is next used.

Sharing a client between threads

A client’s pool is one value behind a std.sync mutex, and its counters are atomics, so threads share a client by sharing the value:

client = Shared(std.http.client.client(config)?)?
job = pool.submit(fetch_all, client.share(), url)?

Each job owns a handle and calls get, send, finish and the rest through it; a connection one thread parks is reused by whichever thread asks next. The lock is held only while a connection is taken out of the pool or put back in: connecting, TLS, writing, waiting for a response and reading a body all run with the lock released, as do closing any connection and the readiness probe before reuse. A request to a slow origin therefore does not delay requests to another through the same client. close_idle takes the connections out one at a time, oldest first, and stops after as many as were idle when it was called, so other threads parking connections cannot keep it running.

max_idle, max_idle_per_origin and idle_timeout bound the shared pool as a whole, and Stats counts every event exactly once. Every request sent, counting replays and redirects, went on a connection counted in opened or in reused, so opened + reused is the number of requests sent. Every connection counted in opened is idle, held by a request or a Response, or counted in discarded once closed, whatever closed it; so after all jobs finish and every response has been finished or dropped, opened is idle() plus discarded. stats() reads each counter without the lock, so a snapshot taken while jobs run may show one counter ahead of another.

Sharing a client is subject to the Shared rules: the connector’s type must be able to cross threads. TcpConnector and HttpsConnector, whose TLS configuration is immutable, qualify. A single-threaded client pays an uncontended lock and unlock per checkout and return, and an atomic increment per counted event; neither allocates.

Replaying requests

A server may close an idle connection at the moment the client reuses it. When a request on a reused connection fails before any byte of a response arrives, by end of input, a reset, or a failed write, the client sends it again on a new connection, at most retries times, but only if:

  • the method is GET, HEAD, OPTIONS, TRACE, PUT or DELETE, and
  • the body is the request’s in-memory body, or a streamed body of which nothing has been read yet (for example, while waiting for 100 Continue).

POST, PATCH and other methods are never replayed, since the server may have acted on them. A failure on a new connection, or after response bytes, is returned, and so is a timeout or cancellation.

Redirects

A 301, 302, 303, 307 or 308 response with a Location field is followed, at most redirects times; one more is TooManyRedirects. With redirects = 0 the redirect response itself is returned.

StatusNext request
303GET without a body (HEAD stays HEAD)
301, 302 after POSTGET without a body
301, 302 otherwise, 307, 308Same method and body

When the body is dropped, so are Content-* fields. A redirect that keeps the body is not followed for a streamed body, which cannot be read twice: send_from returns the 301, 302, 307 or 308 response itself. Location may be absolute, scheme-relative (//host/path), absolute-path or relative; it is resolved against the current URL and . and .. segments are removed. A location that does not parse is InvalidRedirect, another scheme is UnsupportedScheme, and a move from https to http is InvalidRedirect.

When the scheme, host or port changes, the next request and every later one lose Authorization, Cookie, and every field named in config.sensitive. They are not restored on a redirect back. The body of a redirect response is read and discarded, up to 64 KiB, so the connection can be reused; a longer body closes it.

Proxies

ItemMeaning
proxy(url: &str)Result[Proxy, ClientError] for http://host:port, no path or credentials
Proxy.endpoint, authorization: Option[str]Where to connect, and the Proxy-Authorization value

With config.proxy set, every http request goes to the proxy with an absolute-form target (GET http://host/path HTTP/1.1). An https request opens a tunnel instead: the client sends CONNECT host:port HTTP/1.1 to the proxy, and after a 2xx reply runs TLS to the origin through it, verifying the origin’s certificate, and sends an origin-form request. A non-2xx reply is TunnelRefused(status); its body is not read. authorization is sent to the proxy as Proxy-Authorization, on CONNECT for a tunnel, and never to an origin. Nothing is read from the environment: HTTP_PROXY and NO_PROXY are ignored.

Configuration

config() returns a Config with every field public:

FieldMeaningconfig()
limitsstd.http Limits for response heads and bodies, including a proxy’s CONNECT replystd.http.limits()
connect_timeoutResolve the host and connect10 s
write_timeoutAny one write10 s
head_timeoutFrom the end of the request to the end of the final response head30 s
read_timeoutAny one read30 s
idle_timeoutKeep a pooled connection30 s
continue_timeoutWait for 100 Continue before sending a streamed body anyway1 s
max_idlePooled connections in all, at least 0; 0 sends Connection: close16
max_idle_per_originPooled connections for one origin, at least 04
redirectsRedirects to follow, 0 to 10010
retriesReplays of a safe request, 0 to 101
sensitiveFurther field names dropped across originsempty
proxyOption[Proxy]Nothing

Every wait ends at the earliest of the caller’s deadline, its phase timeout and its per-operation timeout. Internal deadline addition saturates at u64.MAX; its bounds are explicit, and debug/optimized boundary tests verify it without allocations. Retry, redirect, response, upload and pool cursor arithmetic uses documented opt-outs justified by configuration, body budgets and validated progress. User connectors and transports retain independent declared arithmetic contracts; these implementation modes do not extend into their callbacks.

Transports

ItemMeaning
protocol Transportread_before and write_before with TcpStream’s parameters, returning Result[i64, ClientError]; write_before takes its source as &[u8]
protocol Connector[T]open(self, endpoint: &Endpoint, deadline, cancel) -> Result[T, ClientError]; the connector is borrowed shared, since threads sharing a client open connections at the same time
Endpoint(scheme, host, port: u16, tunnel: Option[Tunnel])Where to connect: the origin, or the proxy; host is lowercase without brackets
Tunnel(host, port: u16, authorization: Option[str], limits: Limits)The proxy to reach the origin through with CONNECT; limits bounds the proxy’s reply head
dial(&endpoint, &deadline, &cancel)Result[TcpStream, ClientError]: a TCP connection to the endpoint, through its tunnel if it has one
Plain, TcpConnectorA TCP connection; the connector refuses https and tunnels with UnsupportedScheme

A connector decides how to reach an endpoint. dial resolves the host and tries each address in turn. An in-memory connector makes code that uses a client testable without sockets; one that hands out scripted connections keeps them in a Mutex, since open borrows it shared. connector checks both type arguments against the protocols where it is called, so a connector whose open takes self: &mut ... is reported there; like any argument of a bound in another module, a connector and a transport must be pub classes.

HTTPS

import std.http.https supplies a connector that opens http endpoints over TCP and https ones over std.tls. It is a separate module so that a program using only http does not depend on the TLS stack.

ItemMeaning
client(config: Config, tls: ClientConfig)Result[Client[Connection, HttpsConnector], ClientError]
system_client(config: Config)client with the operating system’s trust roots
connector(tls: ClientConfig)HttpsConnector, for std.http.client.connector
secure_error(error: TlsError)The ClientError a TLS failure becomes

Verification is always on: the server’s certificate must chain to the roots in tls and name the URL’s host, a DNS name or IP address, which is also sent as the server name (SNI). There is no switch to skip verification; trust a private CA by building tls from its certificate. Client certificates, protocol versions and revocation lists come from tls. If tls offers ALPN protocols, a server that picks anything but http/1.1 is Tls(Incompatible).

A pooled https connection is keyed by scheme, host and port like any other, so it is reused only for the same server name. A redirect from https to http is InvalidRedirect.

TLS failures map to ClientError:

TlsErrorClientError
Net(Stream(Allocation(a)))Allocation(a)
Any other Net(e)Net(e)
TruncatedNet(Stream(UnexpectedEof)), so a closed pooled connection is replayed as a TCP one is
ClosedNet(Stream(Io(Closed)))
Allocation(e)Allocation(e)
Any certificate, handshake or configuration failureTls(TlsFailure) with the same variant name

Errors

Net(TimedOut) means a caller or configured phase deadline passed. Net(ConnectionTimedOut) means the kernel gave up on the connection; it is propagated as a connection failure, including through HTTPS, rather than treated as the end of the 100 Continue waiting period.

ClientError is Net(NetError) for connection failures, timeouts and cancellation; Http(HttpError) for malformed or oversized responses, with Truncated for a connection that closed early; Url(UrlError); Allocation; Tls(TlsFailure) for a TLS handshake or session failure; TunnelRefused(status) for a proxy that refused CONNECT; Body(StreamError) for a streamed body’s reader failure; and InvalidConfig, InvalidUrl, UnsupportedScheme, InvalidRequest, InvalidRedirect, TooManyRedirects, UnexpectedInterim and WrongBodyLength. TlsFailure has the variants UntrustedCertificate, ExpiredCertificate, WrongHost, InvalidCertificate, RevokedCertificate, Rejected, Protocol, InvalidName, InvalidConfiguration, NoTrustRoots, CertificateRequired and Incompatible; see TlsError. No variant carries a URL or field value, so printing an error does not reveal credentials.

Stats counts, as u64: opened connections; reused, requests sent on a pooled connection; retried replays; redirected hops; and discarded connections, those closed for any reason: the response or server ended reuse, the body was not read to its end, the exchange failed or was replayed, the connection was stale, expired, over a pool limit or had unread bytes buffered, or close_idle closed it. See Sharing a client between threads for the totals they keep. These counters retain atomic fetch_add’s wrapping behavior; checked-default arithmetic does not change that primitive’s contract.

Limits

  • No cookie jar. Read Set-Cookie fields from response.head and send a Cookie field yourself; the client drops it when the origin changes.
  • No content decoding. The client does not send Accept-Encoding, so servers normally send identity bodies. A program that asks for gzip or br checks the response’s actual coding and decodes the body itself with std.compress under an independent decoded-size limit. See Brotli and HTTP.
  • No HTTP/2, and no proxy for https other than an http proxy with CONNECT.
  • No protocol upgrades on pooled connections. WebSocket clients are in std.http.websocket, which opens connections with dial.
  • Connecting tries addresses one after another, not in parallel.

Acknowledgements

  • Design references: Go’s net/http Client and Python’s requests, including dropping credentials on cross-origin redirects.

std.http.cookie

import std.http.cookie reads the Cookie fields of a request and writes Set-Cookie fields (RFC 6265). Reading is lenient; writing is strict, so no value can add an attribute or a header field. The module is checked Plenty in stdlib/std/http/cookie.plenty and adds no runtime primitives; signing uses std.crypto.

import std.time
import std.http.cookie
from std.http import Header
from std.http.cookie import CookieError

def main() -> Result[(), CookieError | AllocError | IoError]:
    headers = [Header("Cookie", "theme=dark; bad value; lang=\"en\"")]?
    received = std.http.cookie.cookies(&headers, 16)?
    for cookie in &received:
        print(cookie.name, cookie.value)?

    mut reply = std.http.cookie.set_cookie("theme", "light")?
    reply.max_age = Some(std.time.seconds(3600u32))
    print(reply.to_string()?)?
    print(std.http.cookie.removal("lang")?.to_string()?)?
    Ok(())
theme dark
lang en
theme=light; Path=/; Max-Age=3600; Secure; HttpOnly; SameSite=Lax
lang=; Path=/; Max-Age=0; Secure; HttpOnly; SameSite=Lax

Reading

ItemMeaning
Cookie(name, value)One cookie a client sent
cookies(&headers, limit)Result[list[Cookie], CookieError]: every valid cookie in the Cookie fields, in order
find(&cookies, name)Option[i64]: index of the first cookie named name

A pair whose name is not a token or whose value has a byte outside RFC 6265’s cookie-octets (control bytes, space, ", ,, ;, \) is skipped, as is a piece without =. The field carries cookies set by every application on the host, and one malformed cookie must not make every request fail. A value wrapped in double quotes loses them. More than limit valid cookies is TooManyCookies. Each field is copied once into a byte buffer and checked with bulk scans; only valid cookies are copied out as text, so a skipped pair allocates nothing.

Values are not decoded. A cookie’s value is whatever the client sends; check it, or sign it, before trusting it.

Writing

ItemMeaning
set_cookie(name, value)Result[SetCookie, CookieError]: a cookie with the defaults below
removal(name)Result with a cookie that deletes name: empty value, Max-Age=0; CookieError for an invalid name
SetCookiePublic fields name, value, path, domain, max_age: Option[Duration], secure, http_only, same_site: SameSite
cookie.to_string()Result[str, CookieError]: the field value, after checking every part
add(&mut head, &cookie)Append a Set-Cookie field to a ResponseHead
SameSiteStrict, Lax, or Unrestricted (SameSite=None)

set_cookie gives Path=/, Secure, HttpOnly, SameSite=Lax and no Max-Age, which ends the cookie with the browser session. Change the fields to relax them; to_string checks the result each time:

RuleError
Name is a non-empty tokenInvalidName
Value has only cookie-octetsInvalidValue
path is empty or starts with / and has only visible ASCII and space, no ;InvalidAttribute
domain has only letters, digits, - and ., and does not start or end with .InvalidAttribute
max_age is a nonnegative DurationPositive fractions round up to whole seconds; zero removes the cookie
SameSite=None needs secureInvalidAttribute
A __Secure- name needs secureInvalidAttribute
A __Host- name needs secure, path / and no domainInvalidAttribute

Rounding max_age is representable even for the largest Duration (u64.MAX nanoseconds becomes 18446744074 seconds).

Empty path and domain are left out. CookieError also has TooManyCookies, Http(HttpError) from add, Crypto(CryptoError) from the HMAC in the signing functions, and Allocation.

Signing

ItemMeaning
sign(&keys, name, value)Result[str, CookieError]: value.<tag>
verify(&keys, name, signed)Result[Option[str], CookieError]: the value when the tag matches under any key, else Nothing
signed_cookie(&keys, name, value)Result[SetCookie, CookieError]: set_cookie(name, sign(...))
find_signed(&cookies, &keys, name)Result[Option[str], CookieError]: verify on each cookie named name, in order; the first whose tag verifies; Nothing when none does
min_key32: shortest key, in bytes

The tag is HMAC-SHA256 over name=value under a key, in unpadded base64url (43 characters) after a .. The name is covered, so a value signed for one cookie is refused under another. keys is a list[Bytes]: sign uses the first, and verify tries every key without stopping at a match, each with the constant-time comparison of Mac.verify, so the time taken depends only on the number of keys. To rotate, put the new key first and keep the old one listed until cookies signed with it have expired.

An empty list or a key shorter than min_key is InvalidKey; sign also checks the name and value as set_cookie does. A malformed or mismatched tag is Nothing, not an error. Signing allocates the result text and a fixed-size HMAC state; verification allocates the decoded tag and the returned value, and nothing per key beyond the HMAC state.

Signing stops changes, not reading: the value travels in clear. It does not expire or bind the cookie to a client; put an expiry time in the value, or keep sessions on the server, when that matters.

Limits

  • No Expires (use max_age), Partitioned, or Priority attributes.
  • No encryption; a signed value is still readable. A session identifier can instead be a random token from std.secrets, looked up on the server.
  • Set-Cookie fields sent by another server are not parsed.

std.http.files

import std.http.files answers std.http.server requests with files below one directory. The module is checked Plenty in stdlib/std/http/files.plenty; it uses std.fs Root handles and file_metadata, and adds no runtime primitives of its own.

ItemMeaning
serve(exchange, &root, path)Result[i64, FilesError]: answer the request with the file at path below root; the status sent
open_file(&root, path)Result[Option[FileStream], FilesError]: the regular file at path, or Nothing when it is refused or missing
media_type(name)str: the Content-Type for name’s extension, from std.mime
http_date(timestamp: &Timestamp)Result[Option[str], FilesError]: an IMF-fixdate such as Sun, 06 Nov 1994 08:49:37 GMT; Nothing before 1970 or after 9999
parse_http_date(text)Result[Option[Timestamp], FilesError]: a timestamp at a whole second, or Nothing unless text is an exact IMF-fixdate
max_path4096: longest accepted path, in bytes
routes.add("GET", "/static/{path...}", 0)?
...
path = exchange.params[0]
std.http.files.serve(exchange, &self.public, &path)?
import std.time
import std.fs
import std.io
import std.http.files
import std.path
from std.http.files import FilesError

def main() -> Result[(), AllocError | IoError | std.fs.FsError | std.io.StreamError | FilesError]:
    parent = std.fs.root(&std.path.from_text(".")?)?
    mut site = std.fs.temporary_dir(&parent, &std.path.from_text("site-")?, 8)?
    for name in ["index.html", ".env"]?:
        mut staged = std.fs.temporary_file(site.root())?
        staged.stream.file.write("<h1>Welcome</h1>\n")?
        staged.publish(&std.path.from_text(name)?)?
        staged.close()?
    for path in ["index.html", ".env", "../index.html", "missing.css"]?:
        found = std.http.files.open_file(site.root(), &path)?
        print(path, found.is_some(), std.http.files.media_type(&path))?
    print(std.http.files.http_date(&std.time.Timestamp(784111777, 0u32))?)?
    for name in ["index.html", ".env"]?:
        site.root().remove_file(&std.path.from_text(name)?)?
    site.cleanup()?
    Ok(())
index.html True text/html; charset=utf-8
.env False application/octet-stream
../index.html False text/html; charset=utf-8
missing.css False text/css; charset=utf-8
Option[str].Some("Sun, 06 Nov 1994 08:49:37 GMT")

Confinement

path is relative and already percent-decoded, as a {name...} capture is. open_file splits it on / and opens each segment through the previous directory’s Root, so every component is opened with O_NOFOLLOW relative to its parent. A path is refused when:

  • it is empty or longer than max_path,
  • a segment is empty, which covers an absolute path and //,
  • a segment starts with ., which covers ., .. and hidden files such as .env or .git,
  • it contains a NUL byte,
  • any component is a symbolic link, the last is not a regular file (a directory, FIFO or device), or opening a component fails because it does not exist, is not a directory where one is needed, is a directory where a file is needed, has too long a name, or is not permitted (ENOENT, ENOTDIR, EISDIR, ENAMETOOLONG, EACCES, ELOOP, ENXIO).

Any other open failure, such as EMFILE, EIO or ENOMEM, is a server fault rather than a refusal: open_file and serve return FilesError.Io.

Cleaning the path as a string would not be enough: a link inside the directory can lead out of it, and a path checked before opening can be replaced before it is opened. Here the open itself refuses links, and the file is opened non-blocking so a FIFO cannot stall the worker. Every refusal is the same 404 Not Found, so a client learns nothing about what lies outside the root or which entries are hidden.

Responses

RequestResponse
Not GET or HEAD405 with Allow: GET, HEAD
Refused or missing path404, plain text
If-None-Match lists the file’s tag, or is *304 with ETag and Last-Modified, no body
No If-None-Match, and If-Modified-Since is not before the file’s time304
Otherwise200 with Content-Type, Content-Length, X-Content-Type-Options: nosniff, ETag, Last-Modified; HEAD sends no body

The tag is weak, W/"size-seconds.nanoseconds" from the open file’s size and modification time, compared weakly as RFC 9110 requires for If-None-Match. If-None-Match takes precedence over If-Modified-Since, which is ignored unless it is an exact IMF-fixdate (the obsolete RFC 850 and asctime forms are refused). Size and time come from std.fs.file_metadata on the open file, so they describe what is sent even if the path is replaced meanwhile. A metadata size above i64.MAX is rejected as FilesError.Http(HttpError.InvalidContentLength) before validators are built, even when a conditional request would otherwise answer 304. No wrapped size, invalid header or file-body read is produced on that path.

The body is read in 16 KiB pieces into one buffer and written as it is read, so memory does not depend on the file’s size. Exactly the size seen at open is sent. A file that shrinks while it is sent is Truncated after the head has gone out; the server then closes the connection.

media_type(name) is std.mime.from_file_name(name): it compares the extension after the last . without case, allocates nothing, and returns application/octet-stream for an extension not in the table.

Errors

FilesError is Io(IoError) from opening a path component with an error outside the refusal list under Confinement or from reading the open file’s metadata, Stream(StreamError) from reading it, Http(HttpError) from response framing/writing or an unrepresentable file size, Truncated, and Allocation. A refused or missing path is not an error.

Limits

  • No Range requests (206), no If-Range, and no strong tags.
  • No directory index: a path naming a directory is 404, so index.html must be named in the route or the handler.
  • No compression, precompressed variants, or Cache-Control; a handler that needs them builds the response itself from open_file, or a reverse proxy adds them.
  • A file is read with ordinary blocking reads on the worker; a slow filesystem holds the worker.
  • Linux only, like std.fs.

Acknowledgements

std.http.proxy

import std.http.proxy finds the client of a request that came through reverse proxies. A forwarding header is believed only from a proxy the program names. The module is checked Plenty in stdlib/std/http/proxy.plenty and adds no runtime primitives. This page also gives the deployment guidance for std.http.server.

import std.http.proxy
import std.net
from std.http import Header
from std.http.proxy import Forwarding, ProxyError

def main() -> Result[(), ProxyError | std.net.NetError | AllocError | IoError]:
    mut proxies = std.http.proxy.proxies(Forwarding.XForwardedFor)?
    proxies.trust("10.0.0.0/8")?
    headers = [Header("X-Forwarded-For", "198.51.100.7, 10.0.0.3"), Header("X-Forwarded-Proto", "https")]?
    for peer_text in ["10.0.0.2:443", "203.0.113.9:443"]?:
        peer = std.net.parse_address(&peer_text)?
        found = std.http.proxy.client(&peer, &headers, &proxies)?
        print(std.net.format_ip(found.address)?, found.scheme, found.proxied)?
    Ok(())
198.51.100.7 Option[str].Some("https") True
203.0.113.9 Option[str].Nothing False

Clients

ItemMeaning
proxies(header: Forwarding)Result[Proxies, ProxyError]: trusts nobody yet
proxies.trust(network)Trust 10.0.0.0/8, fd00::/8 or one address
proxies.trusts(&ip)bool
proxies.max_hopsForwarding entries allowed in one request; 16
ForwardingXForwardedFor (with X-Forwarded-Proto) or Forwarded (RFC 7239)
client(&peer, &headers, &proxies)Result[Client, ProxyError]
client_of(exchange, &proxies)client for an exchange’s peer and fields
Clientaddress: IpAddress, scheme: Option[str], proxied: bool

The rule:

  1. If the peer is not trusted, it is the client, and the forwarding headers are ignored.
  2. Otherwise the entries of the chosen header are read from the right. Each trusted address is a proxy; the first untrusted one is the client.
  3. If every entry is trusted, the leftmost is the client. If there are no entries, the peer is.

Entries left of the client were written by the client. Under X-Forwarded-For they are never parsed as addresses, but they still count toward max_hops. Under Forwarded every element is syntax-checked and counted, so a malformed element or too many elements anywhere in the header fails. The other header is always ignored, so a deployment that uses Forwarded cannot be fooled with X-Forwarded-For.

An entry is 192.0.2.1, 192.0.2.1:8080, 2001:db8::1 or [2001:db8::1]:8080; the port is dropped. An entry that must be read but is none of these, such as unknown or an obfuscated _name, fails with InvalidForwarded, as does a malformed Forwarded element. More than max_hops entries is TooManyHops. Empty comma-separated pieces are skipped. Quoted escapes must have a following byte; a trailing backslash is InvalidForwarded. Scanners stop at the end of a field without advancing an index beyond its length, including at the i64 representation limit. An IPv4-mapped IPv6 address, as a dual-stack listener reports IPv4 peers, is treated as the IPv4 address.

scheme is http or https, lowercased, when a trusted proxy reported it: the client’s element’s proto= for Forwarded, or the last X-Forwarded-Proto value, which the nearest proxy wrote. Any other scheme is InvalidForwarded. It is Nothing when the peer is not trusted.

trust refuses a network with bits set below its prefix length (10.0.0.1/8) with InvalidNetwork, so a typo cannot trust more than intended. client_of fails with UnknownPeer when the exchange has no peer, as with serve_connection on a custom transport.

Deploying behind a reverse proxy

std.http.server serves plain HTTP/1.1. For a public service, run it behind a reverse proxy that terminates TLS:

  • Listen on loopback or a private address, never directly on the public interface. Then every peer is a proxy.
  • Configure the proxy to overwrite, not append to, the header the program reads, or to append only its own peer address. Trust exactly the proxy’s addresses.
  • Let the proxy enforce request rates and connection limits per client; the server’s limits are per connection and per worker.
  • Keep config.limits and the body limits below the proxy’s, so the server answers oversized requests itself.
  • Cookies default to Secure, which browsers send only over HTTPS. The program sees plain HTTP from the proxy; check scheme rather than assuming.

nginx and Caddy configurations tested in front of the server are in tests/fixtures/proxy/; the results are under reverse proxies.

Serving HTTPS directly

Without a proxy, serve TLS from the program with std.http.tls. Then it faces clients itself:

  • Listen on the public address only through std.http.tls.serve; a plain serve on the same host should only redirect or be absent.
  • No proxy is trusted, so leave Proxies empty: client_of returns the peer.
  • Keep the server’s limits, timeouts and idle_connections tight; nothing in front enforces per-client rates.
  • Cookies keep their Secure default, and the scheme is always https.
type HandlerError = std.http.HttpError | std.http.files.FilesError | std.http.cookie.CookieError | std.bytes.BytesError
type SetupError = std.http.server.ServerError | std.bytes.BytesError | AllocError | IoError | std.tls.TlsError | std.net.NetError

pub enum Route:
    Files
    Login
    Me

pub class Site:
    pub public: Root
    pub keys: list[Bytes]

    pub def handle(self, route: &Route, exchange: &mut Exchange[TlsTransport]) -> Result[(), HandlerError]:
        match route:
            case Route.Files:
                path = exchange.params[0]
                std.http.files.serve(exchange, &self.public, &path)?
            case Route.Login:
                mut head = std.http.response(204, "No Content")?
                std.http.cookie.add(&mut head, &std.http.cookie.signed_cookie(&self.keys, "session", "user42")?)?
                exchange.respond(head, &std.bytes.empty()?)?
            case Route.Me:
                found = std.http.cookie.cookies(&exchange.request.headers, 8)?
                match std.http.cookie.find_signed(&found, &self.keys, "session")?:
                    case Some(user):
                        exchange.send_text(200, &user)?
                    case Nothing:
                        exchange.send_text(401, "sign in first")?
        Ok(())

def main() -> Result[(), SetupError]:
    mut table = std.http.server.router[Route]()?
    table.add("GET", "/s/{path...}", Route.Files)?
    table.add("POST", "/login", Route.Login)?
    table.add("GET", "/me", Route.Me)?
    keys = [std.bytes.from_text(&read_text("/etc/site/session.key")?)?]?
    site = Site(std.fs.root(&std.path.from_text("/srv/site")?)?, keys)
    tls = std.tls.server_config(&read_text("/etc/site/chain.pem")?, &read_text("/etc/site/key.pem")?)?
    listener = std.net.listen(&std.net.parse_address("0.0.0.0:443")?, 128)?
    stop = CancellationToken()?
    report = std.http.tls.serve(listener, tls, &table, &site, &std.http.server.config(), &stop)?
    Ok(())

tests/test_std_http_files.rs runs this program on loopback with the test certificates in tests/fixtures/tls.

Static files

std.http.files serves files below a Root. It opens the request path one segment at a time through directory handles, refuses .., hidden names, links and anything but regular files, and answers ETag and If-Modified-Since; see its limits (no Range, no directory index). A reverse proxy can still serve static files when it is already in front.

std.http.server

import std.http.server serves HTTP/1.1 over std.net TCP connections: method and path routing, a fixed pool of worker threads fed by a bounded admission queue, a deadline for each phase of a request, and graceful shutdown. Framing comes from std.http. import std.http.tls serves HTTPS through std.tls. Both modules are checked Plenty in stdlib/std/http/ and add no runtime primitives. The tutorial lesson Serve HTTP requests shows a complete server. Qualification records which production gates the server passes, its measured baselines, and deployment guidance; the server is production-ready.

The separate std.actor.http.server uses supervised connection actors and nonblocking callbacks. Its qualification is independent; the two modules offer different execution and failure models and remain separate public APIs.

import std.bytes
import std.http
import std.http.client
import std.http.server
import std.net
import std.time
from std.http.server import Exchange
from std.net import TcpStream
from std.sync import CancellationToken

pub enum Route:
    Hello

pub class Greeter:
    pub def handle(self, route: &Route, exchange: &mut Exchange[TcpStream]) -> Result[(), std.http.HttpError | AllocError]:
        name = exchange.params[0]
        exchange.send_text(200, &f"hello, {name}"?)?
        Ok(())

type VisitError = AllocError | IoError | std.bytes.BytesError | std.http.client.ClientError | RangeError

def visit(port: u16, stop: CancellationToken) -> Result[(), VisitError]:
    with stop.cancel_on_exit():
        never = CancellationToken()?
        deadline = std.time.deadline_after(&std.time.monotonic_now(), &std.time.seconds(5u32))?
        client = std.http.client.client(std.http.client.config()?)?
        for path in ["/hello/Ada", "/missing"]?:
            reply = client.get(&f"http://127.0.0.1:{port}{path}"?, &deadline, &never)?
            print(reply.status, reply.body.to_text()?.strip()?)?
    Ok(())

def main() -> Result[(), SpawnError[()] | std.http.server.ServerError | std.net.NetError | VisitError]:
    mut routes = std.http.server.router[Route]()?
    routes.add("GET", "/hello/{name}", Route.Hello)?
    listener = std.net.listen(&std.net.loopback_v4(0u16), 16)?
    port = listener.local_address()?.port
    stop = CancellationToken()?
    with spawn(visit, port, stop.share()).drop_job()? as task:
        drop(std.http.server.serve(listener, &routes, &Greeter(), &std.http.server.config(), &stop)?)
        task.join()?
    Ok(())
200 hello, Ada
404 Not Found

Serving

ItemMeaning
serve[R, E, H: Handler[TcpStream, R, E]](listener: TcpListener, router: &Router[R], handler: &H, config: &Config, stop: &CancellationToken)Result[Report, ServerError]: serve until stop is cancelled, then shut down
std.http.tls.serve[R, E, H: Handler[TlsTransport, R, E]](listener, tls: ServerConfig, router, handler, config, stop)serve over TLS; see HTTPS
serve_with[T: Transport, O: Opener[T], R, E, H: Handler[T, R, E]](listener, opener: &O, router, handler, config, stop)serve with a transport that opener makes from each accepted TcpStream
serve_connection[T: Transport, R, E, H: Handler[T, R, E]](stream: T, router, handler, config, stop, cancel: &CancellationToken)Report: serve every request on one connection, on the calling thread
serve_connection_from[T, R, E, H](stream: T, peer: Option[SocketAddress], router, handler, config, stop, cancel)serve_connection that records the connection’s peer address in each exchange
protocol Handler[T, R, E]def handle(self, route: &R, exchange: &mut Exchange[T]) -> Result[(), E]
protocol Transportread_before, write_before and shutdown with the signatures of TcpStream’s; write_before takes its source as &[u8]
reason(status: i64)str: the reason phrase for a common status, or ""

E is the handler’s own error type: HttpError, or a sum such as AllocError | HttpError. Calls infer it from handle, as they infer R from the router; a generic function that passes a handler on declares it too, as in def run[E, H: Handler[TcpStream, Route, E]](handler: &H). The server needs nothing from the error: it answers and counts every Err alike (see When a handler fails) and drops it on the worker. A return type other than Result[(), E] does not satisfy Handler: App does not satisfy std.http.server.Handler: signature of `handle` does not match.

serve takes ownership of the listener and runs on the calling thread. It starts config.workers scoped worker threads, then accepts connections on its own thread and passes each to the queue once it has input (see Idle connections). Every worker shares one borrow of the router, the handler and the config, so handle takes self and route by shared borrow and cannot change them; shared state needs a synchronized type such as a channel or token. A handler class and its handle method must be pub, because serve calls them from another module, and so must the route type its signature names. The route type and the handler must be usable on worker threads, under the ordinary worker eligibility check.

serve fails only before serving: InvalidConfig for a config outside the bounds below, Allocation or Thread if a channel or worker cannot be made, or Net if its selector or wake-up socket cannot be set up. A worker that cannot start stops the workers already running. After that, every problem is answered on the wire where possible, counted in the Report, and logged to config.events; nothing aborts the server.

serve_connection serves one already-open stream. Any class with the three Transport methods works, and an in-memory stream makes handlers testable without sockets; its shutdown can do nothing. The server calls shutdown only with Shutdown.Write, before a lingering close. Its exchanges have no peer; serve_connection_from takes one, for a transport that knows its peer or a test that plays one. stop and cancel have the meanings described under Shutdown.

Routing

ItemMeaning
router[R]()Result[Router[R], ServerError]: an empty table whose routes carry values of type R
router.add(method, pattern, route: R)Result[(), ServerError]: append a route
router.find(method, path)Result[Match, ServerError]: the route for a request
router.route(index)&R: the value of the route at index, in the order routes were added
Match.Found(index, params: list[str])index is the matched route’s; params holds the captures in pattern order, still percent-encoded
Match.NotFoundNo pattern matches the path
Match.MethodNotAllowed(allow: str)Patterns match but not for this method; allow lists the methods, comma-separated

A pattern starts with /. Each segment is literal text, {name}, which captures one nonempty segment, or, last only, {name...}, which captures the rest of the path, possibly empty, slashes included. Names are letters, digits and _. method is a method name or * for any method. A bad method or pattern fails with InvalidPattern, and a route with the same method and the same shape as an earlier one with DuplicateRoute.

Routes are tried in the order they were added; the first match wins, so add specific routes before general ones. A GET route also answers HEAD, with the body left out. Matching compares the path exactly: no case folding, no trailing-slash redirect and no .. normalization. The query string is not part of the path.

Before routing, the server checks the target and Host with std.http.request_target and answers 400 when they break RFC 9112 section 3.2: an HTTP/1.1 request needs exactly one Host, no request may have two, and a Host value must be a host with an optional port. An absolute-form target (GET http://example.com/a HTTP/1.1) is routed on its path, and its authority replaces Host. The check lives in std.http so any server built on it applies the same rules; the parser itself stays permissive so clients and proxies can read any well-formed message.

The server answers 404, or 405 with an Allow field, without calling the handler. A capture that does not percent-decode to UTF-8 gets 400.

Route values

Each route carries a value of the application’s route type R, normally an enum with one variant per route. The server passes the matched route’s value to handle as route, borrowed from the router, and the handler dispatches with a match. Matching is exhaustive, so a variant the handler does not answer is a compile error naming it (non-exhaustive match; missing Route.Hello(_)), and a handler whose route type is not the router’s is rejected at the serve call (conflicting types for R).

pub enum Route:
    Home
    Hello(Greeting)

pub class App:
    pub def handle(self, route: &Route, exchange: &mut Exchange[TcpStream]) -> Result[(), AllocError | HttpError]:
        match route:
            case Route.Home:
                exchange.send_text(200, "home\n")?
            case Route.Hello(greeting):
                greeting.handle(exchange)?
        Ok(())

mut routes = std.http.server.router[Route]()?
routes.add("GET", "/", Route.Home)?
routes.add("GET", "/index.html", Route.Home)?
routes.add("GET", "/hello/{name}", Route.Hello(Greeting("hello, ")))?

A variant’s payload can hold the route’s own handler object and its state, as Hello holds a Greeting; it is used through a shared borrow, like the handler. Several patterns may carry equal values, so two paths can share one variant; DuplicateRoute depends only on method and pattern shape. router needs its type argument written out: router[Route]().

The server does not generate the dispatch. A route enum cannot implement Handler itself, because enums have no methods and protocols constrain only classes, and a router of handler objects of different classes would need existential protocol values, which Plenty does not have. One match in one handler class keeps dispatch static and checked.

Exchanges

The handler receives an Exchange[T] for one request.

Field or methodMeaning
request: RequestHeadMethod, target, version and fields as read
authority: strhost[:port] the request names: an absolute-form target’s authority, else the Host value; empty only for HTTP/1.0 without Host
path, queryThe target’s path and the text after its first ?, both still percent-encoded; an absolute-form target’s scheme and authority are removed
params: list[str]Captures, percent-decoded
peer: Option[SocketAddress]The address the connection came from: a reverse proxy’s, when there is one (see std.http.proxy)
read_body(&mut buffer, start, stop)Result[i64, HttpError]: part of the request body; 0 from a non-empty range is the end, and an empty range returns 0 without reading
read_all(&mut out)Result[i64, HttpError]: append the rest of the body, within limits.max_body
send(status, content_type, &body)Whole response with Content-Type and Content-Length
send_text(status, text)send with text/plain; charset=utf-8
respond(head, &body)Whole response with the caller’s head; Content-Length is added
start(head)Begin a streamed response
write(source: &[u8])Send and flush part of a streamed body
end()Finish a streamed response
stopping()Shutdown has begun; a long response should end soon
cancelled()The drain deadline has passed; reads and writes now fail
status(), sent()Status sent (0 before), body bytes written (0 for HEAD)
stop_token(), cancel_token()Shares of the tokens behind stopping() and cancelled(), for work that outlives a call
switch_protocols(head)Result[(T, Bytes), HttpError]: answer 101 and take the connection; see Switching protocols
detach(head)Result[T, HttpError]: answer with a close-delimited body and take the connection; see Detaching a connection

The response methods return Result[(), HttpError]. A handler answers once. respond refuses a head that declares Content-Length or Transfer-Encoding (ConflictingFraming), and a body on 204 or 304 (LengthMismatch). start streams exactly Content-Length bytes when the head declares it; otherwise it adds chunked framing for HTTP/1.1 or closes the connection after the body for HTTP/1.0. Writing past a declared length, or ending short of it, is LengthMismatch; a HEAD response sends no body, so it may end without writing. A streamed response the handler leaves unended is ended after it returns. Streamed body progress remains exact: a write that would make sent() exceed i64.MAX returns BodyTooLarge before output or response-state mutation, leaving the earlier progress intact. HEAD does not accumulate body bytes.

The first body read sends 100 Continue when the request asked for it. A response sent before the body is read to its end closes the connection after the response, since the next request cannot be found.

exchange.send_text(200, &exchange.query) is rejected: send_text borrows the whole exchange exclusively while exchange.query is borrowed. Bind the field first: query = exchange.query copies the str, sharing its storage.

Switching protocols

switch_protocols(head) sends head, which must have status 101, as given, and hands the connection to the caller with any input that arrived after the request head. The head carries the Upgrade and Connection fields the new protocol needs. The request body must be empty or read to its end (Unfinished otherwise), and the exchange must be unanswered (Broken otherwise). The exchange then counts as answered; the server neither reuses nor closes the connection, and the handler owns it until it returns or drops it. The connection still occupies the worker for as long as the handler runs. std.http.websocket’s accept is built on it.

Detaching a connection

detach(head) sends head with Connection: close added and hands the connection to the caller, which writes a body that ends when it closes the connection: a long-lived stream that the server’s response_timeout would otherwise cut off. The status must be 200 to 999 (InvalidStatusLine otherwise) and allow a body (LengthMismatch for 204 and 304), and the head must not declare Content-Length or Transfer-Encoding (ConflictingFraming). A HEAD request cannot detach (Broken), since its response has no body; answer it with start and end. The rules for the request body and the handler’s ownership are those of switch_protocols; input that arrived after the request head is dropped. A refused head leaves the exchange unanswered. To watch for shutdown, take shares of stop_token() and cancel_token() and pass them to the stream’s waits. std.http.sse’s accept is built on it.

When a handler fails

Handler outcomeResponseCounted as
Ok after answeringAs sentrequests
Ok without answering500handler_errors
Err before answering, after a body read failed413 (too large), 408 (timed out) or 400protocol_errors
Err before answering, otherwise500, connection closedhandler_errors
Err after a streamed response startedConnection closed mid-bodyhandler_errors

Configuration

config() returns a Config with every field public:

FieldMeaningconfig()
workersWorker threads, 1 to 10244
queueAccepted connections waiting for a worker, at least 116
limitsstd.http Limits for every requeststd.http.limits()
idle_timeoutWait for the first byte of the next request5 s
head_timeoutRead a request head, from its first byte10 s
body_timeoutRead a request body, from the end of its head30 s
write_timeoutAny one write10 s
response_timeoutWhole response, from the end of the request head60 s
drain_timeoutShutdown grace for active connections5 s
max_requestsRequests on one connection before it closes, at least 11000
idle_connectionsConnections waiting on the accepting thread for a request, the rest of its head or the rest of a handshake, 0 to 1,000,000256
handshake_timeoutHTTPS only: finish the TLS handshake, from the accept10 s
eventsOption[ChannelSink] for std.log eventsNothing

Limits must be positive (max_body may be 0).

Idle connections

Unless idle_connections is 0, a worker does not wait for a client to send a request. A connection waiting for its first byte, for the next request after a keep-alive response, or for the rest of a request head is parked on the accepting thread’s std.ready selector instead of holding a worker. When the first input arrives it joins the queue, and a worker reads what has arrived. If that is not yet a whole head, the worker hands the connection back with those bytes; the accepting thread then reads the rest as it arrives and queues the connection once the head is complete. A worker serves the request and, if the connection stays open with no next head complete, returns it to be parked again. Ten keep-alive clients can therefore share one worker, and clients that send heads slowly hold no worker at all.

A head is complete for this purpose once its final empty line has arrived, or once it is bound to fail: a line over max_line, a head over max_head, a line ending in a bare LF, or a field line past max_headers. A parked connection with a partial head holds those bytes, at most max_head (64 KiB by default) and usually a 4 KiB buffer. Parked connections close at idle_timeout, or answer 408 at head_timeout once a head has started.

idle_connections caps the parked connections, including those with partial heads, so it bounds how many slow clients the server waits for: a new connection is refused when it is reached. A connection partway through a head or a handshake that a worker hands back is parked even past the cap, since it was admitted while there was room; at most queue + 2 * workers + 64 such connections can be outside the parking slots at once. With 0, nothing is parked: each connection goes to the queue at once and a worker holds it until idle_timeout, reading each head as it arrives, as a thread-per-connection server would.

HTTPS

import std.http.tls
import std.tls

tls = std.tls.server_config(&chain_pem, &key_pem)?
report = std.http.tls.serve(listener, tls, &table, &App(), &config, &stop)?

std.http.tls.serve takes a std.tls ServerConfig and calls a handler of type Handler[TlsTransport, R, E]; TlsTransport.stream is the TlsStream. Each handshake must finish within handshake_timeout of the accept. Workers advance it as the client’s messages arrive, and between them the connection waits parked, so a client that stalls mid-handshake holds no worker. The deadline covers time spent parked and queued, so a client that connects and stays silent is closed at that deadline, not at idle_timeout. After the handshake, connections are served and parked like plain ones: a TLS stream is only parked when it holds no decrypted input, and a partial head is decrypted and gathered on the accepting thread. The plain serve does not import std.tls.

serve_with accepts any Opener[T], the protocol behind std.http.tls:

MethodMeaning
open(self, tcp: TcpStream)Result[T, OpenError]: the transport, made on a worker without waiting for the peer
advance(self, stream: &mut T, deadline: &Deadline, cancel: &CancellationToken)Result[(), OpenError]: advance the opening handshake until it completes or deadline passes (TimedOut); with a deadline already passed, do only what is possible without waiting
handshakes(self)bool: use handshake_timeout for new connections and close, not 503, those refused
watch(self, stream: &T, selector: &mut Selector, key: u64)bool: register a stream for what it waits for; called only after a read or advance that did not wait has timed out, so no input is buffered
unwatch(self, stream: &T, selector: &mut Selector)Remove it

A worker calls advance with an expired deadline and parks the connection while it returns TimedOut; with idle_connections set to 0, or once shutdown has begun, it waits up to handshake_timeout instead. OpenError is TimedOut, Cancelled, Failed (counted in protocol_errors) or Gone (the client left or refused the handshake; not counted).

Deadlines and faults

Every read and write waits under a deadline, so a slow or silent client cannot hold a worker longer than the phase allows. With parking on, a worker waits for neither a request head nor a handshake (see Idle connections).

SituationServer action
No request starts within idle_timeoutClose quietly
HTTPS: no handshake within handshake_timeout of the acceptClose; counted in timeouts
HTTPS: the client closes or sends a fatal alert during the handshake, for example because it rejects the certificateClose; not counted
HTTPS: handshake fails otherwise, for example a plaintext requestClose; counted in protocol_errors
idle_connections connections already waitingA new connection gets 503 with Retry-After: 1 (HTTPS: is closed); counted in rejected. A served connection going idle is closed; one partway through a head or a handshake is parked anyway
Head not complete within head_timeout408, close; counted in timeouts
Malformed head, or a line, head or header count over a limit400, or 431 for size limits, 505 for an unsupported version, 501 for an unsupported transfer coding; close
Ambiguous framing (Content-Length with Transfer-Encoding)400, close
Declared Content-Length over max_body413 before reading, close
HTTP/1.1 without Host, several Host fields, or an invalid Host value400, close
Target neither origin-form (/path) nor http/https absolute-form400, close
Part of a head arrives, then nothingParked without a worker; 408 from the accepting thread at head_timeout, close; counted in timeouts
Body read past body_timeout, or a write past its deadlineClose; counted in timeouts
Queue full503 with Retry-After: 1 (HTTPS: close without a response), close; counted in rejected
accept fails, for example out of file handlesCount in accept_errors, pause 10 ms, keep accepting

Responses written by the server itself carry Connection: close. Allocation failure while serving closes that connection.

Lingering close

Closing a socket while request bytes are still unread makes the kernel reset the connection, and a client that has not yet read the answer can lose it. So after a response that may leave request input unread, the server ends its output instead (Shutdown.Write; over HTTPS, close_notify first) and discards input until the client ends its own, 16 MiB have been discarded, or 2 s (at most write_timeout) have passed, and then closes. The client reads the whole answer and a clean end. The budget is large because a reverse proxy keeps forwarding a refused body up to its own limit, and answers 502 instead of the refusal if the connection resets under it. This applies to the 400, 408, 413 and 431 refusals above, to a handler failure answered with a body-read status, and to a handler response sent before the request body was read to its end.

With parking on, the discarding happens on the accepting thread: the connection is parked in a slot (and so counts toward idle_connections) and closed when its input ends, its budget is spent, or its deadline passes. With idle_connections 0, under serve_connection, or once shutdown has begun, the worker discards before taking its next connection. A parked 408 from the accepting thread lingers the same way. Shutdown closes lingering connections that are parked at once.

Bytes of a pipelined request that arrived with the previous one start that request: its head_timeout runs from the end of the previous response, and an end or stall inside it is answered (400 or 408) rather than closed quietly. Blank lines alone do not start a request.

A head is parsed once it is complete (see Idle connections), so a malformed line followed by nothing gets 408 at head_timeout, not 400 at once. 408 and 503 answers written on the accepting thread get a write budget of write_timeout, at most 100 ms.

Accepted connections have Nagle’s algorithm turned off (TCP_NODELAY). Responses are buffered and flushed whole, so this sends no extra small segments, and a head followed by a body larger than the 4 KiB write buffer is not held back for a delayed acknowledgement. Over HTTPS the server also acknowledges the client’s TLS Finished at once, so a client that leaves Nagle’s algorithm on sends its first request without waiting for a delayed acknowledgement (see std.tls).

Shutdown

Cancelling stop begins shutdown:

  1. serve stops accepting and closes the listener.
  2. Connections already queued are still served. A connection waiting idle for its next request, or parked mid-handshake, closes at once; one whose request has started arriving is handed to a worker, which waits for the rest of its head, and served. Every response carries Connection: close.
  3. Handlers see stopping() and should finish long responses.
  4. At drain_timeout, serve cancels the second token. Every further read or write fails with Cancelled, cancelled() is true, and the connection closes; counted in cancelled.
  5. serve joins every worker and returns the merged Report. Every accepted connection is closed by then. A client may still be reacting to that close: serve returning says nothing about the client’s progress, so join a client task before relying on its output.

Cancellation is cooperative. A handler that loops without reading, writing or checking cancelled() keeps its worker, and serve waits for it.

To drain when a supervisor sends SIGTERM (or on Ctrl-C), run std.signal.cancel_on on a thread beside serve: it cancels stop when the signal arrives. A second signal ends the process at once.

Reports and events

Report counts, as u64: connections, rejected, requests, handler_errors, protocol_errors, timeouts, cancelled, accept_errors and dropped_events. These observation-only counters saturate at u64.MAX, both when incremented and when worker reports are merged; exhausting a counter does not interrupt serving or change an operation’s error. Body-byte progress and framing limits do not saturate.

With events set, each answered request emits an Info event request with fields method, target, status, bytes (body bytes sent) and micros (from head to end of response). Faults emit Warn events: bad request, bad request body, handler failed, handler sent no response, timed out, cancelled, queue full, too many connections, handshake timed out, handshake failed and accept failed, most with a status field. An event the sink cannot take is dropped and counted in dropped_events; logging never blocks serving.

Limits

  • The accepting thread waits on one selector of at most 64 events per wait, and a parked connection costs a list slot, room for one free-slot index, and a selector registration. Parking reuses a vacated slot in constant time; each wait still scans every slot for expired deadlines. A worker serves one connection at a time, from a complete head to the end of the response.
  • idle_connections bounds the slow clients the server waits for: with that many partial heads parked, new connections get 503 until their head_timeout passes. Raise it, and the descriptor limit, for exposed servers; each parked partial head holds at most max_head bytes.
  • HTTPS connections end without a TLS close_notify, except those closed lingering. Responses are length-delimited or chunked, so clients do not depend on it, except for a streamed response to HTTP/1.0, whose end is the close and so cannot be told from truncation.
  • HTTPS has the std.tls limits. The server speaks only HTTP/1.x, so its ServerConfig should offer no ALPN protocol but http/1.1.
  • No Date header, compression, request trailers in the exchange, or HTTP/2. A connection taken by switch_protocols or detach holds its worker until the handler returns, so long-lived WebSockets and event streams count against workers. Forms, JSON, uploads, cookies and forwarded clients are in std.http.body, std.http.cookie and std.http.proxy; static files in std.http.files.
  • A 503 rejection reads only bytes that have already arrived. If more of the request arrives after the close, the client may see a reset instead.
  • A lingering close discards at most 16 MiB in at most 2 s. A client still sending past that, such as a large upload refused with 413 without Expect: 100-continue, can still see a reset; past idle_connections, a refusal closes at once.
  • Captures in params are decoded; path and query are not.
  • Host and absolute-form authorities are checked against the URI grammar, not against names the server answers for. A handler that serves only some hosts compares exchange.authority itself.
  • OPTIONS * and CONNECT targets get 400.
  • A trap in a handler ends the process; see handler errors and traps.

Acknowledgements

HTTP server qualification

This page records what was checked before std.http.server was labelled production-ready, what that showed, and how to deploy it. The server is production-ready: every gate passes; see Gates. HTTP/2 and HTTP/3 need separate adoption decisions and are not part of these gates.

The suites are in tests/test_std_http_server_qualification.rs, with the request corpus in tests/fixtures/http_server/, and in tests/test_std_http_server_proxy.rs, which runs the server behind nginx and Caddy in Docker (see Reverse proxies). The benchmark is examples/http_server_bench.py. Everything runs on loopback; only the proxy tests need a network, once, to pull their images.

The current side-by-side TLS comparison records worker and actor throughput, handshake latency, idle RSS and trap recovery after actor TLS heap isolation and completion-credit changes. The original baselines on this page remain historical measurements of their recorded revision.

Gates

GateStatusEvidence
Parser conformance and a retained corpusPassAbout 80 corpus requests, each replayed in 64-byte reads and one byte at a time; 600 mutated copies. Includes the RFC 9112 section 3.2 Host rules and absolute-form targets
FuzzingPassCoverage-guided libFuzzer runs of the whole connection loop, seeded from the corpus, with every response status checked: 16.9 million inputs in 30 minutes, no finding. A short run is part of the test suite (test_fuzz.rs), beside deterministic corpus mutations here and in test_std_http.rs. See Fuzzing
InteroperabilityPasscurl over plain HTTP and TLS (reuse, sized, chunked and 100-continue uploads, HTTP/1.0, HEAD, chunked and 64 MiB responses, 404/405); std.http.client (pooling, streamed chunked upload, large downloads)
Slow clientsPassIdle and silent clients, partial request heads, partial TLS handshakes, slow readers and slow body senders are cut off at their deadlines without holding the server; heads and handshakes wait parked, not on a worker. See Slow request heads
Refusals without a resetPassRefusals of requests not read to the end (431, 413, 400, a failed body read), each followed by more input, end with the whole answer and a clean end, over plain TCP (parked and on a worker) and TLS (close_notify). See Refusals and resets
Shutdown under sustained loadPassSix clients sending keep-alive requests while shutdown begins, each reusing its connection, with responses still arriving after the stop signal: every response received is whole, the server’s count matches, and serve returns within the drain deadline
Supervisor stopPassWith std.signal.cancel_on, SIGTERM during a streamed response begins shutdown: the response ends whole, serve returns its Report and the program exits 0. A second signal ends the program at once (test_std_signal.rs)
Allocation failurePassEvery allocation on the accepting thread fails in turn; request handling fails in turn through serve_connection (test_std_http_server.rs). No trap, no leak, and the request is served once allocation succeeds
Resource leaksPassNo descriptor survives a load round or a fault round (bad heads, truncated heads, silent and rejected clients), and the allocation-checks runtime finds no live heap after either
Measured baselinesPublishedBaselines, over plain HTTP and HTTPS
Unused backendsPassA JSON service using the server and client links none of rustls/ring, SQLite or zlib-rs
Relocated offline toolchainPassThe compiler binary, copied alone into a fresh directory and run with an empty environment except PATH, builds and runs a JSON service. Installed alone as PREFIX/bin/plenty with a fresh home, it fetches a package’s Git dependency once; with the repository deleted and git replaced by a failing stub, plenty build --offline --locked and plenty run --offline build a CLI/data tool (std.cli, std.csv, std.json, typed std.error reports and exit statuses) and a JSON service over the same dependency. Both run and link no unused backend
TLS and proxy deploymentPassDirect TLS is tested with curl and std.tls clients. Behind nginx 1.30.6 and Caddy 2.11.7: keep-alive on both sides of the proxy, buffered and streamed bodies, 100-continue, forwarded Host and X-Forwarded-*, TLS terminated at the proxy, slow heads, a slow upstream, refusals and a drain on SIGTERM. Opt-in locally (PLENTY_DOCKER_TESTS=1) and run by CI; see Reverse proxies

Supported platforms

x86-64 Linux with the GNU C library, the only target of std.net and std.ready (epoll). Results here were taken on Linux 7.2 (Fedora 44). No other target is supported.

Application error composition

The installed offline package test also checks B14/B71 through its shared CSV summary dependency. The CLI reports invalid integer data with its record number and ParseError, malformed row width with the complete CsvError, and a missing file with its nested stream/I/O cause. Each report retains the operation and path, prints no partial JSON, and exits 1; argument errors still exit 2 and help exits 0.

The JSON service returns those same causes as Context[SummaryError] in its handler error sum and reports them before propagation. One client repeats all three failures and a successful summary three times. The test checks identical CLI/service diagnostics, generic 500 responses without private error details, successful requests after failures, nine handler errors among twelve requests, and no protocol errors. After the client, workers, CSV readers and listener have dropped, the process has the same number of open descriptors as before the run. This qualifies application composition over the CSV/HTTP APIs. The client now uses typed head parsing, writes and fixed/chunked/close-delimited framing. Separate typed-stream and client checks cover exact partial progress, original transport causes, interruption, bounded heads, trailers, compatibility readers and connection recovery; HTTPS checks cover verified and tunnelled requests.

Handler errors and traps

A handler reports a problem by returning Err; the server answers it, counts it and goes on serving (see When a handler fails). A fatal runtime error is different. An index out of range, integer overflow, division by zero, unwrap of an Err, or worker stack exhaustion terminates the whole process at once, without unwinding, with a message such as error: main.plenty:3:11: index out of bounds: index 7, length 3 and exit status 1. Every connection is cut off and no Report is returned.

So in a handler:

  • Check indexes and arithmetic on anything that comes from a request, use optional lookups and ?, and return Err rather than calling unwrap.
  • Run the server under a supervisor that restarts it (systemd Restart=on-failure, a container restart policy).
  • Allocation failure is not a trap: it closes the affected connection.

Baselines

examples/http_server_bench.py builds examples/http_server_bench.plenty with --release, runs it with 4 workers on CPUs 0, 2, 4 and 6 and ApacheBench 2.3 on CPUs 8, 10, 12 and 14 (one hardware thread per physical core, so at most ten threads are busy), and repeats each workload against a fresh server. With --tls it builds examples/https_server_bench.plenty instead, the same handler served by std.http.tls with the ECDSA P-256 certificate in tests/fixtures/tls.

Measured on 2026-10-18 at revision 32c48c0, built by rustc 1.99.0 (target x86_64-unknown-linux-gnu (CPU baseline: x86-64/SSE2)), on an AMD Ryzen 7 7840HS w/ Radeon 780M Graphics with Linux 7.2.9-201.fc44.x86_64, three runs per workload. Other builds were running (load average about 4.5 at the start of each run), so treat the ranges as noise bounds.

WorkloadRequests/s, median (range)p50 / p99 latency, msPeak RSS, KiBFailed
keep-alive, 1 client5560 (5536–5662)0.17 / 0.3662120
keep-alive, 16 clients33314 (32772–33388)0.10 / 2.9662120
keep-alive, 64 clients32853 (29653–32959)0.10 / 9.0362360
new connection per request, 16 clients16106 (16085–16137)0.93 / 1.6462120
keep-alive 16 KiB response, 16 clients24430 (24323–25589)0.14 / 4.3061520
keep-alive 1 KiB POST echo, 16 clients25850 (25783–25897)0.16 / 1.8461600

HTTPS

The same workloads over TLS 1.3 (TLS_AES_256_GCM_SHA384, X25519, ECDSA P-256 server key; rustls 0.23.46 with ring), measured in the same session:

WorkloadRequests/s, median (range)p50 / p99 latency, msPeak RSS, KiBFailed
keep-alive, 1 client3891 (3872–3913)0.20 / 0.5181960
keep-alive, 16 clients25892 (25038–26697)0.13 / 2.0593520
keep-alive, 64 clients24474 (24459–24516)3.33 / 6.19111360
new connection per request, 16 clients (curl)4196 (3939–4235)1.43 / 3.7687920
keep-alive 16 KiB response, 16 clients17858 (17382–19415)0.19 / 3.6992680
keep-alive 1 KiB POST echo, 16 clients19606 (18753–19782)0.91 / 1.4992920

Each new HTTPS connection is a full handshake: the server keeps no session cache and sends no tickets, so clients cannot resume. ApacheBench cannot measure that workload: it counts connections closed without close_notify as failed. The script runs curl 8.18.0 (OpenSSL 3.5.9) instead: four processes on the client CPUs, each with --parallel-max 4, TCP_NODELAY (curl’s default) and no certificate verification. The same curl method against the plain server gave 21397 (21225–22032) requests/s, p50 / p99 0.67 / 1.05 ms, so the handshake, not the client, sets the HTTPS rate.

With 1000 parked keep-alive connections the server’s resident memory grew from 4364 KiB to 6680 KiB over plain HTTP, about 2.3 KiB per idle connection, and from 5928 KiB to 48552 KiB over HTTPS, about 42.6 KiB per idle connection: each TLS session keeps its rustls buffers and keys. Size idle_connections for HTTPS with that in mind.

Latency is per request as the client sees it. Peak RSS is the server’s VmHWM. 16k is a 16 KiB response; the POST echoes a 1 KiB body. Each keep-alive connection closes after max_requests (1000) requests, so ApacheBench reconnects. An earlier plain run, on 2026-10-17 at load average about 8, gave 9239 to 16744 requests/s for the same workloads; the difference is machine load, not a server change.

Limits of these numbers: one machine, loopback only, a client that is itself a bottleneck at high rates, small fixed responses, one certificate type and no TLS session resumption. They are a regression baseline, not a capacity plan.

Before this qualification, a response whose head and body went out as two writes (any body over the 4 KiB write buffer) waited about 40 ms for a delayed acknowledgement: the 16 KiB workload ran at 388 requests/s. The server now turns off Nagle’s algorithm on accepted connections.

Until issue #148 was fixed, a client that leaves Nagle’s algorithm on (ApacheBench, Python’s socket) waited about 40 ms on each new HTTPS connection: a TLS 1.3 server sends nothing after the client’s Finished, so the client held its first request until the kernel’s delayed acknowledgement. ab -n 20 -c 1 without -k took 42–44 ms per request. The server now acknowledges the Finished at once; on 2026-10-18 the same client took 0.76 ms per request over 200 requests (unpinned, not the baseline method above).

Slow request heads

Until issue #132 was fixed, a worker read each request head as it arrived, so a client that sent part of a head held a worker until head_timeout. Now a partial head, like a partial TLS handshake, waits on the accepting thread (see Idle connections). Measured on 2026-10-17 with the scenario of slow_heads_do_not_hold_workers below, also run with the defaults: clients that each send GET / HTTP/1.1\r\nHost: x and stall, then one complete request.

ConfigurationBeforeAfter
2 workers, head_timeout 1 s, 6 slow clientsAnswered after 2954 msAnswered within 2 ms
config() defaults (4 workers, 10 s), 4 slow clientsAnswered after 9959 msAnswered within 2 ms
config() defaults, 64 slow clients503 at once; 44 slow clients got 503, the last 408 came after 50 sAnswered within 2 ms; all 64 got 408 at 10 s

The slow clients still count against idle_connections (default 256): with that many parked, new plaintext connections get 503 until slots become available (new HTTPS connections close). Already-admitted partial heads and handshakes returned by workers can park beyond that admission cap; see the bounded overflow and parking-disabled behavior. With 1000 parked partial heads the server’s resident memory grew from 4548 KiB to 6396 KiB, about 1.9 KiB per connection. Size idle_connections and the descriptor limit for the slow clients an exposed server must outlast. Keep-alive and new-connection throughput showed no change beyond run-to-run noise on a loaded machine (load average about 8). slow_heads_do_not_hold_workers in the qualification suite and partial_heads_wait_without_a_worker and https_partial_handshakes_and_heads_wait_without_a_worker in test_std_http_server.rs check this.

Refusals and resets

Until issue #138 was fixed, a refusal of a request the server had not read to the end closed the socket with request bytes unread, so Linux reset the connection. Measured on 2026-10-17 against examples/http_server_bench.plenty, ten connections per case, each sending the request and then reading to the end:

RequestBeforeAfter
Head with a 20000-byte field (431)10 resets10 clean ends
Content-Length over max_body and 200 KB of body (413)10 resets10 clean ends
Malformed field line and 100 KB more (400)10 resets10 clean ends
Bad chunk size and 100 KB more (400 from a failed body read)10 resets10 clean ends

The results were the same with IDLE=0, where the worker lingers. Before the fix a client reading to the end saw ECONNRESET, sometimes after reading the answer. Now the server ends its output, discards up to 16 MiB (64 KiB until the proxy tests below) for up to 2 s, then closes (see Lingering close). refusals_end_cleanly_instead_of_resetting and tls_refusals_end_with_close_notify in the qualification suite check this.

Fuzzing

fuzz/http_server/main.plenty is a libFuzzer target: plenty --fuzz counts every block the compiled program runs, including the standard library’s, and libFuzzer keeps the inputs that reach new ones. Each input is one client connection served by serve_connection over an in-memory stream. Its first byte selects the read size (1 to 64 bytes), small limits (a 64-byte line, a 256-byte head, 4 fields, a 16-byte body) and a limit of two requests per connection; the rest is what the client sends. The routes are those of the corpus. Response bodies never repeat the request, so the target finds every response head in what the server wrote and traps on a status outside 100, 200, 400, 404, 405, 413, 431, 500, 501 and 505. Traps of the server itself, stack exhaustion and hangs over 10 s are findings too.

fuzz/http_server/run.py builds the target with --release, seeds it with every corpus case in four variants (64-byte reads, 7-byte reads, 1-byte reads, 7-byte reads with small limits) and runs it with the HTTP dictionary fuzz/http_server/http.dict. Findings go to WORK/findings; the default WORK is target/fuzz-http-server. Findings persist across runs: the script lists only the files the current run added, and exits nonzero only for those or for a libFuzzer failure.

cargo build --release --locked
python3 fuzz/http_server/run.py --time 1800 --jobs 2
target/fuzz-http-server/http_server_fuzzer FINDING    # replay one finding

On 2026-10-18, on the machine of the baselines, two jobs ran for 30 minutes each (rustc 1.99.0, clang 22.1.8’s libFuzzer), starting from the corpus of an earlier 10-minute run: 16.9 million inputs, about 4,700 per second per job, with no finding. 3,777 of the program’s 13,878 blocks were reached, nearly all within the first seconds; most of the rest belong to functions the target never calls, such as serve and TLS, which a fuzz build keeps.

To act on a finding, replay it, reduce it with -minimize_crash=1 -runs=100000, and handle it as a parser defect under Security fixes: its request joins the corpus, and so the seeds.

Not covered: TLS records (rustls and ring are fuzzed upstream), the std.ready parking of real sockets, and timeouts, since the in-memory stream never waits.

Reverse proxies

tests/test_std_http_server_proxy.rs puts nginx and Caddy, each in its stock image, in front of the server, and drives them with curl and raw sockets. The server runs on the host (it links the host’s C library, so it does not run on Alpine); the proxy container uses --network host and reaches it on loopback. The configurations are tests/fixtures/proxy/nginx.conf and tests/fixtures/proxy/Caddyfile; the server is tests/fixtures/proxy/server.plenty.

ImageDigest
nginx:1.30.6-alpine3.24sha256:970364228b1e4f932abc22d321932816782cca8a20b85a77552a7cfe9acfb3b5
caddy:2.11.7-alpinesha256:d8542f48d34a9cf4e4c11a478865229840e87e4c96ea3f439101f31a5d35f75f

Each test runs against both proxies. Clients connect from 127.0.0.2, so the handler can tell them from the proxy at 127.0.0.1.

The harness reserves distinct TCP ports and the TLS port’s UDP socket through configuration setup, then releases them immediately before starting Docker. Caddy also opens that UDP port for HTTP/3; this is not HTTP/3 qualification. Process-local logical leases retain both port numbers across that handoff and until the proxy’s Drop cleanup completes, including startup failure and unwind. Concurrent cooperating fixtures reject an already claimed port in either role and select another pair; only registry operations hold the mutex, not Docker startup or scenarios. Deterministic non-Docker tests force reuse during the socket-free gap, check cross-role exclusion and RAII release, and stress eight concurrent reservers (issue #719). These are helper tests, not renewed real-proxy qualification. A separate exact batch on the reviewed port-ownership source passes all eight worker and two actor nginx/Caddy cases with two test threads (local PR #918), including actual container removal and unchanged foreign container state. This is bounded proxy qualification, not a whole-workspace or performance result. Unrelated processes remain outside this ownership registry. Host networking still has a socket handoff window, so readiness checks a unique proxy-local response over both HTTP and verified TLS, without adding upstream requests. An unexpected HTTP status or identity fails immediately rather than being retried. Shutdown setup failures retain response headers and body, listener/upstream ports, container logs and upstream stderr. These checks harden endpoint identification; they do not establish the cause of the intermittent pre-shutdown 400 reported in issue #647.

TestChecks
*_forwarding_keep_alive_and_tlsexchange.authority and Host keep the client’s Host; X-Forwarded-Proto is http or https; client_of, trusting only 127.0.0.1, finds the client and ignores a spoofed X-Forwarded-For; curl reuses its connection to the proxy; twelve requests reach the server over at most three connections; TLS terminated at the proxy, with a 700 KB upload
*_bodies_and_continueSized, chunked and Expect: 100-continue uploads of 700 KB and a 1 MiB chunked download, through a buffering and a streaming location; a client that waits for 100 Continue before sending; a streamed response reaches the client before the handler ends it
*_slow_clients_timeouts_and_refusalsA partial request head is closed by the proxy at its 2 s header timeout and never reaches the server; a handler slower than the proxy’s 2 s upstream timeout gets 504 and the next request is served; five rounds of 413 refusals of 2 MiB bodies and early answers to 1000 KiB bodies the handler does not read, sized and chunked, buffered and streamed, all reach the client
*_shutdown_drainsSIGTERM while a response streams through the proxy: it ends whole, the server exits 0 with its report, and the proxy then answers 502

Differences between the proxies, none of them a server fault:

  • nginx appends to a client’s X-Forwarded-For (203.0.113.9, 127.0.0.2); Caddy, trusting no proxies by default, replaces it (127.0.0.2). With only the proxy trusted, client_of gives 127.0.0.2 for both.
  • nginx sends 100 Continue itself and buffers the body. Caddy streams it and also relays the server’s own 100 Continue, so the client sees two interim responses, which RFC 9110 section 15.2 requires clients to accept.
  • Caddy offers HTTP/2 to TLS clients and speaks HTTP/1.1 to the server.
  • After relaying an answer sent before a large body was read, Caddy closes the client’s connection with the rest of the body unread, so a client still sending may see a reset after the answer. Curl can stop on its upload error before reading the final response. The Caddy refusal and early-answer checks therefore receive concurrently with the upload and require the final status, a single correct Content-Length, and the exact complete body before accepting a reset. A lone 100 Continue, a truncated answer, or a timeout still fails. The nginx checks require curl to exit successfully. The other curl interoperability checks remain strict.
  • nginx reuses connections to the server only with an upstream block with keepalive, proxy_http_version 1.1 and an empty Connection header, as in the test configuration.

The tests found one server defect (issue #142). A refusal, or an answer sent before the body was read, closed lingering after discarding at most 64 KiB. A proxy forwards the whole body regardless, so it was often still sending when the server closed; the connection reset under it, and it answered 502 instead of passing on the server’s answer. The budget is now 16 MiB, still within 2 s. Twenty requests per case, measured on 2026-10-17:

RequestProxy64 KiB budget16 MiB budget
2 MiB sized body, 413nginx, buffering14 of 20 5020
2 MiB chunked body, 413nginx, buffering15 of 20 5020
2 MiB sized body, 413nginx, streaming2 of 20 5020
1000 KiB sized body, early 200nginx, buffering1 of 20 5020
1000 KiB chunked body, early 200nginx, buffering7 of 20 5020
2 MiB bodies, 413, four variantsCaddy1 to 3 of 20 502 each0
1000 KiB chunked body, early 200Caddy1 of 20 5020

With 200 KiB bodies neither proxy saw a 502 under either budget.

Deployment

For a public service, put a reverse proxy in front of the server (see deploying behind a reverse proxy). The proxy also absorbs slow clients beyond what idle_connections bounds:

  • Buffer whole request heads, and preferably bodies, before forwarding (nginx does both by default; keep proxy_request_buffering on). tests/fixtures/proxy/ has tested nginx and Caddy configurations.
  • Connect to the server over loopback or a private network with proxy_http_version 1.1 and keep-alive, so connections are reused.
  • Set the proxy’s client timeouts and connection limits per client; the server’s limits are per connection.

The server itself:

  • Listen on loopback or a private address. Serve HTTPS directly only when no proxy is possible, and then keep head_timeout and handshake_timeout short and idle_connections above the number of slow clients expected.
  • Raise the descriptor limit (LimitNOFILE in systemd) above idle_connections + queue + workers plus the program’s own files; when it runs out, accept fails, is counted in accept_errors and retried every 10 ms.
  • Set events to a std.log channel sink and export the Report counts; rejected, timeouts and accept_errors show overload.
  • Drain on SIGTERM with std.signal, and set the supervisor’s stop timeout (TimeoutStopSec in systemd) above drain_timeout.

Security fixes and backend updates

The HTTP modules are checked Plenty in the compiler’s embedded standard library; a fix reaches a program when it is rebuilt with a fixed compiler. For a defect in parsing, framing or limits:

  1. Record it in the issue register with severity high, and add the triggering request to tests/fixtures/http_server/corpus.txt with its correct answer, so it stays fixed.
  2. Fix it in stdlib/std/http.plenty or stdlib/std/http/, and run this suite with test_std_http.rs, test_std_http_server.rs, test_std_http_client.rs and test_std_http_helpers.rs.
  3. Rebuild and redeploy every program that serves HTTP.

TLS uses vendored rustls and ring, pinned in Cargo.lock. Follow rustls and ring advisories and update them as described under TLS supply, then run test_std_tls.rs, test_std_http_https.rs and the TLS tests here. No other native backend is linked into a program that only serves HTTP.

Reproducing

cargo test --locked --all-features --test integration test_std_http_server_qualification::
PLENTY_DOCKER_TESTS=1 cargo test --locked --test integration test_std_http_server_proxy::
cargo build --release --locked
python3 examples/http_server_bench.py --compiler target/release/plenty --output target/http-bench --runs 3
python3 examples/http_server_bench.py --compiler target/release/plenty --output target/https-bench --runs 3 --tls
python3 fuzz/http_server/run.py --time 1800 --jobs 2

The curl tests are skipped when curl is not installed. The proxy tests need Docker and curl and are skipped, with a message, unless PLENTY_DOCKER_TESTS=1 (see reverse-proxy tests); the proxy_tests job in .github/workflows/ci.yml runs them on every push and pull request. The benchmark needs ApacheBench (ab) and taskset, and with --tls also curl. Fuzzing needs clang’s libFuzzer; without it the libFuzzer tests in test_fuzz.rs print a note and pass, unless PLENTY_FUZZ_TESTS=1. Busy machines distort the benchmark: check the load average first, and keep the total number of busy threads at or below the number of physical cores.

std.http.sse

import std.http.sse sends Server-Sent Events, the text/event-stream format browsers read with EventSource, from a std.http.server handler. The handler turns its exchange into an event stream that outlives the server’s response deadline, then sends events until it closes the stream, the client goes away, or the server shuts down. The module is checked Plenty in stdlib/std/http/sse.plenty and adds no runtime primitives. The tutorial lesson Stream events to browsers shows a complete program. There is no client-side reader. Line-scan and write-progress offsets stay within their respective buffers, including the final line break; no wrapping or saturating lifetime byte counter is used.

import std.time
import std.bytes
import std.http.client
import std.http.server
import std.http.sse
import std.net
from std.http.server import Exchange
from std.http.sse import SseError
from std.net import TcpStream
from std.sync import CancellationToken

pub enum Route:
    Events

pub class Feed:
    pub def handle(self, route: &Route, exchange: &mut Exchange[TcpStream]) -> Result[(), SseError | RangeError]:
        wait_cancel = exchange.cancel_token()
        deadline = std.time.deadline_after(&std.time.monotonic_now(), &std.time.seconds(1u32))?
        mut events = std.http.sse.accept(exchange)?
        events.send("first line\nsecond line", &deadline, &wait_cancel, event="note", id=Some("1"))?
        events.send("done", &deadline, &wait_cancel)?
        events.close()
        Ok(())

type WatchError = AllocError | IoError | std.bytes.BytesError | std.http.client.ClientError | RangeError

def watch(port: u16, stop: CancellationToken) -> Result[(), WatchError]:
    with stop.cancel_on_exit():
        never = CancellationToken()?
        deadline = std.time.deadline_after(&std.time.monotonic_now(), &std.time.seconds(5u32))?
        client = std.http.client.client(std.http.client.config()?)?
        reply = client.get(&f"http://127.0.0.1:{port}/events"?, &deadline, &never)?
        print(reply.body.to_text()?.strip()?)?
    Ok(())

def main() -> Result[(), SpawnError[()] | std.http.server.ServerError | std.net.NetError | WatchError]:
    mut routes = std.http.server.router[Route]()?
    routes.add("GET", "/events", Route.Events)?
    listener = std.net.listen(&std.net.loopback_v4(0u16), 16)?
    port = listener.local_address()?.port
    stop = CancellationToken()?
    with spawn(watch, port, stop.share()).drop_job()? as task:
        drop(std.http.server.serve(listener, &routes, &Feed(), &std.http.server.config(), &stop)?)
        task.join()?
    Ok(())
event: note
id: 1
data: first line
data: second line

data: done

Opening a stream

ItemMeaning
accept(exchange: &mut Exchange[T])Result[EventStream[T], SseError]: answer with head() and return the stream
accept_with(exchange, head: ResponseHead)accept with the caller’s 200 head, such as one with CORS fields
head()Result[ResponseHead, HttpError]: the head accept sends

The response is:

HTTP/1.1 200 OK
Content-Type: text/event-stream
Cache-Control: no-cache
Connection: close

accept_with adds Content-Type and Cache-Control unless the head names them, and refuses a status other than 200 (Http(InvalidStatusLine)) and a head that declares Content-Length or Transfer-Encoding (Http(ConflictingFraming)). A refused head leaves the exchange unanswered, so the handler can still answer it. The request body must be empty or read to its end (Http(Unfinished)).

The body is delimited by the connection’s close rather than chunked: the connection leaves the server through Exchange.detach, so the server’s response_timeout no longer applies, and dropping or closing the stream closes the connection. A HEAD request is answered with the head alone, with the server’s usual framing, and accept returns Closed.

stream.last_event_id is the request’s Last-Event-ID field, which a reconnecting EventSource sends with the last id it received, or Nothing. A handler that keeps a history can resume after it.

Sending

MethodMeaning
send(&data, &deadline, &cancel, event="", id=Nothing)Send one event
send_bytes(data: &[u8], &deadline, &cancel, event="", id=Nothing)send of UTF-8 data, without copying it first
comment(&text, &deadline, &cancel)Send comment lines, which clients ignore
retry(span: &Duration, &deadline, &cancel)Ask the client to wait this long before reconnecting
wait(&deadline, &cancel)Wait until deadline, watching for a disconnect and for shutdown
close()End the response; closing a closed stream does nothing
is_open()Whether events can still be sent

These return Result[(), SseError], except close and is_open. Each sending call writes and flushes whole lines before it returns, within its deadline.

An event is written as the format’s readers parse it:

PartLines
eventevent: name, unless empty; the client’s default type is message
idid: value when given; Some("") resets the client’s last event id
datadata: line for each line; CRLF, CR and LF all end a line
endan empty line, which dispatches the event

send("a\nb", &deadline, &cancel, event="update", id=Some("7")) writes event: update\nid: 7\ndata: a\ndata: b\n\n. Data ending with a line break gets a last, empty data: line, so the client sees the trailing line feed. An event name or id holding CR or LF, or an id holding NUL, is InvalidField; data that is not UTF-8 is InvalidText. These checks happen before anything is written, and the stream stays open.

comment("ping", &deadline, &cancel) writes : ping\n, one line per line of text. A comment is the usual keep-alive: proxies close quiet connections, and a write is how a server notices a client that has gone away.

retry takes a nonnegative Duration; positive fractions round up to whole milliseconds on the wire. The deadline/cancellation pair precedes the optional event and id arguments because required parameters precede defaults.

All waits accept caller cancellation. They preserve the retained server shutdown tokens, checking the caller between native waits of at most 10 ms; this interval is not a scheduling guarantee. Cancellation before a send leaves the stream open and writes nothing. Cancellation during a partial write closes it. wait returns Net(Cancelled) without closing on caller cancellation, so the handler can choose whether to send a final event with another token.

Disconnects and shutdown

A client that closes or resets the connection makes the next write fail with Disconnected. wait reads from the connection while it waits, so it returns Disconnected as soon as the client closes, without a write. Anything the client sends is discarded. A write that times out, or fails part-way through an event, also closes the stream, since the client is left inside an event. Every connection error closes the stream; later calls fail with Closed. Stopping, the field checks above and allocation failures leave it open.

The stream keeps the server’s shutdown tokens. wait ends with Stopping when shutdown begins, and the stream stays open so the handler can send a last event; a later wait returns Stopping at once. When the drain deadline passes, writes and wait end with Net(Cancelled).

A handler sends events between waits:

while True:
    events.send(&next_update()?, &soon()?, &wait_cancel)?
    match events.wait(&in_one_second()?, &wait_cancel):
        case Ok(_):
            pass
        case Err(SseError.Stopping):
            events.send("bye", &soon()?, &wait_cancel, event="shutdown")?
            return Ok(())
        case Err(error):
            return Err(error)

Errors

SseError has these cases:

CaseMeaning
Net(NetError)The connection failed, timed out, or was cancelled
Allocation(AllocError)A buffer could not grow
Http(HttpError)The head was refused or could not be written
InvalidFieldAn event name or id with a line break, or an id with NUL
InvalidTextData that is not UTF-8
DisconnectedThe client closed the connection
ClosedThe stream was closed, or failed earlier
StoppingServer shutdown began while waiting

Limits

  • An open stream holds its server worker until the handler returns, as a WebSocket does, so each one counts against workers. Route event streams to handlers that return when the stream ends.
  • One caller at a time: the methods borrow the stream exclusively, so one thread cannot wait while another sends. A handler that needs events from other threads receives them through a channel between waits.
  • No event history: resuming after last_event_id is the handler’s job.
  • No client-side reader.
  • A stream holds a 256-byte output buffer, grown to the largest event sent, and a 512-byte buffer for discarded input.

std.http.typed

HTTP head parsing, writing and body framing over std.io.typed.Reader[E] and Writer[E] preserve the transport cause in FramingError[E], with no stored-error retrieval API. Use request_body or response_body with a parsed HTTP head to validate framing, select the decoder, enforce body limits and recover a completed persistent connection. The lower-level adapters also remain available for callers that have already validated Content-Length and method/status semantics.

import std.bytes
import std.http
import std.io
import std.io.typed
from std.http.typed import FramingError, read_request, request_body
from std.io.typed import CopyFailure, Partial, ReadFailure, copy_all

type HttpCause = FramingError[ReadFailure[std.io.StreamError]]
type ReadBodyError = Partial[HttpCause] | HttpCause | CopyFailure[HttpCause, std.io.StreamError]

def main() -> Result[(), AllocError | IoError | std.bytes.BytesError | std.io.StreamError | std.http.HttpError | ReadBodyError]:
    wire = std.bytes.from_text("POST /notes HTTP/1.1\r\nHost: example.com\r\nTransfer-Encoding: chunked\r\n\r\n5\r\nhello\r\n0\r\n\r\n")?
    limits = std.http.limits()
    mut conn = std.io.typed.buffered_reader(std.io.typed.reader(std.io.cursor(wire)), 256)?
    head = read_request(&mut conn, &limits)?.ok_or(std.http.HttpError.Truncated)?
    mut body = request_body(conn, &head, &limits)?
    mut output = std.io.typed.writer(std.io.cursor(std.bytes.empty()?))
    mut scratch = std.bytes.with_capacity(16)?
    scratch.resize(16, 0u8)?
    copy_all(&mut body, &mut output, &mut scratch)?
    print(head.method, head.target, std.io.typed.unwrap_writer(output).data.to_text()?)?
    print(body.total(), body.is_done())?
    # The connection comes back only after a complete, reusable message.
    drop(body.finish()?)
    Ok(())
POST /notes hello
5 True

Reading and writing heads

read_request(reader, limits) and read_response(reader, limits) return Result[Option[RequestHead or ResponseHead], Partial[FramingError[E]]]. They share the legacy parser’s grammar, UTF-8 validation and Limits policy. Nothing means EOF before a message; request parsing permits leading blank lines within the head budget. Response parsing returns each interim response separately. Framing fields and request routing remain separate validation steps.

On failure, completed counts bytes consumed for this head, including CRLF and leading request blank lines. It excludes prefetched bytes still in a buffered reader. Transport errors remain Transport(E); malformed syntax and limits are Protocol(HttpError), and an incomplete head is Truncated. Explicit interruption retries without losing the partially read line. Other errors terminate this parse; the count is diagnostic information, not a token for resuming a partially parsed head.

These functions request one byte at a time and never consume body bytes. For sockets, use std.io.typed.buffered_reader so those requests are served from a buffer; its ReadFailure[E] envelope preserves the original transport payload. Pass the same buffered reader to the body constructor to retain read-ahead.

write_request(writer, head), write_response(writer, head), write_chunk(writer, source: &[u8]) and write_last_chunk(writer, trailers) return Result[(), Partial[FramingError[E]]]. Head/trailer serialization shares std.http’s validation and completes before any write. Chunk payload is written directly from the source without copying it. completed counts wire bytes delivered by that operation, including chunk metadata. Transport errors and invalid progress remain distinct; a zero write is Protocol(HttpError.Stream(StreamError.WriteZero)). Partial output must not be blindly replayed. Empty chunks write nothing; write_last_chunk ends a chunked body.

Wire-progress extents are checked before delivery. In particular, a chunk’s size line, payload and final CRLF must all fit i64; representation failure returns Partial(0, FramingError.OutOfRange) before writing any of that chunk. It is not invalid transport progress. Existing transport failures retain their payload and exact previously delivered count.

Head, body and trailer cursor arithmetic uses explicit opt-outs justified by validated ranges, byte budgets and accepted progress. A caller’s arithmetic mode does not change these public result contracts.

From a parsed head

request_body(inner, head, limits) and response_body(inner, head, method, request_keep_alive, limits) return Result[Body[E, R], FramingError[E]], where R satisfies Reader[E]. The transport must be positioned immediately after the head. Pass the whole buffered reader, including any prefetched body or next-message bytes; taking its inner stream would lose those bytes.

These constructors call request_framing or response_framing, respectively. Duplicate/conflicting framing, unsupported transfer coding and invalid lengths remain Protocol(HttpError) failures, including on HEAD and bodyless responses. HEAD, 1xx, 204, 304 and successful CONNECT semantics select an empty body. Requests without declared framing are empty; responses without declared framing are close-delimited. The constructors accept parsed heads: they do not parse bytes, revalidate arbitrary hand-built head syntax, validate request routing/Host, or consume interim responses on the caller’s behalf.

Every mode checks Limits at construction. A declared payload length above max_body fails before any read. Chunked and close-delimited payloads are bounded while reading. At exactly the close-delimited limit, the next nonempty read probes one byte to distinguish EOF from an oversized body; that probe is not delivered or counted, and an oversized body becomes broken. Empty reads never probe. HEAD/bodyless responses do not spend a body budget on their declared representation length.

Body.read follows the same typed reader contract as the low-level adapters. total() counts payload bytes returned successfully; is_done() is true only after successful completion. trailers() borrows the chunked trailer list (empty for other modes); it is complete only once is_done() is true. Interruption preserves state; failures retain concrete transport causes and previously delivered progress through the typed stream helpers. Invalid ranges trap before changing body state or accessing the transport. Other source/framing failures break it.

body.finish() returns Result[R, FramingError[E]]. It returns the complete transport, with buffered bytes preserved, only after a persistent fixed-length, chunked or empty body completes. An unfinished body reports Protocol(HttpError.Unfinished) and remains readable. A failed or previously finished body reports Broken. Successful extraction happens exactly once; subsequent reads report Broken and is_done() becomes false.

NotReusable means completion did not permit another HTTP message: the connection is close-delimited, has a closing Connection policy, is HTTP/1.0 without keep-alive, is a 101 upgrade or is a successful CONNECT tunnel. For responses, pass the originating request’s keep-alive policy as request_keep_alive; both sides must allow persistence. This is message-boundary recovery, not pooling: after an interim response, the next message is still the final response; after reading a request, the server must also honor its response’s closing policy. Upgraded/tunnel streams are deliberately not exposed by this API. Rejected finish calls retain ownership; dropping the body closes or drops its transport, including on incomplete/error paths. Constructors take ownership too, so a construction failure drops the supplied transport.

Low-level framing

ConstructorFraming
fixed_body(inner, length: NonNegative[i64])Result[FixedBody[E, R], FramingError[E]]
close_body(inner)CloseBody[E, R]; reads until transport EOF
chunked_body(inner, limits)Result[ChunkedBody[E, R], FramingError[E]]; bounded chunked decoding with std.http.Limits

All readers expose read(buffer, start, stop) returning Result[i64, StepError[FramingError[E]]]. Invalid caller buffer ranges trap before checking body state or accessing the transport, as required by the Reader contract. A fixed body caps each read to its remaining length and reports Truncated on premature EOF. It never consumes bytes after the body. A close-delimited body treats EOF as successful completion and remembers it, avoiding further transport reads. A close-delimited connection must never be reused.

FramingError distinguishes Transport(E), Truncated, InvalidProgress, OutOfRange, Broken, NotReusable, and Protocol(HttpError). The latter retains chunk syntax, limit, trailer and framing-allocation errors using the existing HttpError variants; transport errors always use Transport(E). Interruption is forwarded as a retry signal without poisoning the body. Transport errors and invalid progress make it broken; later reads report Broken. Truncation also breaks a fixed or chunked body. Chunk syntax, limit and trailer failures also break the chunked body. Invalid buffer ranges do not consume input or break a healthy body. Valid empty reads return zero without reading or marking EOF.

Chunked bodies

chunked_body requires a validated head selecting chunked transfer coding. It checks Limits at construction (max_line and max_head at least two; the types of max_headers and max_body already exclude negative values). Chunk-size lines and trailer fields include CRLF in their line limit. max_head bounds the entire trailer section, including its final CRLF; max_headers bounds trailer fields, and max_body bounds decoded payload. A chunk exceeding the remaining body budget is rejected before its payload is read. Hex sizes are overflow-checked.

The decoder requires CRLF and uses the same extension and field policy as std.http: extensions after ; are ignored after rejecting control bytes other than tab; field names must be tokens, surrounding spaces/tabs are removed from values, and values must be valid UTF-8 without other control bytes or DEL. Obsolete folded fields are rejected. Trailers are metadata, not a second head: they do not change framing or replace validated head fields. Parsed fields are available in body.trailers; treat them as complete only after body.is_done(). body.total() counts decoded payload bytes returned to the caller.

Parsing resumes after Interrupted even within a size line, a CRLF, or a trailer. A successful payload read returns immediately, before parsing subsequent framing. Consequently read_exact and copy_all retain all previously returned payload progress when later metadata or transport fails. Metadata bytes do not count as delivered bytes. As with other readers, an unsuccessful read may have modified scratch storage; only an Ok(count) identifies payload to use.

The terminal blank trailer line completes the body. Metadata reads request one byte at a time, and payload reads are capped to the current chunk; the reader does not request any bytes from the next message. Repeated completed reads return zero without consulting the transport. Empty reads do not advance parsing or imply completion. Construction reserves one line buffer of max_line bytes; trailer strings and their list allocate fallibly while reading. This simple decoder prioritizes bounded, resumable behavior; it has no internal read-ahead buffer or throughput qualification yet.

The adapters own their transports. Connection pooling, TLS setup and cancellation polling belong to std.http.client, which uses these typed head and body paths, including CONNECT and streamed uploads. Cancellation delivered as a typed transport cause survives intact. The client’s legacy Response.read/take_failure bridge remains for compatibility; its internal transport and framing paths do not retrieve stored errors. The low-level adapters do not expose their transports; use the parsed-head constructors and Body.finish for guarded connection recovery. Bridging a legacy head reader preserves only the error type that the legacy reader itself returns.

std.http.websocket

import std.http.websocket speaks the WebSocket protocol of RFC 6455: the opening handshake, as a std.http.server exchange that switches protocols or as a client, then framed text and binary messages, ping and pong, and the closing handshake. std.http.wss adds wss clients through std.tls. Both modules are checked Plenty in stdlib/std/http/ and add no runtime primitives; the handshake’s SHA-1 comes from std.crypto. The tutorial lesson Talk over WebSockets shows a complete program.

import std.time
import std.bytes
import std.http.server
import std.http.websocket
import std.net
from std.http.server import Exchange
from std.http.websocket import Message, WebSocketError
from std.net import TcpStream
from std.sync import CancellationToken
from std.time import Deadline

type ChatError = WebSocketError | std.bytes.BytesError | RangeError | AllocError

def soon() -> Result[Deadline, RangeError]:
    Ok(std.time.deadline_after(&std.time.monotonic_now(), &std.time.seconds(5u32))?)

pub enum Route:
    Chat

pub class Shout:
    pub def handle(self, route: &Route, exchange: &mut Exchange[TcpStream]) -> Result[(), ChatError]:
        wait_cancel = exchange.cancel_token()
        mut socket = std.http.websocket.accept(exchange, &std.http.websocket.config())?
        mut message = std.bytes.empty()?
        while True:
            message.clear()
            match socket.receive(&mut message, &soon()?, &wait_cancel)?:
                case Message.Close(_):
                    return Ok(())
                case _:
                    socket.send_text(&message.to_text()?.upper()?, &soon()?, &wait_cancel)?
        Ok(())

def chat(port: u16, stop: CancellationToken) -> Result[(), ChatError | IoError]:
    with stop.cancel_on_exit():
        never = CancellationToken()?
        url = f"ws://127.0.0.1:{port}/chat"?
        mut socket = std.http.websocket.connect(&url, &std.http.websocket.config(), &soon()?, &never)?
        socket.send_text("hello", &soon()?, &never)?
        mut reply = std.bytes.empty()?
        print(socket.receive(&mut reply, &soon()?, &never)?, reply.to_text()?)?
        socket.close(1000, "bye", &soon()?, &never)?
    Ok(())

def main() -> Result[(), SpawnError[()] | std.http.server.ServerError | std.net.NetError | ChatError | IoError]:
    mut routes = std.http.server.router[Route]()?
    routes.add("GET", "/chat", Route.Chat)?
    listener = std.net.listen(&std.net.loopback_v4(0u16), 16)?
    port = listener.local_address()?.port
    stop = CancellationToken()?
    with spawn(chat, port, stop.share()).drop_job()? as task:
        drop(std.http.server.serve(listener, &routes, &Shout(), &std.http.server.config(), &stop)?)
        task.join()?
    Ok(())
std.http.websocket.Message.Text HELLO

Connections

WebSocket[T] is one open connection over a stream T that reads and writes like a server Transport: TcpStream, or TlsTransport for TLS.

ItemMeaning
socket.receive(&mut out, &deadline, &wait_cancel)Result[Message, WebSocketError]: append the next whole message to out
socket.send(kind, data: &[u8], &deadline, &wait_cancel)Result[(), WebSocketError]: send data as one Kind.Text or Kind.Binary message
socket.send_text(&text, &deadline, &wait_cancel)send of text, copied into a buffer first
socket.ping(data: &[u8], &deadline, &wait_cancel)Send a ping with at most 125 bytes of payload
socket.close(code, &reason, &deadline, &wait_cancel)Run the closing handshake, then shut the stream down
socket.is_open()Whether messages can still be sent
socket.subprotocolOption[str]: the subprotocol both sides agreed on

Message is Text, Binary, or Close(code). Text has been checked as UTF-8. A message sent in several frames is joined before receive returns it. Pings are answered with a pong and pongs are skipped while receiving. When the peer closes, receive answers with the same code and returns Close with the peer’s code, or 1005 when it gave none, and its reason appended to out; the connection is then closed.

send splits a message longer than max_frame into several frames. A Text message must be UTF-8 (InvalidText). Client frames are masked with a fresh key from std.secrets for each frame, as RFC 6455 requires; server frames are not.

close accepts the codes a peer may send: 1000 to 1003, 1007 to 1014, and 3000 to 4999, with a reason of at most 123 bytes (OutOfRange otherwise). It sends the close, skips any messages still arriving until the peer’s close, and shuts the stream down. Closing a connection whose closing handshake already finished returns Ok.

On an error, out keeps only what it held before the call. Every error except Stopping and OutOfRange closes the connection; later calls fail with Closed. A protocol fault first sends a close: 1002 for Protocol, 1009 for TooLarge and 1007 for InvalidText.

Each call has its own deadline. A connection holds one 4 KiB input buffer, grown only to hold input that arrived with the handshake, and one 4 KiB output buffer; a sent message is copied through the output buffer in pieces, so send allocates nothing for any message size.

Configuration

config() returns the defaults.

FieldDefaultMeaning
max_frame65536Largest frame payload received, and the size send splits messages into
max_message1048576Largest message receive joins
subprotocolNothingThe subprotocol a client offers, or a server accepts when the client offers it

Both maxima are Positive[i64], so a zero or negative maximum is a type error. They bound payload bytes, not the prefix already held by the caller’s buffer. An unrepresentable combined buffer extent is Allocation(AllocError.CapacityOverflow), not TooLarge; as with other receive errors, the original prefix is retained and the connection is closed. Frame splitting and mask positions stay within their source ranges even when the configured maximum is i64.MAX.

Servers

accept(exchange: &mut Exchange[T], config: &Config) returns Result[WebSocket[T], WebSocketError]. It checks that the request is a handshake: GET over HTTP/1.1 with no body, Upgrade: websocket, Connection: upgrade, Sec-WebSocket-Version: 13 and one Sec-WebSocket-Key of 16 bytes in base64. Field names and the Upgrade and Connection tokens are compared without case. A request that fails these checks is answered and accept returns Refused(status):

RequestAnswer
Missing websocket in Upgrade, or another version426 Upgrade Required with Upgrade: websocket and Sec-WebSocket-Version: 13
Any other fault400 Bad Request

Otherwise it answers 101 Switching Protocols with the accept value, and the configured subprotocol when the client listed it, and takes the connection with switch_protocols. The server no longer counts or closes the connection; dropping the WebSocket closes it.

The WebSocket keeps the server’s shutdown tokens. A receive waiting for the next message ends with Stopping when shutdown begins, and the connection stays open, so the handler can close with 1001. A wait in the middle of a message, or in send or close, ends with Net(Cancelled) when the drain deadline passes.

The handler keeps its worker for as long as the WebSocket is open, so each open connection counts against workers. Route a WebSocket to a handler that returns when the connection closes.

Clients

ItemMeaning
connect(url: &str, &config, &deadline, &cancel)Result[WebSocket[TcpStream], WebSocketError]: dial and handshake a ws URL
target(url: &str)Result[Target, WebSocketError]: parse a ws or wss URL
dial(&target, &deadline, &cancel)Result[TcpStream, WebSocketError]: a TCP connection to the target
handshake(stream, &target, &config, &deadline, &cancel)Result[WebSocket[T], WebSocketError]: the client handshake over a connected stream
Target.secure, host, portwss or not; the host, lowercase without IPv6 brackets; the port, 80 or 443 by default

target rejects URLs with another scheme (UnsupportedScheme), with credentials or a fragment, without a host, or with port 0 (InvalidUrl). The request target is the URL’s path, / when empty, and its query.

The handshake sends a random 16-byte key and checks the server’s answer: a status other than 101 is Rejected(status), and a 101 without the Upgrade and Connection tokens, with the wrong accept value, with extensions, or with a subprotocol the client did not offer is InvalidHandshake. The deadline bounds the handshake; later calls take their own. cancel ends every later wait with Net(Cancelled).

connect returns UnsupportedScheme for wss URLs: use std.http.wss.

WSS

import std.http.wss runs the client handshake over TLS.

ItemMeaning
connect(url: &str, &tls: ClientConfig, &config, &deadline, &cancel)Result[WebSocket[TlsTransport], WebSocketError] for a wss URL

The server’s certificate is verified against tls’s roots with the URL’s host as the name, as for HTTPS. A TLS failure is Connect(ClientError.Tls(...)). A ws URL is UnsupportedScheme. The module is separate so that a program using only ws does not link TLS. On the server side, accept takes exchanges from std.http.tls.serve unchanged.

Every message operation accepts a caller cancellation token after its deadline. Pre-cancellation returns Net(Cancelled) before changing the connection. Cancellation while receiving or writing closes the connection, since a partial frame cannot safely be resumed as a different operation. The retained server shutdown tokens keep their existing behavior. Native waits keep their shutdown wakeups and check the caller token between slices of at most 10 ms; scheduling and lock contention can delay observation beyond that interval.

Errors

WebSocketError has these cases:

CaseMeaning
Net(NetError)The stream failed, timed out, was cancelled, or ended early
Allocation(AllocError)A buffer could not grow
Http(HttpError)The handshake’s HTTP head could not be read or written
Connect(ClientError)The client could not connect, including TLS failures
Entropy(StreamError)No random bytes for a key or mask
InvalidConfig, InvalidUrl, UnsupportedSchemeSee above
OutOfRangeA ping payload, close code or reason out of bounds
Refused(status)The server refused a request that was not a handshake
Rejected(status)The server answered the client with another status
InvalidHandshakeA 101 answer that does not complete the handshake
ProtocolThe peer broke the framing rules
TooLargeA frame or message over the configured limits
InvalidTextText that is not UTF-8
ClosedThe connection has closed
StoppingServer shutdown began while waiting for a message

Limits

  • No extensions: permessage-deflate is neither offered nor accepted.
  • No proxy for clients; connect dials the URL’s host directly.
  • One caller at a time: receive and the sending methods all borrow the connection exclusively, so one thread cannot wait for a message while another sends.
  • A server WebSocket holds a worker until its handler returns.
  • receive places a control frame’s payload in out while handling it, so out briefly grows by up to 125 bytes.
  • A close answer echoes the peer’s code without its reason.
  • send_text copies its text; send sends bytes the caller already holds.

std.image

import std.image holds images as owned buffers of 8-bit pixels, decodes and encodes PNG files, and crops, converts and resizes images. It has one error type, ImageError.

The module is ordinary Plenty over std.bytes and std.compress, which supplies zlib and CRC-32; it adds no runtime code. Decoding checks the image’s dimensions against caller-supplied limits before it allocates. It is available on the supported x86-64 Linux GNU target.

import std.image
from std.image import ColorType, Filter, ImageError

def main() -> Result[(), IoError | ImageError]:
    # A 4 x 2 RGB image: a red top row over a black bottom row.
    mut image = std.image.new(4, 2, ColorType.Rgb)?
    for x in range(4):
        image.set(x, 0, 0, 255u8)?
    png = std.image.encode_png(&image)?
    decoded = std.image.decode(&png, &std.image.limits())?
    print(decoded.width(), decoded.height(), decoded.pixels() == image.pixels())?
    small = decoded.resize(2, 1, Filter.Bilinear)?
    print(small.get(0, 0, 0)?, small.get(0, 0, 1)?)?
    gray = decoded.convert(ColorType.Gray)?
    print(gray.get(0, 0, 0)?, gray.get(0, 1, 0)?)?
    Ok(())
4 2 True
128 0
54 0

Functions

FunctionResultContract
new(width: Positive[i64], height: Positive[i64], color: ColorType)Result[Image, ImageError]A zeroed image: black, and transparent if it has alpha
from_pixels(width: Positive[i64], height: Positive[i64], color: ColorType, pixels: Bytes)Result[Image, ImageError]Takes pixels as the buffer, without copying
detect(data: &[u8])Option[Format]The format named by the first bytes of data
info(data: &[u8])Result[Info, ImageError]The header of a PNG file, without decoding pixels
decode(data: &[u8], limits: &Limits)Result[Image, ImageError]The image in a whole PNG file, within limits
encode_png(image: &Image, level: i64 = 6)Result[Bytes, ImageError]A PNG file of image, compressed at zlib level
limits()LimitsThe default limits; see Decoding limits

Width and height lie in 1 to 2³¹−1, the PNG maximum, so every image can be encoded. new and from_pixels trap on a dimension above 2³¹−1; convert a computed size with Positive.try_from, or check it first with Limits.check. An image whose size in bytes does not fit in i64 is LimitExceeded, including a colour conversion or mixed-dimension bilinear intermediate whose pixel extent is unrepresentable. from_pixels needs a buffer of exactly width * height * channels bytes, otherwise OutOfRange.

detect recognises PNG, JPEG, GIF and WebP signatures and returns Nothing for anything else. Only PNG is decoded: info and decode return Unsupported for any input that detect does not name as PNG, including an empty or partial signature.

Format is Png, Jpeg, Gif or WebP. Filter is Nearest or Bilinear; see Resizing.

Info is a @copy class with public fields:

FieldMeaning
format: FormatAlways Png
width: i64, height: i64The dimensions in pixels
color: ColorTypeThe colour type decode produces
depth: i64The stored bits per sample: 1, 2, 4, 8 or 16
interlaced: boolWhether the file is Adam7-interlaced

Images

ColorType is Gray, GrayAlpha, Rgb or Rgba. Its channels() method returns 1 to 4 and has_alpha() whether the last channel is alpha. Channels are interleaved in the order of the name. An Image stores rows top to bottom and pixels left to right, width * channels bytes per row with no padding, so the byte of channel c at (x, y) is at (y * width + x) * channels + c. Its fields are private, so the buffer length always matches the dimensions.

MethodResultContract
width(), height()i64The dimensions in pixels
color()ColorTypeThe colour type
pixels()&BytesThe buffer, borrowed
into_pixels(self: Image)BytesGives up the image for its buffer, without copying
get(x, y, channel)Result[u8, ImageError]One sample
set(self: &mut Image, x, y, channel, value: u8)Result[(), ImageError]Replaces one sample
copy()Result[Image, ImageError]An independent image with the same pixels
crop(x, y, width, height)Result[Image, ImageError]A copy of the rectangle whose top-left pixel is (x, y)
convert(color: ColorType)Result[Image, ImageError]A copy in another colour type; see Colour conversion
resize(width, height, filter: Filter)Result[Image, ImageError]A resampled copy; see Resizing

A coordinate or channel outside the image, or a rectangle that is empty or extends past an edge, is OutOfRange. The builtin copy(image) is rejected because the buffer is a resource; use the method.

Decoding limits

decode reads the header and checks it against a Limits before it allocates anything, so a small file that declares a large image fails at once with LimitExceeded:

FieldBound
max_width: NonNegative[i64]Width in pixels
max_height: NonNegative[i64]Height in pixels
max_bytes: NonNegative[i64]The decoded buffer, width * height * channels bytes

std.image.limits() returns 16384 × 16384 pixels and 256 MiB, enough for an 8192 × 8192 Rgba image. Construct Limits(max_width, max_height, max_bytes) for other bounds. max_bytes counts the channels of the decoded colour type: a palette image with transparency decodes to Rgba, four bytes per pixel. check(width, height, color) applies the same test to any dimensions, for example before new or resize with sizes taken from input.

The limits bound what decode allocates: besides the result, it holds two scanlines of at most 8 × width + 1 bytes each, a palette of at most 1 KiB and an Inflater of about 62 KiB. The result is reserved at full size once the header passes. A non-interlaced image is decoded into it row by row, so a file that declares a large image but holds little data writes little memory; an interlaced image is zero-filled first, because its passes write scattered pixels.

decode takes the whole file in memory, so bound the input when reading it as well. Read one byte more than the bound to tell a file that fits from one that does not:

import std.bytes
import std.image
import std.io
from std.bytes import Bytes
from std.image import ColorType, ImageError, Limits

type Problem = AllocError | IoError | std.bytes.BytesError | std.io.StreamError | std.io.Partial | ImageError | OverflowError[i64]

# The whole file, or LimitExceeded if it is longer than `limit` bytes.
def load(path: &str, limit: NonNegative[i64]) -> Result[Bytes, Problem]:
    mut file = std.io.open_file(path, "r")?
    mut data = std.bytes.empty()?
    read = std.io.read_up_to(&mut file, &mut data, limit + 1)?
    file.close()?
    if read > limit.get():
        return Err(ImageError.LimitExceeded)
    Ok(data)

def main() -> Result[(), Problem]:
    banner = std.image.new(3000, 100, ColorType.Rgb)?
    png = std.image.encode_png(&banner)?
    mut out = std.io.open_file("banner.png", "w")?
    std.io.write_all(&mut out, &png)?
    out.close()?

    data = load("banner.png", 1048576)?
    header = std.image.info(&data)?
    print(header.width, header.height, header.depth)?
    previews = Limits(1024, 1024, 4194304)
    match std.image.decode(&data, &previews):
        case Ok(image):
            print(image.width())?
        case Err(ImageError.LimitExceeded):
            print("too large for a preview")?
        case Err(error):
            return Err(error)
    Ok(())
3000 100 8
too large for a preview

Decoding

decode accepts every PNG colour type, bit depth and filter, interlaced or not, and produces 8 bits per channel:

PNG colour typeDecodes toWith tRNS
Grayscale, 1 to 16 bitsGrayGrayAlpha
Truecolour, 8 or 16 bitsRgbRgba
Indexed, 1 to 8 bitsRgbRgba
Grayscale with alpha, 8 or 16 bitsGrayAlphaNot allowed
Truecolour with alpha, 8 or 16 bitsRgbaNot allowed

Samples of 1, 2 and 4 bits scale exactly to 0–255. A 16-bit sample v becomes (v + 128) // 257, the nearest 8-bit level. A tRNS colour key is compared with the stored samples at full depth, so a 16-bit key matches only its exact colour. Palette entries without a tRNS alpha are opaque.

The file must follow the PNG structure exactly; anything else is an error:

  • Signature and header. IHDR must come first, with a valid size, colour type and depth pair, compression and filter method 0, and interlace method 0 or 1.
  • Chunks. Every chunk’s CRC is checked: a mismatch is Checksum. A chunk length over 2³¹−1 or a type that is not four ASCII letters is Corrupt. An unknown critical chunk, one whose type starts with an uppercase letter, is Unsupported; unknown ancillary chunks are skipped after their CRC check.
  • Order. PLTE and tRNS come before the image data, once each, PLTE first, and only for colour types that allow them; an indexed image needs PLTE. The IDAT chunks are consecutive, and their contents form one zlib stream. IEND follows, with no data. Bytes after IEND are ignored.
  • Image data. The zlib Adler-32 is checked: a mismatch is Checksum. A stream that ends early, holds more data than the header describes, or contains an invalid filter type or palette index is Corrupt.
  • Truncation. Input that ends inside a chunk or before IEND is Truncated.

Ancillary chunks are ignored, including gAMA, cHRM, sRGB, iCCP, bKGD, pHYs, text and time chunks, and the animation chunks of APNG, so an animated PNG decodes to its default image.

info reads the signature, IHDR and the chunks before the image data with the same checks, and allocates nothing.

Encoding

encode_png writes an 8-bit, non-interlaced PNG of the image’s colour type: the signature, IHDR, the image data in IDAT chunks of about 64 KiB, and IEND, with no ancillary chunks. level is the zlib level, 0 to 9; anything else is OutOfRange. PNG filter differences and reconstruction use explicit modulo-256 byte arithmetic; cursor and filter-score arithmetic stays bounded by the validated dimensions and held buffers. Level 0 stores rows unfiltered and uncompressed. Above 0, each row is filtered all five ways and the one with the smallest sum of absolute differences is kept, the heuristic the PNG specification recommends. Equal images and levels give equal files.

Colour conversion

convert copies the image into another colour type:

  • Colour to gray uses ITU-R BT.709 luma of the stored values, (2126 × red + 7152 × green + 722 × blue + 5000) // 10000.
  • Gray to colour copies the gray level into all three colour channels.
  • Dropping alpha discards it, without blending against a background.
  • Adding alpha makes every pixel opaque.

Converting to the image’s own colour type returns a copy. Values are used as stored, without gamma decoding.

Resizing

resize takes target dimensions under the rules of new. It does not apply Limits; check sizes taken from input with Limits.check first. Resizing to the same dimensions returns a copy.

  • Nearest copies, for each target pixel, the source pixel under its centre.
  • Bilinear weights source pixels with a triangle filter, one axis at a time. When shrinking, the filter widens by the reduction factor so every source pixel contributes, which avoids the aliasing of point sampling. Weights are fixed-point with 22 fractional bits, following Pillow’s Image.resize.

Images with alpha are resampled with premultiplied colour, so transparent pixels do not tint their neighbours; a fully transparent result has colour 0. Bilinear results equal Pillow’s for images without alpha and are within 1 with alpha, where the two round premultiplied values differently.

Errors

pub enum ImageError:
    OutOfRange
    Corrupt
    Checksum
    Truncated
    Unsupported
    LimitExceeded
    Allocation(AllocError)
VariantMeaning
OutOfRangeA dimension, coordinate, channel, rectangle, buffer length, or level out of bounds
CorruptA malformed PNG file; see Decoding
ChecksumA chunk CRC or zlib Adler-32 mismatch
TruncatedThe input ended inside a chunk or before IEND
UnsupportedA format other than PNG, or an unknown critical chunk
LimitExceededAn image outside the Limits, or an unrepresentable pixel or decompressed-pass extent
Allocation(cause)A buffer could not be allocated

Ownership and threads

An Image owns its buffer; dropping it releases the buffer. from_pixels and into_pixels move a buffer in and out without copying, and pixels borrows it. Every other operation that returns an image returns a new one and leaves its source unchanged. The module has no global state, passes the ordinary worker-effect checks, and may be used from any thread.

Costs

OperationAllocates
new, copy, crop, convert, resize with NearestThe result
resize with BilinearThe result; for each resampled axis, 16 bytes plus 8 bytes per tap for each target position, with 4 taps when enlarging and about 2 × the reduction factor + 2 when shrinking; and an intermediate image of the target width and source height when the height changes and either the width changes or the image has alpha
decodeThe result, two scanlines, the palette, and an Inflater; see Decoding limits
encode_pngThe file, which grows by doubling; a zlib Compressor (about 387 KiB); five filtered copies of a row and a zero row, or one row at level 0; and about 64 KiB of compressed data before each IDAT is framed
detect, info, from_pixels, into_pixels, get, set, Limits.checkNothing

Decoding is linear in the file size and pixel count. Encoding is linear in the pixel count, with zlib slower at higher levels. A bilinear resize costs one multiply per tap per target sample on each axis. Pixel loops are scalar Plenty without SIMD. On a 3000 × 2000 RGB image, decoding took about twice as long as Pillow, encoding at level 6 about 2.7 times, and bilinear shrinking to a third about 6 times.

The module is compiled only into programs that import it.

Limits

  • PNG only: JPEG, GIF and WebP are detected but Unsupported.
  • 8 bits per channel. 16-bit samples are reduced on decode, and there is no 16-bit, palette, low-depth or interlaced output.
  • No colour management: gamma, chromaticity, sRGB and ICC chunks are ignored, and conversion and resizing work on stored values, not linear light.
  • No metadata: text, time, physical size and background chunks are neither returned nor written. No APNG animation frames.
  • Whole files only: no decoding from a std.io reader or row by row.
  • Only nearest and bilinear filters; no box, bicubic or Lanczos.
  • No rotation, flipping, compositing or drawing.
  • One target, x86-64 Linux GNU.

Acknowledgements

  • The W3C PNG Third Edition supplies the format, the filter-selection heuristic and the Adam7 interlace by Adam M. Costello; the decoder and encoder are implemented independently.
  • The Paeth filter is from Alan W. Paeth, “Image File Compression Made Easy”, Graphics Gems II (1991).
  • Bilinear weights follow Pillow’s resampling; the module does not wrap or ship its code. Tests compare decoding, encoding and bilinear resizing with Pillow when it is installed.
  • Gray conversion uses the ITU-R BT.709 weights, as the Rust image crate does.

std.io

import std.io provides byte streams with explicit partial progress: the stream contracts, helpers that retry until a range is complete, fixed-capacity buffering, and adapters for files, the process streams, and memory. Data moves through std.bytes.Bytes ranges; no operation creates a list[u8]. Writes read a &[u8] view, so a buffer, part of one, or a b"..." literal is written as it is, without a copy.

The module is embedded Plenty code over native transport and terminal queries. It is available on the supported x86-64 Linux GNU target.

Stream contracts

A stream is any pub class or enum whose pub methods match one of these protocols. Generic calls specialize for each concrete type. Eligible interfaces can also use explicit protocol boxes, whose method calls dispatch indirectly; this does not introduce an implicit stream object type.

ProtocolMethodResult
Readerread(self: &mut, buffer: &mut Bytes, start: i64, stop: i64)Result[i64, StreamError]
Writerwrite(self: &mut, source: &[u8])Result[i64, StreamError]
Writerflush(self: &mut)Result[(), StreamError]
Seekerseek(self: &mut, target: SeekFrom)Result[i64, StreamError]
Closerclose(self: &mut)Result[(), StreamError]

One call makes at most one attempt and returns the number of bytes it moved.

  • read fills a prefix of buffer[start:stop] and never changes the buffer’s length. A count below the range length is a short read, not an error. Zero for a nonempty range means end of input.
  • write consumes a prefix of source. A short count is normal. Zero for a nonempty source means the stream accepted nothing; write_all reports that as WriteZero instead of looping. To write part of a buffer, pass &buffer[a:b].
  • An empty range or source returns Ok(0) and says nothing about end of input.
  • An error means the call moved no bytes. A stream that fails after moving some bytes returns the count and reports the failure on the next call.
  • Interrupted means nothing moved and the same call can be repeated.
  • seek returns the new byte offset from the start.

A read range must satisfy 0 <= start <= stop <= buffer.len(). An invalid range traps with read: start/stop outside buffer before touching the stream, even at EOF or for an empty range. read_exact applies the same rule with read_exact: start/stop outside buffer. A write source is a view, whose bounds clamp, so it has no range error. A stream that returns a negative count or more than it was offered is reported as InvalidProgress by every helper and buffer, which then stop using that count.

An implementation declares write with the same source: &[u8] parameter and passes the view, or part of it, on to operations that read one, such as Bytes.extend or another stream’s write:

import std.bytes
import std.io
from std.bytes import Bytes
from std.io import StreamError

pub class Tally:
    pub data: Bytes

    pub def write(self: &mut Tally, source: &[u8]) -> Result[i64, StreamError]:
        match self.data.extend(source):
            case Ok(_):
                Ok(len(source))
            case Err(cause):
                Err(StreamError.Allocation(cause))

    pub def flush(self: &mut Tally) -> Result[(), StreamError]:
        Ok(())

def main() -> Result[(), AllocError | std.bytes.BytesError | std.io.Partial | IoError]:
    mut tally = Tally(std.bytes.empty()?)
    std.io.write_all(&mut tally, b"ab")?
    tail = std.bytes.from_text("cdef")?
    std.io.write_all(&mut tally, &tail[:2])?
    print(tally.data.to_text()?)?
    Ok(())
abcd

A helper names the one protocol it needs, such as [R: Reader]. Protocols cannot be combined in one bound, so close_buffered requires Closer and relies on the buffer’s own use of write and flush.

Errors

StreamErrorMeaning
InterruptedThe native call was interrupted (EINTR); retry
UnexpectedEofInput ended before read_exact filled its range
WriteZeroA writer accepted no bytes of a nonempty range
OutOfRangeA seek target or other stream range is invalid
InvalidProgressA stream reported an impossible byte count
Io(IoError)Any other native failure, including Closed, NotReadable, NotWritable, and System(code)
Allocation(AllocError)A buffer could not be allocated or grown

std.io.classify(error: IoError) performs this mapping for code that wraps another native operation.

Helpers that take several steps keep the progress made before a failure:

TypeContents
Partialcompleted: i64 bytes moved, then cause: StreamError
CopyError.Read(count, cause)The reader failed after count bytes reached the writer
CopyError.Write(count, cause)The writer failed after accepting count bytes
CloseError.Flush(cause)Flushing failed; the stream was closed
CloseError.Close(cause)Flushing succeeded; closing failed
CloseError.FlushAndClose(flush, close)Both failed

Seek and binary-field extent calculations handle checked arithmetic through these declared errors. read_up_to reports an unrepresentable buffer extent as Partial(completed, Allocation(CapacityOverflow)) before growing the buffer. If another copy chunk would make the delivered total exceed i64.MAX, copy_all returns CopyError.Read(completed, OutOfRange) before delivering that chunk. The reader may already have consumed it; copying is not transactional or resumable. Internal cursor arithmetic uses explicit opt-outs only after validating ranges, progress and the applicable byte budget.

These types do not convert to each other. Match the result, or use ? in a function whose error sum lists each of them.

Helpers

FunctionResultContract
read_exact(reader, buffer, start, stop)Result[(), Partial]Fill the whole range. End of input first is UnexpectedEof
write_all(writer, source: &[u8])Result[(), Partial]Deliver all of source. No progress is WriteZero. Does not flush
read_up_to(reader, out, limit: NonNegative[i64])Result[i64, Partial]Append at most limit bytes to out; fewer only at end of input
copy_all(reader, writer, scratch)Result[i64, CopyError]Move bytes until end of input through the whole of scratch. Does not flush

All four retry Interrupted and continue after short progress. They stop at the first other error and return the bytes moved so far. After a failed read_exact, buffer[start:start + completed] holds the bytes read. After a failed read_up_to, out has grown by exactly completed bytes.

read_exact, write_all, and copy_all do not allocate. copy_all needs a nonempty scratch, otherwise CopyError.Read(0, OutOfRange); its length is the transfer size. read_up_to grows out by at most 8192 bytes ahead of the data it has received and removes unused bytes before returning, so memory follows the input actually read, not limit.

copy is not used as a name because the builtin copy already exists.

Bounded buffering

FunctionResult
buffered_reader(inner: R, capacity: Positive[i64])Result[BufReader[R], StreamError]
buffered_writer(inner: W, capacity: Positive[i64])Result[BufWriter[W], StreamError]

Each takes ownership of inner and allocates one buffer of exactly capacity bytes. The buffer never grows. The only failure is allocating it. A computed capacity converts with Positive.try_from.

BufReader[R]ResultContract
read(buffer, start, stop)Result[i64, StreamError]Serve held bytes, otherwise make one inner read
read_until(delimiter, out, limit: Positive[i64])Result[i64, StreamError]Append through the first delimiter, at most limit bytes
buffered()i64Bytes held and not yet consumed
capacity()i64The fixed buffer size

read makes at most one inner read per call. A request at least as large as the capacity, made while the buffer is empty, goes straight to the inner stream. read_until returns the count appended: zero is end of input, and a count whose last byte is not the delimiter means the limit or end of input came first. It retries Interrupted. After an error, bytes already appended stay in out and bytes not yet appended stay in the buffer.

BufWriter[W]ResultContract
write(source)Result[i64, StreamError]Hold source, delivering held bytes first when it does not fit
flush()Result[(), StreamError]Deliver every held byte, then flush the inner stream
pending()i64Bytes held and not yet delivered
capacity()i64The fixed buffer size

Held bytes never exceed the capacity and reach the inner stream in order. A range at least as large as the capacity is passed to the inner stream directly after the held bytes, and may be accepted short. Delivery retries Interrupted and short writes. When delivery fails, the undelivered bytes stay held and the range passed to write is not accepted; a later flush resumes at the first undelivered byte.

Nothing is written when a BufWriter is dropped. Call flush or close_buffered; otherwise held bytes are discarded.

FunctionResultContract
seek_reader(reader, target)Result[i64, StreamError]Discard held input and seek. Current is relative to the bytes consumed
seek_writer(writer, target)Result[i64, StreamError]Deliver held bytes, then seek
close_buffered(writer)Result[(), CloseError]Flush, then close the inner stream even if the flush failed

Both classes expose the wrapped stream as the public field inner, for calls such as reader.inner.close(). Reading or writing inner directly bypasses the buffer. The language cannot move a field out of a consumed value, so there is no method returning the inner stream; exchange it with replace(buffer.inner, other).

Adapters

ConstructorStreamProtocols
open_file(path: &str, mode: &str)Result[FileStream, StreamError]Reader, Writer, Seeker, Closer
open_path(path: &Path, mode: &str)Result[FileStream, StreamError]the same, preserving native path bytes
from_file(file: File)FileStreamthe same
stdin()StdinReader
stdout(), stderr()Stdout, StderrWriter
cursor(data: Bytes)CursorReader, Writer, Seeker

open_path accepts a std.path.Path without decoding its bytes. It has the same modes, permissions, symlink behavior, and ownership as open_file. Embedded NUL is StreamError.Io(IoError.InvalidInput) rather than a truncated filename; an invalid mode is StreamError.Io(IoError.InvalidMode). Owner and path allocations are fallible and occur before opening or truncating. Use std.cli.fileinput for buffered fallible text-line iteration across multiple native paths or stdin.

FileStream owns a File, public as file. read and write each make one native call on the descriptor, with no newline translation or text decoding. seek positions by byte offset and returns the new offset; this differs from the opaque text cookies of File.tell. SeekFrom.Start with a negative offset is OutOfRange; another target the system rejects is Io(IoError.System(code)). flush and sync forward to the file. close reports the native close error once and is Ok when repeated; other operations on a closed file are Io(IoError.Closed). The open mode is enforced: Io(IoError.NotReadable) or Io(IoError.NotWritable). Dropping an unclosed FileStream closes the file and discards any close error.

Stdin, Stdout, and Stderr hold no state and never close the process descriptors. Stdin.read makes one native read. Stdout and Stderr share the process streams used by print and the text writers, so output keeps its order across both; flush delivers buffered stdout. Stderr is unbuffered: each Stderr.write makes one native write on descriptor 2, so a closed or read-only stderr is Io(IoError.System(9)) (EBADF).

Each process stream has is_terminal() -> bool. It asks the OS on each call and returns false for redirected, closed or undetectable streams. This query neither reads nor writes bytes and does not imply ANSI escape support or a terminal width. std.cli.progress uses it to keep redirected stderr clean.

Cursor reads from and writes to an owned buffer, public as data, at position(). Reads stop at the end of the buffer. Writes replace existing bytes and extend the buffer at its end; writing past the end first fills the gap with zeros. seek accepts any nonnegative position. A write can report Allocation; it reserves before changing existing bytes.

The same cursor also provides checked sequential binary fields, bounded owned payload reads, and remaining/require_end validation. These methods share its position while retaining stricter per-field failure atomicity.

Ownership and threads

Transport methods take &mut self, so a stream has one user at a time and a buffer passed to read cannot be the stream’s own storage. A read needs &mut Bytes; a write borrows its source shared. A buffered adapter owns its inner stream, and the inner stream cannot be used through another name while the adapter exists. Sharing one stream between threads requires moving it or placing it in a lock; the module adds no synchronization of its own. Writes to Stdout and Stderr from several threads are each atomic with respect to the process buffer, but their order is unspecified.

Costs

Each read or write on FileStream or Stdin is one system call. Buffered adapters make one bulk copy per call between the caller’s range and the fixed buffer. read_until searches held bytes with Bytes.find, one native scan per call. Seeking a BufReader discards up to capacity bytes already read.

AllocatesDoes not allocate
buffered_reader, buffered_writer: one bufferEvery BufReader and BufWriter method except read_until
read_up_to and read_until, when out lacks capacityread_exact, write_all, copy_all, seek_reader, seek_writer, close_buffered
Cursor.write beyond the buffer’s capacityFileStream, Stdin, Stdout, Stderr operations

Limits

  • Only x86-64 Linux GNU is supported. Nonblocking descriptors are not handled: EAGAIN arrives as Io(IoError.System(11)).
  • Stdout writes pass through the Rust standard library’s buffered stdout, which reports EBADF as success: writes to a closed or read-only stdout succeed and discard their bytes.
  • Streams are bytes only. Text decoding, line iteration over streams, and formatted writes are not provided; use read_until and Bytes.to_text.
  • There are no vectored operations, no read-ahead peeking, and no stream object that erases the concrete class.
  • A class cannot declare which protocols its parameter satisfies. A wrong inner stream is rejected at buffered_reader or buffered_writer with a note naming the missing method, not with a protocol diagnostic.
  • The native calls __io_file_read, __io_file_write, __io_stdin_read, __io_std_write, and __io_file_seek are private transport for the standard library, reached through Bytes.read_file and read_stdin, std.bytes.write_file, Stdout.write/Stderr.write, and FileStream.seek. __io_is_terminal is the private process-stream terminal query.

Acknowledgements

Sequential binary fields in a Cursor

std.io.Cursor owns a Bytes buffer and one byte position. Its typed methods use std.encoding to read and write fixed-width fields without caller-maintained offsets. Raw read, write, and seek share this same position; raw reads still allow short reads. No second cursor, stored borrow, native serialization library, or independent position is involved.

Scalar fields

read_u8() and read_i8() take no byte order. For each of u16, i16, u32, i32, u64, i64, f32, and f64, read_<type>(order: Endian) selects explicit little or big endian. The matching writes are write_<type>(value, order, max_bytes: i64 = 1048576); one-byte writes omit order. There is no native-order default or alignment requirement. Signed integers use two’s complement. Floating fields use exact IEEE 754 binary32 or binary64 bits, preserving signed zero, subnormals, infinities and NaN payloads without arithmetic or width conversion, matching std.encoding.

All typed methods return Result[..., BinaryError]. A successful field advances by its exact width. Failed fields leave position, contents, and length unchanged. Earlier successful operations remain committed: this is per-field atomicity, not record rollback. This guarantee is specific to memory; generic stream helpers can report externally visible partial progress.

The byte order belongs to the operation, matching the underlying codecs and making mixed-endian formats explicit without another mutable configuration. Existing buffers are owned by cursor(data); typed operations borrow the cursor for one call. Mutating its public data changes what subsequent reads see without changing the position.

Payloads, bounds, and errors

  • remaining() -> i64 returns bytes after the current position, or zero after seeking past the end.
  • require_end() -> Result[(), BinaryError] rejects remaining bytes without advancing. A position at or beyond the end succeeds.
  • read_bytes(count: i64, max_bytes: i64 = 1048576) returns an owned Bytes copy. It checks the cap and available input before allocation. Returned borrowed views are not supported by the language. An empty read returns an empty owned buffer without moving, even after a seek past the end.
  • write_bytes(source: &[u8], max_bytes: i64 = 1048576) borrows a literal, Bytes, list, or subview and writes its complete range. An empty write does not fill a seek gap or move position.

Every typed write’s max_bytes bounds the final buffer length, including an existing larger buffer and zero-filled seek gaps. Negative caps/counts and position arithmetic overflow are OutOfRange; a valid size exceeding the cap is LimitExceeded. Seek itself remains unrestricted except for its existing nonnegative checked i64 position contract. Writing beyond the current end fills the gap with zeros after successfully allocating the whole extent.

BinaryError variantMeaning
Truncated(position, needed, remaining)The complete requested field is unavailable; all three values are byte counts/offsets
TrailingData(position, remaining)require_end found unconsumed bytes
OutOfRangeNegative count/cap or position arithmetic overflow
LimitExceededRequested owned copy or final write buffer exceeds its cap
Allocation(AllocError)Buffer creation/growth failed

Scalar reads and writes within capacity allocate nothing. Growth uses Bytes allocation, and owned payload reads allocate one buffer and copy count bytes. Scalar operations are constant time; payload copies and zero-filled gaps take time proportional to their size. The caps bound logical bytes, not allocator metadata or resident memory. No synchronization or hidden worker restrictions are added; the owning cursor can be moved to one worker as other owned buffers can. Callers cannot borrow its data as a write source while borrowing the cursor mutably.

See the binary packet tutorial for a runnable example.

std.io.typed

Typed byte streams preserve a caller-selected error payload without side storage. They complement std.io; existing streams keep their APIs.

import std.bytes
import std.io
import std.io.typed as typed
from std.io.typed import Partial, ReadFailure

def main() -> Result[(), IoError | AllocError | std.bytes.BytesError | Partial[ReadFailure[std.io.StreamError]]]:
    mut source = typed.reader(std.io.cursor(std.bytes.from_text("header:body")?))
    mut buffer = std.bytes.with_capacity(6)?
    buffer.resize(6, 0u8)?
    typed.read_exact(&mut source, &mut buffer, 0, 6)?
    print(buffer.to_text()?)?
    match typed.read_exact(&mut source, &mut buffer, 0, 6):
        case Ok(_):
            print("complete")?
        case Err(partial):
            print(partial.completed, partial.cause)?
    Ok(())
header
5 std.io.typed.ReadFailure[std.io.StreamError].UnexpectedEof

Reader[E].read(&mut self, &mut Bytes, start, stop) and Writer[E].write(&mut self, &[u8]) return Result[i64, StepError[E]]. The read range is half-open and must satisfy 0 <= start <= stop <= buffer.len(); an invalid range traps with read: start/stop outside buffer before consulting the transport. read_exact traps with read_exact: start/stop outside buffer. A writer takes the bytes as a view. A successful call reports the bytes moved; a failed call reports no progress from that attempt. StepError.Interrupted requests retry. Source(E) preserves every other cause, including cancellation and allocation failure. Reader Ok(0) is EOF for a nonempty range. Writers need only write, not flush.

OperationResult
read_exact(reader, buffer, start, stop)Result[(), Partial[ReadFailure[E]]]
write_all(writer, buffer: &[u8])Result[(), Partial[WriteFailure[E]]]
copy_all(reader, writer, scratch)Result[i64, CopyFailure[E, G]]

Helpers retry only Interrupted. Partial carries completed: i64 and cause. ReadFailure[E] distinguishes Source(E), UnexpectedEof, InvalidProgress, and OutOfRange; WriteFailure[E] has Source(E), WriteZero and InvalidProgress. Negative or oversized counts are invalid progress. Invalid read ranges and empty reads or writes do not call the underlying stream.

CopyFailure.Read(completed, ReadFailure[E]) and CopyFailure.Write(completed, WriteFailure[G]) preserve the failing operation and independent source/destination error types. Counts mean bytes delivered to the writer, including a short write before failure. A read chunk may already have been consumed when writing fails: this API does not promise resumable copying or retain unwritten bytes. Empty scratch is an OutOfRange read failure. A chunk that would make the delivered total exceed i64.MAX is Read(completed, ReadFailure.OutOfRange) before writing that chunk; it is not an invalid count from the transport. Internal cursor updates use documented bounds-proven opt-outs, without changing the stream’s declared error type. Helpers allocate nothing and require caller-provided storage.

buffered_reader(inner, capacity: Positive[i64]) returns Result[BufReader[E, R], StreamError] for a Reader[E]; construction allocates the buffer, which is its only failure. Its read returns StepError[ReadFailure[E]], preserving a transport cause as Source(ReadFailure.Source(E)) and forwarding Interrupted. Invalid ranges and empty reads do not call the transport. Invalid inner counts become ReadFailure.InvalidProgress. Reads allocate nothing; capacity() and buffered() report the buffer size and unread bytes.

The public inner field permits transport-specific operations, but consuming it discards prefetched bytes. Keep the whole buffered reader when handing a connection from head parsing to body decoding or to the next HTTP message.

reader(inner) wraps an existing std.io.Reader as LegacyReader[R]; writer(inner) wraps a std.io.Writer as LegacyWriter[W]. Their typed cause is StreamError. Only its Interrupted variant becomes a retry signal; other variants survive intact. unwrap_reader(bridge) and unwrap_writer(bridge) consume the bridge and return the owned stream, allowing explicit flushing, seeking or closing. Copying does not flush or close. These bridges cannot recover causes already erased by a legacy adapter: For HTTP client responses, prefer std.http.client.typed_response(response) over bridging Response.read: its typed cause is the complete ClientError. The legacy Response.read/take_failure pair remains a compatibility API.

See typed HTTP framing and the tutorial.

std.json

import std.json decodes and encodes JSON (RFC 8259) under explicit limits, with one error type, JsonError. It has two levels:

  • Events: a resumable Decoder and a validating Encoder that never hold the document. Typed codecs are ordinary functions over them.
  • Value tree: Json, built from and written through the same events.

The module is checked Plenty code with no native primitives of its own. It reads and writes std.bytes buffers and std.io streams, and is available on the supported x86-64 Linux GNU target.

import std.bytes
import std.json
from std.json import Json

def main() -> Result[(), AllocError | IoError | std.bytes.BytesError | std.json.JsonError]:
    bounds = std.json.limits()
    value = std.json.parse_text("{\"name\": \"Ada\", \"langs\": [\"en\", \"fr\"]}", &bounds)?
    match &value:
        case Json.Object(members):
            for member in members:
                match &member.value:
                    case Json.Text(text):
                        print(member.key, "is", text)?
                    case Json.Array(items):
                        print(member.key, "has", len(items), "items")?
                    case _:
                        print(member.key, "is another type")?
        case _:
            print("not an object")?
    mut out = std.json.encoder(&bounds)?
    out.start_object()?
    out.key("id")?
    out.integer(7)?
    out.key("person")?
    out.value(&value)?
    out.end_object()?
    print(out.take()?.to_text()?)?
    Ok(())
name is Ada
langs has 2 items
{"id":7,"person":{"name":"Ada","langs":["en","fr"]}}

Accepted text

The decoder accepts exactly RFC 8259 and one document per input.

RuleBehavior
EncodingUTF-8, validated strictly; no byte-order mark
WhitespaceSpace, tab, \n, \r between tokens only
Top levelAny value, then only whitespace; anything else is TrailingData
ExtensionsNone: no comments, trailing commas, single quotes, NaN, Infinity, hex, or leading +
Numbers`-?(0
StringsBytes below 32 must be escaped; escapes are \" \\ \/ \b \f \n \r \t \uXXXX with digits of either case
SurrogatesA \uD800-\uDBFF escape must be followed by a \uDC00-\uDFFF escape; the pair is one scalar. Anything else is InvalidSurrogate
Duplicate keysCompared after unescaping. Rejected by the tree unless Limits says Keep; never checked by the event decoder

The encoder writes compact text: no whitespace, members in call order. In strings it escapes ", \, and bytes below 32 (\b \f \n \r \t, otherwise \u00xx in lowercase); every other scalar, including / and non-ASCII, is written as UTF-8.

Limits

pub class Limits:
    pub max_depth: NonNegative[i64]
    pub max_input: NonNegative[i64]
    pub max_output: NonNegative[i64]
    pub duplicate_keys: DuplicateKeys   # Reject or Keep

limits() returns Limits(64, 67108864, 67108864, DuplicateKeys.Reject). The fields are public; adjust a copy. A literal converts on assignment; a computed value needs NonNegative.try_from(value)?. Every constructor and whole-document function takes &Limits; decoder and encoder trap when max_depth is above 1024, so the whole-document functions built on them do too.

FieldBoundsWhen exceeded
max_depthArrays and objects open at once. 0 allows scalars onlyDepthExceeded at the opening bracket
max_inputTotal bytes given to one decoderInputTooLarge; the offset is the limit
max_outputTotal bytes one encoder produces, including bytes already takenOutputTooLarge
duplicate_keysWhether the tree accepts a repeated key in one objectDuplicateKey at the repeated key

The depth limit also bounds the encoder’s recursion over a Json tree, so a tree built by hand cannot overflow the stack.

Input and output counters are bounded by the remaining declared budget before addition. Taking or writing output does not reset that budget. Bounded cursor, hex-digit, surrogate and UTF-8 state arithmetic uses explicit, locally justified unchecked: scopes; malformed values still return the JSON faults below, not arithmetic errors. No public arithmetic error type is added.

Errors

pub class JsonError:
    pub fault: JsonFault
    pub offset: i64
    pub line: i64
    pub column: i64

For decoding, offset is the byte offset from 0, and line and column start at 1. Only \n ends a line. A column is one Unicode scalar, not one byte. A syntax fault points at the offending byte; a fault found after a token was read (DuplicateKey, NumberOutOfRange) points at the token’s first byte. UnexpectedEnd points one past the last byte. Position arithmetic is checked: if the next byte or ASCII run would make an offset, line, or column unrepresentable as i64, decoding returns InputTooLarge at the current position without consuming that byte or run. This also applies when max_input itself is i64.MAX; the one-based line and column need their own representation checks.

For encoding, offset is the number of bytes produced when the call failed and line and column are 0. A failed constructor reports 0, 0, 0.

JsonFaultMeaning
UnexpectedByteA byte that cannot start or continue the document here
UnexpectedEndInput ended inside a document, or was empty
TrailingDataA non-whitespace byte after the document
InvalidLiteralA misspelled null, true, or false
InvalidNumberA byte that breaks the number grammar; for Encoder.number, text that is not a JSON number
NumberOutOfRangeAn integer outside i64 or a decimal outside finite f64, in the tree
InvalidEscapeAn unknown escape or a non-hex digit in \uXXXX
InvalidSurrogateAn unpaired surrogate escape
ControlCharacterAn unescaped byte below 32 in a string
InvalidUtf8Malformed, overlong, surrogate, or out-of-range UTF-8 in a string
DuplicateKeyA repeated key under DuplicateKeys.Reject
DepthExceededNesting beyond max_depth
InputTooLarge, OutputTooLargeA size limit was reached; InputTooLarge also covers an unrepresentable input position
NonFiniteNumberA NaN or infinite f64 given to the encoder
InvalidStateA call that the decoder’s or encoder’s state does not allow
Stream(StreamError)A reader or writer failed; InvalidProgress if it reported an impossible count
Allocation(AllocError)Memory was refused

After any error, a decoder or encoder is poisoned: every later call reports InvalidState. Partial trees and buffers are released before the error returns.

Events

pub enum Event:
    Pending
    End
    Null
    Bool(bool)
    Number(str)
    Text(str)
    Key(str)
    StartArray
    EndArray
    StartObject
    EndObject

Number holds the validated source text, so a codec chooses the type and loses nothing: i64.parse, u64.parse, f64.parse, or the text itself. Text and Key hold the unescaped string.

Decoder

decoder(bounds: &Limits) -> Result[Decoder, JsonError].

MethodReceiverResultContract
feed(source: &[u8])exclusiveResult[(), JsonError]Copy a byte view into the unexamined input
finish()exclusive()Declare that no more input will arrive
next()exclusiveResult[Event, JsonError]The next event
offset()sharedi64Bytes examined
depth()sharedi64Arrays and objects open
reject(fault: JsonFault)sharedJsonErrorAn error placed at the first byte of the most recent event’s token

next returns Pending when every fed byte has been examined and the document is incomplete; feed more or call finish. After finish it returns End once the document and trailing whitespace are consumed, and End again on every later call. A number at the end of the fed input stays pending until the next byte or finish shows where it ends. Input may be split anywhere, including inside an escape or a UTF-8 sequence.

read_event[R: Reader](source: &mut Decoder, reader: &mut R) -> Result[Event, JsonError] drives a decoder from a stream. It reads up to 4096 bytes whenever next is Pending, retries Interrupted, treats a zero-length read as end of input, and never returns Pending. Do not mix it with feed on one decoder.

A typed codec reads the events it expects and uses reject for values it will not accept:

def coordinate(decoder: &mut std.json.Decoder) -> Result[i64, JsonError]:
    match decoder.next()?:
        case Event.Number(text):
            match i64.parse(text):
                case Ok(value):
                    Ok(value)
                case Err(_):
                    Err(decoder.reject(JsonFault.NumberOutOfRange))
        case _:
            Err(decoder.reject(JsonFault.UnexpectedByte))

Encoder

encoder(bounds: &Limits) -> Result[Encoder, JsonError]. Every writing method takes an exclusive receiver and returns Result[(), JsonError].

MethodWrites
null(), boolean(value: bool)A literal
integer(value: i64), unsigned(value: u64)Decimal digits
decimal(value: f64)The shortest text that parses back to the same value, as str.from; non-finite is NonFiniteNumber
number(text: &str)The text unchanged, after checking the number grammar
text(value: &str)A string
key(name: &str)A member name; only valid where an object expects one
start_array(), end_array(), start_object(), end_object()Brackets; commas and colons are inserted automatically
event(event: &Event)The call matching a decoder event; Pending and End write nothing
value(value: &Json)A whole tree as one value
MethodResultContract
pending()i64Bytes produced and not yet taken
written()i64Bytes produced in total
is_complete()boolExactly one top-level value has been written and closed
take()Result[Bytes, JsonError]Move the pending bytes out

The encoder accepts only call sequences that form one document: a key outside an object, a value where a key belongs, a mismatched close, or a second top-level value is InvalidState. It does not check member names for duplicates.

write_pending[W: Writer](source: &mut Encoder, writer: &mut W) -> Result[(), JsonError] writes every pending byte with std.io.write_all and keeps the buffer’s capacity. It does not flush. If the writer fails, the bytes it accepted are removed and the rest stay pending.

Value tree

pub enum Json:
    Null
    Bool(bool)
    Int(i64)
    Float(f64)
    Text(str)
    Array(list[Json])
    Object(list[Member])

pub class Member:
    pub key: str
    pub value: Json
FunctionResultContract
parse(source: &[u8], bounds)Result[Json, JsonError]Decode one document from a byte view
parse_text(text: &str, bounds)Result[Json, JsonError]The same, from a str
read_value[R: Reader](reader: &mut R, bounds)Result[Json, JsonError]Decode one document, reading to end of input
encode(value: &Json, bounds)Result[Bytes, JsonError]Compact encoding
encode_text(value: &Json, bounds)Result[str, JsonError]The same, as a str
write_value[W: Writer](writer: &mut W, value: &Json, bounds)Result[(), JsonError]Encode the whole value, then write it; no flush
find(members: &list[Member], key: &str)Option[i64]Index of the first member named key

A number without a fraction or exponent becomes Int and must fit i64; otherwise the document is rejected with NumberOutOfRange, never rounded. Any other number becomes Float, the nearest f64, and must be finite: 1e309 is NumberOutOfRange, while 1e-400 is 0.0. -0 is Int(0); -0.0 keeps its sign. Float values are written with a fraction or exponent (100.0, 1e21), so a tree keeps its Int/Float distinction through a round trip.

Objects keep members in source order, including duplicates under Keep. find is a linear scan; build a dict when an object is searched often.

Ownership and threads

A Json owns its whole tree; moving it moves everything and dropping it releases everything. Json, Decoder, and Encoder are single-owner values with no shared or global state. They can be moved into a worker thread, and the module’s functions pass the ordinary worker-effect checks. One decoder or encoder cannot be used from two threads at once, since every operation needs an exclusive loan.

Sources are shared loans: feed and parse copy what they need and keep no reference to the caller’s bytes. Events own their strings.

Json is recursive, so the builtins that require a finite layout walk reject it: ==, copy, print, and str.repr do not accept a Json. Compare and display trees through encode_text.

Costs

OperationAllocatesTime
decoderThree empty buffersConstant
feedGrows the window by the view; storage is reused once the window is drainedLinear in the view
nextOne str per Number, Text, or Key event; one byte of stack per open container; the scratch buffer grows to the longest tokenLinear in the bytes examined
read_eventA 4096-byte window, onceAs next, plus the reads
parse, read_valueThe events, one list per container, one set of keys per object under Reject, and for parse one copy of the inputLinear in the input, expected
Encoder callsGrow one output buffer; one byte of stack per open container; number text for integer, unsigned, decimalLinear in the bytes written
encode, write_valueThe whole output bufferLinear in the output
encode_textThe output buffer, then a str copyLinear
findNothingLinear in the member count

The decoder and the tree builder use no recursion. Encoder.value recurses once per nesting level, at most max_depth (1024) deep.

An event loop that drains next to Pending before each feed, or uses read_event, holds only the current chunk, the current token, and the depth stack, whatever the document size. A tree costs more than its text: each value is an enum cell and each string and container its own allocation.

The decoder takes a run of unescaped ASCII string content, and the encoder a run of bytes that need no escape, with one std.bytes scan each. Structural bytes, whitespace, numbers, literals, escapes, and non-ASCII string bytes still go through a checked single-element Bytes call. Measured with a release compiler on x86-64 Linux, pinned to one core, on an 8,388,872-byte document of 34,393 objects (687,862 events):

OperationBefore the scansNow
Decoder.next over the whole buffer2.55 s1.58 s
parse3.59 s2.65 s
parse, then encode of the tree4.56 s3.28 s

That is roughly 2 to 5 megabytes per second. The remaining time is the calls made per token and per event, not byte access.

Limits

  • Compact output only: no pretty-printing or key sorting.
  • One document per input: no JSON Lines or concatenated documents. Split the input and use one decoder per document.
  • The tree holds integers as i64 and other numbers as f64. Larger integers and exact decimals are available only as Event.Number text.
  • write_value builds the whole encoding before writing. Use an Encoder with write_pending for bounded output memory.
  • No derived codecs: a typed codec is written by hand over events.
  • No tree equality or path lookup helpers.

std.jwt

import std.jwt signs and verifies JSON Web Tokens (RFC 7519) in the compact JWS form (RFC 7515), with the RFC 7518 and RFC 8037 algorithms that std.crypto provides. Every error is a JwtError.

Upstream code, used as-is. Every MAC and signature is computed and checked by std.crypto, which uses unmodified ring and RustCrypto primitives; std.jwt is checked Plenty code that only parses, validates and dispatches. See std.crypto for this policy.

import std.time
import std.json
import std.jwt
import std.secrets
from std.json import Json, JsonError
from std.jwt import Algorithm, JwtError
from std.secrets import SecretsError

def main() -> Result[(), AllocError | IoError | SecretsError | JsonError | JwtError]:
    secret = std.secrets.token_bytes(32)?
    signer = std.jwt.hmac_signing_key(Algorithm.Hs256, &secret)?
    claims = std.json.parse_text("{\"sub\":\"ada\",\"aud\":\"api\",\"exp\":4102444800}", &std.json.limits())?
    token = std.jwt.sign(&signer, &claims)?

    # The verifying key fixes the algorithm; the header only has to agree.
    key = signer.verifying_key()?
    mut rules = std.jwt.validation(&std.time.wall_now())
    rules.leeway = std.time.seconds(30u32)
    rules.audience = Some("api")
    verified = std.jwt.verify(&token, &key, &rules)?
    print(verified.header.algorithm.name())?
    for member in &verified.claims:
        match &member.value:
            case Json.Text(text):
                print(member.key, text)?
            case _:
                pass

    # Any change to the signed bytes fails the signature check.
    forged = token.replace("eyJzdWIiOiJhZGEi", "eyJzdWIiOiJib2Ii")?
    match std.jwt.verify(&forged, &key, &rules):
        case Err(JwtError.InvalidSignature):
            print("forged token rejected")?
        case _:
            print("unexpected")?
    Ok(())
HS256
sub ada
aud api
forged token rejected

See the signature index for complete declarations.

Keys and algorithms

A key is bound to one algorithm when it is made. sign uses that algorithm, and verify accepts a token only if its header’s alg names the same one. The header never selects the algorithm or the key, so none, an HMAC token checked against an RSA public key, and other algorithm-substitution attacks fail before any cryptography runs.

AlgorithmNameSigning keyVerifying key
Hs256, Hs512HS256, HS512hmac_signing_key(algorithm, &secret)hmac_verifying_key(algorithm, &secret)
Rs256, Rs384, Rs512RS256 to RS512rsa_signing_key(algorithm, RsaPrivateKey)rsa_verifying_key(algorithm, RsaPublicKey)
Ps256, Ps384, Ps512PS256 to PS512rsa_signing_key(algorithm, RsaPrivateKey)rsa_verifying_key(algorithm, RsaPublicKey)
Es256ES256es256_signing_key(P256PrivateKey)es256_verifying_key(P256PublicKey)
EdDsaEdDSA (Ed25519)ed25519_signing_key(&seed)ed25519_verifying_key(&public_key)

Each constructor returns Result[_, JwtError], except the infallible ES256 ones. InvalidKey reports:

  • an algorithm the constructor does not handle, such as rsa_signing_key(Algorithm.Hs256, ...);
  • an HMAC secret shorter than the hash output: 32 bytes for HS256, 64 for HS512 (RFC 7518 section 3.2). Generate one with std.secrets.token_bytes;
  • an Ed25519 seed or public key that is not 32 bytes, or a small-order public key, which could verify forged signatures.

HMAC secrets and Ed25519 keys are copied. RSA and P-256 keys come from std.crypto key import, which enforces their sizes and encodings, and are moved in. SigningKey.verifying_key() returns the matching verifying key: a copy of the secret, or the public half. algorithm() returns a key’s algorithm and Algorithm.name() its JOSE name. Both key types are redacted: printing one shows std.jwt.SigningKey(@redacted) or std.jwt.VerifyingKey(@redacted), never the secret.

HS384 and ES384 are not provided: std.crypto has no SHA-384 HMAC or P-384 curve. Their names are UnsupportedAlgorithm.

Verifying

verify(&token, &key, &rules) returns a Token with the decoded header and the claims as a list[Member] from std.json. It checks, in order:

  1. rules are in range, otherwise InvalidValidation. A max_size above max_token_size traps.
  2. The token is at most rules.max_size bytes, otherwise TooLarge. Nothing is decoded before this check.
  3. The token is three segments separated by dots, each strict unpadded base64url (RFC 4648 section 5) with zero trailing bits. Padding, the standard alphabet, whitespace, and any other byte are Malformed.
  4. The header is a JSON object (NotObject) of strict JSON without duplicate members (Malformed). alg, kid and typ must be strings when present. A missing alg is Malformed; none or an unknown name is UnsupportedAlgorithm. A crit member is CriticalHeader, since no extensions are understood (RFC 7515 section 4.1.11). Other members, including jwk, jku, x5u and x5c, are ignored: they never supply a key.
  5. alg equals the key’s algorithm, otherwise AlgorithmMismatch.
  6. The signature verifies over the first two segments, otherwise InvalidSignature. HMAC tags are compared in constant time by std.crypto’s Mac.verify.
  7. Only now are the claims parsed, with the same strictness as the header: they must be a JSON object without duplicate members.
  8. The claims satisfy the time, issuer and audience rules below.

decode_header(&token, max_size: Positive[i64] = 8192) performs steps 2 to 4 without a key, for choosing a key by kid; a max_size above max_token_size traps. Its result is unauthenticated: use it only to pick among keys the program already trusts, then call verify.

Validation rules

std.jwt.validation(now) returns a Validation with these public fields:

FieldDefaultMeaning
now: Timestampthe argumentA valid wall timestamp at or after the Unix epoch, such as std.time.wall_now()
leeway: DurationzeroAllowed clock skew, up to max_leeway seconds (86400); subsecond precision is retained
require_expiryTrueWhether a token without exp is MissingClaim(Claim.Expiry)
issuerNothingWhen set, iss must be present and equal it
audienceNothingThe audience this program accepts
max_sizedefault_max_size (8192)Token limit in bytes, a Positive[i64] of at most max_token_size (1 MiB)

The clock is explicit so tests and programs choose its source. The claims are checked as RFC 7519 section 4.1 describes:

  • exp: Expired once now - leeway >= exp. A fractional value is rounded down.
  • nbf and iat: NotYetValid when more than leeway seconds in the future. Fractional values are rounded up. An iat in the future is rejected because the token cannot have been issued yet.
  • A time claim that is not a number, or is beyond the i64 range, is InvalidClaim with its Claim.
  • iss, when issuer is set: missing is MissingClaim(Claim.Issuer), a non-string is InvalidClaim(Claim.Issuer), and a different string is WrongIssuer. The comparison is exact.
  • aud is a string or an array of strings; anything else is InvalidClaim(Claim.Audience). It must contain audience, otherwise WrongAudience. A token with aud is WrongAudience when audience is Nothing, since the program is not among its recipients (RFC 7519 section 4.1.3). When audience is set, a missing aud is MissingClaim(Claim.Audience).

Other claims, including sub and jti, are returned unchecked. typ is returned in Header.token_type and not checked.

Signing

sign(&key, &claims, key_id) returns the compact token as str. claims must be a Json.Object, otherwise NotObject; it is encoded compactly in member order, and a value JSON cannot represent, such as a NaN, is Malformed. The header is {"alg":"...","typ":"JWT"}, with kid added when key_id is Some. HS, RS and EdDSA signatures are deterministic; PS and ES signatures use operating-system randomness, and its failure is RandomFailure. sign adds no claims: set exp and iat yourself.

Errors

JwtError has a std.report format method; no message repeats token contents or key material.

VariantMeaning
MalformedNot three strict base64url segments of strict JSON, a header without alg, or claims that cannot be encoded
NotObjectThe header or claims are not a JSON object
TooLargeThe token exceeds max_size
UnsupportedAlgorithmnone or an algorithm this module does not provide
AlgorithmMismatchThe header’s alg is not the key’s algorithm
CriticalHeaderThe header has crit
InvalidSignatureThe signature or MAC does not verify
Expired, NotYetValidThe time claims reject now
MissingClaim(Claim), InvalidClaim(Claim)A required claim is absent, or a claim has the wrong type or range
WrongIssuer, WrongAudienceiss or aud does not match
InvalidKeyA key does not suit its algorithm
InvalidValidationnow or leeway is out of range
RandomFailureRandomness failed while signing
Allocation(AllocError)An allocation failed

Claim is Expiry, NotBefore, IssuedAt, Issuer or Audience.

Memory

verify allocates a copy of the token, the decoded header, claims and signature, and the claims tree; sign allocates the encoded segments and the result. All of these are bounded by max_size when verifying and by the claims when signing. Every allocation is fallible and reported as Allocation; tests fail each allocation in turn and check that no other error results and nothing leaks.

Memory-constrained programs should use ES256 or EdDSA, which allocate nothing inside std.crypto beyond the signature they return. HMAC allocates one 368-byte state per operation. RSA reserves up to 12 KiB per private key while it lives, plus 8 KiB per signature and 12 KiB per verification, and reports a failed reservation as Allocation; see RSA and ECDSA keys.

Limits

  • Only compact JWS. JWE (encrypted tokens), the JSON serialization, unencoded payloads (RFC 7797) and nested tokens are not supported.
  • No JWK or JWKS handling: import keys through std.crypto, which accepts JWK components.
  • No HS384, ES384, ES512 or ES256K.
  • Header nesting is limited to 4 levels and claims to 32.

Arithmetic bounds

Dot offsets refer to found bytes inside the size-limited token. Time comparisons validate a nonnegative clock and at most max_leeway seconds of leeway before subtracting or scaling; large future dates are rejected before nanosecond multiplication. These bounds-proven arithmetic opt-outs keep the existing JWT validation errors, including fractional-second comparisons at integer limits.

Acknowledgements

  • Token format, claims and algorithms follow RFC 7519 (JWT), RFC 7515 (JWS), RFC 7518 (JWA) and RFC 8037 (EdDSA in JOSE). Tests use the RFC 7515 appendix A.1 to A.3 and RFC 8037 appendix A.4 examples.
  • The key-bound API and validation settings were modelled on the Rust jsonwebtoken crate; no code is used.
  • Cryptography is std.crypto’s; see its acknowledgements.

std.log

import std.log provides leveled events with typed fields and a static sink contract. There is no global logger: a program builds a Logger around a sink it chooses and passes the logger, or the sink, to the code that logs. A library that logs takes &mut Logger[S] as a generic parameter, so its caller decides the destination, threshold, and clock. The module is checked Plenty over std.io, std.fmt, and std.time, with no native code of its own. It is available on the supported x86-64 Linux GNU target.

import std.io
import std.log
from std.log import Clock, Level, LogError

def main() -> Result[(), AllocError | IoError | LogError]:
    # Programs usually log to std.io.stderr() with Clock.Wall timestamps.
    sink = std.log.text_sink(std.io.stdout(), 4096)?
    mut log = std.log.logger(sink, Level.Info, Clock.Off)
    log.info("listening", [std.log.integer("port", 8080), std.log.text("host", "db")]?)?
    log.debug("below the threshold", []?)?
    log.warn("slow request", [std.log.floating("seconds", 2.5)]?)?
    print(log.emitted(), log.filtered())?
    Ok(())
level=info msg="listening" port=8080 host="db"
level=warn msg="slow request" seconds=2.5e0
2 1

Events

TypeContents
LevelTrace, Debug, Info, Warn, Error, in increasing severity
ValueText(str), Integer(i64), Unsigned(u64), Floating(f64), Boolean(bool)
Fieldkey: str, value: Value
Eventlevel: Level, message: str, fields: list[Field], time: Option[Timestamp]

All fields are public. text(key, value), integer, unsigned, floating, and boolean build a Field; event(level, message, fields) builds an unstamped Event. An event is wholly owned, so a sink may keep it or move it to another thread. Fields keep the order the caller gave.

level_name(&level) returns the lowercase name, such as "warn". parse_level(&name) accepts exactly those five names and returns UnknownLevel for anything else, including other capitalizations.

Loggers

logger(sink, threshold: Level, clock: Clock) -> Logger[S] takes ownership of sink. The sink is public as the field sink.

Logger[S]ResultContract
enabled(&level)boolWhether an event at level would reach the sink
emit(event)Result[(), LogError]Filter, stamp, and deliver one event
log(level, message, fields)Result[(), LogError]Build and emit an event
debug, info, warn, error (message, fields)Result[(), LogError]log at that level
emitted(), filtered(), failed()u64Events delivered, below the threshold, and rejected by the sink

The diagnostic u64 event counters saturate at u64.MAX; exhaustion never changes filtering, delivery, or the sink error returned to the caller.

An event below the threshold is dropped and counted as filtered; that is Ok(()), not an error. The fields of a filtered event have already been built, so guard expensive fields with enabled first.

ClockStamp for an event whose time is Nothing
OffNone; time stays Nothing
Wallstd.time.wall_now() at emission
Fixed(Timestamp)That timestamp, for reproducible output and tests

An event that already has a time keeps it, so replayed or forwarded events are not restamped. Stamping happens after filtering.

A sink error is returned unchanged and counted as failed. The logger does not retry, buffer, or fall back to another sink. The caller chooses whether a logging failure matters: propagate it, or drop the result and inspect failed() later.

Sinks

A sink is any pub class with this method, resolved at compile time:

ProtocolMethodResult
Sinkemit(self: &mut, event: Event)Result[(), LogError]

The sink owns the event from then on, even on failure.

ConstructorSinkDelivery
text_sink(writer: W, max_line: Positive[i64])Result[TextSink[W], LogError]One logfmt line per event to any Writer
memory_sink(capacity: NonNegative[i64])Result[MemorySink, LogError]Keeps up to capacity events in the public list events
channel_sink(sender: Sender[Event])ChannelSinkQueues the event without waiting
discard()DiscardAccepts and drops every event

Text lines

TextSink writes logfmt: space-separated key=value pairs, then a newline.

time=1700000000.000001234 level=info msg="listening" port=8080 host="db" tls=false load=2.5e-1
  • time appears only for a stamped event: seconds since the Unix epoch, a dot, and exactly nine digits of nanoseconds. A time whose nanoseconds is 1,000,000,000 or more returns InvalidSpec and writes nothing.
  • level is the lowercase name. msg and text values use the debug_text form: double quotes, with ", \, LF, CR, TAB, and other control characters escaped, so one event is always one line.
  • Integers are decimal. Floats use the shortest round-trip scientific form of std.fmt. Booleans are true and false.
  • A key must be nonempty ASCII letters, digits, _, ., or -, and must not be time, level, or msg. Otherwise emit returns InvalidKey and writes nothing.

max_line bounds one line in bytes, newline included; 0 is a compile error. An event whose line would be longer returns TooLarge and writes nothing: lines are never truncated. Each line is built in a buffer reserved at construction, then delivered with one write_all followed by flush. A write error is Output(cause); part of the line may already have reached the writer. TextSink exposes the writer as the public field writer.

For a command-line program, use text_sink(std.io.stderr(), ...). Lines then share the process buffer with print, keep their order relative to it, and are flushed per event.

Memory

MemorySink reserves room for capacity events at construction. Once full, emit returns Full and the event is dropped; the list never grows. Tests inspect sink.events directly.

Channels and threads

ChannelSink sends with send_nowait on a bounded channel. A full queue returns Full and drops the event, so a slow consumer never blocks the logging thread. A queue with no receiver returns Closed. share() returns another handle on the same queue for another thread’s logger.

Drained.delivered and Drained.failed also saturate at u64.MAX, without stopping the consumer or replacing a sink result.

drain(receiver: &Receiver[Event], sink: &mut S) -> Drained delivers queued events to sink until every sender is gone. It waits for events. A sink error is counted in Drained.failed and draining continues; Drained.delivered counts the rest. Run it on one thread to serialize output from many producers:

def consume(receiver: Receiver[Event]) -> Result[Drained, LogError]:
    mut lines = std.log.text_sink(std.io.stderr(), 4096)?
    Ok(std.log.drain(&receiver, &mut lines))

The usual join rules apply: a ChannelSink holds a sender, so drop every handle before joining the draining thread.

Errors

LogErrorMeaning
InvalidSpecAn event time with out-of-range nanoseconds
InvalidKeyA field key TextSink cannot write unambiguously
UnknownLevelparse_level received another name
TooLargeThe line would exceed max_line
FullA bounded sink had no room; the event was dropped
ClosedThe channel has no receiver
Format(FormatError)std.fmt rejected a value
Output(StreamError)The writer failed
Allocation(AllocError)A sink’s storage could not be reserved

Ownership and threads

Every emission takes &mut, so a logger and its sink have one user at a time. Threads do not share one logger: each thread builds its own Logger around a ChannelSink handle from share(), and one thread drains the queue. Events move through the channel without copying. A Logger holding a TextSink over a stream can also be moved into a worker like any owned value.

Costs

AllocatesDoes not allocate
Building fields and the field list, as the caller writes themLogger.emit and the shorthand methods
text_sink: a 32-byte scratch buffer and a max_line line bufferTextSink.emit, apart from the writer’s own storage
memory_sink: a list of capacity eventsMemorySink.emit
channel(...): the queue, made by the callerChannelSink.emit, share, drain

TextSink.emit is linear in the line length; each byte passes through std.fmt and is copied once into the line buffer. The line then costs one write_all and one flush: over stderr() that is one buffered write, and over a FileStream at least one system call. Dropping an event frees its fields.

Limits

  • Only x86-64 Linux GNU is supported.
  • There is no JSON output, no sampling or rate limiting, no per-module thresholds, and no span or context propagation. Compose these from Event and a custom sink.
  • Sinks are static: a logger cannot switch sinks at run time, and there is no sink object that erases the concrete class. Fan-out to two destinations is a custom sink that writes to both.
  • LineBuffer is public only because std.fmt writes into it; it is not meant for direct use.

Acknowledgements

  • Typed fields and caller-chosen sinks are modelled on Go’s log/slog; the five levels match the Rust tracing crate.

std.markdown

import std.markdown parses CommonMark 0.31.2 into a stream of events and renders events as HTML, under explicit limits and with one error type, MarkdownError. It has two levels:

  • Events: a Parser yields Start, End, text, and break events in document order. Programs can inspect, filter, or rewrite them.
  • HTML: an HtmlRenderer turns events into HTML text. to_html, render, and write_html connect a parser to a renderer.

The module is checked Plenty code with no native primitives of its own. It follows the algorithms of the reference implementation, commonmark.js, and renders the HTML that the specification’s examples show. It reads std.bytes buffers, writes std.io streams, and is available on the supported x86-64 Linux GNU target.

Accepted text

RuleBehavior
SyntaxCommonMark 0.31.2: every block and inline construct of the specification, and nothing more
ExtensionsNone: no tables, task lists, strikethrough, footnotes, extended autolinks, or front matter
EncodingUTF-8, validated strictly; invalid input is InvalidUtf8 at the first bad byte
Line endings\n, \r\n, and \r all end a line
NULReplaced by U+FFFD, as the specification requires
Raw HTMLHTML blocks and inline tags pass through unchanged as Html and InlineHtml events
UnicodePunctuation and whitespace classes for emphasis come from Unicode 16.0.0; link labels match after Unicode case folding by lowering then raising
EntitiesAll HTML5 named character references, and decimal and hexadecimal references

Raw HTML is not sanitized. For untrusted input, drop or escape Html and InlineHtml events before rendering, and check link and image destinations, which may use any scheme, including javascript:.

Limits

pub class Limits:
    pub max_input: NonNegative[i64]
    pub max_depth: NonNegative[i64]
    pub max_output: NonNegative[i64]

limits() returns Limits(67108864, 64, 67108864). The fields are public; adjust a copy. Every constructor and whole-document function takes &Limits. A literal converts on assignment; a computed value needs NonNegative.try_from(value)?.

FieldBoundsWhen exceeded
max_inputBytes of Markdown given to one parserInputTooLarge; the offset is the limit
max_depthBlock quotes and list items open at once. 0 allows neitherDepthExceeded on the line that would exceed it
max_outputTotal bytes one renderer produces, including bytes already taken or writtenOutputTooLarge

Inline nesting, such as emphasis inside links inside emphasis, is not limited by max_depth: the parser and renderer walk it without recursion. Running time is linear in the input, including on the inputs that make naive parsers quadratic, such as long runs of unclosed brackets, backticks, or HTML comments. Link destinations may nest parentheses 32 deep, as in cmark.

Errors

pub enum MarkdownFault:
    InvalidUtf8
    InputTooLarge
    OutputTooLarge
    DepthExceeded
    InvalidState
    Stream(StreamError)
    Allocation(AllocError)

pub class MarkdownError:
    pub fault: MarkdownFault
    pub offset: i64
    pub line: i64
Faultoffsetline
InvalidUtf8The first invalid input byte0
InputTooLargemax_input for the raw input limit; 0 (unavailable) for tab-expanded column exhaustion0
DepthExceededThe start of the offending line in the inputThat line, from 1
OutputTooLarge, StreamBytes the renderer had produced0
Others0, or bytes produced for a renderer0

After an error from next or event, later calls to that method return InvalidState. Renderer output already produced remains available through take or write_pending; a stream failure from write_pending can be retried.

Tab-expanded columns remain exact. If advancing a column is unrepresentable, parsing returns InputTooLarge with offset and line both 0: a raw input location is unavailable, not known to be byte 0. Allocation extents that cannot be represented return Allocation(AllocError.CapacityOverflow).

Events

pub enum Tag:
    Paragraph
    Heading(i64)              # level 1 to 6
    BlockQuote
    CodeBlock(str)            # info string; empty for indented code
    BulletList
    OrderedList(i64)          # start number
    Item
    Emphasis
    Strong
    Link(str, str)            # destination, title
    Image(str, str)           # destination, title

pub enum Event:
    Start(Tag)
    End(Tag)
    Text(str)
    Code(str)                 # inline code span
    Html(str)                 # raw HTML block
    InlineHtml(str)           # raw inline tag, comment, or declaration
    SoftBreak
    HardBreak
    Rule
    Done
pub def parser(source: &[u8], bounds: &Limits) -> Result[Parser, MarkdownError]
pub def parser_text(text: &str, bounds: &Limits) -> Result[Parser, MarkdownError]
Parser.next(self: &mut Parser) -> Result[Event, MarkdownError]

parser validates the input and parses its block structure: paragraphs, headings, lists, quotes, code, and link reference definitions. It copies what it needs, so source stays free. next parses inline content one paragraph or heading at a time, so a parser holds the block tree and one leaf’s inline events, not the whole event stream.

Every Start has a matching End with an equal tag. Text, entities, and backslash escapes are resolved: Text holds the characters to show, and adjacent text arrives as one event. A code block’s content, including its final line ending, is one Text event between Start(CodeBlock) and End(CodeBlock); an empty block has none. Html holds a raw block without its final line ending.

Paragraphs directly inside items of a tight list produce no Paragraph events, so a renderer need not track tightness. Link reference definitions produce no events; references to them arrive as Link and Image with the definition’s destination and title. Inside Image, the events between Start and End are its description; HTML renders them as plain text in alt.

After the last event, next returns Done, and keeps returning it.

HTML

pub def renderer(bounds: &Limits) -> Result[HtmlRenderer, MarkdownError]
HtmlRenderer.event(self: &mut HtmlRenderer, event: &Event) -> Result[(), MarkdownError]
HtmlRenderer.pending(self) -> i64
HtmlRenderer.written(self) -> i64
HtmlRenderer.take(self: &mut HtmlRenderer) -> Result[Bytes, MarkdownError]
pub def write_pending[W: Writer](source: &mut HtmlRenderer, writer: &mut W) -> Result[(), MarkdownError]
pub def write_html[W: Writer](source: &mut Parser, writer: &mut W, bounds: &Limits) -> Result[(), MarkdownError]
pub def render(source: &[u8], bounds: &Limits) -> Result[Bytes, MarkdownError]
pub def to_html(text: &str, bounds: &Limits) -> Result[str, MarkdownError]

A renderer appends the HTML for each event to an owned buffer. pending counts bytes not yet taken; written counts every byte produced. take returns the buffer and leaves an empty one. write_pending writes the buffer to a writer; on failure only the bytes the writer refused stay pending, so a retry cannot repeat output. write_html renders every remaining event of a parser, writing whenever 64 KiB is pending; neither function flushes.

The output matches the specification’s examples:

EventHTML
Paragraph, Heading(n)<p>…</p>, <hn>…</hn>
BlockQuote<blockquote> and </blockquote> on their own lines
BulletList, OrderedList(n)<ul>, <ol>, or <ol start="n"> when n is not 1
Item<li>…</li>
CodeBlock(info)<pre><code class="language-w">…</code></pre>, where w is the first word of info; no class when info is empty
Emphasis, Strong<em>, <strong>
Link(d, t)<a href="d" title="t">; no title when t is empty
Image(d, t)<img src="d" alt="…" title="t" />
Code(s)<code>s</code>
Html(s), InlineHtml(s)s, unescaped
SoftBreak, HardBreak, Rulea line ending, <br />, <hr />

Text and attributes escape &, <, >, and ". Destinations in href and src are percent-encoded except for ASCII letters, digits, and !#$%()*+,-./:;=?@_~^, with & written as &amp; and ' as &#x27;. Rewritten events must retain document order and valid tags. A heading level outside 1..6, or unrepresentable nested image depth, returns InvalidState without emitting bytes for that failing event. Earlier output and its exact written total remain intact; the renderer is then broken. Output-budget exhaustion, including unrepresentable byte totals, returns OutputTooLarge without wrapping or saturating the count.

Each block starts on its own line, and the output ends with a line ending unless it is empty.

Example

import std.markdown
from std.markdown import Event, MarkdownError, Tag

def main() -> Result[(), AllocError | IoError | MarkdownError]:
    bounds = std.markdown.limits()
    text = "# Notes\n\nSee [the guide](/guide) and *more*.\n"
    print(std.markdown.to_html(text, &bounds)?)?
    mut events = std.markdown.parser_text(text, &bounds)?
    while True:
        event = events.next()?
        match event:
            case Event.Start(tag):
                match tag:
                    case Tag.Link(destination, _):
                        print("link to", destination)?
                    case _:
                        pass
            case Event.Done:
                break
            case _:
                pass
    Ok(())
<h1>Notes</h1>
<p>See <a href="/guide">the guide</a> and <em>more</em>.</p>

link to /guide

See the runnable Markdown lesson.

Implementation notes

The Unicode punctuation ranges and the HTML5 entity table are generated into the module by scripts/gen_markdown_tables.py from Python’s unicodedata and html.entities; rerun it to move to a newer Unicode version.

The parser and renderer are derived from commonmark.js 0.31.2, Copyright (c) 2014 John MacFarlane, under the BSD-2-Clause license. The punctuation table derives from the Unicode Character Database (Unicode License v3) and the entity table from the WHATWG HTML Standard (BSD-3-Clause for portions in source code). Release archives include these notices in THIRD-PARTY-NOTICES.md.

Acknowledgements

std.template

Bounded text templates, implemented in Plenty. Register named sources once with Engine.add, then render them repeatedly with explicit std.json.Json objects. No files are opened and no Plenty code, shell commands or methods are evaluated.

import std.template
from std.template import TemplateError
from std.json import Json, Member

def main() -> Result[(), TemplateError | AllocError | IoError]:
    bounds = std.template.limits()
    mut templates = std.template.engine(&bounds)?
    templates.add("greeting", "<p>Hello {{ name }}!</p>")?
    values = Json.Object([Member("name", Json.Text("Ada & Grace"))]?)
    print(templates.render("greeting", &values)?)?
    Ok(())
<p>Hello Ada &amp; Grace!</p>

Language

The syntax is a small Jinja-style subset, using Tera as a syntax reference, not a Tera/Jinja compatibility implementation.

FormMeaning
{{ person.name }}Look up an object field; items.0 indexes an array
{{ 'hello' }}, {{ 42 }}, {{ true }}Text, numeric, boolean and null literals
{{ name | upper }}Apply a filter; filters can be chained
{# comment #}Ignore a comment
{% if active %}yes{% else %}no{% endif %}Conditional, with optional else
{% if not active %}...{% endif %}Negate a condition
{% for item in items %}{{ item.name }}{% else %}empty{% endfor %}Array loop; optional else renders only for an empty array
{% block title %}Title{% endblock %}Define a top-level named block
{% extends 'base' %}Inherit from a registered template

Names use ASCII letters, digits and underscores, starting with a letter or underscore. A path’s later segments may be nonnegative array indices. Text literals support \\, \', \", \n, \r and \t. Conditions treat null, false, zero and empty text/arrays/objects as false. Missing fields are errors, including in conditions. Arrays and objects cannot be interpolated directly; loop over an array or select a scalar field.

The innermost loop provides loop.index (one-based), loop.index0, loop.first, loop.last and loop.length. A loop variable shadows an outer value with the same name and is removed when that loop ends. The name loop is reserved for loop metadata. Loops accept a single variable and an array path, without spaces inside that path. Object iteration is not supported.

Filters are upper, lower, trim, length, escape and safe. Text filters require text. length counts Unicode scalars for text and entries for arrays or objects. Filters do not accept arguments. Arithmetic, comparisons, boolean operators other than not, calls, bracket indexing, elif, assignments, includes, macros, raw blocks and whitespace-trimming delimiters are not part of this version. Unsupported syntax is rejected when registering a template, including inside branches that would never execute.

Escaping and inheritance

render(name, context, html=True) escapes &, <, >, " and ' in interpolated values. Literal template text is trusted. escape marks its escaped result safe, avoiding double escaping; safe explicitly trusts the value. upper and lower remove that trust. Use html=False for plain-text emails or generated text.

This escaping is for HTML text and quoted ordinary attributes. It does not validate URLs or make values safe inside JavaScript, CSS, tag names, unquoted attributes or raw HTML. Template authors control safe and literal markup; do not treat untrusted templates as a markup sanitizer.

Escaping individual values

Use escape(text: &str, context: HtmlContext, max_bytes: NonNegative[i64] = 16777216) when assembling a fragment directly or matching another renderer’s exact output. It returns Result[str, TemplateError]. Choose the context from the position where the value will appear; the escaper does not parse surrounding HTML.

HtmlContextEscapesIntended position
Text&, <, >Ordinary HTML text children
DoubleQuotedAttribute&, "Inside a double-quoted ordinary attribute value
SingleQuotedAttribute&, 'Inside a single-quoted ordinary attribute value
TextAndAttributes&, <, >, ", 'The conservative policy used by render and the escape filter

Entities are &amp;, &lt;, &gt;, &quot; and &#x27;, respectively. Other UTF-8 bytes are preserved. The attribute modes do not add delimiters; keep the matching quotes around the complete attribute value. A text-escaped value can still contain quotes and is unsuitable for an attribute without escaping its original input for that attribute’s context.

from std.template import escape, HtmlContext, TemplateError

def main() -> Result[(), TemplateError | AllocError | IoError]:
    name = "Ada & \"friends\""
    title = escape(&name, HtmlContext.DoubleQuotedAttribute, max_bytes=128)?
    child = escape(&name, HtmlContext.Text, max_bytes=128)?
    print(f"<span title=\"{title}\">{child}</span>"?)?
    Ok(())
<span title="Ada &amp; &quot;friends&quot;">Ada &amp; "friends"</span>

The input is borrowed and never modified; success returns ordinary owned text, with no trust marker. Passing that text through Json.Text into render applies normal escaping again. Prefer raw input for template values and these helpers for directly assembled fragments. safe remains an explicit trust assertion by the template author; it does not check an escaping context.

Escaping treats the input as text, including existing entities: &amp; becomes &amp;amp;. Call it once on the original value for the intended context; repeated calls are not idempotent. The new function does not change filter trust: escape uses TextAndAttributes, upper/lower revoke trust, and trim preserves it.

max_bytes caps UTF-8 output, including entity expansion. Oversized input is rejected before allocation because escaping cannot shrink it. Each input byte is scanned once; replacement entities are appended directly and are not revisited. The output cap therefore also bounds source bytes scanned and copied. The function may allocate source/output buffers and returned text; allocation failures are TemplateError.Allocation or an allocation cause inside TemplateError.Bytes. Exceeding the cap returns LimitExceeded, with no partial output. A zero cap accepts only empty text; a dynamic cap uses NonNegative.try_from(value)?.

These contexts do not support HTML comments, tag/attribute names, unquoted attributes, scripts, styles or URL validation. Choosing an enum value is explicit caller responsibility, not proof that its surrounding HTML is valid.

Inheritance

extends must precede every non-whitespace, non-comment construct. An extending template contains only whitespace, comments and top-level blocks. Blocks cannot be nested or repeated within a template. The most-derived definition replaces its ancestor’s entire block; no super() call is provided. Blocks absent from the base’s layout are not rendered. Parent names are literal registry names, not filesystem paths. Missing parents and inheritance cycles are render errors.

Values, limits and errors

The root context must be Json.Object. Build values explicitly with Json and Member, or supply a value from the JSON decoder. Object keys must be unique and float values finite throughout the context. Values are read-only; rendering builds an indexed view and does not modify or deep-copy the caller’s tree. This API needs no serialization derivation. A future derived serializer can provide the same JSON data model without changing template syntax; derived serialization is currently a proposal, not an implemented conversion.

limits() returns:

FieldTypeDefaultBounds
max_sourceNonNegative[i64]1,048,576Total UTF-8 bytes of registered names and sources
max_tokensNonNegative[i64]16,384Total compiled tokens across templates
max_templatesNonNegative[i64]128Registered template count
max_context_nodesPositive[i64]65,536Values in a render context, including its root
max_context_bytesNonNegative[i64]16,777,216UTF-8 bytes of context keys and text values
max_depthPositive[i64]64Syntax/context/render nesting and inheritance chain; at most 64
max_stepsNonNegative[i64]1,000,000Render operations, including context construction, lookup and loop visits
max_outputNonNegative[i64]16,777,216UTF-8 output bytes and transformed text values

A literal converts where a field is expected; a computed value needs Positive.try_from(value)? or NonNegative.try_from(value)?. engine(&bounds) traps when max_depth is above 64 and copies the settings. add compiles and validates before registering; a failure leaves the engine unchanged. Names cannot be empty or registered twice. A render failure returns no partial output and leaves the engine reusable. Limits bound sizes and work counts, not an exact heap-byte quota or wall-clock deadline. Budget totals remain exact: an unrepresentable source, token, context-byte or work total returns TemplateError.LimitExceeded, without wrapping or saturating. Registration commits its totals only after the compiled template is appended successfully. Text filters may allocate intermediate text.

TemplateError distinguishes exceeded limits, syntax with a zero-based source location, missing/duplicate template names, missing values, type mismatches, cycles, allocation failures and byte-buffer errors. Syntax locations are UTF-8 byte offsets at the containing template construct, rather than at an individual expression character. Allocation errors remain explicit, including those wrapped by BytesError.

Acknowledgements

  • Syntax is a subset of Jinja, with Tera as the syntax and behaviour reference.

std.math

std.math provides floating-point functions with recoverable domain and overflow errors, checked rounding to integers, integer algorithms, generic numeric helpers, and constants. Its source is stdlib/std/math.plenty. Importing, the reserved std namespace, and compilation are common to every standard module.

import std.math as math
from std.math import pi, MathError

Domain algorithms report MathError.Domain and MathError.Overflow. Integer magnitude and reduction helpers preserve OverflowError[T], including its wrapped numeric payload. No function in this module allocates.

import std.math as math
from std.math import MathError

def main() -> Result[(), AllocError | IoError | MathError]:
    print(math.sqrt(2.0)?, math.log10(1000.0)?, math.round(2.5)?)?
    print(str.repr(math.sqrt(-1.0))?)?
    print(math.isclose(0.1 + 0.2, 0.3, 1e-9, 0.0)?)?
    print(math.gcd(84u64, 36u64), math.comb(10u64, 3u64)?)?
    print(math.max(3, 9), math.abs_float(-4.5), math.pi)?
    Ok(())
1.4142135623730951 3.0 2
Result[f64, std.math.MathError].Err(std.math.MathError.Domain)
True
12 120
9 4.5 3.141592653589793

Floating-point functions

Inputs are f64, following Python’s double-precision math surface. Explicitly convert integers and f32 inputs. Angles use radians.

FunctionsResult
sqrt(x), cbrt(x)Result[f64, MathError]
exp(x), exp2(x), expm1(x)Result[f64, MathError]
log(x), log2(x), log10(x), log1p(x), log_base(x, base)Result[f64, MathError]
sin(x), cos(x), tan(x), asin(x), acos(x), atan(x), atan2(y, x)Result[f64, MathError]
sinh(x), cosh(x), tanh(x), asinh(x), acosh(x), atanh(x)Result[f64, MathError]
pow(x, y), fmod(x, y)Result[f64, MathError]
floor(x), ceil(x), trunc(x), round(x)Result[i64, MathError]
divmod_float(a, b)Result[(f64, f64), MathError]
fabs(x), copysign(x, y), degrees(x), radians(x)f64
isnan(x), isinf(x), isfinite(x)bool
isclose(a, b, rel_tol, abs_tol)Result[bool, MathError]

MathError.Domain reports invalid real operations, including negative square roots, nonpositive logarithms, infinite trigonometric inputs, and zero finite negative powers. MathError.Overflow reports finite inputs whose result exceeds the floating or integer result range. Underflow succeeds. NaNs propagate through floating results; rounding NaN is a domain error. Rounding infinity or a value outside [-2**63, 2**63) overflows. Floating primitives preserve signed zero; fabs clears its sign and copysign observes it. Predicates and angular conversions have no recoverable failure. Angular conversions follow IEEE arithmetic, including overflow to infinity. Integer .pow remains a distinct exact operation with the language’s recoverable integer arithmetic contract.

round rounds to the nearest integer, choosing the even integer at a halfway value: round(2.5) is Ok(2) and round(3.5) is Ok(4). It takes no decimal places argument. The numeric .round() method keeps the float type and signed zero; the library function checks the rounded value before converting to i64.

divmod_float returns the floor quotient and a remainder with the divisor’s sign, deriving both from the same floating remainder to avoid inconsistent results near integral quotients. Zero remainders retain the divisor’s sign; zero quotients retain the sign of a / b. A zero divisor or any non-finite input is Domain; a quotient beyond the finite float range is Overflow. Floating rounding means a == q * b + r need not hold exactly.

isclose follows Python’s weak symmetric comparison: exact equality succeeds first, unequal infinities fail, and otherwise either relative tolerance or the absolute tolerance may accept the difference. Negative tolerances produce Domain; NaNs are never close. Both tolerances are required parameters; 1e-9, 0.0 matches Python’s defaults.

The module exports pi, e, tau, inf, and nan as immutable f64 module constants. They can also be imported individually. Numeric types retain f64.PI, f64.E, f64.TAU, f64.INF, and f64.NAN, with the same names on f32.

Integer algorithms and generic helpers

FunctionContract
factorial(n: NonNegative[i64]) -> Result[u64, MathError]Inputs above 20 overflow
gcd(a: u64, b: u64) -> u64Euclidean algorithm; gcd(0, 0) is zero
lcm(a: u64, b: u64) -> Result[u64, MathError]Either zero input gives zero; checked multiplication
isqrt(n: u64) -> u64Exact floor square root over the entire u64 range
perm(n: u64, k: u64) -> Result[u64, MathError]Ordered selections; k > n gives zero
comb(n: u64, k: u64) -> Result[u64, MathError]Unordered selections; k > n gives zero; cancels factors before multiplication
abs[T: IntType](x: T) -> Result[T, OverflowError[T]]Integer magnitude; a signed minimum returns its wrapped overflow payload
abs_float[T: FloatType](x: T) -> TFloating magnitude; clears the sign of zero
min[T](a: T, b: T) -> T, max[T](a: T, b: T) -> TCompare two owned comparable values; equal values retain the first
min_all[T](items: &list[T]) -> Option[T], max_all[T](items: &list[T]) -> Option[T]Copy the first extremum; Nothing for an empty list; elements must be copyable and comparable
sum[T: IntType](items: &list[T], start: T) -> Result[T, OverflowError[T]], prod[T: IntType](items: &list[T], start: T) -> Result[T, OverflowError[T]]Sequential integer fold; first overflow returns its wrapped payload; empty lists return Ok(start)
sum_float[T: FloatType](items: &list[T], start: T) -> T, prod_float[T: FloatType](items: &list[T], start: T) -> TSequential IEEE floating fold; empty lists return the explicit start
sum_iter[I, T: IntType](items: &I, start: T) -> Result[T, OverflowError[T]], prod_iter[I, T: IntType](items: &I, start: T) -> Result[T, OverflowError[T]]Borrowed list, set, range, or dictionary-key integer fold
sum_iter_float[I, T: FloatType](items: &I, start: T) -> T, prod_iter_float[I, T: FloatType](items: &I, start: T) -> TBorrowed floating fold; elements must match the start type
divmod[T: IntType](a: T, b: T) -> Result[(T, T), MathError]Same-type integer floor quotient and remainder; zero divisor is Domain, signed minimum divided by -1 is Overflow

Integer combinatorics report Overflow when the result cannot fit u64. Unlike Python, Plenty has no arbitrary-precision integer type. gcd and lcm take two unsigned inputs; convert signed magnitudes explicitly. Generic helpers specialize against operations in their bodies, with normal language errors for unsupported types. They allocate nothing. Integer helpers return ordinary public Result values even when called inside unchecked:; caller modes do not change a callee’s contract. sum_float uses sequential IEEE 754 addition, with no compensated summation guarantee; any NaN element makes the sum NaN, and overflow may produce infinity. The separate float names avoid requiring impossible integer error handling from float callers.

Combinatorics and divmod handle local arithmetic results and preserve their established MathError contract. The infallible unsigned gcd remainder and isqrt loop use explicit unchecked: regions with bounds documented in source. These are proven-bounded operations, not wrapping arithmetic.

Collection helpers borrow their inputs and preserve them. For f32 and f64, min, max, min_all, and max_all follow IEEE 754-2019 minimum and maximum: -0.0 orders below 0.0, so min(0.0, -0.0) is -0.0 and max(-0.0, 0.0) is 0.0, and any NaN input makes the result NaN (Some(NaN) for the list helpers), regardless of its position. The NaN returned is the first NaN input. The pipeline min() and max() terminals share this rule. Other types use ordinary comparisons and retain the first of equal values. Tuple elements retain their existing lexicographic total order, including the ordering of NaN fields; they are not scalar NaNs. Dictionary folds visit keys in insertion order; set order is unspecified. The fold start is always explicit and determines the result type. Generators and custom iterator protocols are not borrowed iterables; strings have nonnumeric elements.

Runtime boundary and current limits

Only numeric methods and constants are compiler/runtime primitives. Platform transcendentals, cancellation-sensitive expm1/log1p, and IEEE sign operations need accurate scalar operations that ordinary arithmetic cannot substitute for. The existing numeric-method boundary exposes these to f32, f64, and generic FloatType functions, so user libraries can use them too. Algorithms, predicates, conversions, tolerance comparisons, guards, and error construction remain Plenty source. Generic arithmetic and integer combinatorics need no runtime additions.

This first module is inspired by CPython’s Doc/library/math.rst and Modules/mathmodule.c, particularly its domain/overflow policy and isclose. It is not the complete Python math API. Missing operations include hypot, dist, fsum, sumprod, frexp, ldexp, nextafter, ulp, remainder, fma, special functions, and newer pi-scaled trigonometry. Accurate summation and bit-level float decomposition need separate contracts and tests; simple arithmetic substitutes must not imply Python’s numerical guarantees. Arbitrary constant expressions and custom iterator protocols remain language design questions. % remains integer-only; use fmod or divmod_float for floats. Powers use .pow() or math.pow; ** is rejected with a diagnostic naming .pow(). Remaining scheduling belongs in the backlog.

Acknowledgements

  • API modelled on Python’s math, including CPython’s domain and overflow policy.

std.mime

import std.mime parses, builds, formats and compares media types such as text/html; charset=utf-8, and maps file extensions to a Content-Type. The module is checked Plenty in stdlib/std/mime.plenty and links no native code. std.http.files uses its extension table. The signature index lists the complete API.

import std.mime
from std.mime import MimeError

def main() -> Result[(), AllocError | IoError | MimeError]:
    value = std.mime.parse("Text/HTML; Charset=\"UTF-8\"; title=\"a \\\"b\\\"\"")?
    print(value.to_string()?)?
    print(value.type_name(), value.subtype(), value.param("charset"), value.param("TITLE"))?
    print(value.matches(&std.mime.parse("text/*")?), value.matches(&std.mime.parse("image/*")?))?
    mut upload = std.mime.new("multipart", "form-data")?
    upload.set_param("boundary", "x7 y")?
    print(upload.to_string()?)?
    print(std.mime.from_file_name("logo.SVG"), std.mime.from_extension("woff2"))?
    Ok(())
text/html; charset=utf-8; title="a \"b\""
text html Option[str].Some("utf-8") Option[str].Some("a \"b\"")
True False
multipart/form-data; boundary="x7 y"
image/svg+xml Option[str].Some("font/woff2")

Media types

ItemMeaning
parse(text)Result[MediaType, MimeError]: the media type in text
new(type_name, subtype)Result[MediaType, MimeError]: a media type without parameters; both names must be tokens
value.type_name(), value.subtype()&str: the lowercase names
value.params()&list[Param]: parameters in order; each Param has name and value
value.param(name)Option[str]: the value of the parameter name, compared without case
value.essence()Result[str, AllocError]: type/subtype without parameters
value.suffix()Result[Option[str], AllocError]: the structured syntax suffix after the last +, such as json in application/ld+json
value.is_wildcard()bool: the type or subtype is *
value.matches(&pattern)bool: value falls within the media range pattern
value.set_param(name, value)Result[(), MimeError]: add a parameter, or replace the value of one with the same name
value.remove_param(name)bool: remove the parameter name; False when absent
value.to_string()Result[str, AllocError]: canonical text

MimeError has two variants: Invalid, for text that is not a media type or a name or value that is not allowed, and Allocation(AllocError).

parse follows the media-type rule of RFC 9110 section 8.3.1:

media-type = type "/" subtype *( OWS ";" OWS [ parameter ] )
parameter  = token "=" ( token / quoted-string )

Spaces and tabs are allowed around the whole text and around each ;, and empty parameters such as a/b;;x=1 are skipped. A quoted-string may hold tab, space, visible ASCII and non-ASCII text, and \ escapes the next character; the stored value is unquoted and unescaped. Everything else returns MimeError.Invalid: a missing or empty type or subtype, a character outside a token, a parameter without = or value, an unclosed quote, text after a closing quote, and a parameter name given twice (compared without case, as RFC 6838 section 4.3 requires).

Type, subtype and parameter names are case-insensitive, so they are stored in lowercase. Parameter values keep their case, except charset, whose values are case-insensitive and are stored in lowercase. == therefore treats Text/HTML;Charset=UTF-8 and text/html; charset=utf-8 as equal. It still compares parameters in order, so a/b; x=1; y=2 and a/b; y=2; x=1 differ.

to_string writes type/subtype, then ; name=value for each parameter in order. A value that is empty or not a token is written as a quoted-string, with " and \ escaped. Parsing formatted text gives an equal value.

set_param requires a token name and a value without control characters other than tab, or returns MimeError.Invalid and leaves the value unchanged. It lowercases the name, and the value of charset.

Media ranges

value.matches(&pattern) reads pattern as a media range, as in an Accept header: */* matches every type, text/* every text subtype, and any other type and subtype must be equal. Each parameter of pattern must appear in value with the same value; extra parameters of value are ignored. So text/html; charset=utf-8 matches text/* and text/html; charset=utf-8, but not text/html; level=1.

std.mime does not parse whole Accept headers: splitting the list and reading q weights belongs to the HTTP layer.

File extensions

ItemMeaning
from_extension(extension)Option[str]: the Content-Type for an extension given without its dot, or Nothing
from_file_name(name)str: the Content-Type for the extension after the last . of name, or application/octet-stream

Both compare without ASCII case and allocate nothing. A name with no ., such as html, has no extension; .html has the extension html. The table is fixed:

ExtensionType
html, htmtext/html; charset=utf-8
csstext/css; charset=utf-8
js, mjstext/javascript; charset=utf-8
jsonapplication/json
txttext/plain; charset=utf-8
csvtext/csv; charset=utf-8
xmlapplication/xml
svgimage/svg+xml
png, jpg, jpeg, gif, webp, icoimage/png, image/jpeg, image/gif, image/webp, image/vnd.microsoft.icon
wasmapplication/wasm
pdfapplication/pdf
woff2, wofffont/woff2, font/woff

The result is Content-Type text, ready for a response header; pass it to parse for a MediaType.

Allocation

parse copies the text once into a byte buffer and allocates the names, values and parameter list of the result. new, set_param, essence, suffix and to_string allocate their results. Every allocation failure returns AllocError and leaves existing values unchanged. to_string checks its fragment-list capacity before allocating; an unrepresentable capacity returns AllocError.CapacityOverflow. Accessors, param, matches, is_wildcard, remove_param and the extension lookups allocate nothing.

Limits

  • No length limit: the caller bounds input, as HTTP header limits do.
  • No content sniffing, and no lookup from a media type back to extensions.
  • The extension table is the fixed list above, not the full IANA registry.

Acknowledgements

std.net

import std.net provides TCP listeners and streams, UDP sockets, IPv4 and IPv6 addresses, and name lookup. Its deadline operations take a Deadline and a cancellation token and wait through std.ready. Immediate TCP operations make one nonblocking attempt; the plain std.io methods wait without a deadline. There is no required event loop or async syntax. Use deadline operations to keep a worker’s waits bounded.

The runtime supplies one primitive per socket call (opcodes 410–418). Selectors and cancellable waits come from std.ready (opcodes 419–422). Addresses, waits, retries and error naming are checked Plenty in stdlib/std/net.plenty.

import std.bytes
from std.io import StreamError
from std.net import NetError, TcpListener, connect, listen, loopback_v4
from std.sync import CancellationToken
from std.time import Deadline, deadline_after, milliseconds, monotonic_now

def greet(listener: &TcpListener, deadline: &Deadline, cancel: &CancellationToken) -> Result[(), AllocError | NetError | StreamError | std.bytes.BytesError]:
    mut stream = listener.accept(deadline, cancel)?
    reply = std.bytes.from_text("hello from the server")?
    stream.write_all_before(&reply, deadline, cancel)?
    stream.close()

def main() -> Result[(), SpawnError[()] | IoError | AllocError | RangeError | NetError | StreamError | std.bytes.BytesError]:
    cancel = CancellationToken()?
    deadline = deadline_after(&monotonic_now(), &milliseconds(5000))?
    listener = listen(&loopback_v4(0u16), 8)?
    address = listener.local_address()?
    with spawn(greet, &listener, &deadline, &cancel)? as server:
        mut stream = connect(&address, &deadline, &cancel)?
        mut buffer = std.bytes.with_capacity(64)?
        buffer.resize(64, 0u8)?
        count = stream.read_before(&mut buffer, 0, 64, &deadline, &cancel)?
        print(buffer.to_text_range(0, count)?)?
        server.join()?
    Ok(())
hello from the server

Addresses

ItemMeaning
IpAddress.V4(bits: u32)An IPv4 address; V4(0x7f000001) is 127.0.0.1
IpAddress.V6(high: u64, low: u64)An IPv6 address as two 64-bit halves in network order
SocketAddress(ip, port: u16, scope: u32)An address and port; scope selects a link-local IPv6 interface and is otherwise 0
ipv4(a, b, c, d)IpAddress from four u8 parts
socket_address(ip, port)SocketAddress with scope 0
loopback_v4(port), loopback_v6(port)127.0.0.1 and ::1
any_v4(port), any_v6(port)0.0.0.0 and ::, which bind every interface of the family
parse_ip(text)Result[IpAddress, NetError]
parse_address(text)Result[SocketAddress, NetError] from 1.2.3.4:80, [::1]:80 or [fe80::1%2]:80
format_ip(ip)Result[str, NetError]
address.to_string()Result[str, NetError], which parse_address reads back

IpAddress is copyable; SocketAddress is a class and moves. Parsing is strict and repairs nothing: IPv4 needs four decimal parts of at most three digits without leading zeros; IPv6 accepts hex groups, at most one ::, and a dotted IPv4 tail. Ports and scopes are decimal. Anything else is InvalidAddress. IPv6 text follows RFC 5952: lowercase, no leading zeros, the first longest run of two or more zero groups written ::, and IPv4-mapped addresses in dotted form (::ffff:192.0.2.1).

Errors

NetError names the outcomes a caller handles differently:

VariantCause
TimedOutThe caller’s deadline passed
ConnectionTimedOutThe kernel gave up on the connection (ETIMEDOUT)
WouldBlockAn immediate TCP operation cannot make progress now (EAGAIN/EWOULDBLOCK); wait for readiness before trying again
CancelledThe token was cancelled
RefusedNothing listens at the address (ECONNREFUSED)
ResetThe peer reset or closed the connection while data was moving (ECONNRESET, EPIPE)
AddressInUseAnother socket is bound to the address (EADDRINUSE)
AddressUnavailableThe address is not local, or IPv6 is unavailable (EADDRNOTAVAIL)
UnreachableNo route to the network or host
NameNotFoundThe resolver found no address for the name
TryAgainThe resolver failed temporarily (EAI_AGAIN), or 32 lookups are already running
HandleLimitThe process or system has no free descriptors (EMFILE, ENFILE)
InvalidAddressAddress text outside the grammar, or an empty or NUL-containing host name
Stream(StreamError)Any other failure, with its native code in StreamError.Io(IoError.System(code)), or allocation failure

TimedOut ends the current wait; it does not mark an established connection as failed, so the caller may retry with another deadline. ConnectionTimedOut is a connection failure: close that stream rather than retrying a wait on it. Both distinctions survive the network-error wrappers in std.tls and the HTTP client.

TCP

Function or methodResult
listen(address: &SocketAddress, backlog: i64)Result[TcpListener, NetError]
listener.accept(deadline, cancel)Result[TcpStream, NetError]
listener.accept_now()Result[TcpStream, NetError]: one nonblocking accept attempt
listener.local_address()Result[SocketAddress, NetError]
listener.close()Result[(), NetError]
connect(address: &SocketAddress, deadline, cancel)Result[TcpStream, NetError]
start_connect(address: &SocketAddress)Result[Connecting, NetError]: starts a connection without waiting
connecting.watch(&mut selector, key), unwatch(&mut selector)Result[(), NetError]: the attempt is ready when it can be finished
connecting.finish_now()Result[(), NetError]: WouldBlock while the attempt continues, then success or why it failed
connecting.stream()TcpStream: consumes a finished attempt
descriptor() on TcpStream, TcpListener, UdpSocket, Connecting, Lookup&File: the handle a selector waits on, for std.ready.Watchable
stream.read_now(buffer, start, stop)Result[i64, NetError]: one nonblocking read attempt into &mut Bytes
stream.write_now(source: &[u8])Result[i64, NetError]: one nonblocking write attempt from a &[u8] view
stream.read_before(buffer, start, stop, deadline, cancel)Result[i64, NetError]: one read; 0 is end of input
stream.write_before(source: &[u8], deadline, cancel)Result[i64, NetError]: offer all of source in one write and return the count accepted; source is a &[u8] view
stream.write_all_before(source: &[u8], deadline, cancel)Result[(), NetError]
stream.read, write, flushThe std.io Reader and Writer methods, returning StreamError on failure; read and write block without a bound
stream.close()Result[(), StreamError], as required by std.io.Closer
stream.shutdown(direction: Shutdown)Result[(), NetError]; Shutdown.Read, Write or Both
stream.set_no_delay(enabled: bool)Result[(), NetError]: send small writes immediately
stream.acknowledge_now()Result[(), NetError]: acknowledge data already received at once, not after the kernel’s delay (Linux TCP_QUICKACK); later data is acknowledged as usual
stream.local_address(), peer_address()Result[SocketAddress, NetError]
  • Port 0 binds a free port; local_address reports it.
  • backlog bounds connections waiting for accept and is clamped to 1..4096; the kernel may round it. Listeners set SO_REUSEADDR, so a port in TIME_WAIT can be reused, but two listeners on one port are AddressInUse. An IPv6 listener accepts only IPv6 connections.
  • accept borrows the listener shared, so several workers can accept from one listener; each connection goes to exactly one of them. A connection reset before it was accepted is skipped.
  • At the descriptor limit, accept returns HandleLimit and the connection stays queued: once the server closes a descriptor, the next accept takes it. A server can back off or shed idle connections instead of failing.
  • Linux queues up to backlog + 1 unaccepted connections (measured on loopback for backlogs 1, 2, 8 and 64). Further connection attempts are dropped, not refused, so connect waits until its deadline and returns TimedOut. The kernel also caps backlog at net.core.somaxconn.
  • connect starts a non-blocking connection and waits for it within the deadline. A refused or unreachable peer is an error value.
  • Accepted and connected descriptors stay nonblocking. read_before and write_before wait for readiness, attempt the transfer, and retry EAGAIN or interruption against the original deadline and cancellation token. A stale readiness report or a full send buffer cannot turn that transfer into an unbounded kernel wait. Plain read and write retain their blocking std.io contract by waiting without a deadline when the socket would block. A read may return fewer bytes than were sent, and a write may accept fewer than offered: loop, or use write_all_before.
  • write_before offers all of source without a chunk-size cap. The nonblocking write accepts only what currently fits in the kernel’s send buffer; a successful count is progress, not a guarantee that the whole view was sent. Pass &buffer[a:b] to write part of a buffer; view bounds clamp.
  • Writing to a connection the peer has closed is Reset, never SIGPIPE.
  • On TimedOut or Cancelled the socket and the caller’s buffer are unchanged, apart from what earlier steps of write_all_before sent.

Closing a stream

TcpStream.close deliberately returns StreamError: its signature satisfies the std.io.Closer protocol. Together with Reader and Writer, this lets a TCP stream work with generic I/O helpers, including close_buffered, which flushes a BufWriter[TcpStream] and closes its underlying stream. Changing close to return NetError would break that protocol contract. Listeners and UDP sockets do not implement Closer; their close methods return NetError.

A function that uses deadline operations and explicitly closes a TCP stream can propagate both error types with NetError | StreamError, as greet does above. Keeping both types preserves the original close failure. Dropping a stream also releases its descriptor, but cannot report a close error.

Waiting on many sockets

stream.watch(selector, key, interest), listener.watch(selector, key) and socket.watch(selector, key, interest) register a socket with a std.ready selector; unwatch(selector) removes it. A listener is readable while a connection waits to be accepted. After selector.wait, each Event names a ready socket by its key. Use accept_now, read_now or write_now to attempt the operation without a second readiness wait. These methods also work without a prior selector wait.

Each immediate method makes at most one socket operation. It never polls, waits for readiness, or retries interruption. WouldBlock means that the accept queue is empty, no input is available, or the send buffer is full. It remains possible after a readiness report; return to the selector so other connections can make progress. No bytes move on WouldBlock. An interrupted attempt returns Stream(StreamError.Interrupted), and an aborted accept returns the native Stream(StreamError.Io(IoError.System(103))) (ECONNABORTED) instead of silently taking another connection. The caller decides when to retry.

A successful read or write may transfer only a prefix of the requested range. Keep the write offset and unsent bytes between events; subscribe to write readiness only while output remains. Zero from a nonempty read range is EOF, not WouldBlock. Empty ranges return zero. Ranges are checked before I/O: 0 <= start <= stop <= buffer.len() for reads, or source.len() for writes; invalid ranges return Stream(StreamError.OutOfRange) and transfer nothing.

Immediate methods take no deadline or cancellation token. Apply the server’s deadline and cancellation policy to its selector waits and between bounded batches of attempts. The deadline-based methods remain useful when one worker can wait for that connection alone.

mut chosen = selector(64).unwrap()
listener.watch(&mut chosen, 0u64)?
# ... accept connections, then stream.watch(&mut chosen, key, Interest.Read)
count = chosen.wait(&deadline, &cancel)?
for index in range(count):
    key = chosen.event(index)?.key

The selector tutorial demonstrates reading the reported connection and handling stale readiness.

UDP

Function or methodResult
bind_udp(address: &SocketAddress)Result[UdpSocket, NetError]
socket.send_to(source: &[u8], target: &SocketAddress, deadline, cancel)Result[i64, NetError]; source is a &[u8] view
socket.receive_from(buffer, start, stop, deadline, cancel)Result[Datagram, NetError]
socket.set_broadcast(enabled: bool)Result[(), NetError]
socket.local_address(), close()As for TCP

A Datagram has public fields size (the datagram’s full length), stored (the bytes written into buffer[start:stop]) and source, and a truncated() method. A datagram longer than the buffer is truncated, never split; the rest is discarded. A datagram too large to send is Stream(StreamError.Io(IoError.System(90))) (EMSGSIZE).

Name lookup

resolve(host: &str, port: u16, deadline: &Deadline, cancel: &CancellationToken) returns Result[list[SocketAddress], NetError]. An IP literal returns that address without a lookup. Otherwise the system resolver (getaddrinfo) is called once and its addresses are returned in its order, each with port. No address is NameNotFound.

The C resolver cannot be interrupted, so the lookup runs on a helper thread with a 1 MiB stack, which writes its result to a socket pair. resolve waits on the reading end within the deadline and the token, so it returns TimedOut or Cancelled on time even while the resolver is still working. An abandoned lookup finishes on the resolver’s own timeouts (/etc/resolv.conf), then discards its result and exits. At most 32 lookups run at once in a process, counting abandoned ones; another is TryAgain. start_lookup(host: &str, port: u16) returns Result[Lookup, NetError]: the same lookup without waiting. lookup.watch(&mut selector, key) reports it ready when the reply arrives, and lookup.finish_now() returns WouldBlock until then and the addresses or error afterwards. Dropping an unfinished Lookup abandons it. std.actor.net uses start_connect and start_lookup so actors connect and resolve without blocking their stage.

The resolver’s result list is allocated by the C library and freed on the helper thread. Test builds with allocation checks wait at exit for every helper thread, abandoned ones included, to free its memory.

Effects and ownership

Opening a socket and resolving a name have the Network effect; operations on an existing socket have the Handle effect. Both are allowed on workers; see native effects on worker threads. A listener, stream or socket owns one descriptor, opened close-on-exec, and closes it when explicitly closed or dropped. Only resolve and start_lookup start a thread, as above.

Example

The tutorial lesson serves an echo connection on a worker and handles a refused connection.

Address arithmetic is bounded by validated record ranges, fixed-width words and decimal/hexadecimal digit counts. Resolver reply-buffer growth checks its next extent before allocation or consuming input; an unrepresentable extent is Stream(StreamError.Allocation(AllocError.CapacityOverflow)). These internal checks preserve the public NetError and stream contracts regardless of the caller’s arithmetic mode.

Limits

  • x86_64 Linux GNU only; elsewhere every operation reports Stream(StreamError.Io(IoError.Unsupported)).
  • No socket timeouts beyond deadlines, multicast, or socket options other than no-delay and broadcast. TLS is in std.tls.
  • An abandoned name lookup keeps its helper thread until the resolver gives up.

Unix-domain sockets

std.net does not provide Unix-domain sockets, by decision rather than as an omission. A path socket leaves a file that someone must remove, and a stale one makes the next bind fail, so it needs a cleanup policy that ownership of the socket alone does not give. The abstract namespace avoids files but is Linux-only. Their distinctive features, passing descriptors and peer credentials, would move ownership of native handles between processes, which the ownership rules do not yet describe. Local communication with a child is covered by std.process pipes, and with another local process by TCP on loopback. Revisit this when a scheduled use case, such as talking to a system daemon, needs them.

std.path — native paths

Import std.path. Path owns an affine list[u8], preserving exact Linux path bytes, including invalid UTF-8, spaces, tabs, CR and LF. It is not a compact byte buffer or a Windows path abstraction. from_bytes(bytes) transfers the list without allocating. from_text(&text) explicitly encodes UTF-8 and returns Result[Path, IoError]. NUL can occur in a lexical Path; filesystem operations reject it with IoError.InvalidInput.

import std.path as path

def main() -> Result[(), AllocError | IoError]:
    base = path.from_text("data/reports")?
    child = path.from_text("2024/summary.csv")?
    full = base.join(&child)?
    print(full.to_string()?, full.is_absolute())?
    print(full.file_name()?.ok_or(IoError.InvalidInput)?.to_string()?)?
    print(full.parent()?.ok_or(IoError.InvalidInput)?.to_string()?)?
    print(len(full.components()?))?
    raw = path.from_bytes([110u8, 111u8, 255u8]?)
    print(raw.to_string()?, raw.to_text().is_err())?
    Ok(())
data/reports/2024/summary.csv False
summary.csv
data/reports/2024
4
no\xff True
OperationResult
path.as_bytes()Shared borrow of the existing list[u8]; no allocation
path.to_text()Checked Result[str, IoError]; never replacement-decodes
path.to_string()Escaped ASCII Result[str, IoError] for diagnostics
path.copy()Detached Result[Path, AllocError]
path.join(&child)Result[Path, IoError]; absolute child replaces the base
path.is_absolute()Whether its Linux spelling begins with /
path.parent(), path.file_name()Result[Option[Path], IoError]
path.components()Result[list[list[u8]], IoError] with lexical components

Lexical operations use Rust’s Unix Path rules. Components remove redundant separators and interior . components; an initial relative . is retained. They preserve .., including across a symlink. Joining preserves the base’s spelling and relative root; it does not canonicalize, resolve links, test existence, or change the working directory. A root has no parent or filename; a has the empty relative path as its parent. All owned output buffers are fallible. OS ABI details stay private.

Display is explicit: to_string() preserves printable ASCII, escapes backslash and double quote, and emits other bytes as \xhh. This representation keeps invalid UTF-8 and control bytes visible without replacement decoding; it is not an input parser. to_text() validates, while as_bytes() allows exact binary transport. Automatic record formatting (print(path), str.repr(path)) is diagnostic list formatting, not a round-trippable filesystem spelling. The method is to_string() rather than display() because print and f-strings call a display method only in its formatter form. Use NUL framing with binary transport when passing arbitrary Linux paths.

The Rust implementation baseline is library/std/src/path.rs; Python’s PurePath/Path separation motivates keeping lexical operations distinct from filesystem observations. Broader buffers and portable path representations remain B12/B39 candidates; issue #76 owns compact buffers/views. Closed issue #61 records the initial raw transport, and issue #49 records the catalogue’s decomposition rather than implementation.

Acknowledgements

std.process

import std.process starts child processes from an argument vector and owns them until they are reaped. Arguments, variables, and directories are operating-system byte strings; pipes carry std.bytes data.

No shell takes part unless the program is one. An argument reaches the child exactly as given: spaces, quotes, $, ;, |, *, and newlines are ordinary bytes. shell is the one, named way to run shell syntax. std.shlex can split a shell-style argument string before passing each resulting word to arg; it does not execute or expand shell syntax.

Request encoding, pipe draining, status decoding, and the reaping rules are checked Plenty. Five native primitives spawn, hand over a pipe end, wait, signal, and poll. The module is available on the supported x86-64 Linux GNU target; elsewhere spawn reports IoError.Unsupported.

import std.process
from std.bytes import BytesError
from std.io import StreamError
from std.process import Stdio

def main() -> Result[(), AllocError | IoError | StreamError | BytesError]:
    mut echo = std.process.command("echo")?
    echo.arg("hello from a child")?
    echo.stdout(Stdio.Piped)
    mut child = echo.spawn()?
    output = child.output(4096)?
    print(output.status.success(), output.stdout.to_text()?.strip()?)?
    Ok(())
True hello from a child

Building a command

FunctionResultContract
command(program: &str)Result[Command, IoError]A command with no arguments
command_bytes(program: &list[u8])Result[Command, IoError]The same for a name that is not UTF-8
shell(script: &str)Result[Command, IoError]/bin/sh -c script

A program name containing / is a path. Any other name is looked up in the directories of the child’s PATH. The name is also the child’s argv[0].

shell passes script to /bin/sh as one argument. The script is shell syntax, so text placed in it is interpreted: never build one from untrusted input. Pass such data as an argument or a variable instead.

Command methods take self: &mut Command:

MethodResultEffect
arg(value: &str)Result[(), IoError]Append one argument
arg_bytes(value: &list[u8])Result[(), IoError]Append one argument of any bytes
arg_path(value: &Path)Result[(), IoError]Append a path losslessly
env(name: &str, value: &str)Result[(), IoError]Set a variable for the child
env_bytes(name: &list[u8], value: &list[u8])Result[(), IoError]The same for any bytes
env_remove(name: &str), env_remove_bytes(name: &list[u8])Result[(), IoError]The child does not inherit it
env_clear()()The child inherits nothing; earlier edits are dropped
current_dir(path: &Path)Result[(), IoError]The child’s working directory
stdin(mode: Stdio), stdout(mode: Stdio), stderr(mode: Stdio)()Choose a stream
  • A child inherits the parent’s environment and working directory unless the command says otherwise. Edits apply in order, so the last one for a name wins. Nothing changes in the parent.
  • A piece containing a NUL byte, an empty program or directory, and a variable name that is empty or contains = are IoError.InvalidInput. A failed method leaves the command exactly as it was.
  • An empty argument is valid and is passed as an empty argument.
  • Allocation failure is IoError.Data(DataError.Allocation(...)). Unrepresentable argument, environment and combined request sizes preserve AllocError.CapacityOverflow in that cause, before mutation or starting a child.
  • Appending a terminated byte string reserves only its new bytes and NUL, not bytes already held. Existing capacity can satisfy the operation without an allocation; failed growth leaves the original command unchanged.

Stdio selects each standard stream:

ModeThe child’s stream
Stdio.InheritThe parent’s own stream. The default
Stdio.Null/dev/null
Stdio.PipedA new pipe whose other end the Child owns

Spawning

command.spawn() borrows the command and returns Result[Child, IoError]. A command is reusable: each call starts another child.

A program that cannot be started is an error value, not a child that exits with a conventional code: a missing program or directory is IoError.System(2), a file without execute permission IoError.System(13). A failed spawn leaves no process and no open pipe.

Pipe ends are close-on-exec, so one child never inherits another child’s pipes. Other descriptors the program opened are close-on-exec as well.

Close-on-exec still leaves a short window: a child forked by one thread holds a copy of every descriptor open at that moment until its exec closes them. If another thread had a program open for writing then, running that program just after closing it fails with “text file busy” (IoError.System(26)). spawn retries that error after waits of 1, 2, 4, 8, 16, 32 and 64 ms, then reports it, so a busy program delays spawn by at most 127 ms. Spawns in one program take turns, and a retrying spawn keeps its turn while it waits, so no new child can pick up a descriptor meanwhile; other threads’ child operations other than a blocking wait wait with it. spawn takes no deadline or cancellation token. Waiting allocates nothing; each attempt repeats the brief host allocation described under Limits. A program another process keeps open for writing still fails after the last wait.

Child

A Child is the single owner of one process. It is not copyable.

MethodResultContract
id()i64The process id
take_stdin(), take_stdout(), take_stderr()Option[File]Move a pipe end out, once
close_stdin()Result[(), IoError]End the child’s input
wait()Result[ExitStatus, IoError]Close stdin, block until the child ends, reap it
try_wait()Result[Option[ExitStatus], IoError]Reap if ended; Nothing while running
wait_timeout(budget: &Duration, cancel: &CancellationToken)Result[Option[ExitStatus], WaitError]Nothing when the budget ran out
signal(number: i32)Result[(), IoError]Send signal 1 to 64
terminate()Result[(), IoError]SIGTERM: a request the child may ignore
kill()Result[(), IoError]SIGKILL: cannot be caught
communicate(input: &[u8], limit: NonNegative[i64])Result[Output, StreamError]Feed stdin, collect both outputs, wait
output(limit: NonNegative[i64])Result[Output, StreamError]communicate with no input

Methods that reap or move a pipe take self: &mut Child; id and the signal methods borrow.

Pipes

A taken pipe end is an ordinary File: stdin is write-only, stdout and stderr are read-only, and none can seek. Use Bytes.read_file and Bytes.write_file directly, or wrap the end with std.io.from_file for the retry helpers and buffering.

  • A read returns the bytes that have arrived, possibly fewer than asked, and 0 at end of input, which is when the child and every process that inherited the stream have closed it.
  • A write to a child that no longer reads is IoError.System(32) (EPIPE). The program is not sent SIGPIPE for it.
  • The child sees end of input only when stdin is closed. wait and communicate close a stdin the Child still holds; a taken end is the caller’s to close.

Reading one pipe to the end while the child blocks writing another, or writing a large input while the child blocks writing output, is a deadlock: a pipe holds 64 KiB. Use communicate, which serves all three in one loop.

Waiting and status

ExitStatus describes how the child ended:

MethodResultContract
success()boolExit code 0
code()Option[i32]The exit code, 0 to 255; Nothing if a signal ended it
signal()Option[i32]The terminating signal; Nothing if it exited
  • The first successful wait, try_wait, or wait_timeout that sees the child ended reaps it. The status is kept, and later calls return it without asking the operating system.
  • wait blocks without a limit. It does not drain pipes: a child blocked writing to a full pipe that nobody reads never ends.
  • wait_before(deadline: &Deadline, cancel: &CancellationToken) polls try_wait with pauses of at most 10 ms. wait_timeout constructs one total deadline from a duration and delegates to it. Both leave stdin open and the child running when cancelled or timed out. WaitError.Cancelled only ends this wait; it never sends a signal. WaitError.Io preserves operating-system errors; an overflowing budget is WaitError.Io(IoError.InvalidInput).
  • Signals reach only a child that has not been reaped. Afterwards the methods return Ok(()) and send nothing, so a number can never reach an unrelated process that received the same id. A number outside 1 to 64 is IoError.InvalidInput.
  • A signalled child still has to be waited for.

Collecting output

communicate writes input to a piped stdin, appends piped stdout and stderr to two new buffers as data arrives, closes each stream at its end, then waits. Output has public fields status: ExitStatus, stdout: Bytes, and stderr: Bytes.

  • Streams that are not piped, or whose end was taken, are skipped and their buffers stay empty.
  • limit bounds each output separately. A stream that produces more is StreamError.OutOfRange; at most limit + 1 of its bytes were buffered. The child is then still running and still owned: kill or wait for it.
  • A child that stops reading ends the feed without an error. Its status says what happened.
  • Interrupted calls are retried. Other failures are the StreamError of the failing transfer.
  • Memory is the two outputs, grown in 8 KiB steps. Input is written in 4 KiB pieces and is not copied. A next-buffer extent that cannot fit i64 returns StreamError.Allocation(AllocError.CapacityOverflow) before growing or reading that chunk, including when the extra limit-probe byte is unrepresentable.

Readiness flags and process identifiers use bit masks. Argument counts, status subtraction and pipe cursors use explicit arithmetic opt-outs justified by successful storage growth, validated statuses and concrete file progress. Caller arithmetic modes do not change these boundaries.

Ownership and reaping

Plenty has no user destructors, so dropping a Child closes the pipe ends it still holds and does nothing else: the process is neither stopped nor reaped. Its runtime record, and its zombie once it ends, last until the program exits. A build with runtime allocation checks reports that record as a leak. Every child should end in a wait, try_wait, or wait_timeout that returns a status.

A child that outlives the program is adopted by the system as usual.

Limits

  • Linux x86-64 GNU only. Starting a child uses the host allocator briefly for its argument vector; that one allocation is outside fallible accounting.
  • No redirection of a stream to a file or to another child, and no merging of stderr into stdout. shell can do both.
  • Timed child waits poll; scheduling and lock contention can delay return beyond the 10 ms polling interval.
  • No process groups, sessions, user or group changes, resource limits, or detached children.
  • communicate holds whole outputs in memory. For streaming, take the pipes and give each its own thread.

Acknowledgements

std.random

import std.random provides deterministic pseudo-random generators that the caller creates, seeds, owns, and passes explicitly, with one error type, RandomError.

The module is checked Plenty code with no native primitives. It never reads the operating system, the clock, or any process-wide state: there is no global generator and no automatic seeding. It is available on the supported x86-64 Linux GNU target.

A generator’s output is predictable from its seed, and from a few outputs. Do not use it for keys, tokens, passwords, or anything an adversary must not guess; use std.secrets. Collection hashing is unrelated to both modules.

import std.random
from std.random import RandomError

def main() -> Result[(), AllocError | IoError | RandomError]:
    mut rng = std.random.seeded(2026u64)
    print(rng.between(1, 7)?, rng.between(1, 7)?, rng.next_bool())?
    mut deck = ["A", "B", "C", "D", "E"]?
    rng.shuffle(&mut deck)
    print(deck)?
    print(rng.sample_indices(10, 3)?)?
    mut replay = std.random.seeded(2026u64)
    print(replay.between(1, 7)?)?
    Ok(())
6 5 True
["D", "A", "E", "C", "B"]
[9, 7, 1]
6

Generators

FunctionResultContract
seeded(seed: u64)RngExpand the seed into a full state; every seed is valid
from_state(a: u64, b: u64, c: u64, d: u64)Result[Rng, RandomError]Restore a state taken from state(); all zeros is ZeroState

Rng has private fields, so these two functions are the only way to make one. To seed from the operating system, pass std.secrets.random_u64() to seeded and record the seed if the run must be repeatable.

Methods that draw take self: &mut Rng.

MethodResultContract
next_u64()u64One step; all 64 bits are uniform
next_u32()u32The high half of one step
next_bool()boolThe top bit of one step
next_f64()f64A multiple of 2^-53 in [0, 1)
below(bound: u64)Result[u64, RandomError]Uniform in [0, bound); zero is EmptyRange
between(low: i64, high: i64)Result[i64, RandomError]Uniform in [low, high); low >= high is EmptyRange
index(count: i64)Result[i64, RandomError]Uniform index into count elements; zero is EmptyRange, negative traps
shuffle[T](items: &mut list[T])()Uniform permutation in place
sample_indices(count: i64, amount: i64)Result[list[i64], AllocError]amount distinct indices below count, in selection order; a negative count or amount outside 0..=count traps
fill(target: &mut Bytes, start: i64, stop: i64)()Overwrite target[start:stop]
state()(u64, u64, u64, u64)The current state words; takes &Rng

Ranges are half-open and never clamp. between covers any ordered pair of i64 bounds, including i64.MIN to i64.MAX. fill follows the range rule. An argument that no call could accept, such as a negative count, is a caller bug and traps with a message naming the method; an empty range is EmptyRange because it can come from data, such as an empty list.

below, between, and index are unbiased: values of a step that would make some results more likely are rejected and another step is drawn. A draw takes fewer than two steps on average for every bound.

shuffle is Fisher-Yates. It exchanges elements with swap, so it works for any element type and never copies or drops an element. To pick k of n elements, call sample_indices(n, k) and index the collection.

A rejected request (EmptyRange or a failed allocation) does not advance the generator. Lists of fewer than two elements and empty fill ranges also consume no steps.

Algorithm and stability

Rng is xoshiro256** 1.0: 256 bits of state and a period of 2^256 - 1. seeded fills the state with four successive SplitMix64 outputs, as the algorithm’s authors recommend.

The sequences are version-stable. For a given seed or state, every method above returns the same values in every Plenty release, on every thread and run. The derived methods are part of that contract:

MethodDefinition
next_u32next_u64() >> 32
next_boolBit 63 of next_u64()
next_f64(next_u64() >> 11) * 2^-53
below(bound)Draw next_u64() until the value is at least 2^64 mod bound; return it modulo bound
between(low, high)low + below(high - low), computed in u64
shuffleFor last from len - 1 down to 1: swap last with below(last + 1)
sample_indicesOver the list 0..count, for position from 0 below amount: swap position with position + below(count - position); return the first amount
fillEach step supplies eight bytes, least significant first; a final partial group takes the low bytes of one more step

A different algorithm would be a new constructor, not a change to seeded.

Errors

pub enum RandomError:
    EmptyRange
    ZeroState
VariantMeaning
EmptyRangeThe requested range holds no value
ZeroStatefrom_state was given the all-zero state, which never advances

Bounded draws validate NonZero[u64] before rejection sampling. Generator mixing still uses explicit wrapping operations; list/sample/fill cursors use documented bounds-proven arithmetic opt-outs. Neither changes the version-stable draw sequence, failure-state guarantees or public error types.

Ownership and threads

A generator is an ordinary owned value of four u64 words stored inline. Drawing needs an exclusive loan, so two parts of a program cannot interleave draws from one generator by accident; give each its own, seeded differently. The module holds no global state, passes the ordinary worker-effect checks, and a generator can be moved into a worker thread. There is no built-in way to derive independent streams from one seed: use distinct seeds.

Costs

OperationAllocatesTime
seeded, from_state, stateNothingConstant
next_u64, next_u32, next_bool, next_f64, below, between, indexNothingConstant; expected constant for the bounded draws
shuffleNothingLinear, one bounded draw per element
fillNothingLinear, one checked Bytes.set per byte
sample_indices(count, amount)One list of count indicesLinear in count

sample_indices is proportional to count, not amount. Throughput has not been measured on an idle machine.

Limits

  • One algorithm. No jump or split functions for parallel streams.
  • Normal, exponential and floating uniform distributions are provided by std.stat, using an explicitly supplied Rng. Weighted choice is not provided.
  • No f32 draws and no inclusive ranges.
  • sample_indices returns indices; there is no element-sampling or reservoir form.

Acknowledgements

  • Uses the xoshiro256** generator by David Blackman and Sebastiano Vigna, seeded with SplitMix64 by Guy Steele, Doug Lea and Christine Flood.

std.stat

import std.stat supplies descriptive statistics over &list[f64] and normal, uniform and exponential distributions. It is checked Plenty code; there is no native statistics dependency or global random state.

import std.random
import std.stat
from std.stat import StatError

def main() -> Result[(), AllocError | IoError | StatError]:
    values = [2.0, 4.0, 4.0, 4.0, 5.0, 5.0, 7.0, 9.0]?
    print(std.stat.mean(&values)?, std.stat.median(&values)?, std.stat.pstdev(&values)?)?
    print(std.stat.quantiles(&values)?)?
    scores = std.stat.normal(100.0, 15.0)?
    print(scores.cdf(100.0)?, scores.inverse_cdf(0.5)?)?
    mut rng = std.random.seeded(42u64)
    draw = scores.sample(&mut rng)?
    print(draw > 0.0 and draw < 200.0)?
    Ok(())
5.0 4.5 2.0
[4.0, 4.5, 5.5]
0.5 100.0
True

Observations

All observations must be finite. Empty data returns StatError.Empty, and NaN or infinity returns NonFinite. Convert integer observations explicitly to f64; integers beyond 2^53 may lose precision during conversion.

FunctionMeaningAllocation
mean(values)Arithmetic meanNone
pvariance(values), pstdev(values)Population variance/deviation, denominator nNone
variance(values), stdev(values)Sample variance/deviation, denominator n-1; needs two observationsNone
median(values)Middle value, or midpoint of the middle pairCopy and sort
quantile(values, p)Inclusive linear interpolation, 0 <= p <= 1Copy and sort
quantiles(values, n: Positive[i64] = 4)The n-1 cuts at i/nCopy/sort plus output list

Quantiles use Hyndman–Fan type 7: interpolate at sorted index p*(length-1). This corresponds to Python’s statistics.quantiles(method="inclusive"), not its default exclusive method. A singleton yields its one value at every probability. n=1 produces an empty list after validating the data. The original list is never mutated. Sorting takes O(n log n) time and O(n) additional space; the other functions take O(n) time and constant space.

Mean uses scaled, compensated summation. Deviation uses a corrected two-pass calculation with scaled differences. These avoid many overflow/cancellation problems of summing raw values or squares, but are still approximate f64 calculations, not exact arithmetic. Underflow may round to zero. An unrepresentable result is Overflow; deviation can succeed when variance overflows. A one-element sample returns InsufficientData.

Distributions

Constructors validate parameters and return Result[Distribution, StatError]:

ConstructorParameters
normal(mean=0.0, sigma=1.0)Finite mean and strictly positive finite standard deviation
uniform(low=0.0, high=1.0)Finite ordered bounds; width and reciprocal width must be representable
exponential(rate=1.0)Strictly positive finite rate

All three provide pdf(x), cdf(x), and inverse_cdf(p), returning Result[f64, StatError]. NaN observations are rejected; infinite observations have their usual limiting densities and probabilities. Probabilities must be finite and in [0,1]. Normal inverse endpoints are negative/positive infinity; exponential’s upper endpoint is infinity. Uniform inverse endpoints are its bounds, and its density includes both bounds (irrelevant to integrals). Parameters are private and cannot be changed after construction.

Normal tails use a central power series and Laplace’s continued fraction for the Mills ratio; inversion bisects the smaller tail. Regression checks cover central values, tails through probability 1e-100 and inverse round trips with a 2e-13 relative tolerance. This is a tested tolerance, not a correctly-rounded guarantee for every f64. Upper-tail CDFs eventually round to one; lower tails underflow near 39 standard deviations. Inversion uses 64 iterations of the bounded tail evaluation and is intended for ordinary statistical use, not high-throughput vector analysis. Exponential uses expm1/log1p to retain small-probability precision. Finite-result arithmetic overflow is reported.

Sampling and reproducibility

Pass a mutable std.random.Rng to sample. Uniform returns f64; normal and exponential return Result[f64, StatError] because the transformed value can overflow. An error leaves the generator unchanged. All constructors, queries and sampling operations allocate nothing.

DistributionDraw policy
UniformOne next_f64() draw u; interpolate between bounds
ExponentialOne draw u; -log1p(-u)/rate
NormalTwo draws u,v; Box–Muller mean + sigma*sqrt(-2*log1p(-u))*cos(tau*v); no cached partner

The draw counts and these transformations are version-stable; adding distributions does not change any std.random sequence. Transcendental functions follow the platform math implementation, so samples are reproducible within the same math environment, not promised bit-identical across platforms or libm updates. Rounding can put a uniform sample exactly at the upper bound. No sampling function reads the clock, OS entropy, or a global generator.

Errors and scope

StatError has Empty, InsufficientData, NonFinite, Domain, Overflow and Allocation(AllocError). Invalid distribution parameters and probabilities are Domain; failures to allocate sorted/output lists retain the original allocation cause, including CapacityOverflow for an unrepresentable quantile output allocation. Validated list lengths, positive quantile counts and fixed iteration budgets bound internal integer arithmetic; no primitive arithmetic errors are added to the public contracts. These operations have no hidden retry.

This module does not provide hypothesis tests, regression, weighted statistics, discrete distributions, arbitrary precision or dataframe aggregation. Use std.dataframe for bulk analytical queries.

Algorithm references: Python statistics, statrs distributions, and Lee’s analysis of the Laplace fraction. See the statistics tutorial for examples.

Acknowledgements

std.plot

std.plot draws layered charts as deterministic SVG files or byte streams. It is implemented in Plenty and needs no plotting runtime, R installation, display server, or external shared library. Import it with import std.plot as gg.

This is the core/SVG portion of the plotting proposal, not the complete first milestone or a drop-in replacement for ggplot2. See the plotting tutorial for runnable examples.

import std.plot as gg
from std.plot import Column, PlotError

def main() -> Result[(), AllocError | IoError | PlotError]:
    data = gg.table([
        ("day", Column.Num([1.0, 2.0, 3.0, 4.0]?)),
        ("visits", Column.Num([120.0, 150.0, 90.0, 180.0]?)),
    ]?)?
    chart = gg.ggplot(data, gg.aes("day", "visits"), [
        gg.geom_line(),
        gg.geom_point(),
        gg.labs(title="Daily visits"),
    ]?)?
    notes = chart.save("visits.svg", width=6.0, height=4.0)?
    print(notes)?
    svg = read_text("visits.svg")?
    print(svg.startswith("<svg"), "Daily visits" in svg)?
    Ok(())
[]
True True

Data and ownership

table(list[(str, Column)]) consumes named columns without copying their lists. Columns have equal lengths and unique nonempty names; an empty table has height zero. Column.Num, Int, Text, and Bool contain lists of f64, i64, str, and bool. MaybeNum and MaybeText contain lists of Option[f64] and Option[str]. Table.height() and Table.names() inspect the table.

Table.share() gives another immutable owner of the same columns without allocating. ggplot consumes a table. Plot.plus(components) borrows the original plot and returns an independent specification, sharing its table and copying its mappings and components. Both plots remain usable.

columns(&dictionary) copies a dict[str, list[f64]], retaining its key order. rows(&records, fields) evaluates typed accessors and builds columns. Construct fields with num, integer, text, and flag; each takes a name and an owned callback such as def(row: &Reading) -> f64: row.temperature. These constructors allocate a boxed callback. rows consumes the field list and retains no callbacks or borrowed records. This also supports lists of tuples and derived columns.

Mappings and components

aes(x, y, colour=, color=, fill=, group=, ymin=, ymax=) maps column names, not expressions. colour and color are aliases; supplying both nonempty names is an error. An empty name leaves that mapping unset. Numeric columns are continuous; text and bool columns are discrete. factor("name") makes a numeric column discrete. Numeric factors sort by numeric value; other levels sort by their string value. Int positions and continuous colours convert to f64, with its usual precision limit; factor labels preserve exact integers.

ggplot(table, mapping, components) eagerly checks names, required aesthetics, column kinds, and component settings. Constants belong in a geom, such as geom_point(colour="red"); a colour name inside aes would instead be looked up as a column.

Every geom accepts data=, mapping=, and inherit_aes=True. Omitted data uses the plot table; supplied data is an owned table and can have different columns and a different row count. A layer mapping overrides only its nonempty aesthetics; other aesthetics inherit the plot mapping. Use inherit_aes=False to start with an empty mapping. Empty names do not clear individual inherited aesthetics. Inherited names are looked up in the layer’s table, so a summary table without the plot’s grouping column normally needs inherit_aes=False. Only effective layer mappings are validated.

Position and colour scales train across all layers, so a category has the same position and colour wherever it appears. Mixing discrete and continuous mappings on one scale is an error. plus preserves the original specification and shares the immutable plot and layer tables.

ConstructorBehavior
geom_pointPoints; requires x and y; size is a diameter in millimetres
geom_lineConnects points sorted by x within each group
geom_pathConnects points in input order within each group
geom_colBars with supplied continuous y heights
geom_barCounts rows at each x, per group; omit y
geom_histogramCounts continuous x in equal-width bins; omit y
geom_errorbarSupplied ymin/ymax intervals with caps
geom_linerangeSupplied ymin/ymax intervals without caps
geom_ribbonFilled band between ymin and ymax, sorted by x within groups

See uncertainty and intervals for bound validation, missing-value gaps, grouping, and transformations. These geoms draw supplied bounds; they do not estimate confidence intervals.

Each geom accepts colour/color, alpha, and na_rm; bars also accept fill and position. Lines and interval geoms use linewidth; ribbons also accept fill. Numeric settings accept integer or float arguments. Supported constant colours are black, white, red, green, blue, grey, gray, orange, purple, yellow, pink, brown, navy, transparent, and #RRGGBB or #RRGGBBAA. More R colour names are not yet supported.

Discrete colour/fill mappings default to an eight-colour palette, repeated for additional levels. Continuous mappings default to an RGB gradient from #132B43 to #56B1F7. Legends are automatic. This is not yet ggplot2’s variable-size HCL palette or its continuous colour-bar guide.

Manual palettes and gradients

scale_colour_manual(values) and scale_fill_manual(values) consume a dict[str, str] mapping level labels to colours. For example, gg.scale_fill_manual({"North": "navy", "South": "#D55E00"}?) keeps each region’s colour stable when rows are reordered or a region is absent. Dictionary order does not change the sorted legend order. Every observed level must have an entry; extra entries are allowed. Numeric categories need factor and keys matching their displayed labels. Invalid colours, missing entries, and continuous mappings return errors from ggplot or plus.

scale_colour_gradient(low="#132B43", high="#56B1F7") and scale_fill_gradient(...) set continuous endpoints. Both endpoints must be six-digit #RRGGBB colours, with either hex letter case. Interpolation is in encoded RGB channels, truncating to integer channels; it is not ggplot2’s Lab interpolation. The finite data minimum and maximum map to the endpoints; a constant column maps to the midpoint. Legend swatches use the same mapping. Discrete data with a gradient returns WrongKind.

The scale_color_manual and scale_color_gradient spellings are aliases. The last scale component for each aesthetic wins, regardless of its position relative to geoms. plus can replace a scale without changing the original plot. Colour and fill are independent; different palettes get separate legends even when they map the same column. Constant geom colours override mappings. Missing values are still omitted, rather than given a palette colour. Scale limits, custom breaks, na_value, and palette functions are not yet exposed.

Default groups follow the interaction of discrete mappings; an explicit group replaces this rule. Bar positions use colour/fill groups independently of x. position_identity, position_stack, position_fill, and position_dodge select unadjusted, stacked, proportion, and side-by-side bars. Positive and negative stacks accumulate separately. Stacks follow reverse group order, matching the top-to-bottom discrete legend. Dodge reserves one slot for every group, including when a group is absent at a particular x. Bar width is in data units (default 0.9); automatic numeric resolution-based widths are deferred.

Histogram bins defaults to 30 and may be 1–10,000. Bins span the finite x range; they are left-closed/right-open, except that the final bin includes the maximum. An identical-value input uses a unit-width bin centred on that value. Empty bins do not produce rectangles. This deliberately small binning API does not yet implement ggplot2’s boundary/centre/binwidth alignment or computed statistics. Count/bin layers require discrete colour/fill mappings. If an explicit group combines different mapped colours in one output bar, rendering reports a conflict instead of selecting one arbitrarily.

Rows with missing or non-finite mapped values are omitted and counted in returned notes; na_rm=True suppresses the note. These observation-only diagnostic counts saturate at i64.MAX; semantic row counts, groups, and numerical aggregates do not saturate. A missing y interrupts a line/path within its group. Numeric data magnitudes above 1e100 are rejected during construction.

Labels, coordinates, and themes

facet_wrap and facet_grid split layers into panels, with fixed or free position scales and shared colour legends. See facets for panel ordering, independent statistics and repeated reference layers.

labs(title=, x=, y=, colour=, fill=) overrides nonempty labels; default axis and legend labels come from column names. Later nonempty label settings win.

theme_grey, theme_minimal, and theme_classic select grey-panel, white-grid, and plain-axis styles. They accept base_size in points and legend_position (Legend.Right or Legend.Hidden). A later theme replaces the earlier theme. These are compact styles, not the proposal’s complete theme-element tree.

coord_flip() exchanges the displayed axes. coord_cartesian(xlim=(low, high), ylim=(low, high)) clips and zooms after counting and stacking, preserving statistics. Omit either pair to use its automatic range. Only one coordinate component is allowed. Coordinate limits are expressed in original data units, including on transformed axes. See position scales for transforms, category order, breaks, labels, and the distinction between scale limits and coordinate zoom.

Layout plans text boxes before writing any SVG. Margins adapt to axis labels, titles and a right-hand legend. Titles, tick labels and legend labels wrap at word boundaries; an overlong word splits between Unicode scalars. Explicit newlines start new lines, and other whitespace is normalised. Tick boxes must not overlap. A chart that cannot fit returns InvalidSetting before writing to the supplied writer; increase dimensions, shorten labels, use fewer breaks, or hide the legend. No ticks or legend entries are silently removed.

SVG text names Liberation Sans, Arial, Helvetica, sans-serif; no fonts or font metrics ship. Deterministic character advances define each text box, and SVG textLength with lengthAdjust="spacingAndGlyphs" constrains the displayed width to that box. This may stretch/compress glyphs with a fallback font. The guarantee concerns the planned boxes in conforming SVG viewers, not identical glyph outlines across fonts. Complex-script shaping, font-specific vertical metrics, rotated ticks, and multi-column legends remain unsupported.

Output and failures

plot.save("chart.svg", width=6.0, height=4.0, units=Units.In, dpi=300.0) writes SVG and returns Result[list[str], PlotError]. write_svg(&mut writer, ...) accepts any std.io.Writer and also returns notes. Both borrow the plot, so it can be rendered repeatedly. Units.Cm, Mm, and Px are also supported; dpi converts pixel dimensions to physical dimensions and never rasterises SVG. Dimensions must be between 2 and 100 inches on each axis.

Rendering prepares per-layer rows, writes SVG fragments directly to the writer, and flushes it. save uses a 16 KiB file buffer. It does not collect the complete SVG unless the caller selects a memory cursor. Output uses two-decimal coordinates, stable element order and identifiers, and no timestamps or machine-specific font discovery. Identical inputs with the same toolchain produce identical bytes.

save truncates an existing destination, including before layout preflight. A rendering, allocation, or I/O failure can leave partial output; it is not an atomic file replacement. For transactional publication, write a temporary file and rename it after success. A stream failure is PlotError.Write(completed, cause), where completed counts bytes in the failing fragment, not the entire plot. Flush/close failures use Io.

Other errors are UnknownColumn, DuplicateColumn, ShapeMismatch, MissingAesthetic, WrongKind, Conflict, InvalidSetting, Allocation, and Bytes. No library messages are printed. ggplot catches mapping and setting mistakes; rendering can additionally reject unsupported numeric magnification, XML-forbidden text, or insufficient layout space. Allocation failure is always returned; no infallible native plotting allocator is involved.

Current scope

Not yet implemented: after_stat, statistical smoothing, date scales and other colour scales, full themes and font metrics, PNG/PDF, dataframe input, or record reflection. These remain part of the accepted proposal; priorities are tracked in B80.

Acknowledgements

  • ggplot2 supplies the model for the public names, layered composition, aesthetic mappings, grouping, and stacked-bar semantics. Its source was consulted during implementation; no ggplot2 code or data ships.
  • Hadley Wickham’s A Layered Grammar of Graphics describes the data/mapping/layer/scale/coordinate separation used by the module.

Plot position scales

std.plot trains shared axes across every layer’s effective data and mapping. The last scale component for each axis wins, including components added with plus. Scale names supply axis titles; labs overrides them.

import std.bytes
import std.io
import std.plot as gg
from std.bytes import BytesError
from std.plot import Column, PlotError

def main() -> Result[(), AllocError | IoError | PlotError | BytesError]:
    data = gg.table([
        ("requests", Column.Num([10.0, 100.0, 1000.0, 10000.0]?)),
        ("latency", Column.Num([2.0, 3.0, 5.0, 9.0]?)),
    ]?)?
    chart = gg.ggplot(data, gg.aes("requests", "latency"), [
        gg.geom_point(),
        gg.scale_x_log10(
            name="Requests per second",
            breaks=Some([10.0, 1000.0]?),
            labels=Some(["10", "1k"]?),
        ),
        gg.scale_y_continuous(name="Latency (ms)", limits=(0.0, 10.0)),
    ]?)?
    mut output = std.io.cursor(std.bytes.empty()?)
    drop(chart.write_svg(&mut output)?)
    svg = output.data.to_text()?
    print(">1k<" in svg, ">100<" in svg, "Latency (ms)" in svg)?
    Ok(())
True False True

Continuous axes

scale_x_continuous and scale_y_continuous accept name, breaks (Option[list[f64]]), labels (Option[list[str]]), limits=(low, high), and transform. Transform.Identity, Log10, Sqrt, and Reverse are supported. The scale_{x,y}_log10, _sqrt, and _reverse helpers select those transforms directly.

Breaks and limits use original data units; ticks are positioned using the transform and labelled in original units. Explicit labels require explicit breaks of equal length. Breaks must be distinct, finite and in the transform domain. Omit breaks (or pass Nothing) for automatic ticks; use breaks=Some([1.0, 10.0, 100.0]?) for specific values and breaks=Some([]?) to hide ticks. Supply labels as Some([... ]?). Limits must be finite, increasing and inside the transform domain. Without breaks, ticks use 1/2/5 steps in transformed space and automatic ranges have padding.

Explicit labels allow units, percentages, or date strings. Automatic calendar breaks, date columns and formatting callbacks are not implemented.

Discrete axes

scale_x_discrete and scale_y_discrete accept name, limits (an optional ordered list of distinct level strings), and labels. Pass, for example, limits=Some(["South", "North"]?). Limits set category order, retain named unused categories, and omit rows in unlisted categories. Labels must match the resulting level list. Without limits, levels are sorted as described under mappings; numeric factors keep numeric ordering.

Limits and transformations affect statistics

Continuous scale limits omit out-of-range input values before statistics. Values outside a transform’s domain (for example zero on a log axis) are omitted too, with removal notes unless na_rm=True. Missing/omitted values interrupt lines. Discrete limits similarly exclude unlisted categories.

Histograms bin transformed x values. Supplied bar heights are transformed before stacking; computed counts are transformed before position adjustment. Consequently log-transformed stacks sum logarithms, not original heights; use unstacked bars when that is the intended interpretation. Bar baselines are zero in transformed space (one in original units for log10).

coord_cartesian limits only zoom after these computations and use original data units even with transformed scales. Use coordinate limits when statistics must still include values outside the view. coord_flip exchanges the displayed axes after applying their scales; x scale settings still describe the x mapping.

Plot facets

std.plot can split a plot into labelled panels. Facets use column names, not formulas.

  • facet_wrap("site", ncol=2) wraps one facet variable across a rectangular layout. nrow and ncol default to zero (automatic). If both are supplied, their product must accommodate every panel.
  • facet_grid(rows="region", cols="period") uses one variable per dimension. Either dimension can be omitted. Every row/column combination is retained, including combinations with no observations.

Levels sort using the mapping rules: text lexically, numbers numerically. Missing/non-finite facet values form a separate panel labelled (missing); they are not discarded as missing aesthetics. A literal string "(missing)" remains a distinct category. Headings show the variable name and its value. The last facet component wins, including when added with plus.

Layers and statistics

Facet columns are read from each layer’s effective table, independently of aes. At least one layer must contain all requested facet variables. A layer missing a facet variable repeats across its levels. This makes a reference line without a site column appear in every site panel. A partially specified grid layer repeats only across the missing dimension.

Counts, histogram bins and bar positions are computed separately per panel. Colour/fill scales train over all layers and panels and use one shared legend. Position scales are trained before statistics; their final ranges include panel-specific counts, stacks and interval bounds.

Fixed and free axes

Both constructors accept scales=Scales.Fixed (the default), Free, FreeX, or FreeY.

For wrapped facets, free axes train independently in each panel. In a grid, x scales are shared within each column and y scales within each row, even when free. This retains grid alignment. Explicit scale or coordinate limits still apply to every panel. Discrete scale limits retain their explicit order. Fixed histogram x scales use common bin boundaries; free x scales train their bin ranges independently within each scale group.

coord_flip exchanges displayed axes after scale training. FreeX always refers to the x mapping, so its displayed direction changes under flipping.

Layout and limits

All panel layouts are preflighted before any SVG is written. Panel sizes are equal, with common margins so plotting areas align. Axis titles and ticks repeat in each panel. The plot title and legend appear once. Increase output dimensions when labels or panels cannot fit; panels are not silently omitted.

There is a maximum of 1,000 panels. Multiple variables in one wrap/dimension, custom labellers, explicit facet ordering, proportional panel space, margins panels, and interior-axis suppression are not implemented. To choose an order, use a numeric grouping column and provide its meaning in the surrounding report.

Rendering materialises panel tables with fallible allocations and shares text values; selections containing every row share the original table. Broadcast layers still repeat their prepared geometry. Memory and preparation time therefore grow with the number of panels and repeated observations; this is not a lazy dataframe view. SVG fragments are still streamed.

Plot uncertainty and intervals

std.plot draws caller-supplied lower and upper bounds:

  • geom_errorbar draws a vertical interval with end caps.
  • geom_linerange draws the interval without caps.
  • geom_ribbon fills the area between bounds, sorted by x within each group.

Map bounds with aes(x="time", ymin="lower", ymax="upper"). A y mapping is optional and ignored by these geoms, so they can inherit a centre-value mapping shared with points or a line. All three accept layer data/mapping overrides, colour, alpha, linewidth and na_rm. Ribbons also accept fill. They do not compute statistical estimates, errors or confidence levels.

Bounds and transformations

Both bounds must be continuous columns. Finite ymin > ymax is an error at construction. Equal bounds are valid. Both endpoints contribute to axis training, including in facets.

Position transforms and limits apply to each bound. A row with a missing, non-finite, out-of-domain or scale-excluded endpoint is omitted, with a note unless na_rm=True. Use coordinate limits to clip a visible interval without removing its data. Reversed axes and coord_flip preserve the interval.

Linewidth is in millimetres. Error-bar width is the total cap width in transformed x units (category spacing for discrete x); it defaults to 0.2 and currently must be in (0, 1]. On a log x axis it therefore describes a distance in log space. To display horizontal intervals, use coord_flip. Independent xmin/xmax mappings are not implemented.

Ribbon grouping and gaps

Ribbons require continuous x. Discrete colour/fill mappings or an explicit group define separate bands. Colour and fill must be constant within each continuous segment of a group; differing styles produce a rendering error instead of arbitrarily choosing one. Duplicate x values within a segment also produce an error. Use distinct x values for a well-defined band.

Missing bounds at a finite x split a ribbon into separate polygons. A segment needs at least two valid rows to form a polygon. Coordinates are joined by straight edges; no smoothing, interpolation between groups, or statistical fitting occurs. Ribbons use identity positioning, not stacking.

std.linalg — dense linear algebra

std.linalg provides finite f64 matrices, column vectors, multiplication, linear solves and decompositions. Storage and the public API are written in Plenty; the substantial numerical algorithms use faer 0.24.4 through an optional shared engine. Constructing, inspecting and transposing, adding, subtracting and scaling matrices works without installing that engine.

import std.linalg as la

def main() -> Result[(), AllocError | IoError | la.LinalgError]:
    a = la.matrix(2, 2, &[2.0, 1.0, 1.0, 1.0]?)?
    b = la.vector(&[5.0, 3.0]?)?
    x = a.solve(&b)?
    print(x.to_list()?)?
    gram = (a.transpose()? * a)?
    print(gram.rows(), gram.cols(), gram.to_list()?)?
    print((2.0 * a)?.get(0, 0)?)?
    Ok(())
[2.0, 1.0]
2 2 [5.0, 3.0, 3.0, 2.0]
4.0

Matrices and vectors

matrix(rows, cols, values) copies a flat row-major list. Both dimensions must be positive and the list must have exactly rows * cols entries. vector(values) creates an n × 1 matrix. zeros(rows, cols), identity(n) and diagonal(values) create common matrices. Inputs must be finite.

Matrix stores its values privately in column-major order. rows() and cols() return dimensions; get(row, col) and mutable set(row, col, value) use zero-based indices and reject negative or out-of-range indices. to_list() returns a row-major copy. transpose() and copy(matrix) allocate independent copies. Neither returns a borrowed view. Matrices have ordinary Plenty ownership and can be passed, returned, or stored in other classes.

Arithmetic

Arithmetic borrows its operands and returns a fresh matrix in Result[Matrix, LinalgError]. Use (a + b)?: parentheses put error propagation around the complete operation. Vectors follow the same rules because they are one-column matrices.

ExpressionNamed equivalentMeaning
(a + b)?a.add(&b)?Elementwise addition; identical shapes required
(a - b)?a.subtract(&b)?Elementwise subtraction; identical shapes required
(-a)?a.negate()?Negate every entry
(a * b)?a.matmul(&b)?Matrix product, not elementwise multiplication
(2.0 * a)?a.scale(2.0)?Multiply every entry by a finite f64 scalar

Scalar multiplication uses scalar-left syntax. a * 2.0 is unsupported: the current language permits one __mul__ signature, which Matrix reserves for matrix products. Use a.scale(2.0)? when scalar-right order reads better. There is no broadcasting, elementwise multiplication, scalar addition, or matrix division. Nonfinite scale factors return NonFinite; nonfinite computed entries return Overflow.

No in-place arithmetic hooks are provided: these operations can fail, whereas += and similar hooks must return unit. Explicitly assign a = (a + b)? or call a named method. Each operator allocates its own result; chains are not fused and intermediate results consume storage. Dispatch is a direct borrowed method call with no dispatch allocation or implicit operand copy, but the result-producing wrappers are not promised to inline.

Products, solves and factors

OperationResult and contract
a.matmul(&b)Matrix product; a.cols() == b.rows()
a.solve(&b)Solve square A X = B with partial-pivot LU; multiple RHS columns supported
a.lu()Reusable Lu; rectangular matrices supported
factor.solve(&b)Reuse a square LU factorization
factor.lower() / upper()Explicit m × k unit-lower and k × n upper factors; k = min(m,n)
factor.permutation()Row indices p such that A[p, :] = L U
a.qr()(q, r), full m × m orthogonal Q and m × n upper-trapezoidal R; A = Q R
a.cholesky()Lower L with A = L Lᵀ; requires symmetric positive-definite A
a.svd()(u, values, v), thin factors with k columns; A = U diag(values) Vᵀ
a.eigh()(values, vectors) for symmetric A; eigenvectors are columns and values ascend
a.eig()(real, imag, vectors_real, vectors_imag) for any square real A; right eigenvectors are columns

SVD singular values are nonnegative and descending. General eigenvalues have no promised order. eig() represents complex results with separate real and imaginary parts; it does not require a complex-number scalar type. Decomposition signs/phases, eigenvalue order in eig(), and bases in repeated eigenspaces are not stable identifiers. Compare residuals rather than exact factor entries.

Every operation preserves its inputs, including on failure. Results own their storage. solve is preferable to forming an explicit inverse; factor once when solving repeatedly. There are no implicit broadcasting, indexing overloads, sparse matrices, empty matrices, or GPU operations.

Errors and numerical limits

Operations that can fail only to allocate, such as to_list and transpose, return AllocError; the others report it as LinalgError.Allocation(cause). Plenty matrix allocations and native scratch allocations are fallible. Errors carry no dynamically allocated diagnostic message.

LinalgError distinguishes Shape, Bounds, NonFinite, Singular, NotSymmetric, NotPositiveDefinite, NoConvergence, Overflow, NativePanic, and Allocation(AllocError). Nonpositive dimensions and products above i64.MAX // 16 return Shape before allocation; these checks do not expose primitive arithmetic errors. Internal index arithmetic is bounded by that validated extent. Invalid indices return Bounds before reading or mutating storage. Constructors/setters reject NaN and infinity; numerical results containing NaN or infinity report Overflow. Symmetry is checked with exact f64 equality between the two triangles; symmetrize deliberately if your data is approximate.

LU reports Singular on an exactly zero diagonal pivot at solve time. Near singularity is not reliably diagnosed: a finite result can be inaccurate. No automatic regularization, rank tolerance, conditioning estimate, least-squares solver or pseudoinverse is provided. General eigenvectors of a defective matrix need not form a basis. Scaling and residual checks remain the caller’s numerical responsibility. Results may differ in their final bits across CPUs/releases.

Why Plenty uses a faer fork

Plenty uses cjrh/faer-rs’s caller-workspace branch, based on faer 0.24.4 at 0ecd846. Upstream’s dense, triangular and indexed GEMM dispatches can allocate packing buffers infallibly inside the private-gemm-x86 backend. A failure can abort the process before Plenty can return AllocError, even when matrix outputs and decomposition scratch were allocated fallibly.

The workspace API change adds fallible caller-owned packing buffers and a scoped borrow used by all three dispatches. It avoids initializing the backend’s owning thread-local cache, uses allocation-free cache discovery, and restores nested scopes on unwinding. On CPUs without the optimized x86 path, scoped calls use allocation-free nano-gemm. Plenty enters the scope with Par::Seq; the fork does not make every upstream owned/parallel API fallible. Assembly kernels and numerical algorithms are unchanged. The fork includes the small backend as a path dependency because part of the change belongs in that separate upstream crate.

Plenty separately reuses these workspaces in a fixed four-slot pool. That performance optimization is in the engine, not a second fork kernel variant. It avoids paying for a large reservation on every small operation while keeping allocation failure recoverable on cold/concurrent misses. See the engine contract for its bounded retained-memory cost. The earlier approach of disabling optimized GEMM is no longer used.

Measured on the development x86-64 machine, correctness-checked warm matrix products retained approximately upstream throughput. Seven-sample medians for 16-byte-strided inputs were:

Square matrixFork with workspace reuseUnmodified faerEarlier disabled-GEMM workaround
32 × 320.00482 ms0.00473 ms0.00483 ms
128 × 1280.0947 ms0.0942 ms0.136 ms
512 × 5124.43 ms4.59 ms12.12 ms

These are machine-specific measurements, not a promised speedup over upstream. The same release compiler, baseline x86-64 target, wrapper and CPU affinity were used; matrix creation was outside timing and workspace acquisition was inside. A separate matched 700-product run at 512 × 512 counted 29.51 billion user instructions for both fork and upstream, versus 92.04 billion for the workaround (including identical benchmark setup and validation). This supports restoring the optimized path rather than attributing the improvement solely to CPU timing noise. Cold workspace allocation is outside these warm measurements. The reproducible driver is plenty-linalg/bench.py.

The branch is selected in plenty-linalg/Cargo.toml; Cargo.lock pins the exact revision, including the backend (currently 8e1e786). The fork’s design/provenance note keeps the source import separate from the API change for upstream review. The intent is to return to upstream releases when equivalent workspace support is available. Dependency updates must recheck allocation denial, numerical residuals, concurrent calls and the non-AVX fallback before changing the pin.

See the linear algebra lesson for runnable solve and factorization examples, and the generated signatures for every declaration.

Install and ship the linear algebra engine

Native operations in std.linalg require libplenty_linalg_v1.so. Linux release archives include it in bin/ beside the compiler, with bin/linalg-notices/ holding engine.json and dependency notices. The compiler finds it there without configuration; set PLENTY_LINALG_ENGINE to an absolute path to select a different library from the same release. An embedded compiler can set CompileOptions.linalg_engine, which takes precedence. Type checking and unreachable native calls need no engine. Pure matrix construction/access does not link it either. Programs reaching a native operation link only this engine, without pulling in the Polars engine or faer code into the runtime archive.

From a source checkout, build it once:

For the library plus local --dist metadata/licenses, use just engine linalg. just build builds the compiler and both engines; see the local build recipes. The lower-level command below installs only the library.

python3 -B scripts/build_native_engine.py linalg --out target/release

The isolated plenty-linalg/ Cargo workspace has a pinned lockfile. The build targets baseline x86-64; compatible SIMD is selected by the underlying kernels. It runs sequentially and creates no thread pool. The large algorithm and SIMD implementation justifies a native dependency instead of reimplementing it in Plenty.

Allocation contract

The facade owns matrices in Plenty lists. Engine calls borrow strided numeric views for one call and retain no handles or references. Faer’s low-level APIs receive scratch allocated with MemBuffer::try_new; other scratch buffers use Vec::try_reserve_exact. High-level infallible allocating faer owners are never used. Native scratch lives outside Plenty’s allocation accounting, but allocation failure is reported as AllocError.OutOfMemory.

The faer fork adds fallibly allocated, caller-owned packing workspaces. The engine acquires one before entering a kernel, then supplies it to all three optimized GEMM dispatch paths through a scoped borrow. This avoids initializing the backend’s owning thread-local cache. Rayon is disabled. Native tests deny allocations, including on the first call, and exercise blocked decompositions and CPUs without AVX.

A fixed four-slot pool reuses workspaces. Its mutex is held only while taking or returning a buffer, never during numerical computation; concurrent calls receive independent buffers. The pool itself allocates nothing. A cache miss allocates fallibly, and a full pool drops returned excess buffers. The baseline x86 build retains at most roughly 16 MiB of packing storage until process exit; active concurrent calls may temporarily require additional buffers. Output and decomposition-scratch allocation costs remain visible through Result.

Source builds fetch the fork once, with the exact Git revision pinned in plenty-linalg/Cargo.lock. For offline builds, first run cargo fetch --locked --manifest-path plenty-linalg/Cargo.toml while online, then preserve both Cargo’s registry and Git caches. Prebuilt engine users need neither Git nor a source checkout.

Unexpected Rust panics are caught at the C boundary and reported as NativePanic; this is a guard against internal bugs, not an OOM recovery mechanism. Engine operations write fresh Plenty-owned output buffers which are discarded on error, leaving caller inputs intact.

Deployment

Prefer explicit distribution output. With a release compiler, the bundled engine and notices need no configuration:

plenty --compile analysis.plenty --dist my-app
./my-app/analysis

my-app must not exist. Plenty copies only reached engines, validates the packaged metadata and library checksum, and includes all engine notices:

my-app/
  analysis
  libplenty_linalg_v1.so
  linalg-notices/
    engine.json
    LICENSE-MIT
    LICENSE-APACHE
    THIRD-PARTY-NOTICES.md
    licenses/rust/...

Distribution output searches only $ORIGIN (the executable’s directory), with no build-machine fallback. Ordinary --compile FILE -o OUT still searches $ORIGIN then the build-time engine directory; manual deployment requires copying the exact library and notices into the same layout. A sibling lib/ directory is not searched automatically. PLENTY_LINALG_ENGINE affects compilation, not an already-built program. Check relocation without the original installation and without stale LD_LIBRARY_PATH or LD_PRELOAD. Users need the host C libraries, not Rust, Cargo, Python, or Plenty. Missing plla_v1_ symbols indicate an old engine.

Prebuilt releases target x86-64 and ARM64 GNU/Linux with glibc 2.28 and libgcc_s.so.1. Compile and test your application on your oldest supported distribution: a compatible engine cannot lower the application’s own glibc requirement. See the shared native-engine contract.

std.ready

import std.ready waits for a native handle, or any of several handles, to become readable or writable, bounded by a Deadline and optionally by a cancellation token. It is the blocking model for I/O on worker threads: each worker makes bounded blocking waits, and no event loop, callback registration or async syntax is involved. See native effects on worker threads.

The runtime supplies four primitives (opcodes 419–422): a single ppoll on a handle’s descriptor with a nanosecond timeout, and opening, registering with and waiting on a selector. Deadlines, interruption retries, cancellation and event decoding are checked Plenty in stdlib/std/ready.plenty.

import std.bytes
import std.io
import std.process
from std.process import Stdio
from std.ready import WaitError, read_before
from std.sync import CancellationToken
from std.time import deadline_after, milliseconds, monotonic_now

def main() -> Result[(), AllocError | IoError | RangeError | WaitError | std.bytes.BytesError | str]:
    mut command = std.process.shell("echo ready")?
    command.stdout(Stdio.Piped)
    mut child = command.spawn()?
    pipe: Result[File, str] = child.take_stdout().ok_or("missing piped stdout")
    mut output = std.io.from_file(pipe?)
    cancel = CancellationToken()?
    deadline = deadline_after(&monotonic_now(), &milliseconds(5000))?
    mut buffer = std.bytes.with_capacity(16)?
    buffer.resize(16, 0u8)?
    count = read_before(&mut output, &mut buffer, 0, 16, &deadline, &cancel)?
    print(count, buffer.to_text_range(0, count)?.strip()?)?
    child.wait()?
    Ok(())
6 ready

Types

TypeContents
InterestRead, Write or ReadWrite
ReadinessPublic bool fields readable, writable, hangup, error
WaitErrorTimedOut, Cancelled, or Stream(StreamError)
SelectorAn owned set of registered handles; see Selectors
EventPublic fields key: u64 and readiness: Readiness
WatchableProtocol: descriptor(self) -> &File, the handle a selector waits on

hangup and error are reported whatever the interest. On Linux a pipe whose writer has closed reports hangup without readable; a read then returns end of input. error means a read or write will report the pending error.

Functions

FunctionResult
wait_now(file: &File, interest: Interest)Result[Readiness, WaitError]: one immediate poll
wait_until(file: &File, interest: Interest, deadline: &Deadline, cancel: &CancellationToken)Result[Readiness, WaitError]
wait(file: &File, interest: Interest, deadline: &Deadline, cancel: &CancellationToken)Result[Readiness, WaitError]
read_before(stream: &mut FileStream, buffer: &mut Bytes, start: i64, stop: i64, deadline: &Deadline, cancel: &CancellationToken)Result[i64, WaitError]
write_before(stream: &mut FileStream, source: &[u8], deadline: &Deadline, cancel: &CancellationToken)Result[i64, WaitError]
selector(capacity: i64)Result[Selector, WaitError]

wait_until is a compatibility alias of wait, with the same cancellation semantics. wait_now does not block or allocate a token; no readiness is TimedOut.

  • A handle that is already ready succeeds, even when the deadline has passed. Otherwise the wait returns TimedOut once the deadline passes.
  • A token cancelled before the call returns Cancelled even when the handle is ready. One cancelled during the wait wakes it at once: the wait polls the token’s wake descriptor alongside the handle, and cancel() makes that descriptor readable. On loopback, 50 cancellations of a blocked accept woke it in 27 microseconds on average and 80 at worst.
  • An interrupted wait (EINTR) is retried against the same deadline.
  • A closed File is Stream(StreamError.Io(IoError.Closed)). An invalid descriptor is reported the same way.
  • read_before waits for input and then makes one FileStream.read; zero is end of input.
  • write_before waits for space and then makes one write of at most 4096 bytes. This cap bounds pipe writes; it does not guarantee space in a blocking socket’s send buffer. It returns the bytes written; loop to write more.
  • A wait borrows the handle and allocates nothing. On TimedOut or Cancelled, the handle, the stream position and the caller’s buffer are unchanged.

Regular files are always ready; readiness is useful for pipes, terminals and sockets. These generic FileStream transfer helpers neither change descriptor mode nor retry a transfer after stale readiness. A blocking descriptor can still block after a readiness report, for example when another reader consumed the input. Use std.net’s transfer methods for TCP: its descriptors stay nonblocking and its deadline methods retry against the original budget.

Selectors

A selector waits on many handles in one call, so one worker can serve many idle connections. It owns one epoll descriptor and an event buffer of 16 bytes per event; it starts no thread.

MethodResult
selector.watch(file: &File, key: u64, interest: Interest)Result[(), WaitError]: register, or change the key and interest
selector.unwatch(file: &File)Result[(), WaitError]; a handle that is not registered is not an error
selector.wait(deadline, cancel)Result[i64, WaitError]: how many events are ready, at least 1
selector.event(index: i64)Result[Event, WaitError]: event index of the last wait
selector.close()Result[(), WaitError]
  • capacity is the most events one wait reports, clamped to 1..64. Further ready handles stay ready for the next wait.
  • Registrations are level-triggered: a handle is reported by every wait while it is ready, until it is read, written or unwatched. Closing a registered handle removes it.
  • wait follows the single-handle rules: ready handles are reported even when the deadline has passed, an earlier cancellation wins, a later one wakes the wait at once, and EINTR is retried.
  • An index outside the last wait’s count is Stream(StreamError.OutOfRange).
  • watch, unwatch and wait need the selector exclusively (&mut); handles are borrowed shared only for the call. A registered handle may still be read, written or closed by its owner.
  • std.net streams, listeners and UDP sockets have watch and unwatch methods that report NetError.
  • Watchable names a handle by the File it waits on. std.net’s TcpStream, TcpListener, UdpSocket, Connecting and Lookup implement it, and std.actor’s Context.watch takes any Watchable.

For TCP, follow an event with accept_now, read_now or write_now. They make one nonblocking attempt without a second ppoll. If the event is stale, NetError.WouldBlock returns control to the selector loop instead of blocking other connections. Keep an offset after a partial write, and stop watching for write readiness when the output is drained. EOF is zero from a nonempty read range, not WouldBlock.

examples/net_wait_bench.plenty measures one worker serving N idle loopback connections, one ready at a time, with a selector against a loop that polls each connection with an expired deadline. Mean round trip in nanoseconds over 20,000 one-byte echoes, pinned to eight cores:

ConnectionsSelectorPolling loop
128,66821,958
1629,14330,192
12830,048134,935
102431,5151,099,546

The selector’s cost does not grow with idle connections; the polling loop’s grows linearly.

Readiness flags are tested with bit masks, not integer division. Selector storage and event offsets use documented bounded arithmetic: at most 64 records of 16 bytes. Caller arithmetic modes do not alter these contracts.

Cancellation descriptors

A token’s first cancellable wait creates its wake descriptor, an eventfd opened close-on-exec, which the token keeps until it is dropped. A program therefore holds one extra descriptor per token that has waited. If the descriptor cannot be created (for example at the descriptor limit), the wait falls back to blocking at most 10 milliseconds at a time and rechecking the token, as before.

Example

from std.ready import WaitError, read_before
from std.sync import CancellationToken
from std.time import deadline_after, milliseconds, monotonic_now

def drain(stream: FileStream, cancel: CancellationToken) -> Result[i64, WaitError | OverflowError[i64]]:
    mut owned = stream
    mut buffer = std.bytes.with_capacity(64).unwrap()
    buffer.resize(64, 0u8).unwrap()
    deadline = deadline_after(&monotonic_now(), &milliseconds(5000)).unwrap()
    mut total = 0
    while True:
        count = read_before(&mut owned, &mut buffer, 0, 64, &deadline, &cancel)?
        if count == 0:
            return Ok(total)
        total = total + count
    Ok(total)

The tutorial lesson runs a complete program.

Limits

  • x86_64 Linux GNU only; elsewhere every wait reports IoError.Unsupported.
  • Selectors are epoll instances; there is no edge-triggered or one-shot mode, and a wait reports at most 64 events.
  • A token does not wake an unbounded read, write or child wait.
  • Sockets and name lookup are in std.net.

Acknowledgements

  • Selector is modelled on Python’s selectors module and the Rust mio crate, from which Interest and caller-chosen event keys come.

std.regex

import std.regex compiles regular expressions and searches str text and std.bytes buffers with them. A search runs in time linear in the input for a compiled pattern: there is no backtracking. Every resource a pattern or search can consume has an explicit limit. Pattern and search errors use RegexError; arithmetic on freely constructed Match offsets has the separate contract described below.

The parser, compiler and matching engines are Plenty code in the module: a Pike VM, bounded ASCII prefix filters, a lazy DFA and a one-pass matcher. The runtime supplies only primitives: Unicode properties and single-scalar case mappings from the Rust toolchain’s Unicode tables, and reads of borrowed UTF-8 by byte offset. They do not allocate, except the one that copies a byte range into a new str, which can fail with an allocation error.

Alphabetic, Lowercase, Uppercase, White_Space, Numeric, Control, and case mappings use the Unicode version in the Rust compiler that builds Plenty (char::UNICODE_VERSION). Their results may change when that toolchain is upgraded. Mark uses the independently pinned Unicode 17.0.0 tables in std.unicode, as does that module’s normalization. The two data versions need not match; ASCII classes and byte-offset rules are unchanged by Unicode table upgrades.

import std.regex
from std.regex import RegexError

def main() -> Result[(), AllocError | IoError | RegexError]:
    date = std.regex.compile("(?P<year>\\d{4})-(?P<month>\\d{2})-(?P<day>\\d{2})", &std.regex.limits())?
    text = "Released 2024-03-15, patched 2024-04-02."
    print(date.is_match(text)?)?
    match date.captures(text)?:
        case Some(groups):
            match groups.name("year"):
                case Some(year):
                    print("year", year.text(text)?)?
                case Nothing:
                    print("no year")?
        case Nothing:
            print("no date")?
    print(len(date.find_all(text, 10)?))?
    print(date.replace_all(text, "${day}/${month}/${year}", 1000)?)?
    Ok(())
True
year 2024
2
Released 15/03/2024, patched 02/04/2024.

Functions and types

ItemResultContract
limits()LimitsThe default limits, listed below
compile(pattern: &str, bounds: &Limits)Result[Regex, RegexError]The compiled pattern, or its first error

Limits(max_pattern_bytes, max_nesting, max_repeat, max_instructions, max_memory, max_work) has public NonNegative[i64] fields in that order. A literal converts where a field is expected; a computed value needs NonNegative.try_from(value)?.

FieldDefaultBoundsExceeded as
max_pattern_bytes65536Pattern length in bytesLimit.PatternBytes
max_nesting128Depth of nested groups; at most 1000Limit.Nesting
max_repeat1000Each count in {n}, {n,}, {n,m}Limit.Repetition
max_instructions100000Program instructions plus class items, after repetitions are expandedLimit.Instructions
max_memory67108864Matcher scratch bytes for one searchLimit.Memory
max_work100000000Matcher steps in one callLimit.Work

compile traps when max_nesting is above 1000. Matcher base scratch memory is 8 × (7n + 2ns + 2s + 8) bytes for n instructions and s capture slots (two per group, counting the whole match); compile checks it for the largest s the regex can use, so no later search fails on memory. An eligible search may also allocate a 66,048-byte DFA cache, only when it fits in the remaining max_memory budget. Smaller budgets select the VM without rejecting an otherwise valid pattern. The VM scratch remains available for fallback; all allocations remain fallible. The estimate is checked before comparison: an unrepresentable word estimate also exceeds every representable byte budget and reports Limit.Memory. Actual unrepresentable allocation extents report Allocation(AllocError.CapacityOverflow) before any matcher array is allocated. Work is checked against its remaining budget before increment, including when max_work is i64.MAX; exhaustion remains Limit.Work, not an arithmetic error. Repetition-count overflow likewise remains Limit.Repetition.

RegexError is:

VariantMeaning
Syntax(SyntaxError)Invalid pattern or replacement template; SyntaxError(offset, message) gives the byte offset into it
LimitExceeded(Limit)A limit was exceeded; also Limit.Output and Limit.Matches for the per-call limits below
OutOfRangeA start offset outside the input or inside a UTF-8 sequence, a Match outside its haystack, or a Cursor from another regex
Allocation(AllocError)Allocation failed

Regex

All searches are leftmost-first, as in Perl and the Rust regex crate: of the matches that start earliest, the one the pattern prefers wins (a|ab matches a in ab; greedy repetitions prefer more).

MethodResultContract
group_count()i64Groups, counting group 0 (the whole match)
group_index(name: &str)Option[i64]The number of a named group
instructions()i64Program size, for choosing max_work
is_match(text: &str)Result[bool, RegexError]Whether any match exists; stops at the first
find(text: &str)Result[Option[Match], RegexError]The first match
find_at(text: &str, start: i64)Result[Option[Match], RegexError]The first match at or after start
captures(text: &str)Result[Option[Captures], RegexError]The groups of the first match
captures_at(text: &str, start: i64)Result[Option[Captures], RegexError]As captures, from start
cursor()Result[Cursor, RegexError]Iteration state for next_match
next_match(text: &str, cursor: &mut Cursor)Result[Option[Match], RegexError]The next match, or Nothing after the last
find_all(text: &str, max_matches: i64)Result[list[Match], RegexError]Every match in order
split(text: &str, max_pieces: i64)Result[list[str], RegexError]The text between matches
replace(text: &str, template: &str, max_bytes: i64)Result[str, RegexError]text with the first match replaced
replace_all(text: &str, template: &str, max_bytes: i64)Result[str, RegexError]text with every match replaced

find_at and captures_at start at a scalar boundary; assertions such as ^ and \b still see the text before start. Iteration (next_match, find_all, split, replace_all) follows the Rust regex crate: after an empty match, the next search starts one scalar later, and an empty match where the previous match ended is skipped. So a* on baaa matches at 0–0 and 1–4, and split of "añ" by the empty pattern gives "", "a", "ñ", "".

Per-call limits: find_all fails with Limit.Matches rather than return more than max_matches matches, and split rather than return more than max_pieces pieces. replace and replace_all fail with Limit.Output when the result would exceed max_bytes bytes.

A replacement template inserts group n for $n or ${n}, a named group for ${name}, and $ for $$; a group that did not take part inserts nothing. Any other $, including $name, an unknown group, or an unclosed ${, is a Syntax error at the $. The template is checked before any search.

Bytes

is_match_bytes(data), find_bytes(data, start), captures_bytes(data, start), next_match_bytes(data, cursor), and find_all_bytes(data, max_matches) search a Bytes buffer. Valid UTF-8 sequences are scalars as in text; each byte of an invalid sequence (including an encoded surrogate) is one position that nothing matches, not even \P{Any} or [^a]. Offsets are byte offsets into data, and start may be any offset from 0 to data.len(); a start inside a sequence makes its remaining bytes invalid positions.

Match, Captures and Cursor

Match has public fields start and end (byte offsets, end exclusive) and methods len() -> Result[i64, OverflowError[i64]], is_empty(), text(haystack: &str) and bytes(data: &[u8]). len() checks end - start: public fields may contain arbitrary offsets, so representable negative lengths are retained and unrepresentable differences return an arithmetic error. The text and byte methods still copy the matched part and report OutOfRange for a haystack it does not fit. Matches returned by a successful search have ordered, nonnegative offsets and a representable length.

Captures has len() (equal to group_count()), get(index: i64) and name(name: &str), each Option[Match]. Group 0 is the whole match; a group outside the pattern or not taking part is Nothing. A group inside a repetition holds its last iteration.

Capture indexing validates the group number before multiplying it by two; out-of-range indices, including i64.MAX, return Nothing. Internal opt-outs are bounded by validated scalar widths, pattern/input spans, configured nesting, successful instruction emission and preflighted matcher array extents. Unlimited replacement does not maintain a redundant total-replacements counter.

A Cursor holds the matcher scratch and position, so a loop over next_match allocates scratch only when it is created. Use it with the regex that made it; a different pattern, memory budget or work budget returns OutOfRange before consulting cached transitions.

Pattern syntax

SyntaxMeaning
x, \x for punctuation xA literal scalar
\n \t \r \f \v, \xHH, \x{H...}Control characters and a scalar by hexadecimal value
.Any scalar except \n; any scalar under s
[abc], [a-z], [^...]A class, range, or negated class; - is literal first or last, ] must be escaped
\d \DASCII digit [0-9], or not
\w \WASCII word scalar [0-9A-Za-z_], or not
\s \SASCII space [\t\n\v\f\r ], or not
\p{Name} \P{Name}Has the Unicode property, or not; also inside classes
xy, x|ySequence; alternatives, the first preferred
x* x+ x?0 or more, 1 or more, 0 or 1; greedy
x{n} x{n,} x{n,m}Counted repetition
x*? x+? x?? x{n,m}?Lazy: prefer fewer
(x), (?P<name>x), (?<name>x)Capturing group; names match [A-Za-z_][A-Za-z0-9_]* and are unique
(?:x), (?flags:x)Non-capturing group, optionally with flags
(?flags)Set flags until the end of the enclosing group
^ $Start and end of text; of a line under m
\A \zStart and end of text
\b \BASCII word boundary, or not

Properties are Alphabetic, Lowercase, Uppercase, White_Space, Numeric, Control, Mark (general category M), ASCII, and Any. Flags are i (case-insensitive), m (multi-line ^ $), and s (. matches \n); (?i-m) sets i and clears m.

Case-insensitive matching uses simple, one-scalar case mappings: a scalar matches another when their lowercase or uppercase mappings agree, so (?i)k matches the Kelvin sign and (?i)σ matches Σ and ς, but (?i)ss does not match ß.

These are errors rather than silently different behavior: look-around ((?=, (?!, (?<=, (?<!), backreferences (\1), nested classes and POSIX classes ([[:alpha:]]), an empty class ([]), a repetition of a repetition (a**; write (?:a*)*), a range with a class end ([\d-z]), the x flag, and unknown escapes, properties, or flags. A pattern with an unmatched ) or ( is an error too.

Costs

Search is linear in the input for a fixed compiled pattern. The Pike VM visits each instruction at most once per input position in each of its two sets. Accelerators can add a bounded scan before transferring to the VM; there is no unbounded backtracking. max_work caps matcher steps in each call: find, captures, and is_match run one search; find_all, split, and replace_all count every search they make; each next_match call is counted separately. A native literal batch reserves the sum of its maximum search spans before starting, charges each needle’s actual search span, and retains that charge across fallback. Even a match near the beginning can charge an entire window; insufficient budget returns Limit.Work before the batch. Short tails use the scalar filter. Work includes accelerated byte scans and uncached DFA transitions, and is retained across fallback. Exact step counts depend on the engine and its cache. Iteration can rescan text after a match, so choose max_work for the whole call.

The compiler selects these accelerators without changing match semantics:

  • A case-sensitive literal prefix uses substring search in text. For programs of at most 256 instructions, a bit-parallel filter can instead recognize up to eight mandatory ASCII positions across alternatives and positive range classes. It admits possible matches for the matching engine to confirm; nullable, case-insensitive or non-ASCII paths shorten or disable this filter. Up to eight paths can occupy separate eight-bit lanes, preventing false candidates made by mixing alternatives. Extraction visits at most 2,048 instructions and keeps at most 17 pending paths; exceeding those bounds retains the position-wise union.
  • When the extracted paths expand to at most eight ASCII literals of the same three-to-eight-byte width, a 64-byte scalar probe returns nearby matches promptly. Longer scans then use native substring search in windows of at most 1,024 bytes. Windows overlap by width - 1, and all literals are searched before selecting the earliest candidate. The engine still confirms candidates and resolves alternative priority. Literal storage is bounded by eight short strings; it is compiled metadata, not per-search scratch. Wider classes or excessive path products retain the scalar filter.
  • For ASCII-only programs of at most 62 instructions without assertions, a lazy DFA answers existence searches. When every match has the same byte length, it also supplies match offsets for searches, cursors and replacements. At its initial state it uses the same mandatory-prefix filter to skip impossible starts. Its 64-state cache is local to the matcher and reused by a cursor. Cache exhaustion disables the DFA for that matcher and resumes the VM.
  • For small programs without assertions, a one-pass table accelerates searches that need only whole-match offsets when each ASCII byte has at most one preferred consuming path. It retains greedy/lazy ordering. Non-ASCII input transfers to the VM. Failed anchored attempts can consume at most one input length before fallback, preventing quadratic rescanning. The table has at most 62 × 129 entries and is built only with sufficient memory headroom.

ASCII accelerators borrow the text’s UTF-8 bytes once per search and read the view directly, without allocating or calling the runtime for each byte. Scalar decoding and assertions retain their Unicode boundary semantics.

Captures requiring subgroup positions use the Pike VM, with prefix filtering where applicable. Unsupported patterns continue to use the VM. Each search allocates scratch; a Cursor reuses it. No new public API or opt-in flag is needed. The regex-redux measurements exercise cleaning, counting and replacement with the same patterns as the Rust regex reference; remaining gaps include Unicode DFA transitions and the remaining scalar scan lowering (issue #760).

Choices

The module uses its own matcher rather than the Rust regex crate because that crate allocates infallibly and would abort on allocation failure, and its Unicode tables would enlarge every program that imports std.regex. The narrower memchr::memmem::find primitive uses constant space and no allocation. Its SIMD byte search and portable fallbacks justify native code; Plenty’s current SIMD API has no byte-vector search operations. Literal extraction, batching, work limits and all matching policy remain Plenty. The syntax is the subset of the Rust crate’s syntax with the same meaning, except that Perl classes and \b are ASCII-only, which the tests check against the crate.

Acknowledgements

std.report

std.report provides bounded human-readable reports using the same std.fmt.Formatter[W] and FormatError as value display and f-strings. It provides rendering helpers for the canonical std.fmt.Formattable protocol; it does not own another formatter, protocol, or error family. Import formatter types and constructors from std.fmt.

import std.report
from std.fmt import FormatError, Formatter

pub class DiskFull:
    path: str
    needed: u64

    pub def format[W](self, output: &mut Formatter[W]) -> Result[(), FormatError]:
        output.text("disk full writing ")?
        output.text(&self.path)?
        output.text(": need ")?
        output.unsigned(self.needed, 10)?
        output.text(" more bytes")

def main() -> Result[(), IoError | FormatError]:
    error = DiskFull("out.csv", 4096u64)
    print(std.report.to_string(&error, 256, 4)?)?
    Ok(())
disk full writing out.csv: need 4096 more bytes

Formatting contract

The structural Formattable protocol requires a public shared-receiver method on a public class or enum, since its bound is checked in std.fmt:

pub def format[W](self, output: &mut Formatter[W]) -> Result[(), FormatError]:
    pass

The writer parameter belongs to the method. A single implementation can render to a string, stderr, or a caller-owned writer, and can call all the scalar and format-spec methods described in std.fmt. For example, output.float_spec(value, &std.fmt.spec(precision=2, kind="f")) is available inside a report. Scratch for advanced numbers can allocate; this does not alter the allocation-free text/integer/boolean stderr path.

render[T: Formattable, W](&T, &mut Formatter[W]) borrows the value, enters one depth level, calls format, and restores depth even on error. Depth is in 0..=64; zero permits direct scalar writes but rejects render. Nested values should use render to participate in the depth limit. Direct user-method calls do not count depth automatically.

An anonymous sum satisfies Formattable when every normalized member satisfies it. Dispatch borrows the active member and calls its writer-generic format method without moving the payload or allocating a wrapper. The sum adds no nesting level beyond the one entered by render. Aliases, reordered members and singleton normalization preserve this behavior. Direct sum.format(&mut output) uses the same dispatch without entering a depth level. A missing, private, or mismatched method on any member fails the bound, even when that member is inactive. This forwarding is specific to std.fmt.Formattable; other protocols, including display/debug, are not automatically forwarded.

Helpers and destinations

OperationBehavior
std.report.render(&value, &mut output)Borrowed report into any Formatter[W]
std.report.to_string(&value, max_bytes: NonNegative[i64], max_depth: NonNegative[i64])Create a string formatter, render, and finish
std.fmt.formatter(writer, scratch, max_bytes, max_depth)Transfer a writer and caller-provided scratch into a formatter
std.fmt.string_formatter(max_bytes, max_depth)Create an allocating Formatter[Cursor]
std.fmt.stderr_formatter(max_bytes, max_depth)Create an allocation-free Formatter[Stderr]
std.fmt.finish(&output)Copy a string formatter into an owned str; takes &Formatter[Cursor]

The formatter does not add punctuation, flush stdout, close a stream, or lock a whole record. Each write counts toward its byte budget. completed() records the accepted prefix. has_failed() records terminal stderr failure; subsequent emissions on that formatter return FormatError.OutputClosed without writing. Other writer destinations retain their own recovery policy and do not set this flag. Formatting limits and allocation failures leave the formatter usable. See errors and partial output for shared errors and retry rules. In particular, finishing stderr is rejected by its type.

For allocation-free stderr reports, use text, integer, unsigned, and boolean. Text writes use the native descriptor-2 operation directly, retry interruptions, and count successful short writes. A closed or read-only stderr returns Output(StreamError.Io(IoError.System(9))) (EBADF). On supported x86-64 Linux the write suppresses SIGPIPE for that operation, so a closed pipe returns an error and normal cleanup runs. Advanced formatting lazily acquires scratch and can fail with Allocation(...).

The string formatter stages a complete UTF-8 text emission before calling its cursor. Failed staging or cursor growth preserves the valid UTF-8 output and byte count. finish borrows the formatter, so an allocation failure can be retried. A whole record can still have a successful prefix before a later emission fails; do not replay it blindly.

Current boundaries

This protocol selects human-readable entrypoint reports; types without it retain structural rendering. std.error.Context[E] supports the protocol when its cause does. display/debug remain the value presentation hooks; sharing the formatter does not change which hook an entrypoint chooses. The tutorial shows enum and sum reporting.

The completed reporting proposal records the original reporting design and its later unification.

std.secrets

import std.secrets returns unpredictable bytes from the operating system as buffers, text tokens, and integers, with one error type, SecretsError.

Use it for session tokens, keys, nonces, and password-reset links. For repeatable simulation and sampling use std.random; the two modules share no state and no code path.

Entropy comes from the operating system. Plenty implements no random generator for secrets: native OS services supply every byte.

Policy is checked Plenty code over one native primitive, the intrinsic __io_entropy_read, which is reserved for the standard library and makes a single OS request. Linux uses getrandom(2) (glibc 2.25 or later, Linux 3.17 or later). The macOS runtime uses getentropy, with at most 256 bytes per request; Windows uses the system-preferred BCryptGenRandom generator, with at most u32::MAX bytes per request. Short counts are normal and the Plenty layer retries until its requested range is full. See platform support for compiler qualification; an entropy backend alone does not qualify a port.

import std.secrets
from std.secrets import SecretsError

def main() -> Result[(), IoError | SecretsError | OverflowError[u64]]:
    session = std.secrets.token_urlsafe(32)?
    reset_code = std.secrets.token_hex(16)?
    roll = std.secrets.below(6u64)? + 1u64
    # The values differ on every run; their shapes do not.
    print(len(session), len(reset_code), roll >= 1u64 and roll <= 6u64)?
    Ok(())
43 32 True

Functions

FunctionResultContract
fill(target: &mut Bytes, start: i64, stop: i64)Result[(), SecretsError]Overwrite target[start:stop] with secret bytes
token_bytes(count: i64)Result[Bytes, SecretsError]A new buffer of exactly count secret bytes
token_hex(count: i64)Result[str, SecretsError]count secret bytes as 2 * count lowercase hex digits
token_urlsafe(count: i64)Result[str, SecretsError]count secret bytes as unpadded base64url, ceil(4 * count / 3) characters
random_u64()Result[u64, SecretsError]A uniform u64
below(bound: u64)Result[u64, SecretsError]Uniform in [0, bound) without modulo bias; zero is EmptyRange

count is a number of bytes, not characters, and has no default: choose it for the secret’s purpose. Thirty-two bytes is ample for a bearer token. A negative count is a size that cannot be allocated, so it is Allocation(AllocError.CapacityOverflow); zero gives an empty result without consulting the system. fill follows the range rule. The text forms use the std.encoding codecs.

Failure policy

  • No fallback. If the system cannot supply entropy, the operation returns SecretsError.Entropy. It never substitutes the clock, an address, a seeded generator, or /dev/urandom.
  • Blocking. Linux waits until its kernel generator has been initialized, which matters only very early in boot. The API has no nonblocking form.
  • Short and interrupted requests are retried until the range is full. A request that reports zero bytes for a nonempty range is UnexpectedEof; a count larger than the request is InvalidProgress.
  • No partial secrets. When fill fails, the whole requested range is zero. A failed token call releases its buffer and returns only the error.
  • Scrubbing. token_hex, token_urlsafe, random_u64, and below zero their intermediate byte buffer before releasing it. The values they return, and a token_bytes buffer, are the caller’s to protect: Plenty does not zero memory on drop, and the encoder’s own intermediate buffer is not scrubbed.

Sources

The functions above are shorthand for methods on Source[R], which applies the same policy to any byte reader.

ItemContract
os()Source[OsEntropy], the operating-system source
source_from[R: Reader](inner: R)Source[R] over a std.io reader
Source[R].fill, token_bytes, token_hex, token_urlsafe, next_u64, belowAs the functions above, with self: &mut Source[R]; next_u64 is random_u64
Source[R].innerThe reader, public so its owner can inspect or replace it
os_entropy()OsEntropy, a std.io Reader whose read makes one request and may return a short count

source_from exists to test code against scripted short reads and failures, and to use a hardware device opened as a stream. The secrecy of its output is exactly that of the reader supplied; only os() is backed by the kernel generator. OsEntropy has a private field and can only be made by os_entropy() and os().

next_u64 reads eight bytes as a little-endian integer. below draws next_u64() until the value is at least 2^64 mod bound and returns it modulo bound.

Errors

pub enum SecretsError:
    OutOfRange
    EmptyRange
    Entropy(StreamError)
    Allocation(AllocError)
VariantMeaning
OutOfRangeA range outside the buffer; the source was not consulted
EmptyRangebelow(0)
Entropy(cause)The source failed; cause is the StreamError
Allocation(cause)A token or scratch buffer could not be allocated, including CapacityOverflow for a negative count

From the operating system, cause is StreamError.Io(IoError.System(code)) with the getrandom error number: for example 38 (ENOSYS) on a kernel without the call, or under a sandbox that denies it.

Ownership and threads

Targets are exclusive loans. Source[OsEntropy] and OsEntropy hold no descriptor and no buffered bytes; creating one costs nothing and they need no closing. The module has no global state, passes the ordinary worker-effect checks, and may be used from any thread. Each request goes to the kernel, so threads never share generator state in the process, and a forked child cannot repeat its parent’s values.

below validates its bound into NonZero[u64] before rejection sampling, so its remainder operations are statically infallible. Reading an eight-byte word uses one documented bounded shift-offset opt-out. Neither changes the entropy source, retry policy, zeroization or public error type.

Costs

OperationAllocatesSystem requests
fillNothingOne per 33,554,431 bytes at most; one for up to 256 bytes
token_bytes(count)One buffer of count bytesAs fill
token_hex, token_urlsafeThe token buffer, the encoded buffer, and the strAs fill
random_u64One 8-byte scratch bufferOne
belowAs random_u64, per attemptFewer than two expected

Nothing is buffered between calls, so every operation is at least one system call. To draw many small values, fill one buffer and decode it, or seed a std.random generator when the values need not be secret. Throughput has not been measured on an idle machine.

Limits

  • Linux only, through getrandom(2); there is no /dev/urandom path for kernels older than 3.17.
  • No nonblocking or deadline-bounded request.
  • No key derivation; see backlog B67. Digests, HMAC, and constant-time comparison are in std.crypto.
  • No zero-on-drop buffer type.
  • No choice or shuffle helpers; combine below with indexing.

std.shlex

std.shlex splits text containing shell-style words and quotes one argument for a POSIX shell. It is written in Plenty and never starts a process, reads the environment, or expands shell expressions.

Prefer std.process.Command.arg when you already have individual arguments: pass them directly, without quoting. Use split for an argument string supplied by a configuration file or user.

Interface

FunctionResultContract
split(text: &str, comments: bool = False)Result[list[str], ShlexError]Remove word quoting and escapes, returning owned arguments
quote(text: &str)Result[str, ShlexError]Encode one argument for a POSIX shell
import std.shlex
from std.shlex import ShlexError

def main() -> Result[(), ShlexError | AllocError | IoError]:
    print(std.shlex.split("--name 'Ada Lovelace' --literal '$HOME'")?)?
    print(std.shlex.quote("two words")?)?
    Ok(())
["--name", "Ada Lovelace", "--literal", "$HOME"]
'two words'

Splitting rules

  • Space, tab, carriage return and line feed separate unquoted words. Other Unicode whitespace is ordinary data; this whitespace convention follows Python’s shlex rather than a shell’s command grammar.
  • Single quotes preserve everything until the next single quote. Double quotes preserve whitespace and single quotes. Adjacent quoted and unquoted parts join into one word, and '' or "" creates an empty word.
  • Outside quotes, backslash preserves the next scalar. Inside double quotes, it escapes only $, backtick, double quote, backslash and line feed; before anything else, the backslash is retained.
  • Backslash followed by line feed is removed outside single quotes. Alone, that continuation does not create an empty word.
  • With comments=True, an unquoted # at the start of a word skips the rest of that line. a#b, quoted hashes and escaped hashes stay literal. Comments are disabled by default.

There is no variable, command, arithmetic, tilde or wildcard expansion. Operators such as ;, |, > and && are ordinary word contents, not a pipeline or a command separator. Dollar-single-quoted strings, substitutions, here-documents and shell grammar are not supported. This is not a shell script validator, nor an exact implementation of Python’s shlex modes.

Quoting and errors

quote leaves nonempty ASCII words containing only letters, digits and _@%+=:,./- unchanged. Otherwise it uses single quotes and escapes embedded single quotes by closing and reopening the quoted sections. Empty text becomes ''. For every accepted input, split(quote(text)?)? returns exactly one word equal to text.

Quoted text is for one argument in a noninteractive POSIX shell (sh -c or a script), not for an entire script or for Windows shells. Control characters remain literal; do not paste or feed this output to an interactive shell. It does not make an executable name, option, path, or shell program appropriate for an application. Prefer argument-vector spawning; when a shell is needed, passing data through arguments or environment variables also avoids building scripts from data.

ShlexError has UnclosedQuote(position), TrailingEscape(position), Nul(position) and Allocation(AllocError). Positions are zero-based Unicode scalar indices, matching Plenty string indexing. The quote/escape position points to its opening character. Both functions reject NUL, including in a comment, because POSIX process arguments cannot contain it. Allocation failures return Allocation(cause); no partial result is exposed and intermediate storage is released.

Splitting takes linear time and uses a reusable scalar-piece list plus the returned words. Quoting takes linear time and may allocate intermediate text when escaping apostrophes. These are whole-string APIs; callers handling untrusted, arbitrarily large input should bound that input before parsing.

The API draws on Python’s shlex and the Rust shlex crate, with the specific splitting rules above rather than a compatibility mode switch.

Acknowledgements

std.sketch

Bounded summaries for streams too large to retain. These pure-Plenty algorithms own their memory and random state. They do not wrap Apache DataSketches or share its serialized formats. Also see xor filters and cuckoo filters.

from std.sketch import SketchError, hyperloglog, hash_u64, quantiles

def main() -> Result[(), SketchError | AllocError | IoError | ZeroDivisionError]:
    mut users = hyperloglog()?
    mut latency = quantiles()?
    for i in range(10000):
        user = u64(i % 2500)
        users.add(hash_u64(user))
        latency.add(f64(i % 100))?
    print(f"about {users.estimate():.0f} distinct users"?)?
    print("median", latency.quantile(0.5)?, "max", latency.quantile(1.0)?)?
    Ok(())
about 2511 distinct users
median 47.0 max 99.0

Distinct identities

hyperloglog(precision=12, seed=0) creates a dense HyperLogLog. Precision is 4–18; storage is exactly 2**precision packed one-byte registers plus Bytes metadata. add(key: u64) and merge(other) allocate nothing. Duplicate keys are harmless. Merge requires equal precision and seed and leaves the receiver unchanged on SketchError.Incompatible. Estimate and merge cost O(registers).

estimate() uses the original HyperLogLog harmonic estimate with linear counting for small cardinalities. relative_standard_error() returns 1.04/sqrt(m): an asymptotic standard error, not a guaranteed error bound. There is no HLL++ bias table or near-64-bit-universe correction. Use this for ordinary cardinality estimation, not exact counts or estimates approaching the hash-space size. See the HyperLogLog paper.

Keys are numeric identities. hash_u64(key, seed=0) is a stable SplitMix64 mixer, not a cryptographic hash. The algorithms are designed for ordinary data, not adversarially chosen keys. Applications mapping text or compound values to u64 must choose and persist their own stable hash convention; collisions in that mapping cannot be undone by a sketch.

Quantiles

quantiles(capacity: Positive[i64] = 1024, seed=0) creates Quantiles, a uniform sample without replacement using reservoir Algorithm R. Capacity is up to 1,048,576 finite f64 values; a larger capacity traps. add(value) rejects NaN/infinities; count() is the number of accepted values and retained() is at most capacity. Construction reserves the sample buffer; updates allocate nothing, maintain it in sorted order, and cost O(capacity) when replacing a sample. Queries cost O(1).

quantile(fraction) accepts a finite number in [0,1]. Empty streams return SketchError.Empty. Endpoints return the exact minimum/maximum. Other fractions use the nearest-rank order statistic of the retained sample (no interpolation). Results are exact before sampling starts. Afterwards they are probabilistic; there is no deterministic rank-error guarantee or tail-relative guarantee. Increase capacity to improve accuracy. This first API has no quantile merge. It is a simple reservoir, not KLL, REQ, or a t-digest.

The sketch owns a std.random.Rng. The seed, accepted input order, and capacity determine its sample. Serializing and restoring preserves continued sampling. The random generator’s existing version-stability contract is unchanged.

Frequent identities

frequent_items(capacity: Positive[i64] = 128) creates a FrequentItems Misra–Gries summary with up to 65,536 counters; a larger capacity traps. It accepts unit-weight u64 identities; there is no weighted update or merge in this API. Construction allocates two fixed arrays. add(key) and frequency(key) allocate nothing and cost O(capacity).

frequency(key) returns Frequency(key, lower, upper) with deterministic bounds on the true count. error_bound() is the number of full counter decrements, at most count() // (capacity + 1). A missing candidate has lower bound zero and upper bound error_bound(). candidates() allocates a list of retained nonzero counters in storage order, not frequency order. Every key whose true frequency exceeds error_bound() appears in that list. A second pass over the original data can compute exact counts for these candidates.

Errors and persistence

InvalidArgument covers invalid configuration or nonfinite input. Count overflow returns Limit without accepting the update: these numerical counters stay exact, never wrap or saturate, and a rejected update leaves samples, extrema, RNG state and frequency bounds unchanged. Nonfinite quantile input is still InvalidArgument, even when the count is exhausted. Constructors, readers and updates that can also fail for another reason report allocation failure as SketchError.Allocation(cause); candidates() and to_bytes() fail only to allocate and return AllocError. All allocation failures remain recoverable.

to_bytes() writes a versioned Plenty format: PSH1, PSQ1, or PSF1. Integers use little-endian u64 words. Quantile values use 32-byte, space-padded, shortest round-trip scientific ASCII slots; this preserves finite f64 values, including signed zero, without host-endian assumptions. These are checkpoint formats, not interchange formats with third-party sketch libraries.

hyperloglog_from_bytes(data, max_precision=18), quantiles_from_bytes(data, max_capacity: Positive[i64] = 1048576), and frequent_items_from_bytes(data, max_capacity: Positive[i64] = 65536) reject unknown versions, bad lengths, invalid state and trailing bytes. Pass a smaller caller limit when reading untrusted checkpoints. All three readers take a &[u8] view. Validated precision/capacity bounds keep checkpoint sizes and offsets representable; frequent-item accounting is validated against the serialized total before adding each counter. Formats are version-tagged: future incompatible formats need a new version and must not silently reinterpret these bytes. They contain no checksum or authentication; structurally valid corruption can change estimates.

The crate audit for issues #496–#498 found infallible allocations in datasketches 0.5.0 (hll/array8.rs, req/compactor.rs), xorf 0.13.0 (prelude/mod.rs), and scalable_cuckoo_filter 0.5.1 (bits.rs). Small direct implementations keep Plenty’s recoverable allocation policy without a native dependency or ABI.

Acknowledgements

  • Uses the HyperLogLog algorithm by Philippe Flajolet, Éric Fusy, Olivier Gandouet and Frédéric Meunier (2007).
  • Uses reservoir Algorithm R as described by Jeffrey Vitter (1985).

std.sketch.xor

A static approximate-membership filter over u64 keys. build(keys, max_keys: NonNegative[i64], seed=0) accepts duplicates, sorts/deduplicates a fallible copy, and returns an XorFilter. The input is not mutated. max_keys bounds input entries, not just distinct keys; a max_keys above 16,777,216 traps.

from std.sketch import SketchError
import std.sketch.xor as xor

def main() -> Result[(), SketchError | AllocError | IoError]:
    keys = [10u64, 20u64, 30u64, 20u64]?
    filter = xor.build(&keys, 1000)?
    print(filter.len(), filter.contains(20u64), filter.contains(25u64))?
    Ok(())
3 True False

contains(key) allocates nothing and reads three fingerprint positions. Inserted keys have no false negatives for an intact filter. Absent keys may return true: approximately 1/256 with well-distributed hashes. This is not an exact set, security boundary, or an adversarial-key guarantee. len() counts distinct input keys; the empty filter always returns false.

Construction uses three disjoint blocks, graph peeling, and reverse XOR8 fingerprint assignment. The persistent filter has 3 * (1 + unique_count // 2) one-byte slots, plus metadata. At the maximum key count, each block has 8,388,609 slots. Position reduction multiplies a 32-bit hash fragment by this bounded block size without overflow; only the SplitMix64 mixer and attempt-seed schedule use explicit modular arithmetic. Scratch memory is O(input entries + slots), considerably larger than the final filter. Sorting is O(n log n), followed by up to 32 O(n) peeling attempts. Failure to peel returns SketchError.ConstructionFailed; allocation failures return SketchError.Allocation(cause). See Graf and Lemire’s xor-filter paper.

This first implementation deliberately uses more construction slack than a tuned xor/fuse filter. It is immutable after construction. For changing data, see cuckoo filters.

to_bytes() returns Result[Bytes, AllocError]. from_bytes(data, max_keys: NonNegative[i64]) reads a &[u8] view and returns SketchError. The PSX1 format stores seed, unique count, block size (little-endian u64), then fingerprint bytes. Readers validate the count/shape and exact length before allocation, and reject values beyond the caller’s limit. It is a Plenty-specific versioned checkpoint, not the xorf crate’s representation. It has no integrity checksum: valid-looking corruption can introduce false negatives.

Acknowledgements

  • Uses the xor filter by Thomas Mueller Graf and Daniel Lemire (2020).

std.sketch.cuckoo

cuckoo(initial_buckets=64, max_buckets: Positive[i64] = 1048576, seed=0) creates a scalable approximate-membership filter for u64 keys. Initial buckets must be a power of two between 1 and 1,048,576. The total bucket limit is at least that large; a limit above 16,777,216 traps. Each bucket has four two-byte fingerprints. Layers double geometrically; the caller’s limit counts buckets across all layers. The validated bucket budget bounds byte offsets and the live occurrence count (at most four occurrences per bucket); these sizes and counts do not wrap or saturate.

insert(key) returns the layer index receiving that occurrence. Every call inserts an occurrence, including duplicates. contains(key) searches all layers. len() counts live inserted occurrences, not distinct keys; layers() and buckets() expose current storage growth.

An existing layer is tried with at most 256 eviction steps. A failed eviction chain is rolled back completely before trying another layer or allocating a new one. Allocation failure or SketchError.Limit preserves the filter and its membership. Construction allocates bounded eviction scratch; insertions into existing layers and membership/deletion queries allocate nothing.

Keep the layer index that insert returns; remove needs it:

from std.sketch import SketchError
from std.sketch.cuckoo import cuckoo

def main() -> Result[(), SketchError | AllocError | IoError]:
    mut seen = cuckoo()?
    receipt = seen.insert(42u64)?
    seen.insert(7u64)?
    print(seen.len(), seen.contains(42u64))?
    print(seen.remove(42u64, receipt)?, seen.len(), seen.contains(7u64))?
    Ok(())
2 True
True 1 True

Deletion requires the insertion receipt

Keep the index returned by insertion alongside the authoritative record. remove(key, layer_index) requires a known live occurrence inserted into that layer of this filter. It returns Result[bool, SketchError]; an invalid layer index is InvalidArgument. The layer indices survive serialization and remain stable during growth. Empty layers are retained.

A positive contains is insufficient evidence for deletion: it may be a false positive. Also, a key may falsely match an earlier layer even when it really exists in a later layer. Supplying the receipt avoids that cross-layer ambiguity. Removing an absent occurrence, reusing a receipt, or using the wrong filter/layer can erase another key’s fingerprint and cause false negatives. Do not deduplicate inserts using contains; doing so breaks deletion accounting. Applications that cannot retain this information should use the filter only for membership checks.

Within a layer, equal fingerprints sharing a bucket share both candidate buckets, so deleting one of the corresponding known occurrences remains sound. The fingerprints do not identify which colliding occurrence was removed.

Accuracy, cost and persistence

Intact filters have no false negatives under the deletion precondition. False positives are approximately 8/65536 per full layer for ordinary hashes, increasing with the number of layers. Zero fingerprints map to one, causing a small bias. This is not an adversarial-key or security guarantee. Queries cost O(layers), with up to eight slot comparisons per layer. Growth initializes a new array and may be slower than a successful insertion into an existing layer.

to_bytes() writes PSC1, seed, total configured limit, initial bucket count, layer count, and occurrence count as little-endian u64, followed by little-endian u16 slots in layer order. from_bytes(data, max_buckets: Positive[i64]) reads a &[u8] view, validates shape, counts, length and the caller limit before constructing storage, and returns SketchError. The caller limit bounds the serialized configured maximum, not only today’s occupancy. Serialization fails only to allocate and returns AllocError. The format is versioned, Plenty-specific and unauthenticated.

Acknowledgements

std.signal

import std.signal lets a program receive SIGTERM and SIGINT as an event instead of being ended by them. Supervisors such as systemd and container runtimes stop a service with SIGTERM; a terminal sends SIGINT on Ctrl-C. A program that never calls listen keeps the default action: either signal ends it at once.

The runtime supplies one primitive (opcode 500): it installs the handler and returns the program’s end of a socket pair. Waiting, decoding and the cancel_on helper are checked Plenty in stdlib/std/signal.plenty.

import std.process
import std.signal
from std.ready import WaitError
from std.signal import SignalError
from std.sync import CancellationToken
from std.time import deadline_after, monotonic_now, seconds

def main() -> Result[(), AllocError | IoError | RangeError | SignalError | WaitError]:
    mut signals = std.signal.listen()?
    # Play the supervisor: a shell sends SIGTERM to its parent, this program.
    mut kill = std.process.shell("kill -TERM $PPID")?
    kill.spawn()?.wait()?
    deadline = deadline_after(&monotonic_now(), &seconds(5))?
    received = signals.wait(&deadline, &CancellationToken()?)?
    print("caught", received)?
    signals.close()?
    Ok(())
caught std.signal.Signal.Terminate

Types and functions

ItemContents
SignalInterrupt (SIGINT) or Terminate (SIGTERM)
SignalErrorAlreadyListening, or Stream(StreamError)
SignalsThe handle on which caught signals arrive
listen()Result[Signals, SignalError]
cancel_on(signals: Signals, stop: CancellationToken)Result[Option[Signal], WaitError]
MethodResult
signals.wait(deadline: &Deadline, cancel: &CancellationToken)Result[Signal, WaitError]: the next signal
signals.received()Result[Option[Signal], WaitError]: a signal already caught, without waiting
signals.watch(selector: &mut Selector, key: u64)Result[(), WaitError]: register for reading
signals.unwatch(selector: &mut Selector)Result[(), WaitError]
signals.close()Result[(), WaitError]

wait follows the std.ready rules for deadlines and cancellation. After a selector reports the handle, received takes the signal. WaitError is the std.ready type.

Behavior

  • listen catches both signals for the whole process, whichever thread they arrive on. Each caught signal is sent as one byte on a socket pair; the handler makes no allocation and takes no lock.
  • A process listens once. A second listen is AlreadyListening, even after the first handle is closed.
  • Only the main thread may call listen: a worker that reaches it is rejected at compile time, like std.env.set_current_dir. See native effects. A worker may own or borrow the Signals handle and wait on it.
  • After the first signal, both signals have their default action again, so a second one ends the process at once. A drain that takes too long can be cut short with a second Ctrl-C.
  • Closing or dropping the handle puts the default action back in effect: the next signal ends the process as if it had never listened.
  • The handler is installed with SA_RESTART, so most interrupted system calls resume. std.ready waits retry EINTR against their deadline.
  • Child processes started by std.process start with the default action: exec resets caught signals, and the signal mask is not changed. A Ctrl-C at a terminal goes to the whole foreground process group, children included.
  • The handle holds one descriptor, opened close-on-exec. The runtime keeps the other end for the life of the process.

Stopping a server

cancel_on waits for a signal and cancels stop, returning which signal it was. If something else cancels stop first, it returns Nothing. Run it on its own thread beside std.http.server.serve, whose shutdown and drain begin when stop is cancelled:

signals = std.signal.listen()?
stop = CancellationToken()?
with spawn(std.signal.cancel_on, signals, stop.share()).drop_job()? as watcher, stop.cancel_on_exit():
    served = std.http.server.serve(listener, &routes, &handler, &config, &stop)
    stop.cancel()
    watcher.join()?
    report = served?

Leaving the block joins the watcher, which waits until a signal arrives or stop is cancelled. stop.cancel_on_exit() cancels stop on every exit, and the explicit stop.cancel() comes before the in-block join; see cancel on every exit. The tutorial lesson runs a complete program.

Limits

  • x86-64 and AArch64 GNU/Linux; elsewhere listen is Stream(StreamError.Io(IoError.Unsupported)).
  • Only SIGTERM and SIGINT. There is no SIGHUP reload, no other signal, and no way to catch a signal more than once.
  • A library compiled from Plenty does not catch signals unless its code calls listen; the host’s dispositions are its own.

Acknowledgements

  • Uses D. J. Bernstein’s self-pipe trick, with a socket pair in place of the pipe.
  • Uses the Rust libc crate for Linux signal action and socket bindings (MIT OR Apache-2.0).

std.simd — portable explicit lanes

Use this module when an algorithm needs explicit lane operations. Start with ordinary column pipelines for bulk clist queries: their automatic SIMD handles vector widths and tails without changing the source algorithm.

Values and construction

i32x4, u32x4, f32x4, and f64x2 are nominal, copyable values. Their names describe scalar types and logical lane counts, not an instruction set. Construct them with i32x4.Lanes(a, b, c, d) or the matching module factory, for example splat_i32x4(value). Construction, arithmetic, comparison and selection allocate nothing. Values can be passed, returned, nested in other values and stored in collections under the ordinary enum rules.

lane(index) returns Option[Scalar]; indices outside 0..lane_count return Nothing, including negative indices. to_tuple() copies the logical lanes in order, without allocation. Ordinary == compares complete values; use the simd_* methods below when each lane needs its own comparison result.

Construct distinct lanes, then add the same value to each with explicit wrapping:

import std.simd as simd
def main() -> Result[(), IoError]:
    values = simd.u32x4.Lanes(0u32, 1u32, 2u32, u32.MAX)
    print(values.wrapping_add(simd.splat_u32x4(1u32)).to_tuple())?
    Ok(())
(1, 2, 3, 0)

Arithmetic and masks

OperationContract
add, sub, mulInteger vectors return Result[Vector, OverflowError[Lane]]; float vectors stay infallible
Float divScalar IEEE division, including infinities and NaNs
Integer wrapping_add, wrapping_sub, wrapping_mulModular lane arithmetic
Integer wrapping_sum()Modular sum of all lanes
Float sum_ordered(initial)Left fold from initial, in increasing lane order
simd_eq/ne/lt/le/gt/ge(other)Lane comparisons returning a compatible mask

Integer checked operations execute in lane order and return the first overflowing lane’s scalar OverflowError, including its wrapped value. Callers handle this ordinary Result; an unchecked: caller does not change the method’s contract. Floating operations do not implicitly fuse or reassociate. NaN comparisons, signed zero and subnormals follow the target’s scalar rules; NaN payload identity is not promised.

The integer reduction API is explicitly wrapping_sum(): it associates as (((lane0 + lane1) + lane2) + lane3), wrapping each addition. There is no ordinary fallible integer reduction method. Float sum_ordered(initial) also folds left in lane order, without reassociation.

Comparisons on 32-bit lanes produce mask32x4; f64x2 produces mask64x2. Masks contain logical booleans, constructed directly with Lanes(...) or by comparison. There are no integer-mask conversions or public native bit encoding.

mask.select(yes, no) selects each lane from two vectors of the same type and the matching mask shape. It evaluates both operands before selection: it does not suppress side effects, allocation failures or traps while computing them. any(), all() and count() summarize a mask. invert(), and_mask(other), or_mask(other) and xor_mask(other) combine logical lanes.

Select a floor for the lanes below it; the input stays unchanged:

import std.simd as simd
def main() -> Result[(), IoError]:
    values = simd.f32x4.Lanes(18.0f32, 21.0f32, 16.0f32, 24.0f32)
    floor = simd.splat_f32x4(20.0f32)
    below = values.simd_lt(floor)
    print(below.count(), below.select(floor, values).to_tuple())?
    Ok(())
2 (20.0, 21.0, 20.0, 24.0)

Like other generic Plenty helpers, select[T] forwards structurally to yes.selected(mask, no). The module’s vector implementations of that helper are private. A user-defined type may supply its own public selected method; its effects and allocation behavior then belong to that implementation, and the compiler does not replace it with a vector intrinsic.

Safe list access

load_i32x4(values, offset=0) and the corresponding factories for other vector types accept a shared list borrow and return Option[Vector]. They return Nothing unless a whole vector fits. vector.store(&mut values, offset) returns True after replacing all lanes, or False without changing the list if the whole vector does not fit. Negative offsets are invalid. Bounds are checked before adding lane offsets, including for i64.MAX.

These operations neither allocate nor grow the list. They access ordinary list elements individually; lists are not packed native arrays. Returned vectors own copies of their scalar lanes, so no borrow persists after loading. Stores take an exclusive borrow and preserve length and capacity. Process full groups and then the scalar tail explicitly when writing a manual kernel.

Handle a load that may not fit. A failed store leaves every element unchanged:

import std.simd as simd
def main() -> Result[(), AllocError | IoError | OverflowError[i32]]:
    mut values = [1i32, 2i32, 3i32, 4i32]?
    match simd.load_i32x4(values):
        case Some(lanes):
            print(lanes.store(&mut values, 1))?
            print(lanes.add(simd.splat_i32x4(10i32))?.store(&mut values, 0))?
        case Nothing:
            print("A full vector does not fit")?
    print(values)?
    Ok(())
False
True
[11, 12, 13, 14]

Rust relationship and initial limits

The model follows Rust’s portable std::simd: fixed logical lanes, splats, per-lane arithmetic and simd_* comparisons, masks, and selection. This first Plenty module has concrete types rather than Simd[T, N] and uses methods rather than overloaded arithmetic operators. Enum factories use module functions and the Lanes variant because Plenty has no static enum methods. Unlike Rust’s wrapping integer operators, add/sub/mul preserve Plenty’s checked arithmetic; modular operations have explicit wrapping_* names.

The implementation uses existing inline enums, whose payload stores each lane in a tagged scalar slot. A four-lane payload currently occupies 64 bytes, plus the ordinary value handle; this is not a packed 16-byte vector or a native register ABI. Mask variants expose logical booleans, not hardware masks. Neither vector nor mask layout is a C, serialization, Arrow or Parquet contract.

On x86-64, the default optimized profile lowers wrapping arithmetic, float arithmetic, comparisons and selection to baseline 128-bit SIMD. Integer wrapping_sum reduces lanes directly without calling the scalar module body. Numeric vector locals without mutable borrows stay in vector registers across supported operations and loops. Values materialize in ordinary tagged storage at calls, containers and observable borrows; parameters, masks, generators and mutable-borrowed locals retain that storage path. This is an internal optimization, not a different ownership rule. Checked integer arithmetic, mask summaries, ordered float reductions and list access keep their ordinary scalar implementations. On other compiler hosts, the module’s scalar bodies remain available. The AArch64 implementation currently uses these scalar bodies, including in optimized builds; ARM vector acceleration is separate work. The same source semantics apply with and without optimization. No runtime ISA selection or newer CPU feature is required.

Storage boundaries, copying and lane assembly can dominate short operations. Explicit SIMD is not a blanket speedup over scalar code, and this module does not replace the packed-column lowering used by clist. Wider vectors, packed views, generic lane counts, gathers/scatters, casts, shifts and integer division are not in this initial surface. See the tutorial and the implementation evidence.

Acknowledgements

  • Design follows Rust’s portable std::simd.

std.sync

Exports CancellationToken, the existing cooperative cancellation facility, and Mutex[T]. The imported and prelude CancellationToken types are identical; no wrapper, allocation, or new effect permission is introduced by importing. Mutex is not a prelude name: each use needs an import, through any item or module alias. Supported target: x86_64 Linux GNU.

CancellationToken() -> Result[CancellationToken, AllocError] creates an owned shared state in one fallible allocation.

MethodResultEffect
share()CancellationTokenExplicitly retain the same state
cancel()()Permanently request cancellation and wake token waiters
is_cancelled()boolObserve the request
wait()()Wait until cancellation is requested
wait_timeout_ms(timeout_ms: u64)boolObserve cancellation within the budget
cancel_on_exit()with guardCancel on every exit from the with block; see cancel on every exit

Methods borrow the handle, allocate nothing, and copy no captured data. Share and observation have constant bookkeeping; cancellation also wakes waiters. Lock contention and scheduling have no bounded latency. Tokens are safe to move or explicitly share into eligible workers; borrowed handles retain ordinary loans. Dropping the last handle releases the state but does not request stopping. Cancellation is idempotent and does not reset.

A token does not interrupt instructions, cancel a future, or wake a channel wait. Workers must return cooperatively, using bounded waits where needed. std.time supplies deadline arithmetic and timeout conversion.

Typed deadline waits

Prefer these waits when a caller supplies a total budget and cancellation. They borrow the handles, allocate nothing, and preserve the deadline across wakeups. deadline and cancel are the last two parameters.

OperationResult
recv_before(&receiver, &deadline, &cancel)Result[T, WaitError]
send_before(&sender, value, &deadline, &cancel)Result[(), SendBeforeError[T]]
select_recv_before(&first, &second, &deadline, &cancel)Result[Selected[A, B], WaitError]
future_before(&future, &deadline, &cancel)Result[(), WaitError]
cancelled_before(&token, &deadline, &cancel)Result[(), WaitError]

WaitError has TimedOut, Cancelled and Disconnected (channels only). Every SendBeforeError[T] variant (TimedOut, Cancelled, Disconnected) returns the original unsent value, without copying it. Cancellation visible before an attempt wins even over readiness. Otherwise an already-ready value or disconnection wins over an expired deadline. Selection keeps the builtin’s first-receiver priority. Future observation leaves its result owned by the future and does not cancel its worker. Token observation leaves its state intact.

Channel and future primitives cannot wake on a separate token, so these wrappers sleep in slices of at most 10 ms, rechecking cancellation between slices. This is a polling interval, not a hard latency guarantee under scheduler or lock contention. Positive fractional milliseconds round up when passed to the native wait, so a final slice can overrun by less than 1 ms. No deadline is restarted.

Low-level millisecond methods have explicit unit names: send_timeout_ms, recv_timeout_ms, wait_timeout_ms on futures and tokens, and select_recv_timeout_ms. The original _timeout spellings remain compatibility aliases with the same millisecond units. They do not accept a separate token.

Mutex[T]

A value that workers update through shared references or Shared[Mutex[T]] handles, one lock holder at a time. See mutexes for the rules and the deadlocks the compiler rejects.

from std.sync import Mutex

class Stats:
    count: i64
    total: i64

def record(stats: &Mutex[Stats], value: i64) -> Result[(), OverflowError[i64]]:
    with stats.lock() as data:
        data.count = data.count + 1
        data.total = data.total + value
    Ok(())

def main() -> Result[(), SpawnError[()] | IoError | OverflowError[i64]]:
    stats = Mutex[Stats](Stats(0, 0))
    with spawn(record, &stats, 3)? as first, spawn(record, &stats, 4)? as second:
        first.join()?
        second.join()?
    final = stats.into_inner()
    print(final.total)?
    Ok(())
7

The explicit joins handle each worker’s arithmetic Result; merely leaving a spawn scope waits for completion but does not propagate the worker’s error. An arithmetic failure unlocks the mutex on exit, retaining earlier mutations.

OperationResultEffect
Mutex[T](value), Mutex(value)Mutex[T]Store value unlocked, inline
with mutex.lock() as value:binds value: &mut TWait for the lock, hold it for the block
mutex.into_inner()TConsume an owned mutex and return its value

T is any type a generic class field can hold: no references, generators, or borrowed environments. A mutex is affine, with no copy, equality, or formatting. It is stored inline (a 32-bit state word beside T); construction, locking, and unlocking allocate nothing. An uncontended lock and unlock are inline atomic instructions; a contended one waits in the kernel.

lock() is valid only as a with manager. It borrows the mutex shared, so no mut is needed, and every exit from the block unlocks. There is no try_lock, timed lock, poisoning, reentrant locking, or fairness guarantee. into_inner() needs no lock: owning the mutex proves no other thread can hold it.

Atomic integers and booleans are in std.atomic. The shared owner Shared[T] is a prelude type. Semaphores, condition variables, and read-write locks are not exported.

Acknowledgements

  • Mutex uses mutex 2 from Ulrich Drepper’s “Futexes Are Tricky”.
  • Uses the Rust libc crate for the target-specific Linux futex syscall number (MIT OR Apache-2.0).

std.system

Bounded Linux system-information snapshots, implemented in Plenty over std.fs and std.io. No subprocess, background sampler, native dependency, or global cache is involved.

local() -> Result[Procfs, IoError]
open_procfs(path: &std.path.Path) -> Result[Procfs, IoError]

local() opens /proc. open_procfs opens another procfs mount or a directory of captured files; this also supports deterministic tests. Procfs owns a directory handle and closes it on drop. It resolves reads relative to that handle, following the rooted filesystem API’s no-follow rules.

import std.system
from std.path import from_text
from std.system import SystemError

def main() -> Result[(), IoError | SystemError]:
    # A captured procfs file gives fixed values; std.system.local() reads /proc.
    write_text("loadavg", "0.50 0.25 0.10 2/180 4242\n")?
    source = std.system.open_procfs(&from_text(".")?)?
    load = source.load_average()?
    print(load.one_minute, load.runnable_tasks, load.last_pid)?
    live = std.system.local()?
    print("CPUs found:", live.cpu_count()? > 0)?
    Ok(())
0.5 2 4242
CPUs found: True

Queries

Every method returns Result[T, SystemError]. Calls read fresh data; two calls are not one atomic snapshot. Limits are named optional arguments.

MethodResultSourceDefaults
cpu_count()i64 online logical CPU recordsstatmax_bytes=1048576
memory()Memorymeminfomax_bytes=65536
load_average()LoadAverageloadavgmax_bytes=4096
uptime()Uptimeuptimemax_bytes=4096
block_devices()list[BlockDevice]partitionsmax_bytes=1048576, max_devices=4096
disk_io()list[DiskIo]diskstatsmax_bytes=1048576, max_devices=4096
process_ids()sorted list[u32]numeric directory entriesmax_entries=65536
process(pid: u32)ProcessPID/statusmax_bytes=65536

The generated signature index lists every public record field. For example:

source = std.system.local()?
memory = source.memory(max_bytes=65536)?
print(memory.total_bytes, memory.available_bytes)?

Units and meaning

  • Memory reports total/free physical bytes, optional available physical bytes, and total/free swap bytes. MemAvailable is a kernel estimate; Nothing means the field is unavailable, not zero. Free memory excludes reclaimable caches and is usually not the useful estimate of allocation headroom.
  • LoadAverage has 1-, 5-, and 15-minute load values, runnable/total task counts, and the last created PID. Load is not CPU-utilization percent.
  • Uptime.seconds is elapsed boot time, including suspend according to procfs. idle_cpu_seconds sums idle time over CPUs and can exceed elapsed time.
  • BlockDevice gives major/minor device numbers, a name, and size_bytes. Whole disks and partitions can appear; summing them double-counts storage. /proc/partitions reports 1024-byte blocks, so capacity has that granularity. This is not mounted filesystem capacity or free space.
  • DiskIo gives reads/writes completed, bytes read/written, currently in-flight requests, and busy milliseconds. Sector counts are converted using Linux’s fixed 512-byte accounting unit. These are raw counters, not rates. Counters can reset on reboot/device replacement or wrap; detect resets before computing differences. Whole disks and partitions overlap here too. New trailing diskstats fields are ignored; the modern eleven-counter prefix is required.
  • Process gives PID, parent PID, the kernel-reported name and state code, real UID, thread count, and optional virtual/resident bytes. Name escaping from status is preserved, including literal backslash escapes; it is not a command line or executable path. Missing memory fields, for example on kernel threads, yield Nothing. Resident memory is an approximate kernel counter.

The CPU count is not an affinity/cgroup-aware worker-pool size. Memory, load, uptime and devices describe what the mounted files expose, often the host even in a container. PIDs follow the mount’s PID namespace and permissions. This API does not infer a container’s effective CPU or memory quota.

Limits, failures, and races

max_bytes is a Positive[i64]; i64.MAX traps, as does process with pid 0. Entry and device bounds are NonNegative[i64]. A byte-limited read probes one extra byte to distinguish exact-limit EOF from truncation, since proc files often report size zero. Exceeding any bound returns Limit, never a partial successful result. process_ids counts all visited directory entries toward its bound, including non-PID names. Parsing and result storage allocate fallibly; byte limits bound input size, not exact peak memory.

enum SystemError:
    Io(IoError)
    Stream(std.io.StreamError)
    Allocation(AllocError)
    Limit
    Malformed(str)

I/O errors keep their original cause, including missing and inaccessible files. Allocation errors can also occur inside the existing Io/Stream envelopes. Invalid UTF-8, missing required fields, duplicate selected keys, invalid numbers and unit-conversion overflow return Malformed(field_or_source). Unknown meminfo/status keys are ignored. Numeric conversions never truncate or wrap. Scaling checks the full u64 product and preserves the field-specific Malformed error on overflow; counts and byte-limit probe offsets are bounded by their validated limits rather than saturated.

Enumerating PIDs does not read each process’s status or silently skip failures. Call process(pid) for the records wanted, choosing how to handle disappearance or access denial. It opens the PID directory once before reading status, avoiding mixing files from a reused path; the process may still exit and the read may fail. A returned PID is neither a stable identity nor authority to signal a process.

This first interface has no CPU-utilization sampling, filesystem free-space query, command-line decoding, cgroup accounting, or process control. For children owned by the program, use std.process.

Formats follow the kernel’s procfs documentation, disk I/O accounting, and partitions ABI. See the tutorial.

std.tempfile

Owned temporary files and directories, with automatic cleanup and explicit fallible cleanup methods. Prefer this module for scratch storage; use std.fs for descriptor-rooted staging and atomic publication.

import std.fs
import std.path as path
import std.tempfile as tempfile

def main() -> Result[(), AllocError | IoError | std.fs.FsError]:
    parent = path.from_text(".")?
    with tempfile.directory(&parent, prefix="build-")? as scratch:
        note = scratch.path().join(&path.from_text("note.txt")?)?
        write_text(&note.to_text()?, "draft")?
        print(read_text(&note.to_text()?)?)?
        scratch.cleanup()?
        print(std.fs.metadata(scratch.path()).is_err())?
    Ok(())
draft
True

Constructors

FunctionResultDefaults
default_dir()Result[Path, IoError]First nonempty TMPDIR, TEMP, TMP; otherwise /tmp
file(parent: &Path = default_dir()?)Result[TemporaryFile, IoError]Unnamed file
named_file(parent, prefix="plenty-", suffix="")Result[NamedTemporaryFile, FsError]Parent is default_dir()
directory(parent, prefix="plenty-", max_entries: Positive[i64] = 100000, max_depth: Positive[i64] = 64)Result[TemporaryDirectory, FsError]Parent is default_dir()

parent is a borrowed Path; prefix and suffix are borrowed strings. Explicit parents bypass environment selection. Relative parents are resolved against the current directory. Environment paths preserve native bytes. Selection is fresh on every call, without caching or a trial file: an unusable configured directory causes construction to fail, rather than silently falling back elsewhere.

Named resources use 16 random bytes encoded as 32 hexadecimal characters, between the prefix and suffix. Prefix and suffix together may occupy at most 200 UTF-8 bytes and must contain neither / nor NUL. These are byte limits, not scalar counts; invalid components or a combined length above 200 return FsError.InvalidInput before creating a resource. Directories accept only a prefix. Creation retries up to 16 name collisions. Files are created with owner-only permissions, directories with owner-only access (subject to the process umask). No API returns an uncreated, supposedly unique filename.

Files

TemporaryFile.stream() lends a mutable std.io.FileStream. Its context manager also yields that stream. close() returns Result[(), StreamError]; dropping the owner closes the descriptor. The file has no pathname.

NamedTemporaryFile.path() returns a borrowed absolute Path, usable by another program while the owner is alive. The file supports read, write, flush, sync, and seek with the same signatures as FileStream, and works with std.io reader/writer helpers. Its context manager yields the named owner itself. The underlying descriptor stays private for cleanup identity checks.

close() returns Result[(), FsError]: it removes the name, then closes the stream. An already removed name is accepted. A different file at that name is rejected; a failed removal leaves the owner armed for another cleanup attempt. keep() returns Result[Path, AllocError] and disarms removal after successfully copying the path. The stream remains open until closed or dropped. Call keep() before cleanup; it does not recreate an already removed file.

Directories

TemporaryDirectory.path() returns a borrowed absolute Path; root() borrows its descriptor-rooted std.fs.Root. Its context manager yields the directory owner. cleanup() returns Result[(), FsError] and recursively removes contents and the directory. Successful cleanup is idempotent. keep() returns an owned path and disarms cleanup, allocating the path before changing ownership policy.

Each cleanup attempt is bounded by max_entries and max_depth, both Positive[i64]; directory traps on a max_depth above 64. The root directory counts toward both budgets. A limit or I/O failure can leave a partially removed tree; cleanup remains armed for a later attempt. Symlinks are removed without traversing their targets. Before traversal, the parent entry must still identify the owned directory; otherwise cleanup returns FsError.Replaced without traversing the replacement.

Cleanup and platform limits

Context exits and destructors perform best-effort cleanup. Use explicit close()? or cleanup()? inside the context when failure must be reported. Normal return and ? propagation run cleanup; process termination and runtime traps do not promise destructor execution. Cleanup itself can fail to allocate.

These APIs currently use the Linux facilities of std.fs: unnamed and named files require filesystem O_TMPFILE support; named publication additionally requires /proc/self/fd. There is no fallback that exposes an intermediate predictable name. Memory-to-disk spooling is not provided.

Parents must be trusted against concurrent hostile mutation. Identity checks detect replaced entries, but are not atomic with deletion; they do not make cleanup safe against an attacker racing renames. Named files may also be removed by external temporary-directory cleaners. Keep the owner alive while another program uses its path.

See the temporary-resource tutorial for complete examples.

Acknowledgements

  • Type names follow Python’s tempfile; keep and fallible close follow the Rust tempfile crate.

std.test

import std.test provides assertions, fixtures, and test doubles for the tests that plenty test runs, with one error type, TestError. The module is checked Plenty code. Any program may import it; it depends on the runner only for reading PLENTY_TEST_SEED and for failure injection.

A test file with main passes when it exits with status zero, so a plain program can run assertions:

import std.test
from std.test import TestError

def test_split() -> Result[(), AllocError | TestError]:
    parts = "a,b,,c".split(",")?
    std.test.equal(&len(parts), &4)?
    std.test.equal(&parts[2], &"")?
    std.test.check(parts[0] < parts[1], "parts keep their order")?
    Ok(())

def main() -> Result[(), AllocError | IoError | TestError]:
    test_split()?
    print("test_split passed")?
    Ok(())
test_split passed

Assertions

An assertion that holds returns Ok(()) without allocating. One that fails flushes standard output, writes the position of the call, assertion failed: , and an explanation to standard error, and returns Err(TestError.Assertion). The explanation is written at once, so it survives propagation or handling by the caller. Tests can return TestError or a sum containing it.

parse.plenty:12:5: assertion failed: left == right
  left: 4
 right: 5

The file is relative to its source root, tests/ for a test file, as in trap reports. The position is the user code that called the assertion, even through standard-library helpers: the caller location that trap notes use. A user helper that wraps an assertion reports the helper’s line, not its caller’s.

FunctionPasses whenFailure explanation
fail(message: &str)nevermessage
check(condition: bool, message: &str)conditionmessage
equal[T](left: &T, right: &T)left == rightleft == right, then left: and right: lines
not_equal[T](left: &T, right: &T)left != rightleft != right, then a both: line
less[T](left: &T, right: &T)left < rightleft < right, then left and right lines
close(left: f64, right: f64, tolerance: f64)left == right or abs(left - right) <= tolerancethe three values; NaN on either side fails, equal infinities pass
is_ok[T, E](result: &Result[T, E])result is Okresult is Ok, then an error: line
is_err[T, E](result: &Result[T, E])result is Errresult is Err, then a value: line

Values are shown with str.repr, so T must support equality or ordering as the function needs. A value whose type str.repr rejects, such as a Shared[T] owner or a std.sync.Mutex, is shown as <value cannot be formatted>; is_ok and is_err therefore accept every Result, whatever the branch they do not expect holds. Showing a value allocates; when that fails the line reads <no memory to show the value>. A failure to write to standard error is ignored.

Under plenty test, a test that returns Err(TestError.Assertion), or the Failure.Unspecified that ? makes of it, gets no further error: line: the explanation above is the whole report.

Fixtures

FunctionResultContract
scratch()Result[Scratch, TestError]A new empty directory named plenty-test- plus 32 random hex digits, created with mode 0700 under $TMPDIR when it is an absolute path, otherwise under /tmp
seed()Result[u64, TestError]PLENTY_TEST_SEED as a decimal u64, or 0 when it is unset; any other value writes a message to standard error and returns TestError.InvalidSeed
rng()Result[Rng, TestError]std.random.seeded(seed()?)

plenty test --seed N sets PLENTY_TEST_SEED for every test it runs, so a failure seen under one seed repeats under it.

Scratch owns its directory:

MethodResultContract
path()&PathThe directory’s absolute path
root()&RootA std.fs.Root anchored at the directory
file(name: &str)Result[str, TestError]The absolute path text of name inside the directory, for builtins such as write_text and open
cleanup()Result[(), TestError]Remove the directory and everything in it; later calls do nothing. Takes &mut Scratch

Dropping a Scratch runs cleanup and ignores its error. Removal uses std.fs.remove_tree with budgets of 100000 entries and depth 64; beyond them cleanup reports TestError.Fixture(FsError.Limit) and leaves the rest in place. A process that traps or is killed skips destructors, so its scratch directory remains, except under plenty test, which gives each test its own TMPDIR and removes it when the test ends.

Test doubles

Code that reads the clock or moves bytes can take its source as a protocol bound, so a test passes a double and a program passes the real thing. Calls are resolved at compile time; a double costs nothing in the program that does not use it.

fake_clock(nanoseconds_after_origin: u64) returns a FakeClock, a std.time.Clock whose time moves only when the test moves it. Its instants are on the instant_at timeline.

MethodResultContract
now()InstantThe current fake instant
advance(span: &Duration)Result[(), RangeError]Move forward by span; past the end of the timeline is RangeError.OutOfRange and nothing moves. Takes &mut FakeClock
set(nanoseconds_after_origin: u64)()Move to any instant, including backwards. Takes &mut FakeClock

fake_stream(input: Bytes) returns Result[FakeStream, AllocError], a std.io Reader and Writer over memory. Reads return input from its start; writes append to output. Both fields are public. It injects the partial progress and failures that a file or socket produces only under load:

MethodResultContract
limit(bytes: i64)()Each later read or write moves at most bytes bytes; 0 or less removes the limit
fail_after(after: i64, error: StreamError)Result[(), OverflowError[i64]]The call after calls from now (0 is the next) returns Err(error) and moves nothing. One failure is pending at a time; a later request replaces it
calls()i64read, write, and flush calls made so far, including failed ones
read, write, flushas std.ioThe stream protocol methods

A range outside its buffer traps with read: start/stop outside buffer before the call is counted. flush does nothing but count and fail when told. Writing allocates as output grows; that failure is StreamError.Allocation.

fail_after checks the absolute call index before replacing the pending fault. On overflow it returns the wrapped OverflowError[i64] and changes neither schedule nor counter; callers must handle its result. read, write and flush return StreamError.OutOfRange without changing data, counter or pending fault if the call counter cannot advance. No caller arithmetic mode changes these boundaries. Existing signed offsets are retained: a representable past index does not match a future call.

from std.time import Clock, Instant

def overdue[C: Clock](clock: &C, since: &Instant) -> bool:
    clock.now().saturating_duration_since(since).as_seconds() > 30

def test_overdue() -> Result[(), std.test.TestError | RangeError]:
    mut clock = std.test.fake_clock(0u64)
    since = clock.now()
    std.test.check(not overdue(&clock, &since), "fresh")?
    clock.advance(&std.time.seconds(31u32))?
    std.test.check(overdue(&clock, &since), "after 31s")?
    Ok(())

A program calls the same function with std.time.monotonic_clock().

Failure injection

Executables that plenty test builds, or that are compiled with --test-hooks, can make the C library’s allocation and file calls fail, so a test reaches the error paths of code that allocates or does file I/O. Injection belongs to the calling thread.

FunctionEffect
fail_allocations_after(count: u32)After count more allocations succeed, each fails until restored: the operation reports AllocError.OutOfMemory
restore_allocations()Allocations succeed again
fail_files_after(count: u32, code: i32)After count more file calls succeed, each fails with errno code until restored: the operation reports IoError.System(code)
restore_files()File calls succeed again

All four return Result[(), TestError]. A later call replaces the budget. Counted file calls are opening any path, and reading, writing, syncing, or truncating a regular file; standard streams, pipes, and sockets never fail, so a failing test can still report. Other filesystem calls, such as renaming or listing a directory, are not counted. How many calls an operation makes is not part of its contract, so a test should fail from the first call (count 0) or check only that some call failed. A failure that ? propagates leaves the budget in place; restore it before an assertion that must allocate to show its values.

def test_save_reports_a_full_disk() -> Result[(), std.test.TestError]:
    scratch = std.test.scratch()?
    name = scratch.file("state.txt")?
    std.test.fail_files_after(0u32, 28i32)?
    saved = write_text(&name, "state")
    std.test.restore_files()?
    full: Result[i64, IoError] = Err(IoError.System(28i32))
    std.test.equal(&saved, &full)?
    Ok(())

Use plenty --compile server.plenty --test-hooks -o server to exercise a real program’s refusal paths from an external client. The program can read its own test configuration and arm a budget immediately before admission, then restore it before reporting the result. plenty build --test-hooks and plenty run --test-hooks enable the same controls for project executables. This also enables file failure injection. The flag does not arm any failures by itself. Keep it out of production builds.

Without this opt-in or plenty test, these functions write error: failure injection needs plenty test or --test-hooks to standard error and return Err(TestError.Unavailable). Default executables contain no interposition code. Project and compiler caches distinguish instrumented builds from default builds.

Memory-limited integration runs

On Linux, playpen can bound the memory of the real executable and its descendants through the systemd user manager:

playpen --memory-limit 128M --memory-swap-max 0 --private-tmp false -- ./server

Build before launching, so the limit measures the program rather than the compiler. Disabling the private temporary directory lets a test driver share temporary fixtures with the program. Use a suitable limit for the workload and check both the exit status and the application’s expected response.

The cgroup limit is an integration resource ceiling: exhausting it can kill the process instead of returning AllocError. Use --test-hooks and fail_allocations_after for deterministic error-path checks, including under playpen. The budget is a count of allocation attempts, not bytes or a live-heap limit; it cannot measure peak memory or prove a program stays below a byte budget. Use an external memory profiler for those measurements.

Errors

VariantMeaning
AssertionAn assertion failed; its explanation is already on standard error
Fixture(FsError)Creating or removing a scratch directory failed, or a name or path was invalid
Allocation(AllocError)Memory for a fixture ran out
InvalidSeedPLENTY_TEST_SEED is set but is not a decimal u64
UnavailableFailure injection was requested without plenty test or --test-hooks

Limits

  • Injected failures cover allocation and the file calls listed above, on the calling thread only. Allocation on other threads, such as executor workers, and network calls are not injected; use FakeStream at the stream boundary.
  • Calls the C library makes internally, such as fopen inside another C function, are not intercepted; only the program’s and runtime’s own calls are.

std.text

Bounded reusable text construction, joining, scalar wrapping, and byte-set scans. Word wrapping and indentation add Python-style paragraph tools. Import std.text; every allocating operation returns Result[..., TextError].

OperationBehavior
builder(max_bytes: NonNegative[i64])New empty Builder
Builder.byte_len(), Builder.limit()Current UTF-8 byte count and configured maximum
Builder.append(text: &str)Append a copy, or leave contents unchanged on failure
Builder.clear()Empty the builder, retaining its allocation
Builder.to_text()Independent UTF-8 string snapshot; builder remains reusable
Builder.finish()Final UTF-8 string; consumes the builder on success or failure
join(pieces: &list[str], separator: &str, max_bytes: NonNegative[i64])Separator between pieces, including empty pieces; empty list yields empty text
hard_wrap(text: &str, width: Positive[i64], max_bytes: NonNegative[i64])Hard wrap at width Unicode scalars, inserting LF
find_in(text: &str, set: &[u8])Option[i64]: scalar position of the first ASCII scalar whose byte is in set
find_not_in(text: &str, set: &[u8])Option[i64]: scalar position of the first scalar that is not such a member: the end of a run

TextError has InvalidLimit, LimitExceeded, Allocation(AllocError), and Bytes(BytesError) variants. Limits and widths are domain types, so InvalidLimit only reports wrapper indentation or a placeholder that cannot fit. A literal converts where a limit is expected; a computed value needs NonNegative.try_from(value)?. Byte-buffer failures preserve their typed cause; normalization uses the direct allocation variant. Limits apply to output bytes, not characters. Appending checks the complete input size without allocating, then appends atomically. Appends of at most four UTF-8 bytes reserve capacity before writing directly, without a temporary encoding; larger appends encode a temporary buffer. Empty appends allocate nothing. Invalid UTF-8 must be handled at byte decoding; str is always valid UTF-8. Embedded NUL is preserved. Public builder operations preserve complete UTF-8.

Hard wrapping preserves every input scalar and existing LF, including final LF. It never inserts an extra LF before an existing LF. It does not collapse spaces, find word boundaries, calculate terminal display width, or keep grapheme clusters together: a combining mark can begin a new line. Width and string positions are scalar counts; the output limit and byte_len are byte counts.

Segmentation and display width

The official unicode_text package provides UAX #29 grapheme cluster, word and sentence boundaries, and terminal display width following the Rust unicode-width crate, with Unicode 17.0.0 data matching std.unicode. It is plain Plenty; add it as a git dependency:

[dependencies]
unicode_text = { git = "https://github.com/plentylang/unicode-text.git", tag = "v0.1.0" }
import unicode_text
from unicode_text import Ambiguous

def main() -> Result[(), AllocError | IoError]:
    text = "Hi 👩‍🔬! 日本語のテキスト。"
    print(unicode_text.grapheme_list(&text)?)?
    print(unicode_text.word_list(&text)?)?
    for span in unicode_text.sentences(text):
        print(span.start, span.stop)?
    print(unicode_text.width(&text, Ambiguous.Narrow))?
    print(unicode_text.width("±1", Ambiguous.Wide))?
    Ok(())
["H", "i", " ", "👩‍🔬", "!", " ", "日", "本", "語", "の", "テ", "キ", "ス", "ト", "。"]
["Hi", "日", "本", "語", "の", "テキスト"]
0 8
8 17
25
3

graphemes, word_bounds, words and sentences are generators of Span values carrying scalar positions and UTF-8 byte offsets, and allocate nothing; the *_list forms return Result[list[str], AllocError]. width takes an explicit Ambiguous.Narrow or Ambiguous.Wide for East Asian Ambiguous characters and, like unicode-width, counts each control scalar as one column; scalar_width returns Nothing for a control. The package README lists the complete API, policies and data update procedure.

find_in and find_not_in are the str forms of the Bytes scans: set is a Bytes buffer or a bytes literal, borrowed, and the whole text is scanned in one native call that allocates nothing. Only ASCII scalars can be members; set bytes from 0x80 match nothing, so a non-ASCII scalar never matches find_in and always stops find_not_in. Results are scalar positions, as for str.find, and there are no bounds arguments. Empty text gives Nothing; an empty set makes find_in Nothing and find_not_in Some(0) for nonempty text. Use them to validate a character class without a per-scalar loop:

import std.text

def token(text: &str) -> bool:
    text.byte_len() > 0 and std.text.find_not_in(text, b"!#$%&'*+-.^_`|~0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz").is_none()

def main() -> Result[(), AllocError | IoError]:
    print(token("GET"), token("G T"), token("GÉT"), std.text.find_in("a=b", b"="))?
    Ok(())
True False False Option[i64].Some(1)

find_in costs O(text bytes), plus counting the scalars before a match; find_not_in costs O(bytes up to the first non-member).

The builder owns a compact Bytes buffer. finish(self: Builder) consumes it; use to_text if the builder must survive an allocation failure for retry or remain available for reuse. Both operations currently copy the bytes into a new string; finish does not promise zero-copy conversion. Other methods borrow it; append and clear require exclusive access, and ordinary loan checking rejects mutation while shared access remains live. There is no global mutable state or thread affinity. Appending takes O(input bytes) amortized time; inputs longer than four UTF-8 bytes need a temporary encoding. Small appends reuse spare capacity, and reserve geometrically when they grow; growth can temporarily retain old and replacement allocations. Storage capacity grows geometrically and may exceed the logical byte limit; the limit is not an exact resident-memory quota. to_text copies O(output bytes). Joining and hard wrapping take O(input + output bytes) amortized time, plus output storage. Hard wrapping writes scalars directly into geometrically grown storage.

Algorithms are checked Plenty using std.bytes; no new OS backend is required. Sources and storage primitives are bundled for offline compilation with a relocated matching toolchain on x86_64 Linux GNU. See the runnable lesson.

See word wrapping and indentation for wrap, fill, shorten, dedent, indent, indent_with, and reusable TextWrapper.

Internal scalar and UTF-8 cursors use explicit arithmetic opt-outs justified by actual input spans and checked builder budgets. Hyphen scanning clamps lookahead to the input and adds an ASCII run’s length to the byte cursor, rather than forming a potentially overflowing sum of unrelated offsets. Width/placeholder validation and input/output limits retain their existing TextError contracts; caller arithmetic modes do not change them.

Acknowledgements

Word wrapping and indentation

The paragraph tools in std.text follow Python’s textwrap. They operate on owned UTF-8 strings through borrowed inputs, return independent results, and report failures through TextError. wrap finds word boundaries and returns lines; hard_wrap inserts literal line breaks at scalar counts.

Quick use

import std.text

def main() -> Result[(), IoError | std.text.TextError]:
    print(std.text.wrap("Hello there, how are you?", 12)?)?
    print(std.text.fill("Hello there, how are you?", 12)?)?
    print(std.text.shorten("Hello  world!", 11)?)?
    print(std.text.indent(&std.text.dedent("    first\n      second")?, "> ")?)?
    Ok(())
["Hello there,", "how are you?"]
Hello there,
how are you?
Hello [...]
> first
>   second
FunctionResult and policy
wrap(text, width=70, max_bytes=1048576, max_input_bytes=1048576)list[str], without appended line terminators
fill(text, width=70, max_bytes=1048576, max_input_bytes=1048576)The same lines joined with LF
shorten(text, width, placeholder=" [...]", max_bytes=1048576, max_input_bytes=1048576, fix_sentence_endings=False, break_long_words=True, break_on_hyphens=True)Collapse Unicode whitespace, then keep enough whole chunks to fit one line with the placeholder when truncated
dedent(text, max_bytes=1048576, max_input_bytes=1048576)Remove a common literal space/tab prefix from nonblank LF-delimited lines
indent(text, prefix, max_bytes=1048576, max_input_bytes=1048576)Prefix each nonblank line, retaining original line endings
indent_with(text, prefix, predicate, max_bytes=1048576, max_input_bytes=1048576)Prefix lines selected by a Callable[[&str], bool]

Text and prefix arguments are borrowed &str. Boolean options on shorten are independent flags; pass changed flags by name. shorten first collapses runs of Unicode White_Space to single ASCII spaces and trims the ends. Its wrapping and truncation then follow the same rules as a one-line wrapper; turning off long-word breaking does not disable the width used for truncation.

dedent ignores whitespace-only lines when finding the common margin and normalizes their bodies to empty strings. Tabs and spaces are different characters; a tab is not expanded to a number of spaces. It preserves LF positions and the rest of nonblank lines, including their CR characters. indent uses the builtin splitlines rules, including CRLF and Unicode line separators. A final terminator does not create an extra line. The predicate sees each full line, including its terminator, and is invoked once per line, in order. To indent even blank lines:

import std.text

def main() -> Result[(), AllocError | IoError | std.text.TextError]:
    result = std.text.indent_with("one\n\ntwo\n", "+ ", def(line: &str) -> bool: True)?
    print(str.repr(result)?)?
    Ok(())
"+ one\n+ \n+ two\n"

Reusable configuration

text_wrapper(width=70, max_bytes=1048576, max_input_bytes=1048576) returns a TextWrapper without allocating. Its public fields below may be changed between calls. wrap(text) and fill(text) borrow both the wrapper and input; they retain neither results nor input and do not modify configuration.

FieldDefaultMeaning
width: Positive[i64]70Line width in Unicode scalars, including indentation
initial_indent: str""Prefix on the first produced line
subsequent_indent: str""Prefix on other lines
expand_tabs: boolTrueExpand tabs to tab stops before wrapping
tabsize: NonNegative[i64]8Tab stop interval; zero deletes tabs when expansion is enabled
replace_whitespace: boolTrueReplace ASCII tab/LF/VT/FF/CR with spaces after expansion
fix_sentence_endings: boolFalseChange a single inter-chunk space after an ASCII lowercase sentence ending to two spaces
break_long_words: boolTrueSplit chunks that cannot fit on a line
drop_whitespace: boolTrueDiscard whitespace chunks at line ends and at the start of subsequent lines
break_on_hyphens: boolTruePrefer compound-hyphen and double-dash boundaries
max_lines: Option[Positive[i64]]NothingOptional truncation limit, with a placeholder on the last line
placeholder: str" [...]"Truncation suffix; leading whitespace is removed when it must stand alone
max_bytes: NonNegative[i64]1048576Total UTF-8 output bytes
max_input_bytes: NonNegative[i64]1048576Input and expanded intermediate byte limit
import std.text

def main() -> Result[(), IoError | std.text.TextError]:
    mut wrapper = std.text.text_wrapper(16)
    wrapper.initial_indent = "* "
    wrapper.subsequent_indent = "  "
    print(wrapper.fill("one two three four five")?)?
    wrapper.max_lines = Some(1)
    wrapper.placeholder = "..."
    print(wrapper.fill("one two three four five")?)?
    Ok(())
* one two three
  four five
* one two...

Wrap a single paragraph at a time. With whitespace replacement enabled, LF becomes a space. With replacement disabled, original control characters may remain inside a returned line. Leading paragraph whitespace is preserved when it fits with content, and empty or whitespace-only paragraphs normally produce no lines. Whitespace within a paragraph is otherwise preserved, rather than collapsed. Disable break_long_words to keep long words intact even when they exceed width. Indentation at least as wide as width can also exceed it: long-word breaking still consumes at least one scalar per iteration.

These operations count scalars, not terminal columns or grapheme clusters; combining marks may begin a line. Hyphen scanning uses the existing runtime Unicode Alphabetic and Numeric properties plus underscore. These properties follow the Rust toolchain’s Unicode version, as in std.regex, and are not a promise of exact CPython Unicode-category equivalence. Ordinary word splitting recognizes ASCII whitespace only; nonbreaking spaces stay inside chunks. Blank-line classification, trimming, and shorten use Plenty’s Unicode White_Space property, which excludes Python’s extra U+001C–U+001F whitespace classifications. Sentence detection is the intentionally limited English heuristic of lowercase ASCII letter, ., !, or ?, optionally a closing quote.

Errors, limits, and cost

All operations return Result[..., TextError]. Widths and max_lines are Positive[i64]; byte limits and tabsize are NonNegative[i64], so a literal outside those ranges does not compile and a computed value needs Positive.try_from(value)? or NonNegative.try_from(value)?. Indentation or a placeholder that cannot fit the width is InvalidLimit. A valid cap exceeded by input, expansion or output is LimitExceeded. Invalid configurations are checked when an operation is called; constructing or editing a wrapper is infallible. Indent strings and an active placeholder are each bounded by max_input_bytes, as is the prefix for indent and indent_with.

max_bytes counts the sum of the UTF-8 bytes in wrap/TextWrapper.wrap lines. For fill, the inserted LF bytes count too. max_input_bytes bounds the original input and tab-expanded intermediate separately, so even shorten rejects an input above that cap instead of allocating an unbounded temporary. A huge tab size is checked against the cap before expanding. Chunk and line counts are bounded by the input scalar count; sentence correction can add at most one extra space per input scalar. The limits bound logical data, rather than exact resident memory: list slots, geometric capacities, temporary string copies, and old allocations during growth also take space.

Scanning and copying are O(input + expanded input + output), with storage proportional to the capped intermediate, output, and per-chunk/per-line metadata, plus four bytes per expanded input scalar for the splitter’s lookaround array. A split long word retains scalar and UTF-8 byte cursors, avoiding repeated suffix scans and copies without another scalar array. Unchanged whitespace preparation shares the immutable input; tab expansion fills each space run in one bulk operation. Complete chunks are reused during line assembly, and dedent keeps only the common ASCII margin rather than allocating one list per line. shorten uses native whitespace splitting and joining and releases that word list before wrapping. Small builder appends write bytes directly into geometrically grown storage; empty appends allocate nothing. Allocation failure is recoverable as Allocation(AllocError) or Bytes(BytesError) with the original byte-buffer cause. Failure discards the partial result and preserves borrowed inputs and wrapper configuration; a predicate’s own effects are not undone. fill materializes the bounded line list before joining it, so it retains line metadata and an additional output copy. No global state, native worker, regex engine, or additional dependency is needed.

Performance qualification

examples/textwrap_bench.plenty in the repository provides seven repeatable workloads. Optimized Plenty executables were compared against commit 7d253e5b on Linux x86-64, AMD Ryzen 7 7840HS, pinned to CPU 0. Each number below is the median time per operation from three alternating before/after process pairs, each doing three operations, at workload scale 262144 (approximately 256 KiB of input). Input construction and printing are outside the timed region; both binaries produced the same byte-count checksums. Both used 8 MiB input/output limits to cover expanded tabs.

WorkloadBeforeAfterSpeedup
Prose fill109.39 ms27.71 ms3.95×
Long Unicode word31.67 ms7.02 ms4.51×
Long dash run94.71 ms15.70 ms6.03×
Tab-heavy fill296.89 ms101.99 ms2.91×
Dedent12.56 ms3.62 ms3.47×
Indent7.60 ms3.73 ms2.04×
Shorten110.22 ms18.96 ms5.81×

The 16, 64 and 256 KiB scale probes showed approximately linear growth. At the largest scale, peak whole-process RSS for dash wrapping fell from 15,684 to 4,072 KiB, and tab wrapping from 27,132 to 14,032 KiB; RSS includes input construction, allocator retention and all three repetitions. At scale 65536, allocation-failure budget probes required 28,912 successful checked allocations for prose fill instead of 236,498, and 6,725 for shorten instead of 309,813. These counts measure Plenty’s injected allocation checks, not operating-system allocation calls. The remaining costs include eager chunk/line storage and encoding larger pieces for the builder. These are local measurements; elapsed time and allocator retention depend on the platform.

Acknowledgements

  • CPython textwrap provides the algorithms and API model, adapted to bounded Plenty storage and a scanner instead of regex splitting. Copyright (C) 1999–2001 Gregory P. Ward; Copyright (C) 2002 Python Software Foundation; PSF-2.0. The complete license is bundled in release third-party notices.

std.thread

Explicit imports for scoped threads. Exports: spawn, ThreadError, and SpawnError[F]. These are the same compiler facilities and types available in the prelude, including through module and item aliases. Importing allocates nothing and introduces no wrapper or new execution model. Supported target: x86_64 Linux GNU.

from std.thread import spawn, SpawnError

def total(values: &list[i64], start: i64, stop: i64) -> Result[i64, OverflowError[i64]]:
    mut sum = 0
    for i in range(start, stop):
        sum += values[i]
    Ok(sum)

def main() -> Result[(), AllocError | IoError | SpawnError[()] | OverflowError[i64]]:
    values = [n for n in range(1, 101)]?
    with spawn(total, &values, 0, 50)? as low, spawn(total, &values, 50, 100)? as high:
        print(low.join()? + high.join()?)?
    Ok(())
5050

Use with spawn(worker, arguments...)? as task: for a named function, or with spawn(job)? as task: for an owned zero-argument job. task.join() waits and moves the result once; leaving the scope joins automatically. Tasks remain lexical handles and cannot escape. Borrowed arguments retain their loans until join. The ordinary worker eligibility, foreign-effect, and channel hang checks apply to every imported spelling.

A start with only owned or copied arguments, spawn(job) or spawn(worker, arguments...), reports SpawnError[F].Unavailable(job, code), retaining the job for recovery; a named start’s job binds its arguments. .drop_job() on the start drops the job and reports the job-free SpawnError[()]. A start that borrows, such as spawn(total, &values, 0, 50), reports SpawnError[()].Unavailable((), code) and drops its moved arguments, since a loan cannot be returned in an error. ThreadError remains the payload of PoolError.Thread. See owned jobs and start failures. Job handoff and bookkeeping allocate no Plenty heap storage; native thread creation uses OS resources. There is no hidden clone. Start/join bookkeeping has fixed overhead apart from argument/result transfer and cleanup, but scheduling and joining have no bounded latency. A join waits as long as the worker needs.

Scoped spawn(...)? propagates the start error like any ?: the enclosing Result’s error must be the start error, or an anonymous sum containing it. spawn(...).unwrap() traps on a failed start.

The prelude spellings remain supported. There is no detached thread, global pool, forced interruption, or import-only exemption from ownership checks. For examples see explicit concurrency imports.

std.time

std.time provides nonnegative spans, monotonic instants, deadlines, and wall-clock timestamps as four distinct types, with clock reads and sleep. The arithmetic is ordinary embedded Plenty code. The runtime supplies only two clock reads and one sleep. No operation allocates.

Every time value is a small class, so it is affine: pass it by reference, as every operation here does, and use copy for an independent value. == compares values structurally. Ordering has no operator; compare as_nanoseconds() results or use the operations below.

from std.time import deadline_after, instant_at, milliseconds, monotonic_now, seconds, sleep

def main() -> Result[(), RangeError | IoError]:
    start = monotonic_now()
    sleep(&milliseconds(20))
    elapsed = monotonic_now().duration_since(&start)?
    print(elapsed.as_milliseconds() >= 20)?

    # A fake timeline makes deadline arithmetic exact.
    deadline = deadline_after(&instant_at(0), &seconds(5))?
    later = instant_at(seconds(2).as_nanoseconds())
    print(deadline.remaining(&later).as_milliseconds(), deadline.expired(&later))?
    Ok(())
True
3000 False

Durations

A Duration is a span of u64 nanoseconds, from zero to just over 584 years. A negative span is unrepresentable.

OperationResultContract
nanoseconds(count: u64)DurationExact
microseconds(count: u32), milliseconds(count: u32), seconds(count: u32)DurationExact; a u32 count always fits
span.as_nanoseconds()u64Exact
span.as_microseconds(), as_milliseconds(), as_seconds()u64Whole units, discarding the remainder
span.as_timeout_ms()u64Milliseconds rounded up
span.is_zero()bool
span.checked_add(&other)Result[Duration, RangeError]Error above the maximum
span.checked_sub(&other)Result[Duration, RangeError]Error when other is longer
span.saturating_sub(&other)DurationZero when other is longer
span.checked_mul(factor: u64)Result[Duration, RangeError]Error above the maximum

Constructors never fail and arithmetic never traps or wraps. Build a span of minutes or longer by multiplying, as in seconds(60).checked_mul(5). There is no float conversion, division, or text format.

Instants and deadlines

An Instant is a point on a monotonic timeline: it never moves backwards and is unaffected by changes to the system clock. Its origin is unspecified, so an instant means something only relative to another from the same timeline.

OperationResultContract
monotonic_now()InstantRead the process’s monotonic clock
instant_at(nanoseconds_after_origin: u64)InstantA point on a caller-defined timeline
at.checked_add(&span)Result[Instant, RangeError]Error past the end of the timeline
at.duration_since(&earlier)Result[Duration, RangeError]Error when earlier is later
at.saturating_duration_since(&earlier)DurationZero when earlier is later

A Deadline is the instant at which a total budget ends. Code that receives a &Deadline can make several waits without restarting the budget.

OperationResultContract
deadline_after(&start, &budget)Result[Deadline, RangeError]Error past the end of the timeline
deadline_at(&at)Deadline
deadline.expired(&now)boolTrue once now reaches the deadline
deadline.remaining(&now)DurationZero once expired

A zero budget is expired at its own start. Deadline operations take the current instant as an argument and read no clock, so their results depend only on their inputs. instant_at supplies the fake clock for simulations and tests; mixing its instants with monotonic_now() readings is meaningless and undetected.

The monotonic clock is Linux CLOCK_MONOTONIC, reported as nanoseconds since the process’s first reading. It does not advance while the machine is suspended. Readings are comparable across threads of one process and not across processes.

Clocks

Clock is the protocol for code that reads the time through a value instead of calling monotonic_now(), so a test can control what it sees:

pub protocol Clock:
    def now(self) -> Instant:
        pass

monotonic_clock() returns a MonotonicClock, whose now() is monotonic_now(); it holds no state. std.test.fake_clock returns a clock that moves only when told. A function written as def f[C: Clock](clock: &C) is compiled once per clock type, with no dynamic dispatch.

Wall-clock timestamps

A Timestamp has public fields seconds: i64, counted from the Unix epoch in UTC, and nanoseconds: u32 in [0, 1_000_000_000). Times before the epoch use floor seconds: half a second before it is Timestamp(-1, 500000000). std.fs metadata returns this type.

OperationResultContract
wall_now()TimestampRead the system clock once
Timestamp(seconds, nanoseconds)TimestampUnchecked field construction
stamp.duration_since(&earlier)Result[Duration, RangeError]Error when earlier is later, or the span exceeds a Duration
stamp.checked_add(&span)Result[Timestamp, RangeError]Error past i64 seconds

The system clock can be stepped in either direction, by an administrator or by time synchronization. A later reading may therefore be earlier, and duration_since reports that as an error instead of a negative or wrapped span. Measure elapsed time and timeouts with instants. A timestamp cannot be converted to an instant or compared with one.

A timestamp whose nanoseconds is out of range is an error in both operations. There is no subtraction of a span, leap-second table, parsing, or formatting here; the tz package provides calendars, zones, and text.

Civil time and time zones

Calendar dates, wall-clock times, IANA time zones, and RFC 3339 text live in the official tz package, not in std. It bundles a pinned release of the IANA time-zone database as Plenty source, so results do not depend on the host’s zone files, and a program that never looks up a zone links no zone data. Its API follows Rust’s jiff. Add it as a Git dependency:

[dependencies]
tz = { git = "https://github.com/plentylang/tz.git", tag = "v0.3.0" }

A tz.Zoned holds a Timestamp with its zone. Adding calendar units keeps the wall-clock reading; a local time that a transition skips or repeats is resolved by the tz.Disambiguation the caller passes:

import tz

def main() -> Result[(), tz.TzError | FormatError | AllocError | IoError]:
    new_york = tz.zone("America/New_York")?
    london = tz.zone("Europe/London")?
    meeting = tz.datetime(2024, 3, 9, 9, 30)?.to_zoned(&new_york, tz.Disambiguation.Reject)?
    next_week = meeting.checked_add(&tz.span(weeks=1), tz.Disambiguation.Reject)?
    print(f"{next_week}"?)?
    print(f"{next_week.in_zone(&london)?}"?)?
    print(meeting.span_until(&next_week).hours())?
    Ok(())
2024-03-16T09:30:00-04:00[America/New_York]
2024-03-16T13:30:00+00:00[Europe/London]
167

span_until measures elapsed time: the 167 hours above include the hour the DST transition skips. zoned.until(&other, largest, disambiguation) and since instead give a difference in calendar units up to largest (tz.Unit.Year down to Nanosecond), computed on the wall clock as jiff does; calendar units need both values in the same zone.

tz.zoned(&stamp, &zone) and zoned.timestamp() convert between the two types.

tz.zone(name) can return any zone, so it links the whole database (about 150 KB). The tz.zones module has one function per canonical zone, and each links only that zone’s data. The functions cannot fail:

import tz
from tz.zones import america_new_york

def main() -> Result[(), tz.TzError | AllocError | IoError]:
    new_york = america_new_york()
    meeting = tz.datetime(2024, 3, 10, 2, 30)?.to_zoned(&new_york, tz.Disambiguation.Compatible)?
    print(meeting.offset().seconds())?
    Ok(())

This prints -14400: 02:30 falls in that day’s gap, and Compatible moves it forward to 03:30 EDT. The package README lists the API, how zone names map to function names, its data version, and how the data is regenerated.

Sleeping and timed waits

OperationResultContract
sleep(&span)()Block this thread for at least the span
sleep_until(&deadline)()Block until the deadline; return at once if expired

Sleep measures the monotonic clock, resumes after signals, and cannot be cancelled. Scheduling may extend it; nothing shortens it. Workers may sleep and read both clocks.

Prefer std.sync’s typed waits: recv_before, send_before, select_recv_before, future_before and cancelled_before take a shared deadline and a cancellation token. The wrappers keep one total budget and preserve unsent values on every failure.

Low-level timed waits have explicit _timeout_ms names and accept u64 milliseconds. as_timeout_ms() rounds up so a positive remainder does not become a zero-budget retry loop. For example, token.wait_timeout_ms(span.as_timeout_ms()) observes that token without a second cancellation source. The old _timeout names remain aliases.

An expired deadline yields a zero budget, which those operations define as an immediate check where readiness still wins.

Errors, costs, and limits

Additions, products and non-saturating differences handle local checked arithmetic and map unrepresentable results to the existing RangeError.OutOfRange. Infallible unit conversions, saturating differences and fractional timestamp arithmetic use explicit unchecked: suites with bounds documented in source. Their guards prove nonzero divisors and representable results; no operation relies on silent wrapping except the explicit two’s-complement difference used for ordered timestamp seconds.

Every recoverable failure is the built-in RangeError.OutOfRange: the result is not representable in its type. Time arithmetic therefore propagates with ? alongside T.try_from. Clock reads and sleep cannot fail.

Each operation takes constant time and allocates nothing, including clock reads and sleep. Values hold no OS resource and may move between threads.

Supported target: x86_64-unknown-linux-gnu. See the lessons Measure time and wait and Share one deadline across waits.

Design decisions

  • Four types, no conversions between clocks. A span, a monotonic point, a budget end, and a wall time have different valid operations. Distinct types reject using a wall time for a timeout at compile time.
  • Integer nanoseconds. One exact representation with no rounding rule. Unit constructors take u32 so that construction is infallible; arithmetic is where overflow can occur, and it is always checked.
  • No hidden clock reads. expired and remaining take now. A function’s clock reads are visible at its call sites, and a fake clock needs no protocol, global state, or test-only mode.
  • Explicit rounding names. as_milliseconds discards and as_timeout_ms rounds up. Neither is a default.
  • Existing waits are unchanged. Millisecond budgets, cancellation tokens, and their wakeup rules remain the single mechanism; this module supplies the arithmetic that feeds them.
  • One timestamp type. std.fs reuses Timestamp instead of declaring its own, so file times and clock readings subtract directly.

Acknowledgements

  • Duration, Instant and their checked and saturating operations are modelled on Rust’s std::time.

std.tls

import std.tls runs TLS 1.2 and 1.3 over a std.net TCP stream. A client verifies the server’s certificate chain against chosen trust roots and checks that it names the server; a server presents a certificate chain and key, and may ask clients for theirs. Handshakes, reads, and writes take the same deadline and cancellation token as the TCP stream under them. The module has one error type, TlsError, and is available on the supported x86-64 Linux GNU target.

Upstream code, used as-is. The TLS protocol, certificate verification and all cryptography are rustls with its ring provider, compiled into the runtime. Plenty implements none of them; this is deliberate, as for std.crypto.

A rustls session never touches a socket: std.tls moves ciphertext between the session and the TCP stream in checked Plenty, so every wait is a std.net wait.

import std.bytes
import std.tls
from std.net import NetError, TcpListener, connect, listen, loopback_v4
from std.sync import CancellationToken
from std.time import Deadline, deadline_after, milliseconds, monotonic_now
from std.tls import ServerConfig, TlsError

# Test-only CA and a localhost leaf it signed; never reuse a published key.
const CA_PEM: str = "-----BEGIN CERTIFICATE-----\nMIIBmjCCAT+gAwIBAgIULOQ2Qxr8CVZY2bG0uRYKKEdMD3swCgYIKoZIzj0EAwIw\nGTEXMBUGA1UEAwwOUGxlbnR5IFRlc3QgQ0EwIBcNMjQwMTAxMDAwMDAwWhgPMjEy\nNDAxMDEwMDAwMDBaMBkxFzAVBgNVBAMMDlBsZW50eSBUZXN0IENBMFkwEwYHKoZI\nzj0CAQYIKoZIzj0DAQcDQgAEkiB3Ogehrrt6YbQghwDPv6tkhE+z9zTCVih8fvtR\nZ1/691Nfh7/J9IgaNgNSTMJjGTolzGnjaCUDqoARlFyM+6NjMGEwHQYDVR0OBBYE\nFOTFBDgWXHj/qmVJDMlAHYCEZFX5MB8GA1UdIwQYMBaAFOTFBDgWXHj/qmVJDMlA\nHYCEZFX5MA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgIEMAoGCCqGSM49\nBAMCA0kAMEYCIQD41ATQnIjKkKMLQAb9AyoyOFmljYZyz348XJEIbqMgKwIhAL3m\nmWd8On5gvkfytspatH5mDWOcSyW+au7vuIC2gHhO\n-----END CERTIFICATE-----\n"
const LEAF_PEM: str = "-----BEGIN CERTIFICATE-----\nMIIBnDCCAUKgAwIBAgIBATAKBggqhkjOPQQDAjAZMRcwFQYDVQQDDA5QbGVudHkg\nVGVzdCBDQTAgFw0yNDAxMDEwMDAwMDBaGA8yMTI0MDEwMTAwMDAwMFowFDESMBAG\nA1UEAwwJbG9jYWxob3N0MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE8THNQeTi\nJ6Eq0tSxtHW5Q4TCsbIqB3MOtM98ERyKgl6kHeHCeDZYKx8B+VRGT2jkWuTeN24Q\nljwUfh7uh7Iaz6N+MHwwGgYDVR0RBBMwEYIJbG9jYWxob3N0hwR/AAABMAkGA1Ud\nEwQCMAAwEwYDVR0lBAwwCgYIKwYBBQUHAwEwHQYDVR0OBBYEFEIFC2Fyd4cfAKTx\nMiNCMdDkqDNbMB8GA1UdIwQYMBaAFOTFBDgWXHj/qmVJDMlAHYCEZFX5MAoGCCqG\nSM49BAMCA0gAMEUCIAUF3HH43lnirgSN1Vr4twZrFmXKBxlbYNQni/h4ZaxhAiEA\n5rD/yE1B4IsbEGCMlyAeX507leXfGY9+wrHaxikU/w0=\n-----END CERTIFICATE-----\n"
const LEAF_KEY: str = "-----BEGIN PRIVATE KEY-----\nMIGHAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBG0wawIBAQQglvSEE3H7v+mInxSF\nAyWFDKYpzpXr20wH8mfFY3bCaB2hRANCAATxMc1B5OInoSrS1LG0dblDhMKxsioH\ncw60z3wRHIqCXqQd4cJ4NlgrHwH5VEZPaORa5N43bhCWPBR+Hu6HshrP\n-----END PRIVATE KEY-----\n"

def serve(listener: &TcpListener, config: &ServerConfig, deadline: &Deadline, cancel: &CancellationToken) -> Result[(), AllocError | TlsError | NetError | std.bytes.BytesError]:
    mut stream = std.tls.accept(listener.accept(deadline, cancel)?, config, deadline, cancel)?
    reply = std.bytes.from_text("hello over TLS")?
    stream.write_all_before(&reply, deadline, cancel)?
    stream.close_before(deadline, cancel)

def main() -> Result[(), SpawnError[()] | IoError | AllocError | RangeError | TlsError | NetError | std.bytes.BytesError]:
    cancel = CancellationToken()?
    deadline = deadline_after(&monotonic_now(), &milliseconds(5000))?
    client = std.tls.client_config(&std.tls.roots_from_pem(&CA_PEM)?)?
    server = std.tls.server_config(&LEAF_PEM, &LEAF_KEY)?
    listener = listen(&loopback_v4(0u16), 8)?
    address = listener.local_address()?
    with spawn(serve, &listener, &server, &deadline, &cancel)? as worker:
        tcp = connect(&address, &deadline, &cancel)?
        mut stream = std.tls.connect(tcp, &client, "localhost", &deadline, &cancel)?
        mut buffer = std.bytes.with_capacity(64)?
        buffer.resize(64, 0u8)?
        count = stream.read_before(&mut buffer, 0, 64, &deadline, &cancel)?
        print(buffer.to_text_range(0, count)?)?
        worker.join()?
    Ok(())
hello over TLS

Ciphertext cursors are bounded by their fixed buffers, and each native TLS or TCP transfer advances by at most the remaining slice. Their small arithmetic opt-outs do not change TlsError, partial progress after a failed wait, or closing the TCP stream after a failed close-notify.

Trust roots and configuration

FunctionResultContract
roots_from_pem(pem: &str)Result[TrustRoots, TlsError]The CERTIFICATE blocks of pem as trust anchors
system_roots()Result[TrustRoots, TlsError]The operating system’s certificate bundle
client_config(roots: &TrustRoots)Result[ClientConfig, TlsError]Client settings verifying servers against roots
server_config(chain_pem: &str, key_pem: &str)Result[ServerConfig, TlsError]Server settings presenting chain_pem, leaf first, with its private key

roots_from_pem skips blocks of other types. PEM that does not decode is InvalidCertificate. A decoded certificate that cannot serve as a trust anchor is skipped, since system bundles may hold some; when none is usable the result is NoTrustRoots. TrustRoots.len() counts the anchors kept.

system_roots reads the first readable file of:

  1. /etc/ssl/certs/ca-certificates.crt (Debian, Ubuntu, Arch, Alpine)
  2. /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem (Fedora, RHEL)
  3. /etc/ssl/ca-bundle.pem (openSUSE)
  4. /etc/ssl/cert.pem

None readable is NoTrustRoots. It does not consult environment variables; read another bundle with read_text and pass it to roots_from_pem. The bundle is read at the call, so a long-running program sees updates only when it builds new roots.

server_config accepts a PKCS#8, PKCS#1, or SEC1 private key. A chain or key that does not parse, an empty chain, or a key that does not match the leaf is InvalidConfiguration. Servers do not ask clients for certificates.

Both configurations use TLS 1.2 and 1.3 with rustls’s default cipher suites and key-exchange groups. Neither resumes sessions: clients keep no session cache and servers issue no tickets. A configuration is immutable and may be borrowed by any number of connections and workers.

A TLS 1.3 server therefore sends nothing after the client’s Finished. So that a client with Nagle’s algorithm on does not hold its first request until the kernel’s delayed acknowledgement (about 40 ms on Linux), a server stream acknowledges the client’s last handshake flight at once (Linux TCP_QUICKACK), unless a reply or the client’s first request is already there to carry the acknowledgement.

Builders

A builder gathers further settings; build checks them together and returns the configuration. A builder with no further calls builds the same configuration as client_config or server_config.

FunctionResult
client_builder(roots: &TrustRoots)Result[ClientBuilder, TlsError]
server_builder(chain_pem: &str, key_pem: &str)Result[ServerBuilder, TlsError]
MethodOnContract
certificate(chain_pem: &str, key_pem: &str) -> Result[(), TlsError]bothClient: the chain presented when a server asks. Server: another chain to choose by name
verify_clients(roots: &TrustRoots, certificates: ClientCertificates) -> ()serverAsk clients for a certificate and verify it against roots
application_protocol(name: &str) -> Result[(), TlsError]bothOffer or accept name in ALPN, after the names already given
minimum_version(version: TlsVersion) -> ()bothTls13 refuses TLS 1.2 peers
cipher_suite(suite: CipherSuite) -> ()bothAllow suite; once any is allowed, only allowed suites are used
revocation_list(crl_pem: &str) -> Result[(), TlsError]bothReject peer certificates that a CRL in crl_pem names
build() -> Result[ClientConfig or ServerConfig, TlsError]bothThe configuration
pub enum ClientCertificates:
    Required
    Optional

pub enum TlsVersion:
    Tls12
    Tls13

pub enum CipherSuite:
    Tls13Aes256GcmSha384
    Tls13Aes128GcmSha256
    Tls13Chacha20Poly1305Sha256
    EcdheEcdsaAes256GcmSha384
    EcdheEcdsaAes128GcmSha256
    EcdheEcdsaChacha20Poly1305Sha256
    EcdheRsaAes256GcmSha384
    EcdheRsaAes128GcmSha256
    EcdheRsaChacha20Poly1305Sha256

Client certificates. With verify_clients, a server asks for a client certificate and verifies its chain against roots, which are usually a private authority rather than the system bundle. With Required, a client that presents none fails with CertificateRequired on the server. With Optional, it is accepted, and peer_certificate() is Nothing. A client has at most one chain: a second certificate call is InvalidConfiguration. In TLS 1.3 the client finishes its handshake before the server has checked its certificate, so a rejected client sees Rejected at its first read.

Certificates by name. A server with several chains presents the first whose leaf names the host the client asked for through server name indication (SNI), or the first chain when none does or the client sent no name (clients connecting by IP address send none).

Application protocols. A server picks the first of its names, in the order given, that the client offers. A client that offers names, none of which the server accepts, fails with Incompatible on the server and Rejected on the client. When either side gives no names, none is chosen. A name is 1 to 255 bytes; others are InvalidConfiguration.

Versions and suites. Suites are the ring provider’s, listed in its default order of preference. The client’s preference decides among suites both sides allow. Allowed suites that cover no allowed version are InvalidConfiguration at build. Peers with no version or suite in common fail with Incompatible, or Rejected on the side that receives the alert.

Revocation lists. revocation_list adds each X509 CRL block of crl_pem; PEM without one is InvalidConfiguration. A certificate that a list from its issuer names is RevokedCertificate. Certificates whose issuer has no list are accepted, and a list’s next-update time is not enforced, so a stale list still applies. A server revocation_list without verify_clients is InvalidConfiguration at build. Lists are read once; load fresh ones by building a new configuration.

Connecting

FunctionResultContract
connect(tcp: TcpStream, config: &ClientConfig, server_name: &str, deadline: &Deadline, cancel: &CancellationToken)Result[TlsStream, TlsError]Complete a client handshake over tcp
accept(tcp: TcpStream, config: &ServerConfig, deadline: &Deadline, cancel: &CancellationToken)Result[TlsStream, TlsError]Complete a server handshake over tcp

Both take ownership of the TCP stream and return only after the handshake completes. server_name is a DNS name or an IP address; the server’s leaf certificate must name it in its subject alternative names. A name that is neither is InvalidName. Certificates are checked against the wall clock at the handshake.

When the handshake fails for a TLS reason, the alert rustls prepared is sent if the deadline allows, so the peer sees Rejected. On any failure the TCP stream is closed when the partial session is dropped.

start_connect(tcp, config, server_name) and start_accept(tcp, config) return a stream without waiting. Its handshake then runs inside its first reads and writes, in handshake_before, or in try_handshake; no application data passes before the peer is verified.

Streams

MethodResultContract
handshake_before(deadline: &Deadline, cancel: &CancellationToken)Result[(), TlsError]Finish the handshake. One that times out keeps its progress for a later call, so with a deadline already passed it does only the work possible without waiting
read_before(buffer: &mut Bytes, start: i64, stop: i64, deadline: &Deadline, cancel: &CancellationToken)Result[i64, TlsError]Decrypted bytes into buffer[start:stop], waiting for at least one
write_before(source: &[u8], deadline: &Deadline, cancel: &CancellationToken)Result[i64, TlsError]Encrypt and send at most 16 KiB of source; the count taken
write_all_before(source: &[u8], deadline: &Deadline, cancel: &CancellationToken)Result[(), TlsError]Send all of source
close_before(deadline: &Deadline, cancel: &CancellationToken)Result[(), TlsError]Send close_notify, then close the TCP stream
shutdown(direction: Shutdown)Result[(), TlsError]End the TCP stream in one direction without closing it; ending writes sends close_notify first without waiting, and ciphertext the socket does not take at once is lost. Reading continues
read(buffer: &mut Bytes, start: i64, stop: i64)Result[i64, StreamError]std.io Reader; waits without a deadline
write(source: &[u8])Result[i64, StreamError]std.io Writer; waits without a deadline
flush()Result[(), StreamError]Nothing to do: writes are sent as they are made
close()Result[(), StreamError]close_before without a deadline
application_protocol()Result[Option[str], TlsError]The protocol chosen through ALPN
server_name()Result[Option[str], TlsError]On a server, the name the client sent through SNI; Nothing on a client
peer_certificate()Result[Option[Bytes], TlsError]The DER encoding of the peer’s verified leaf certificate
version()Option[TlsVersion]The negotiated version
cipher_suite()Option[CipherSuite]The negotiated suite

Each source is a &[u8] view: a Bytes buffer, part of one, or a b"..." literal, written without a copy.

The session details are Nothing until the handshake has settled them. peer_certificate() is Nothing on a server that did not verify clients or whose client presented none.

Ranges follow the range rule. A read of zero bytes means the peer sent close_notify. A connection that ends without one is Truncated (StreamError.UnexpectedEof through read), because an attacker can cut a TCP connection short. Code that frames its own messages, such as Content-Length bodies, may treat a truncation after the last expected byte as an ordinary end.

close_before does not wait for the peer’s close_notify. It closes the TCP stream even when sending close_notify fails, and then returns that error. After a close, writes are Closed. A failed write has sent an unknown prefix of its range.

The std.io methods block as TcpStream’s do, so TlsStream works with std.io helpers and std.http framing. Through them, TLS failures appear as StreamError.Io(IoError.Other); use the _before methods to see the TlsError.

Without waiting

These methods never wait, so one thread can drive many streams from a std.ready selector:

MethodResultContract
try_handshake()Result[bool, TlsError]Advance the handshake; true once complete
try_read(buffer: &mut Bytes, start: i64, stop: i64)Result[Option[i64], TlsError]Decrypted bytes if any can be had now; Some(0) is close_notify
try_write(source: &[u8])Result[Option[i64], TlsError]Encrypt at most 16 KiB; Nothing, taking nothing, while the handshake is incomplete or earlier ciphertext is unsent
try_flush()Result[bool, TlsError]Send buffered ciphertext; true once none is left
interest()InterestWhat to wait for: ReadWrite while ciphertext waits to be sent, else Read
watch(selector: &mut Selector, key: u64, interest: Interest)Result[(), TlsError]Register the TCP stream
unwatch(selector: &mut Selector)Result[(), TlsError]Remove it

A try_ result of Nothing or false means the stream needs the socket. These methods use TCP’s immediate operations, without polling with an expired deadline. WouldBlock or interruption leaves the TLS progress buffered and returns Nothing or false; other network errors, including ConnectionTimedOut, propagate. Do all the work available without waiting, then register interest() again and wait: rustls may hold decrypted bytes the socket no longer signals, and a write may leave ciphertext that only a writable socket can take. Ciphertext a try_ call could not send stays buffered for the next call. Registrations are level-triggered, so ReadWrite on an idle stream wakes at once; register Read once interest() says so.

A read_before or handshake_before that times out at its deadline has also done all the work available, so a stream may be watched after one. std.http.server parks HTTPS connections this way, between handshake messages and while idle.

Sessions

A Session is a TLS session without a socket: the caller moves ciphertext through its own buffers. It never waits and owns no descriptor, so it suits event loops that already own the socket. std.actor.tls runs one over an actor connection. client_session(&config, server_name) and server_session(&config) make one with the same verification as connect and accept.

MethodResultContract
feed(source: &[u8])Result[i64, TlsError]Process received ciphertext; the count used, short while decrypted bytes wait for read
feed_end()Result[(), TlsError]The peer’s stream ended; without close_notify, read then fails with Truncated
take(target: &mut Bytes, start: i64, stop: i64)Result[i64, TlsError]Ciphertext to send; 0 when none waits
read(target: &mut Bytes, start: i64, stop: i64)Result[Option[i64], TlsError]Decrypted bytes; Nothing while none is ready, Some(0) for close_notify
write(source: &[u8])Result[i64, TlsError]Encrypt at most 16 KiB; short once 64 KiB of ciphertext waits for take; Closed after close
close()()Queue close_notify for take
handshaking(), wants_write(), wants_read()boolHandshake incomplete; ciphertext waits for take; more ciphertext is useful

application_protocol, server_name, peer_certificate, version and cipher_suite are as for TlsStream. While the handshake runs, write holds plaintext in the session; check handshaking() first to bound it. On a TLS failure the session has queued an alert: send what take returns before closing the socket. A session is allocated where it is made; in an actor handler that is the actor’s heap.

Read ranges follow the Reader contract: invalid buffer bounds trap with read: start/stop outside buffer before I/O or handshake work. The nonblocking session take operation also validates its caller buffer first and traps with take: start/stop outside buffer. Empty ranges must still be within the buffer.

Errors

pub enum TlsError:
    Net(NetError)
    UntrustedCertificate
    ExpiredCertificate
    WrongHost
    InvalidCertificate
    Rejected
    Protocol
    Truncated
    InvalidName
    InvalidConfiguration
    NoTrustRoots
    Closed
    Allocation(AllocError)
    RevokedCertificate
    CertificateRequired
    Incompatible
VariantMeaning
Net(cause)The TCP operation ended, including deadline TimedOut, kernel ConnectionTimedOut, and Cancelled
UntrustedCertificateThe peer’s chain does not lead to a trust root, or a signature is wrong
ExpiredCertificateA certificate in the chain is expired or not yet valid
WrongHostThe leaf certificate does not name server_name
InvalidCertificateA certificate is malformed or unusable for TLS, or PEM does not decode
RejectedThe peer sent a fatal alert, usually because it rejected our certificate
ProtocolThe peer broke the TLS protocol
TruncatedThe connection ended without close_notify
InvalidNameserver_name is neither a DNS name nor an IP address
InvalidConfigurationA chain, key, name, list, or combination of builder settings is unusable
NoTrustRootsNo usable trust anchor was found
ClosedThe stream was closed by this side
Allocation(cause)A buffer or runtime object could not be allocated
RevokedCertificateA revocation list names the peer’s certificate
CertificateRequiredA server requiring client certificates received none
IncompatibleThe peers share no version, cipher suite, or application protocol

Deadlines and cancellation

Every _before method passes its deadline and token to each std.net call it makes, so a handshake against a silent peer ends with Net(NetError.TimedOut) and a cancelled token with Net(NetError.Cancelled). The kernel’s connection timeout is Net(NetError.ConnectionTimedOut) and is not resumable. The try_ methods also return this error; they do not mistake it for a socket that would block. Use one deadline for a whole exchange, as with std.net. Ciphertext that a timed-out write did not send stays buffered and goes out with the next call, but which plaintext the peer has received is unknown; after a timeout in the middle of an exchange, close or drop the stream unless the protocol can recover.

Ownership and threads

TrustRoots, ClientConfig, ServerConfig, TlsStream, and Session each own one runtime object; dropping it releases the object, and dropping a stream closes its socket without close_notify. The configurations, builders, streams and sessions are redacted: printing one shows its type name and (@redacted), never key or session state. A stream may move to a worker. Trust roots, builders and configurations belong to the heap that allocates them. Outside actors, sessions share the parsed configuration. An actor session reconstructs a heap-local configuration from owned source material so a trap cannot retain another heap’s configuration. Starting a session draws randomness and verifying a certificate reads the clock; both are allowed in workers.

The __tls_* intrinsics behind the session are reserved for the standard library. Writing uses __tls_write, which reads the caller’s &[u8] view in place.

Costs

ItemMemory
TlsStreamTwo 16 KiB ciphertext buffers, plus rustls’s session state and its buffer for the incoming record (up to about 17 KiB)
Sessionrustls’s session state and its record buffers; see std.actor.tls for measurements
ConfigurationOwned certificate/key/trust/policy recipe plus parsed configuration; shared by sessions outside actors
TrustRootsThe parsed anchors; a full system bundle holds about 120

rustls allocates through the runtime’s Rust allocator, not Plenty’s fallible allocation, so an allocation failure inside rustls stops the program. rustls has no fallible allocation interface, and stable Rust aborts when the global allocator fails, so this is not recoverable. Inside an actor, exhaustion instead traps that actor and discards its heap, including partial TLS construction. Allocation failures in std.tls itself are Allocation. Actor sessions also pay for reconstructing a private configuration, including key parsing; see actor TLS memory.

Plaintext is copied once into rustls and ciphertext once into the outgoing buffer; incoming ciphertext is decrypted into rustls’s buffer and copied to the caller.

Supply

The runtime builds these crates from unmodified copies under plenty-runtime/vendor (ring’s pregenerated assembly is trimmed to the x86-64 ELF files), so building the compiler needs no network:

CrateVersionLicence
rustls0.23.46Apache-2.0, ISC, or MIT
rustls-webpki0.103.15ISC
rustls-pki-types1.15.1MIT or Apache-2.0
ring0.17.14Apache-2.0 and ISC, with OpenSSL and BoringSSL notices
untrusted, zeroize, once_cell0.9.0, 1.9.1, 1.21.4ISC; MIT or Apache-2.0; MIT or Apache-2.0
getrandom, libc0.2.17, 0.2.186MIT or Apache-2.0

ring’s C and assembly sources are compiled by build.rs with the system C compiler into a static archive, as SQLite’s are. The versions are pinned in Cargo.lock, and a test fails if the two disagree. To update, replace the directories with new releases, run cargo update -p CRATE --precise VERSION, and rerun tests/test_std_tls.rs. Follow rustls security advisories: an update is a change to the compiler, and programs pick it up when rebuilt.

Programs whose main never reaches std.tls code contain none of rustls or ring, even when they import a module that imports std.tls; the linker drops the unreachable functions and the TLS runtime only they call.

Example

The tutorial lesson serves an echo over TLS on a worker and shows a client rejecting the wrong host name. Mutual TLS adds client certificates and application protocols.

Limits

  • No certificate pinning; verify peer_certificate() bytes yourself.
  • No session resumption or 0-RTT data: a shared session cache is memory without a bound or an owner, and 0-RTT data can be replayed.
  • No OCSP: rustls does not check stapled responses, and fetching them would reach the network behind the program’s back. Revocation is by CRL only.
  • No choice of key-exchange groups, and no TLS 1.0 or 1.1.
  • Outside actors, allocation failure inside rustls stops the program; actor TLS contains it by discarding the actor heap.
  • HTTPS in std.http is not built in; wrap a TlsStream with its framing.
  • One target, x86-64 Linux GNU.

Acknowledgements

  • Wraps rustls 0.23.46 with the ring 0.17.14 crypto provider and rustls-webpki 0.103.15, vendored in the runtime; see Supply for licences.

  • The CA bundle paths are drawn from the Linux list in Go’s crypto/x509.

  • Uses zeroize 1.9.1 (MIT or Apache-2.0) to clear retained encoded private keys on normal destruction.

std.toml

import std.toml parses TOML 1.0 documents into a value tree and writes a tree back as TOML text, under explicit limits, with one error type, TomlError.

For editing human-maintained files, document_text and document create a Document that preserves the source layout. Use the value tree writer below for generated files.

The module is checked Plenty code with no native primitives of its own. It reads and writes std.bytes buffers and std.io streams, and is available on the supported x86-64 Linux GNU target.

import std.toml
from std.toml import Toml

def main() -> Result[(), AllocError | IoError | std.toml.TomlError]:
    bounds = std.toml.limits()
    config = std.toml.parse_text("title = \"demo\"\n\n[server]\nport = 8080\nhosts = [\"a\", \"b\"]\n", &bounds)?
    match &config:
        case Toml.Table(entries):
            for entry in entries:
                match &entry.value:
                    case Toml.Text(text):
                        print(entry.key, "is", text)?
                    case Toml.Table(fields):
                        print(entry.key, "has", len(fields), "entries")?
                    case _:
                        print(entry.key, "is another type")?
        case _:
            print("not a table")?
    print(std.toml.encode_text(&config, &bounds)?.rstrip()?)?
    Ok(())
title is demo
server has 2 entries
title = "demo"

[server]
port = 8080
hosts = ["a", "b"]

Accepted text

The parser accepts TOML 1.0 and rejects everything else, including TOML 1.1 additions.

RuleBehavior
EncodingUTF-8, validated strictly before parsing; no byte-order mark
Newlines\n or \r\n; a \r alone is an error everywhere
Comments# to end of line; control characters other than tab are ControlCharacter
KeysBare (A-Za-z0-9_-), basic-quoted, or literal-quoted parts joined by ., with optional spaces or tabs around each .
StringsBasic, literal, multi-line basic, and multi-line literal. A newline right after an opening """ or ''' is dropped; \r\n inside a multi-line string is kept as written
Escapes\b \t \n \f \r \" \\ \uXXXX \UXXXXXXXX, hex digits of either case, Unicode scalar values only. In a multi-line basic string, \ before a line end removes the line end and the whitespace and newlines that follow
Control charactersBytes 0-8, 10-31, and 127 are rejected in every string and comment, except newlines in multi-line strings
IntegersDecimal with optional sign, or 0x, 0o, 0b with lowercase prefix and no sign. _ only between digits; no leading zeros. Must fit i64
FloatsInteger part, then a fraction, an exponent, or both; inf, nan with optional sign. Must be finite unless written as inf or nan
Booleanstrue, false
Date-timesRFC 3339 with T, t, or one space between date and time; seconds required; Z, z, or ±HH:MM offset. Fractional digits beyond nanoseconds are truncated. Year 0000-9999; the day must exist in its month; second 60 is accepted at any time
ArraysMixed element types, newlines and comments between elements, optional trailing comma
Inline tablesOne line, no trailing comma; complete as written

Tables follow the definition rules of the specification:

SituationFault
A key assigned twice in one tableDuplicateKey
A [table] header repeated, or naming a table already made by dotted keys or an inline tableDuplicateTable
A header or dotted key that passes through a value, an inline table, or an array that is not an array of tablesKeyConflict
A dotted key that adds to a table made by a [header] other than the current oneKeyConflict
[[name]] where name is not an array of tablesKeyConflict

A header may name a table that an earlier header created implicitly, such as [a] after [a.b], once. A header path that passes through an array of tables continues in its last table.

Value tree

pub enum Toml:
    Bool(bool)
    Int(i64)
    Float(f64)
    Text(str)
    DateTime(DateTime)
    Array(list[Toml])
    Table(list[Entry])

pub class Entry:
    pub key: str
    pub value: Toml

pub enum DateTimeKind:
    OffsetDateTime
    LocalDateTime
    LocalDate
    LocalTime

pub class DateTime:
    pub kind: DateTimeKind
    pub year: i64
    pub month: i64
    pub day: i64
    pub hour: i64
    pub minute: i64
    pub second: i64
    pub nanosecond: i64
    pub offset_minutes: i64

A document is a Toml.Table. Tables keep their entries in the order each key first appeared in the text, including keys added later by a header, so [a.b] before [a] gives a the entry b first. An array of tables is an Array of Table values. DateTime fields that its kind does not use are 0; offset_minutes is minutes east of UTC, and Z is 0.

FunctionResultContract
limits()LimitsThe defaults below
parse(source: &[u8], bounds: &Limits)Result[Toml, TomlError]Parse one document from a buffer
parse_text(text: &str, bounds)Result[Toml, TomlError]The same, from a str
read_value[R: Reader](reader: &mut R, bounds)Result[Toml, TomlError]Read to end of input, then parse
encode(value: &Toml, bounds)Result[Bytes, TomlError]TOML text for a table
encode_text(value: &Toml, bounds)Result[str, TomlError]The same, as a str
write_value[W: Writer](writer: &mut W, value: &Toml, bounds)Result[(), TomlError]Encode the whole value, then write it; no flush
find(entries: &list[Entry], key: &str)Option[i64]Index of the first entry named key

read_value reads chunks of up to 4096 bytes, retries Interrupted, and treats a zero-length read as end of input. It reads at most one byte past max_input before reporting InputTooLarge.

Writing

The writer produces text that parses back to the same values:

  • Each table lists its plain entries as key = value lines first, then each table entry as a [path] section and each non-empty array whose elements are all tables as [[path]] sections. A section header is omitted for a non-empty table whose entries are all sections. Inside one group, entries keep their order. The plain-first order is required: a plain key written after a header would belong to that header’s table.
  • Other arrays and tables nested in them are written inline on one line.
  • Keys are bare when they are non-empty and use only bare-key bytes; otherwise they are basic strings.
  • Strings are basic strings. ", \, and control characters are escaped (\b \t \n \f \r, otherwise \u00XX in uppercase hex); every other scalar is written as UTF-8.
  • Floats are the shortest text that parses back to the same value, as str.from, with .0 added when that text has no point or exponent; infinities and NaN are inf, -inf, and nan.
  • Date-times use T, Z for offset 0, and the shortest fraction that keeps the nanoseconds.

Comments, blank lines, key quoting, number spellings, and inline-or-section choices of the source are not kept. The writer rejects a root that is not a table (NotATable), a table with a repeated key (DuplicateKey), and a DateTime whose fields are out of range or whose unused fields are not 0 (InvalidDateTime).

Limits

pub class Limits:
    pub max_depth: NonNegative[i64]
    pub max_input: NonNegative[i64]
    pub max_output: NonNegative[i64]

limits() returns Limits(64, 67108864, 67108864). The fields are public; adjust a copy. A literal converts on assignment; a computed value needs NonNegative.try_from(value)?. Every function takes &Limits and traps when max_depth is above 1024.

FieldBoundsWhen exceeded
max_depthTables and arrays nested inside the document’s root table. 0 allows only plain values at the root. An array of tables counts two levels: the array and each tableDepthExceeded
max_inputBytes in one documentInputTooLarge; the offset is the limit
max_outputBytes one encoding producesOutputTooLarge

The depth limit also bounds the parser’s and writer’s recursion, so neither a document nor a tree built by hand can overflow the stack.

Input cursor advances consume validated bytes, and lookahead beyond the integer representation is treated as beyond the input. Bounded cursor, digit and calendar arithmetic uses explicit opt-outs with local range proofs. Stream reads check the full allocation extent, including the extra byte used to detect oversize input, before resizing or invoking the reader; an unrepresentable extent reports Allocation(AllocError.CapacityOverflow). Existing parser, output-size and atomic document-edit errors remain unchanged.

Errors

pub class TomlError:
    pub fault: TomlFault
    pub offset: i64
    pub line: i64
    pub column: i64

For parsing, offset is the byte offset from 0, and line and column start at 1. Only \n ends a line. A column is one Unicode scalar, not one byte. If a one-based line or column cannot fit in i64, the fault is InputTooLarge at the still-valid byte offset, with both line and column 0 meaning unavailable. Coordinates are not saturated or wrapped. A syntax fault points at the offending byte. A fault about a whole key or header (DuplicateKey, DuplicateTable, KeyConflict, and a DepthExceeded that a key or header causes) points at the first byte of its key or [. A DepthExceeded that an array or inline table causes points at its opening bracket, and a fault about a number or date-time at the value’s first byte. UnexpectedEnd points one past the last byte. An allocation failure before scanning starts points at offset 0, line 1, column 1; one while assembling the value after a successful scan points one past the last byte.

For writing and for stream failures, offset is the number of bytes produced, written, or read when the call failed, and line and column are 0. NotATable reports 0, 0, 0.

TomlFaultMeaning
UnexpectedByteA byte that cannot start or continue the document here
UnexpectedEndInput ended inside a key, value, or header
InvalidUtf8Malformed, overlong, surrogate, or out-of-range UTF-8, anywhere
ControlCharacterA forbidden control character in a string, key, or comment
InvalidEscapeAn unknown escape, a bad hex digit, or an escape that is not a Unicode scalar
InvalidNumberA token that breaks the integer or float grammar
NumberOutOfRangeAn integer outside i64 or a float outside finite f64
InvalidDateTimeA malformed date-time, or one naming a day or time that does not exist
DuplicateKeyA key defined twice in one table
DuplicateTableA table defined twice
KeyConflictA key or header that would extend a value or a closed table
DepthExceededNesting beyond max_depth
InputTooLarge, OutputTooLargeA size limit was reached
NotATableencode was given a root that is not a table
Stream(StreamError)A reader or writer failed; InvalidProgress if it reported an impossible count
Allocation(AllocError)Memory was refused

Partial trees and buffers are released before an error returns.

Ownership and threads

A Toml owns its whole tree; moving it moves everything and dropping it releases everything. There is no shared or global state, so trees can be moved into a worker thread. parse copies its source and keeps no reference to it.

Toml is recursive, so the builtins that require a finite layout walk reject it: ==, copy, print, and str.repr do not accept a Toml. Compare and display trees through encode_text. DateTime is an ordinary class and can be printed and compared.

Costs

OperationAllocatesTime
parse, parse_textOne copy of the input; one node per key, table, and value; one index entry per named key; one scratch buffer as long as the longest key, string, or number; then the treeLinear in the input, expected
read_valueThe input buffer, grown in chunks of 4096 bytes, then as parse without the copyAs parse, plus the reads
encode, write_valueThe whole output buffer; one key set per table with two or more entries; a temporary copy of each key and string and a temporary str per float, each released before the nextLinear in the output, expected
encode_textThe output buffer, then a str copyLinear
findNothingLinear in the entry count

The parser takes runs of plain string, comment, and bare-key bytes with one std.bytes scan each. A table lookup goes through an index keyed by parent and name, so a table with many keys costs the same per key as a small one. Measured with a release compiler on x86-64 Linux, parsing and then writing a 2,688,896-byte document of 20,000 [[item]] tables and one table of 20,000 keys takes about 2 seconds, close to std.json on input of the same size and shape.

Limits of this module

  • The value-tree writer normalizes formatting; use Document for edits that preserve unrelated source text.
  • No event or streaming interface: the whole document and its tree are held in memory. TOML’s tables can be extended anywhere in a file, so no part of the tree is final before the end of input.
  • Integers are i64 and floats f64, as TOML 1.0 requires of implementations; a larger integer is an error, never rounded.
  • No tree equality, path lookup, or typed decoding helpers; a typed reader is written by hand over the tree with find.
  • TOML 1.1 additions (newlines and trailing commas in inline tables, \e, \xHH, optional seconds) are rejected.

Editing TOML documents

std.toml.Document retains validated TOML 1.0 source text and a parser index. It takes inspiration from toml_edit, but is implemented in checked Plenty with fallible allocation, alongside the existing parser and writer.

document(source: &[u8], bounds: &Limits) -> Result[Document, TomlError]
document_text(text: &str, bounds: &Limits) -> Result[Document, TomlError]
Document.text() -> Result[str, TomlError]
Document.get(path: &list[str]) -> Result[Option[Toml], TomlError]
Document.set(path: &list[str], value: &Toml) -> Result[(), TomlError]
Document.insert(path: &list[str], value: &Toml) -> Result[(), TomlError]
Document.remove(path: &list[str]) -> Result[bool, TomlError]

Mutation methods take &mut Document. Paths contain decoded key components: ["package", "version"] addresses package.version, whereas ["a.b"] addresses the single quoted key "a.b". get returns an owned value, or Nothing for a missing key. An empty path reads the complete root table; mutating the root is rejected with KeyConflict.

set replaces an existing value or inserts a missing one. insert requires absence (DuplicateKey otherwise). For insertion of a scalar or array, its parent table must already exist. Insert a Toml.Table to create a section, including empty tables and implicit ancestors. Existing section tables can only be replaced with tables; replacing a section with a scalar is rejected. remove returns whether a key existed and removes a table’s descendant statements, even when its sections are separated in the source.

import std.toml
from std.toml import Toml

def main() -> Result[(), AllocError | IoError | std.toml.TomlError]:
    bounds = std.toml.limits()
    source = "[package]\nname = \"demo\"  # keep this comment\nversion = \"0.1.0\"\n"
    mut document = std.toml.document_text(source, &bounds)?
    document.set(&["package", "version"]?, &Toml.Text("0.2.0"))?
    document.insert(&["package", "edition"]?, &Toml.Int(2026))?
    print(document.text()?.rstrip()?)?
    Ok(())
[package]
name = "demo"  # keep this comment
version = "0.2.0"
edition = 2026

Preservation contract

  • Parsing followed by text returns the exact original source, including comments, whitespace, key spelling, line endings and absence of a final newline.
  • Replacing an assignment changes only its value token. Its key, spaces around =, trailing comment and newline remain byte-for-byte unchanged. The new value uses the normal writer’s spelling; replacing an array or inline table replaces its entire token, including comments inside that token.
  • Insertion adds a canonically formatted entry at the end of its parent section, or appends a new section. New text uses LF line endings; original CRLF bytes remain untouched. A missing newline at an insertion boundary is added.
  • Removing a key or section removes the complete associated statement lines, including their trailing comments. Standalone comments and blank lines remain exactly where they were relative to other surviving text. The editor does not guess which standalone comment belongs to a removed key.
  • Replacing a section removes its old statements and appends a newly formatted section. Unrelated statements and standalone comments retain their bytes.

Boundaries and costs

Arrays, including arrays of tables, are atomic values. get can read them and remove can delete an entire array of tables, but a path through an array returns KeyConflict; there is no implicit “last row” selection. Inline-table fields can be read individually, but edits must replace the whole inline table. They have no independent document statements to insert or remove. There is no comment-editing, key-renaming, array-index path, or custom-formatting API.

Each mutation builds a candidate source and reparses it before committing. All errors, including TomlFault.Allocation(AllocError), leave the document unchanged. Unsupported paths return KeyConflict; syntax conflicts discovered by validation retain the parser’s specific fault and candidate-source location. This is an in-memory editing API, not an atomic file-writing API.

The document retains the source, arena and statement spans. Editing costs a full source copy and parse; get copies the requested tree. This deliberately favours a small, checked implementation for manifests over large-document incremental editing. Limits are captured at construction: input and depth limits apply to both original and candidate text, and output limits bound candidate construction. No Rust dependency or infallible allocator wrapper is introduced.

Acknowledgements

std.xml

std.xml reads and writes bounded XML 1.0 events over std.io byte streams. It is implemented in Plenty so buffer, string, attribute-list, and nesting-stack allocation failures remain explicit. It keeps a current token and a stack of open names, without building a document tree.

Events and format

Attribute(name: str, value: str) has public fields. Events own their strings and attribute lists; they remain valid after another reader call.

EventMeaning
Declaration(Option[bool])XML 1.0 declaration; optional standalone yes/no
Start(str, list[Attribute])Open an element with ordered attributes
Empty(str, list[Attribute])Empty element, without a separate end event
End(str)Close the matching element
Text(str)Decoded character data, including whitespace
CData(str)CDATA content without delimiters
Comment(str)Comment content without delimiters
ProcessingInstruction(str, str)Target and data; separator whitespace is omitted
EofComplete, well-formed document

Input must be UTF-8, optionally with a leading UTF-8 BOM. A declaration is optional; when present it must be first, specify version 1.0, and use UTF-8 if it names an encoding. The standalone flag is retained. There must be exactly one root element; comments, processing instructions, and XML whitespace may surround it. Names use XML 1.0 Fifth Edition character ranges. Duplicate attribute names and mismatched closing names are errors.

Literal CRLF and CR normalize to LF. Literal whitespace in attributes becomes spaces. The five predefined entities (amp, lt, gt, apos, quot) and decimal/hexadecimal character references are decoded in text and attributes; references to tab, LF, or CR in attributes retain those characters. Comments, CDATA, and processing instructions do not expand references. XML-forbidden characters, malformed UTF-8, and malformed references are rejected.

DTD declarations, including internal subsets and external entities, are rejected. The module never loads an external resource. Names with colons and xmlns attributes are preserved literally: it does not resolve namespaces or check namespace bindings. It does not validate against a schema or DTD, build a DOM, query XPath, canonicalize XML, or support encodings other than UTF-8. Events preserve content rather than original spelling, quote choices, entity spellings, or declaration formatting.

Limits

limits() returns a Limits with public fields. Constructors copy the limits; changing the original afterwards has no effect.

FieldTypeDefaultMeaning
max_depthNonNegative[i64]64Open nesting including an empty element; at most 1024
max_inputNonNegative[i64]67108864Total input bytes consumed, including BOM
max_outputNonNegative[i64]67108864Total encoded bytes written
max_tokenPositive[i64]1048576Raw bytes per reader event, or encoded bytes per writer event, including markup
max_attributesNonNegative[i64]4096Attributes per element

A literal converts where a field is expected; a computed value needs NonNegative.try_from(value)? or Positive.try_from(value)?. reader and writer trap when max_depth is above 1024. A zero depth permits no root and therefore no complete document. The input limit applies to readers and the output limit to writers. The XML declaration has its own fixed grammar of at most three fields. The BOM is checked against the token limit separately from the first event.

A reader uses a fixed 4096-byte read buffer and may read ahead that far. Size limits constrain consumption, rather than the inner stream’s position. Text between markup tokens is one event and must fit max_token; it is not split into arbitrarily sized chunks. Memory is bounded by the buffer, current event, and at most max_depth retained names, plus events retained by the caller. These limits bound input-controlled work and storage, not elapsed time or a stream that repeatedly reports interruption.

Input/output position sums are bounded by their remaining byte budgets before advancing, even when a configured budget is i64.MAX. Internal cursor and suffix arithmetic uses explicit opt-outs justified by held-buffer bounds; character-reference arithmetic is guarded by the Unicode scalar ceiling. Oversized numeric references remain InvalidReference, and byte/token exhaustion remains InputTooLarge, OutputTooLarge, or TokenTooLarge rather than exposing arithmetic errors. Partial writer progress retains the original transport cause and exact accepted-byte offset.

Reading and writing

reader[R](inner: R, bounds: &Limits) -> Result[Reader[R], XmlError] owns a stream with a public read method matching std.io.Reader. Reader.next(&mut self) -> Result[Event, XmlError] retries interrupted reads, checks reported progress, and returns Eof repeatedly after successful EOF. Reader.offset() counts consumed input bytes, before line normalization.

writer[W](inner: W, bounds: &Limits) -> Result[Writer[W], XmlError] owns a stream with public write and flush methods matching std.io.Writer. Writer.write(&mut self, event: &Event) -> Result[(), XmlError] validates and encodes one event before writing it. It checks nesting, root count, attribute uniqueness, names, characters, and event ordering. Text and attribute values are escaped; attribute tab/LF/CR and text CR inside elements are written as character references. Comments containing -- or ending in -, CDATA containing ]]>, and processing instructions containing ?> are rejected. Raw CR in comments, CDATA, processing instructions, or document whitespace still follows XML line normalization when read back.

Writer.finish(&mut self) -> Result[(), XmlError] verifies that the root was completed and flushes the stream. It is idempotent after success. Writing Eof calls finish; further writes are rejected. Writer.offset() counts bytes accepted by the inner writer. Both wrappers expose their owned inner stream; avoid reading or writing it directly while using the wrapper, since that bypasses the wrapper’s position and document state.

A writer can stream a reader’s events straight through, but this is not an atomic file replacement: earlier events and a partially written event remain visible on failure. Use a separately managed temporary destination when that matters. See the XML lesson.

Errors

XmlError has public fields fault: Fault and offset: i64. The offset is zero-based consumed input or accepted output bytes at detection. Validation may detect a malformed token after consuming it; this is not necessarily the byte containing the original mistake.

Fault distinguishes syntax, unexpected EOF, invalid names/characters/UTF-8, invalid references, unknown entities, duplicate attributes, mismatched ends, forbidden doctypes, unsupported declarations, each resource bound, invalid writer state, and Stopped. Allocation(AllocError) and Stream(StreamError) retain their causes, including stream allocation errors. All allocation in the module is fallible. A malformed or oversized reader event, failed writer event, or failed finish permanently stops that wrapper; subsequent operations return Stopped. Successfully finished writers instead reject additional writes with InvalidState.

Acknowledgements

  • W3C XML 1.0 Fifth Edition supplies the character, name, document, reference, and normalization rules implemented independently.
  • quick-xml informs the streaming event API; the module does not wrap or ship its code.

std.unicode

Normalization and a deliberately small classification surface, pinned to Unicode 17.0.0 independently of the host Rust compiler. version() returns the string "17.0.0".

FunctionResult
nfd(text: &str, max_scalars: NonNegative[i64])Canonical decomposition
nfc(text: &str, max_scalars: NonNegative[i64])Canonical decomposition and composition
nfkd(text: &str, max_scalars: NonNegative[i64])Compatibility decomposition
nfkc(text: &str, max_scalars: NonNegative[i64])Compatibility decomposition and canonical composition
combining_class(scalar: &str)Option[u32]: canonical combining class of exactly one scalar
is_mark(scalar: &str)Option[bool]: General_Category is Mark for exactly one scalar
import std.unicode
from std.text import TextError

def main() -> Result[(), AllocError | IoError | TextError]:
    decomposed = std.unicode.nfd("café", 64)?
    print(len("café"), len(decomposed), std.unicode.nfc(&decomposed, 64)? == "café")?
    accent = decomposed[4]
    print(std.unicode.is_mark(&accent), std.unicode.combining_class(&accent))?
    print(std.unicode.nfkc("file ①", 64)?)?
    Ok(())
4 5 True
Option[bool].Some(True) Option[u32].Some(230)
file 1

Normalization returns Result[str, std.text.TextError]. It borrows input and returns independent text. The explicit scalar limit bounds the input, fully decomposed intermediate, and result. Exceeding any gives LimitExceeded. The limit is NonNegative[i64], and a limit above i64.MAX // 4 traps. An empty string fits a zero limit. Output UTF-8 is consequently at most four times the limit. Normalization’s explicit arithmetic opt-outs rely on that validated ceiling, successfully appended scalar counts, and positive insertion-sort indices. They do not bypass scalar limits or allocation failures, and normalization adds no arithmetic error variants to TextError. The limit counts Unicode scalars, not bytes or grapheme clusters, and canonical composition does not let an oversized decomposition bypass it.

Classification allocates nothing and returns Nothing for zero or multiple scalars. Mark and nonzero combining class are distinct properties. This module does not offer locale collation, case folding, or general categories beyond Mark. Grapheme, word and sentence segmentation and display width are in the official unicode_text package, described with text segmentation and width. Existing string casing and ASCII classification remain the text builtins. Compatibility normalization can erase meaningful presentation distinctions; callers choose it explicitly. No implicit normalization occurs in equality, hashing, paths, decoding, or text builders.

Stable canonical ordering and blocked composition follow UAX #15, including Hangul and composition exclusions. Checked Plenty performs the algorithms; allocation-free scalar lookups reuse bundled unicode-normalization 0.1.25 tables and primitives. Canonical ordering uses stable insertion: O(n²) worst-case time for adversarial descending combining runs, O(n) otherwise, where n is decomposed scalar count. Storage is O(n), with two scalar lists and bounded UTF-8 construction; allocation failure is recoverable. Choose a suitably small limit for untrusted input. No hidden global state or thread affinity is introduced; ordinary ownership and worker-effect checking still apply.

The complete Unicode 17 NormalizationTest.txt fixture checks all twenty normalization equalities for each row through one compiled Plenty harness. Unicode upgrades are deliberate source changes: replace the pinned upstream sources, update the version function, fixture and licenses together, and rerun the corpus, allocation tests, tutorial tests and packaging tests. Do not derive table versions from the host toolchain. Vendoring preserves the dependency-free runtime build and avoids the upstream iterator’s infallible scratch allocation. Provenance and exact update steps are recorded beside the vendored sources.

The native tables add no external libraries or build-time network access and ship inside both runtime archives. Supported target: x86_64 Linux GNU. See the lesson and std.text for bounded output construction and error variants.

Acknowledgements

std.units

import std.units as u provides the SI base dimensions, named derived dimensions, units, physical constants and functions over the built-in Quantity[T, D]. The module is ordinary embedded Plenty: dimensions are pub dimension declarations and units are pub const quantities. Nothing in it allocates.

import std.units as u
from std.units import Velocity, km, hour, m, s

def main() -> Result[(), IoError | AllocError]:
    speed: Quantity[f64, Velocity] = 90.0 * km / hour
    braking = 6.0 * m / (s * s)
    distance = speed * speed / (2.0 * braking)
    print(speed, speed.to(u.knot))?
    print(f"{distance.to(m):.1f} m, {(speed / braking).to(s):.2f} s"?)?
    Ok(())
25.0 m/s 48.59611231101511
52.1 m, 4.17 s

Every unit is a Quantity[f64, D] holding its size in the coherent SI unit, so km stores 1000.0. Construction multiplies by the unit and to divides by it; both are folded at compile time when the operands are constants. A conversion between two non-coherent units divides two rounded values, so it can round: u.foot.to(u.inch) is 12.000000000000002. The stored values themselves are the nearest f64 to each exact definition.

Printing shows the coherent value and SI symbols in SI order (kg, m, s, A, K, mol, cd), so a newton prints as kg*m/s^2; see formatting.

Dimensions

BaseCoherent unit
Masskg
Lengthm
Times
CurrentA
TemperatureK
Amountmol
Luminositycd

Derived dimensions are aliases; each names a different dimension, so diagnostics can name it:

GroupDimensions
Geometry and motionArea, Volume, Wavenumber, Velocity, Acceleration, Frequency, Momentum
MechanicsForce, Pressure, Energy, Power, Action, Density, SurfaceTension, Gravitation
FlowMassFlowRate, VolumeFlowRate, DynamicViscosity, KinematicViscosity
HeatHeatFlux, SpecificEnergy, Entropy, SpecificHeatCapacity, ThermalConductivity, StefanBoltzmann
ChemistryMolarMass, MolarEnergy, MolarHeatCapacity, Concentration, PerAmount, CatalyticActivity
ElectromagnetismCharge, Voltage, ElectricField, Capacitance, Permittivity, Resistance, Conductance, MagneticFlux, MagneticFluxDensity, Inductance, Permeability
LightIlluminance, LuminousEfficacy

SI gives some different kinds of quantity one dimension. Energy also describes torque, Frequency also describes activity in becquerels, and SpecificEnergy also describes absorbed and equivalent dose. A program that needs to keep such kinds apart declares its own base dimension.

Units

SI units, their prefixed forms, eV and Wh use their symbols. Other units use names. Micro is written u: um, us, uF.

DimensionUnits
Massug, mg, g, kg, tonne, dalton, pound, ounce
Lengthpm, nm, um, mm, cm, m, km, inch, foot, yard, mile, nautical_mile, angstrom, au, light_year, parsec
Timeps, ns, us, ms, s, minute, hour, day, week, year
CurrentuA, mA, A, kA
TemperaturemK, K, rankine
Amountumol, mmol, mol, kmol
Luminositycd, lm
Onerad, sr, degree, arcminute, arcsecond
Area, Volumehectare, acre; litre, millilitre, cubic_metre, gallon (US liquid)
Velocitykm_per_hour, mph, knot
FrequencyHz, kHz, MHz, GHz, Bq
ForcemN, N, kN, MN, pound_force
PressurePa, hPa, kPa, MPa, GPa, mbar, bar, atm, torr, mmHg, psi
EnergymJ, J, kJ, MJ, GJ, Wh, kWh, MWh, GWh, eV, keV, MeV, GeV, calorie, kilocalorie, btu
PoweruW, mW, W, kW, MW, GW, horsepower
ChargenC, uC, mC, C
VoltageuV, mV, V, kV, MV
CapacitancepF, nF, uF, mF, F
Resistance, Conductanceohm, kohm, Mohm; uS, mS, S
MagnetismWb; uT, mT, T; nH, uH, mH, H
Otherlx, Gy, Sv, kat

There is no milliohm constant, because mohm could be read as milli or mega; write 1e-3 * u.ohm. Angles are dimensionless, as in SI, so u.degree is Quantity[f64, One] holding π/180. calorie is the thermochemical calorie (4.184 J), btu the International Table BTU, horsepower the mechanical horsepower (550 ft·lbf/s), mmHg the conventional millimetre of mercury and year 365 days. Celsius and Fahrenheit are not units here; see temperature.

std.units.catalogue adds every SI prefix on every SI unit and the other named units of NIST SP 811. It is a separate module so that only programs importing it pay for its 768 constants.

Constants

ConstantValueStatus
delta_nu_Cs9192631770 Hz, caesium hyperfine frequencyExact SI defining constant
c299792458 m/s, speed of lightExact SI defining constant
h6.62607015e-34 J·s, Planck constantExact SI defining constant
e1.602176634e-19 C, elementary chargeExact SI defining constant
k_B1.380649e-23 J/K, Boltzmann constantExact SI defining constant
N_A6.02214076e23 /mol, Avogadro constantExact SI defining constant
K_cd683 lm/W, luminous efficacyExact SI defining constant
hbarh / (2π)Exact, rounded to f64
RN_A * k_B, molar gas constantExact, rounded to f64
sigma5.670374419e-8 W/(m²·K⁴), Stefan–Boltzmann constantExact value truncated as CODATA lists it
g_n9.80665 m/s², standard gravityExact by definition
atm101325 Pa, standard atmosphereExact by definition
G6.67430e-11 m³/(kg·s²), gravitational constantCODATA 2022
m_e, m_p, m_uElectron, proton and atomic mass constantCODATA 2022
epsilon_0, mu_0Vacuum permittivity and permeabilityCODATA 2022

Functions

FunctionResultContract
abs[T: IntType, D](x)Result[Quantity[T, D], OverflowError[T]]std.math.abs of integer storage; a signed minimum returns its overflow
abs_float[T: FloatType, D](x)Quantity[T, D]std.math.abs_float of floating storage
clamp[Num, D](x, low, high)Quantity[Num, D]math.min(math.max(x, low), high), so low > high gives high; any storage
sum[D](items: &list[Quantity[f64, D]])Quantity[f64, D]Neumaier compensated sum; an empty list gives zero
mean[D](items: &list[Quantity[f64, D]])Option[Quantity[f64, D]]sum(items) / len(items); Nothing for an empty list
sqrt[D](x: Quantity[f64, D])Result[Quantity[f64, Root[D, 2]], MathError]std.math.sqrt, with its errors
cbrt[D](x: Quantity[f64, D])Result[Quantity[f64, Root[D, 3]], MathError]std.math.cbrt, with its errors
hypot[D](a, b)Quantity[f64, D]sqrt(a*a + b*b) scaled to avoid intermediate overflow; infinity if either side is infinite
exp, log, log2, log10Result[Quantity[f64, One], MathError]Take Quantity[f64, One]; std.math errors
sin, cos, tanResult[Quantity[f64, One], MathError]Take an angle as Quantity[f64, One] in radians
asin, acos, atanResult[Quantity[f64, One], MathError]Return an angle in radians
atan2[D](y, x)Result[Quantity[f64, One], MathError]Both arguments have dimension D; the angle of (x, y)

Every argument is a copied quantity, except sum and mean, which borrow the list. Functions built on std.math take f64 storage, as std.math does. abs takes integer storage and abs_float floating storage, as in std.math; clamp accepts any storage. The generic std.math functions min, max, min_all and max_all already accept quantities, so this module does not repeat them. math.sum and math.prod take plain integers and math.abs a plain integer, so they do not accept quantities. A square or cube root whose dimension is not whole is a compile error at the call that needs it:

import std.units as u
import std.math as math

def main() -> Result[(), math.MathError]:
    side = u.sqrt(2.0 * u.litre)?
    Ok(())
`Root[Pow[std.units.Length, 3], 2]`: not a whole dimension

A root derives its dimension, so a formula’s result is checked. Falling through a tunnel across a uniform-density Earth is simple harmonic motion: the trip to the centre takes a quarter period, (π/2)·sqrt(R/g), and the speed there is sqrt(g·R):

import std.units as u
from std.units import km, m, s, minute
import std.math as math

def main() -> Result[(), IoError | AllocError | math.MathError]:
    radius = 6371.0 * km
    to_centre = f64.PI / 2.0 * u.sqrt(radius / u.g_n)?
    top_speed = u.sqrt(u.g_n * radius)?
    print(f"{to_centre.to(minute):.1f} min, {top_speed.to(km / s):.2f} km/s"?)?
    Ok(())
21.1 min, 7.90 km/s

Dimensionless results stay quantities, so an angle or a ratio combines with other quantities without .get():

import std.units as u
from std.units import m
import std.math as math

def main() -> Result[(), IoError | AllocError | math.MathError]:
    rise = 3.0 * m
    run = 4.0 * m
    angle = u.atan2(rise, run)?
    slope = u.hypot(rise, run)
    print(f"{angle.to(u.degree):.2f} degrees, {slope}"?)?
    print(slope * u.cos(angle)?, u.sqrt(rise * run)?)?
    lengths = [rise, run, slope]?
    print(u.sum(&lengths), u.mean(&lengths))?
    Ok(())
36.87 degrees, 5.0 m
4.0 m 3.4641016151377544 m
12.0 m Option[Quantity[f64, std.units.Length]].Some(Quantity[f64, std.units.Length](4.0))

Lists

FunctionResultContract
wrap_list[D](values: list[f64])list[Quantity[f64, D]]Consumes values and returns the same list with quantity elements
strip_list[D](items: list[Quantity[f64, D]])list[f64]Consumes items and returns the same list with plain elements

Use these at the boundary with code that produces or expects plain numbers. Both are infallible and run in constant time: a quantity row is stored exactly as its f64, so the conversion replaces only the list’s type descriptor. No element is read or copied, and nothing is allocated; see representation. D is given explicitly to wrap_list and inferred by strip_list:

import std.units as u
from std.units import Length, km

def main() -> Result[(), IoError | AllocError]:
    laps = u.wrap_list[Length]([400.0, 400.0, 1609.344]?)
    total = u.sum(&laps)
    print(f"{total.to(km):.3f} km"?)?
    metres = u.strip_list(laps)
    print(metres)?
    Ok(())
2.409 km
[400.0, 400.0, 1609.344]

Temperature and time

FunctionResultContract
from_celsius(degrees: f64)Quantity[f64, Temperature]degrees + 273.15 kelvin
to_celsius(t)f64t in kelvin minus 273.15
from_fahrenheit(degrees: f64)Quantity[f64, Temperature]Through Celsius: (degrees - 32) * 5/9 + 273.15
to_fahrenheit(t)f64Through Celsius: (t - 273.15) * 9/5 + 32
from_duration(span: &Duration)Quantity[f64, Time]Seconds in f64
to_duration(t: Quantity[f64, Time])Result[Duration, RangeError]Rounds to the nearest nanosecond; RangeError.OutOfRange for a negative, NaN or too-large time

Celsius and Fahrenheit have offset zeros, so they are not multiplicative units. They convert at the boundary to an absolute temperature; a temperature difference in degrees Celsius has the same size as one in kelvin. Going through Celsius keeps whole Celsius values exact: from_fahrenheit(212.0) is 373.15 K.

A std.time.Duration counts nanoseconds as u64. from_duration converts whole seconds and the remainder separately, but the result is still an f64: spans longer than about 104 days (2^53 nanoseconds) no longer resolve single nanoseconds.

import std.units as u
import std.time as time

def main() -> Result[(), IoError | RangeError]:
    boil = u.from_fahrenheit(212.0)
    print(boil, u.to_celsius(boil), u.to_fahrenheit(u.from_celsius(-40.0)))?
    timeout = u.to_duration(1.5 * u.minute)?
    print(timeout.as_seconds(), u.from_duration(&time.milliseconds(250)))?
    print(u.to_duration(-1.0 * u.s).is_err())?
    Ok(())
373.15 K 100.0 -40.0
90 0.25 s
True

Limits

  • Units and functions use f64 storage. A quantity with f32 storage converts with Quantity[f32, D].from(q).
  • wrap_list and strip_list take f64 storage only.
  • Named units are not printed: a joule prints as kg*m^2/s^2.

Acknowledgements

  • Exact unit definitions are from the SI Brochure (9th edition) and NIST SP 811, appendix B.
  • Measured constants are the CODATA 2022 recommended values.
  • misu and Frink informed the choice of units; Frink’s samples suggested the tunnel example.

std.units.catalogue

import std.units.catalogue as uc adds every SI prefix on every SI unit and the other named units of NIST SP 811 (2008), appendix B, to std.units. Its units are the same Quantity[f64, D] values over the std.units dimensions, so they mix freely with std.units units and constants. Nothing in it allocates.

import std.units as u
import std.units.catalogue as uc

def main() -> Result[(), IoError | AllocError]:
    gold = 3.0 * uc.troy_ounce
    print(f"{gold.to(u.g):.2f} g"?)?
    battery = 3000.0 * uc.milliampere_hour * (3.7 * u.V)
    print(f"{battery.to(u.Wh):.1f} Wh"?)?
    lot = 2.5 * uc.survey_acre
    print(f"{lot.to(u.acre):.6f} international acres"?)?
    print(uc.Gm.to(u.km), uc.fs.to(u.ps))?
    Ok(())
93.31 g
11.1 Wh
2.500010 international acres
1000000.0 0.001

The module holds 768 constants. Constant evaluation costs about 10 ms of compile time for a program that imports it, and nothing for one that does not.

Prefixed SI units

The module defines each of the 24 SI prefixes, quecto (q, 10⁻³⁰) to quetta (Q, 10³⁰), on each SI base unit and each coherent derived unit with a special name except the steradian and the degree Celsius: g (prefixes apply to the gram), m, s, A, K, mol, cd, rad, Hz, N, Pa, J, W, C, V, F, ohm, S, Wb, T, H, lm, lx, Bq, Gy, Sv and kat. Names follow std.units: prefix symbol then unit symbol, with micro written u and deca da, as in ug, dam, GPa and Mohm. Each value is a power-of-ten literal times the coherent unit, so it is the nearest f64 to the exact multiple.

Two prefixed symbols cannot be names. The attosecond is attosecond, because as is a keyword, and the milliohm is milliohm, because std.units rejects mohm as readable as milli or mega.

Prefixed forms that std.units also defines, such as km, hold the same values. The catalogue also adds microlitre, centilitre, decilitre, hectolitre, kilolitre, megalitre, meV, TeV and TWh.

Named units

GroupUnits
Lengthfermi, micron, mil, microinch, point, pica, printers_point, printers_pica, survey_foot, survey_mile, chain, rod, fathom
Areaare, barn, circular_mil, survey_acre, darcy
Volumestere, acre_foot, barrel, quart, pint, cup, gill, fluid_ounce, tablespoon, teaspoon, bushel, peck, dry_quart, dry_pint, imperial_gallon, imperial_gill, imperial_fluid_ounce, cord, register_ton
Masscarat, grain, pennyweight, troy_ounce, troy_pound, short_hundredweight, long_hundredweight, short_ton, long_ton, assay_ton, slug
Timeshake, sidereal_day, sidereal_hour, sidereal_minute, sidereal_second, sidereal_year, tropical_year
Forcedyne, kilogram_force, ounce_force, poundal, kip, ton_force
Pressuretechnical_atmosphere, ksi, cmHg, inHg, ftHg, mmH2O, cmH2O, inH2O, ftH2O
Energyerg, calorie_IT, kilocalorie_IT, btu_th, therm_EC, therm_US, quad, ton_of_tnt
Powerhorsepower_electric, horsepower_metric, ton_of_refrigeration
Viscositypoise, centipoise, rhe, stokes, centistokes
Electricity and magnetismampere_hour, milliampere_hour, faraday, mho, debye, gauss, gamma, maxwell, oersted
Radiationcurie, rad_dose, rem, roentgen
Lightfootcandle, phot, stilb, lambert, footlambert
Anglerevolution, gon, angular_mil
Othergal, kayser, tex, denier

Each unit is built from its definition rather than from SP 811’s rounded factor: survey_foot is 1200/3937 m, survey_acre 43,560 square survey feet, btu_th the thermochemical calorie per gram kelvin times a pound and a degree Fahrenheit, and slug one pound-force second squared per foot. A unit whose size is measured rather than defined, such as sidereal_day (86,164.0905 s) or tropical_year (365.24219 days, J2000), uses the current value.

Some units need a choice of name or variant:

  • point and pica are the desktop-publishing point (1/72 in) and pica (1/6 in); printers_point (0.013837 in) and printers_pica are the older American printer’s units.
  • rad_dose is the rad of absorbed dose (0.01 Gy). u.rad is the radian.
  • gal is the gal of acceleration (0.01 m/s²); the US gallon is u.gallon.
  • quart, pint, cup, gill, fluid_ounce, tablespoon and teaspoon are US liquid measures; dry_quart, dry_pint, peck and bushel are US dry measures. Imperial measures have an imperial_ prefix.
  • calorie_IT and kilocalorie_IT are International Table calories; u.calorie is the thermochemical calorie. btu_th is the thermochemical BTU; u.btu is the International Table BTU.
  • The conventional mercury and water columns (cmHg, inH2O and so on) use 13,595.1 kg/m³ and 1,000 kg/m³ under standard gravity.
  • faraday is e·N_A·mol, exact in the 2019 SI; SP 811 printed the earlier CODATA value.

Compound units have no constants: write u.btu * u.inch / (u.s * u.foot * u.foot * u.rankine) for a BTU-inch per second square foot degree Fahrenheit. Square and cubic customary units are products too, such as u.foot * u.foot.

Dimensions

The catalogue declares the dimensions its units need that std.units lacks. Each names a distinct normal form, as the std.units aliases do.

DimensionDefinitionUnits
LinearDensityu.Mass / u.Lengthtex, denier
ElectricDipoleMomentu.Charge * u.Lengthdebye
MagneticFieldStrengthu.Current / u.Lengthoersted
Exposureu.Charge / u.Massroentgen
Luminanceu.Luminosity / u.Areastilb, lambert, footlambert

rhe has type Quantity[f64, One / u.DynamicViscosity].

Omissions

  • The CGS electrostatic and electromagnetic units (abampere, statvolt, biot, franklin, gilbert, unit pole and the rest), whose systems give charge different dimensions. The gauss, maxwell and oersted remain, because their SI equivalents are unambiguous and they are still in use.
  • Variants defined at a historical reference temperature: the mean, 39 °F, 59 °F and 60 °F BTUs, the mean, 15 °C and 20 °C calories, the perm, and mercury and water columns at a stated temperature.
  • Units without a fixed definition: the boiler and water horsepowers and the clo.
  • Duplicates of std.units units under another name, such as the metric ton and kilopond.

Checks

Tests compare every catalogue unit, and the named units of std.units, with the factor SP 811 table B.8 prints: within half a unit in its last digit for a rounded factor, and within four ulps for an exact one. Another test checks that each prefix exists on each SI unit with the right value and dimension.

Acknowledgements

  • Units and exact factors are from NIST SP 811 (2008), appendix B, a US government work, and the SI Brochure (9th edition); the ronna, quetta, ronto and quecto prefixes were added by the 27th CGPM (2022).
  • Frink’s unit file served as a checklist; no text was taken from it.

std.url

import std.url parses and formats absolute URLs, percent-encodes data for a named URL component, and splits and builds query strings. It has one error type, UrlError.

The module is checked Plenty code over std.bytes, with no native primitives of its own. It is available on the supported x86-64 Linux GNU target. A URL is not a filesystem path: std.path is a separate type and nothing converts between them.

import std.url
from std.url import Component, Pair, UrlError

def main() -> Result[(), AllocError | IoError | UrlError]:
    url = std.url.parse("https://example.com:8443/search?q=caf%C3%A9&page=2#top", 2048)?
    print(url.scheme, url.host, url.port, url.path)?
    match &url.query:
        case Some(query):
            for pair in std.url.query_pairs(query)?:
                print(pair.key, "=", pair.value)?
        case Nothing:
            print("no query")?
    print(std.url.encode("a b/c", Component.Segment)?)?
    print(std.url.query_text(&[Pair("q", "fish & chips"), Pair("n", "1")]?)?)?
    Ok(())
https Option[str].Some("example.com") Option[i64].Some(8443) /search
q = café
page = 2
a%20b%2Fc
q=fish%20%26%20chips&n=1

The accepted syntax

std.url implements the generic syntax of RFC 3986, with these stated differences. It is not the WHATWG URL Standard and does not imitate browsers.

RuleBehavior
SchemeRequired. Relative references such as /a/b, //host/a, and ?q are MissingScheme
CharactersASCII only, exactly the characters RFC 3986 allows in each component. Spaces, controls, "<>\^{} and the backtick, [/] outside an address literal, and all non-ASCII text are InvalidByte
Escapes% must be followed by two hex digits of either case, or the URL is InvalidEscape
UserinfoEverything before the first @ of the authority. A second @ is an error, never a choice of host
HostA registered name, or [IPv6 address], or [v…] (IPvFuture). Empty is allowed. A dotted IPv4 address is an ordinary registered name
IPv6The RFC 3986 IPv6address grammar: one optional ::, an optional trailing dotted IPv4 address whose octets have no leading zeros. Zone identifiers (%25eth0) are rejected
PortNarrower than the RFC: one to five digits, no leading zero, at most 65535. An empty port (http://h:/) is InvalidPort
Internationalized namesNo IDNA processing. A Unicode host must already be Punycode (xn--...) or percent-encoded UTF-8
NormalizationNone. Case, . and .. segments, escapes, default ports, and empty components are kept exactly as written

Because nothing is normalized and the port has one spelling, every accepted URL formats back to the identical text.

Url

pub class Url:
    pub scheme: str
    pub userinfo: Option[str]
    pub host: Option[str]
    pub port: Option[i64]
    pub path: str
    pub query: Option[str]
    pub fragment: Option[str]

Components hold the text between the delimiters, still percent-encoded and without the delimiters themselves. Nothing means the delimiter was absent; Some("") means it was present with nothing after it, so s:, s:?, and s:// are three different values. host is Nothing exactly when the URL has no // authority. An address literal keeps its brackets: [::1].

FunctionResultContract
std.url.parse(text: &str, max_bytes: NonNegative[i64])Result[Url, UrlError]Split and validate text, whose UTF-8 length must not exceed max_bytes
url.to_string()Result[str, UrlError]Validate every component and join them

parse checks the length first: LimitExceeded for a longer text. It then reports the first violation from left to right.

to_string exists so that a Url built or edited by a program cannot produce text that parses differently. It applies the rules of parse to each field, and also returns:

  • MissingHost when userinfo or port is present without host;
  • InvalidPort when port is outside 0 to 65535;
  • InvalidByte when host is present and a nonempty path does not start with /, or host is absent and path starts with //.

A successful to_string result always parses back to equal components.

Compare URLs only after deciding what equivalence means for your scheme. The module does not lowercase schemes or hosts and does not decode escapes, so HTTP://A/%7e and http://a/~ are different values.

Percent encoding

FunctionResultContract
encode(text: &str, component)Result[str, UrlError]Escape the UTF-8 bytes of text for component
encode_bytes(source: &[u8], component)Result[Bytes, UrlError]The same for a byte view
decode(text: &str)Result[str, UrlError]Replace each %hh; the result must be UTF-8
decode_bytes(source: &[u8])Result[Bytes, UrlError]Replace each %hh in a byte view; an InvalidEscape index is relative to the view

Ranges follow the std.bytes range rule.

Letters, digits, and -._~ are never escaped. % is always escaped, so encoding text that already contains escapes encodes it again. Escapes are written with uppercase hex digits. Each Component additionally keeps:

ComponentAlso keptUse for
Userinfo!$&'()*+,;=A user name or a password; : is escaped, so join them yourself
Segment!$&'()*+,;=:@One path segment; / is escaped
Path!$&'()*+,;=:@/A whole path whose / are separators
Query!$&'()*+,;=:@/?A whole query string
QueryPart!$'()*,;:@/?One key or one value; &, =, and + are escaped
Fragment!$&'()*+,;=:@/?A fragment

Userinfo output is also valid in a registered host name.

Decoding is the same for every component: %hh becomes one byte and every other byte is copied. + stays +; the application/x-www-form-urlencoded reading of + as a space is not implemented. decode returns InvalidUtf8 when the bytes are not UTF-8; use decode_bytes for binary data.

Decoded text can contain any character, including /, ?, NUL, and line breaks. Split a URL into components before decoding, and never parse decoded text as a URL or a path.

Query strings

pub class Pair:
    pub key: str
    pub value: str
FunctionResultContract
query_pairs(query: &str)Result[list[Pair], UrlError]Split at &, split each piece at its first =, decode both sides
query_text(pairs: &list[Pair])Result[str, UrlError]Join key=value with &, each side encoded as QueryPart

Every &-separated piece is one pair, in order, including empty pieces and repeated keys. A piece without = has an empty value, so a and a= read the same; query_text always writes the =. The empty query has no pairs. query_pairs accepts any byte that is not part of a malformed escape; pass it a Url.query to get the stricter component check.

Errors

pub enum UrlError:
    LimitExceeded
    MissingScheme
    MissingHost
    InvalidByte(i64)
    InvalidEscape(i64)
    InvalidHost(i64)
    InvalidPort(i64)
    InvalidUtf8
    Allocation(AllocError)
VariantMeaning
LimitExceededThe text is longer than max_bytes
MissingSchemeNo scheme and : before the first /, ?, or #
MissingHostto_string was given userinfo or a port without a host
InvalidByte(index)The byte at index is not allowed where it appears
InvalidEscape(index)The % at index is not followed by two hex digits
InvalidHost(index)The address literal starting at index is malformed, or the byte at index follows its ] and is not :
InvalidPort(index)The port starting at index is not an accepted number
InvalidUtf8Decoded bytes are not UTF-8
Allocation(cause)Storage could not be allocated

An index is a UTF-8 byte offset into the input. For to_string it is an offset into the text that would have been produced; for the byte-range functions it is a position in the source buffer, not relative to start. A failed operation releases anything it allocated. Wrap a result with std.error.context to name the operation.

Ownership and threads

Url and Pair own their strings, so they are resources: assignment moves them, and the ordinary loan rules apply to their fields. All inputs are shared loans. The module holds no global state; its functions pass the ordinary worker-effect checks and may run in any thread.

Costs

Every operation is linear in its input and allocates in proportion to it.

OperationAllocates
parseOne copy of the text, then one string per present component. A text of more than max_bytes UTF-8 bytes is rejected before the copy, so the copy never exceeds max_bytes
to_stringOne growing buffer, a temporary per component, and the result
encode_bytes, decode_bytesOne buffer of exactly the output size, after validation
encode, decodeA Bytes copy of the text, the coded buffer, then the str
query_pairsA copy of the query, the list, and two strings per pair
query_textOne growing buffer and two temporaries per pair

Encoded output is at most three times its input. The encoder counts literal bytes, then checks 3 * escaped + literals before allocating its result. Unrepresentable output sizes are Allocation(AllocError.CapacityOverflow). Parser, decoder and query cursors use documented bounds-proven arithmetic; terminal literal runs and final query pairs stop at the input endpoint instead of advancing past it. A serializer error offset that cannot fit i64 also returns allocation-capacity overflow. Caller arithmetic modes do not change these contracts.

Every byte goes through checked single-element Bytes calls, several times per byte. An earlier release-compiler measurement, before checked-default adoption, on x86-64 Linux pinned to one core found that parsing one 91-byte URL with all seven components took about 49 µs: roughly half a microsecond per input byte.

Limits

  • No relative references and no reference resolution (RFC 3986 section 5).
  • No normalization or equivalence: no case folding, dot-segment removal, escape normalization, or default-port knowledge.
  • No IDNA or Punycode conversion, and no scheme-specific rules such as http requiring a host.
  • No IPv6 zone identifiers. IPv4 addresses are not distinguished from registered names.
  • No form encoding (+ as a space) and no ; pair separator.
  • Query pairs are a list. There is no keyed lookup helper; scan the list.

std.uuid

UUID values for records, requests and files, with random v4 and timestamp-prefixed v7 generation. The implementation is checked Plenty using std.bytes, std.secrets and std.time. No extra native library is linked. Start with the tutorial; the signature index lists the complete API.

import std.bytes
import std.secrets
import std.uuid

def main() -> Result[(), AllocError | IoError | ParseError | std.bytes.BytesError | std.secrets.SecretsError]:
    id = std.uuid.parse("6BA7B810-9DAD-11D1-80B4-00C04FD430C8")?
    print(id.to_string()?, id.version(), id.is_rfc_variant())?
    same = std.uuid.from_bytes(&id.to_bytes()?)?
    print(same == id)?
    fresh = std.uuid.v4()?
    print(len(fresh.to_string()?), fresh.version(), fresh == id)?
    Ok(())
6ba7b810-9dad-11d1-80b4-00c04fd430c8 Option[u8].Some(1) True
True
36 Option[u8].Some(4) False

Values and conversion

Uuid owns two private u64 fields in network order: most significant half first. It has ordinary class move semantics; == and != compare the complete 128-bit value. All methods borrow their receiver.

OperationResult and contract
parse(text: &str)Result[Uuid, ParseError]; exactly 36 ASCII characters in 8-4-4-4-12 hyphenated hex form, accepting either digit case
from_bytes(source: &[u8])Result[Uuid, ParseError]; exactly 16 bytes, in big-endian/network order; accepts a view of Bytes, a list or a byte literal
from_u64_pair(high: u64, low: u64)Uuid; preserves the supplied halves
nil()Uuid; all zero bits
id.to_string()Result[str, BytesError]; lowercase hyphenated text
id.to_bytes()Result[Bytes, BytesError]; an owned 16-byte buffer
id.write_bytes(target: &mut Bytes, offset: i64)(); overwrites exactly 16 existing bytes; no growth or allocation
id.as_u64_pair()(u64, u64); a copyable, numerically ordered sorting key
id.compare(other: &Uuid)i64; -1, 0, or 1 in unsigned 128-bit order
id.is_nil()bool; all bits are zero
id.is_rfc_variant()bool; the two leading variant bits are 10
id.version()Option[u8]; the raw four-bit version for the RFC variant, otherwise Nothing
id.unix_milliseconds()Option[u64]; the leading 48 bits for RFC-variant v7, otherwise Nothing

Parsing and byte conversion preserve all bits, including nil, max, unrecognized versions and non-RFC variants. Parsing does not validate generation history. Malformed text and wrong byte lengths return ParseError.Invalid. There is no trimming, URN/braced/compact spelling, little-endian GUID convention, or implicit normalization. write_bytes validates the whole range before writing; invalid offsets trap with Uuid.write_bytes: offset outside target buffer before writing.

Parsing, reading bytes, comparison, and numeric construction do not allocate. to_bytes allocates its result; to_string uses a 36-byte scratch buffer and allocates its result. Allocation failures are BytesError.Allocation. Normal print(id) retains structural class formatting: use print(id.to_string()?)? for canonical text. Class relational operators and dictionary keys are not available; use compare, or sort_by_key with as_u64_pair(). Use canonical text when a dictionary key is needed.

Generation

v4() -> Result[Uuid, SecretsError]
v7() -> Result[Uuid, RangeError | SecretsError]

v4 obtains 16 OS-random bytes and sets the version and variant, leaving 122 random bits. v7 reads wall_now() once and obtains 10 random bytes: its layout is a 48-bit Unix millisecond timestamp and 74 random bits, plus version/variant. Both propagate allocation and entropy errors from std.secrets; neither falls back to a pseudorandom seed, and neither maintains a process-wide counter.

The clock value must have nonnegative seconds, nanoseconds below one billion, and a millisecond value at most 281474976710655. Sub-millisecond fractions are rounded down; invalid timestamps return RangeError.OutOfRange before requesting entropy. Generation uses a short-lived entropy buffer; a batch generator with reusable scratch is not part of this API.

v7 sorts by its recorded millisecond, not by call order. Values in the same millisecond have random order, and a backwards wall-clock adjustment can yield a smaller UUID. There is no strict monotonicity or mathematical uniqueness guarantee. The timestamp is visible to anyone reading the identifier. UUIDs are identifiers, not authentication credentials; use std.secrets tokens for that purpose.

For deterministic tests or an explicitly chosen entropy device:

v4_from[R: Reader](source: &mut Source[R]) -> Result[Uuid, SecretsError]
v7_from[R: Reader](at: &Timestamp, source: &mut Source[R]) -> Result[Uuid, RangeError | SecretsError]

Source is std.secrets.Source, constructed with source_from(reader) or os(). These functions apply the same masking and timestamp checks as the convenience calls. Production callers must supply cryptographic entropy; replaying a test source can repeat identifiers. Validation happens before any read, and short/interrupted reads and failures follow the secrets source’s existing policy. No clock or entropy callback is stored in a UUID.

References and scope

The layout and test vectors follow RFC 9562, especially sections 5.4/5.7 and appendices A.3/A.6. API references are Python’s uuid and Rust’s uuid::Uuid. Plenty deliberately accepts fewer text spellings than either reference and does not adopt Python’s v7 counter or Rust’s stateful now_v7 ordering policy. Name-based v3/v5 and v1/v6/v8 generators are not implemented.

Arithmetic bounds

Parsing and formatting use fixed 36-byte text and 16-byte wire layouts. Output offsets are validated before subtraction or mutation, and UUIDv7 validates the 48-bit millisecond range before requesting entropy. Bounds-proven arithmetic opt-outs preserve existing parse, range, entropy, and allocation errors without adding arithmetic errors to the public API.

Acknowledgements

Toolchain and implementation

Building and running covers compiler commands, native linking, supported targets, and library artifacts.

Architecture describes the frontend, checking, specialization, Cranelift emission, runtime packaging, storage, and cleanup mechanisms. The support index links to implemented contracts and their principal limits.

Contributor practices live in Development.

Building and running programs

Plenty compiles ahead of time through Cranelift. Checking needs no native linker; running and executable builds use a cc-compatible driver and the runtime archive embedded in the compiler. Program builds invoke neither Cargo nor rustc.

  • Installing a release: release archives, requirements, and PATH setup.
  • Commands: checking, running, executable and library output.
  • Warnings: lints that checking reports, such as redundant-copy.
  • Neovim: file detection, syntax highlighting, editing defaults, and language server setup.
  • Language server: plenty lsp diagnostics, hover, go to definition, and completion.
  • Packages: plenty.toml, path dependencies, plenty.lock, profiles, and the plenty build, run, check, and test commands.
  • Application exits: statuses and returned-error reports.
  • Native linking and targets: driver arguments, object output, runtime extraction, and supported platforms.
  • Native engines: when a large native library ships as a separate shared library, and the contract it follows.
  • Application distributions: explicit relocatable executable, engine and notices bundles with --dist.
  • Library artifacts: generated files, contract inspection, and publication guarantees.
  • Fuzzing: coverage-instrumented libFuzzer builds with --fuzz.

For source-level entrypoint rules, see source and modules. For runtime packaging, see architecture.

Installing a release

Each Plenty release on the project’s GitHub Releases page provides a compiler archive for each Linux target, such as plenty-VERSION-x86_64-unknown-linux-gnu.tar.gz or plenty-VERSION-aarch64-unknown-linux-gnu.tar.gz, and its .sha256 checksum. Download the archive matching your machine’s architecture. Extract it anywhere and put its bin/ directory on PATH. No installer, root access, fixed location, Rust, Cargo, or source checkout is needed. The README in the archive repeats these instructions.

The Linux archive includes the native engines for std.dataframe and std.linalg in bin/, where the compiler finds them. Programs that do not reach an engine’s native operations do not depend on it. To ship a program that does, follow application deployment.

For a step-by-step download guide and persistent PATH examples for Bash, Zsh, and Fish, start with Setting up.

plenty-VERSION-x86_64-unknown-linux-gnu/
  bin/plenty                  compiler and project tool
  bin/plentyc                 single-file compiler
  bin/libplenty_dataframe_v1.so
                              Polars engine for std.dataframe
  bin/dataframe-notices/      its manifest and licenses
  bin/libplenty_linalg_v1.so  faer engine for std.linalg
  bin/linalg-notices/         its manifest and licenses
  README.md                   requirements and installation
  LICENSE-MIT                 Plenty's license: MIT or Apache-2.0,
  LICENSE-APACHE                at your option
  THIRD-PARTY-NOTICES.md      licenses of software built into the compiler
  licenses/rust/              the Rust standard library's notices

The standard-library sources and ordinary runtime are built into the executables, so the compiler works from any directory. Dataframe and linear algebra operations use the shared engines in bin/; keep them beside the compiler.

Requirements

RequirementDetail
Platformx86-64 or ARM64 (AArch64) Linux with glibc 2.28 or later: RHEL/AlmaLinux/Rocky 8, Debian 10, Ubuntu 18.10, Fedora 29, or newer
KernelThe Rust standard library’s minimum, 3.2 on x86-64 and 4.1 on ARM64; old kernels are not tested
CPUAny x86-64 CPU (the original SSE2 instruction set) or any ARMv8-A CPU; the compiler and generated programs use only that baseline
LinkingA cc driver, linker, startup files, and C library development files

Rust’s AArch64 Linux target sets the 4.1 kernel minimum. ARM64 archives are built and tested on native ARM runners; page sizes other than 4 KiB are not tested.

musl distributions such as Alpine are outside the native target set; see native linking and targets.

Plenty generates machine code itself and runs cc only to link it with the embedded runtime and the system C library, as described in native linking. It is not a self-contained toolchain. Install the linking prerequisites:

# Debian, Ubuntu
sudo apt-get update && sudo apt-get install gcc libc6-dev

# Fedora, RHEL, AlmaLinux, Rocky Linux
sudo dnf install gcc tar gzip

# Arch Linux
sudo pacman -Syu --needed gcc

Release checks run these commands in clean AlmaLinux 8.10 (glibc 2.28), Debian 12 and 13, Ubuntu 24.04, Fedora 44, and Arch Linux containers for x86-64, and AlmaLinux 8.10 and Debian 13 for ARM64, then compile and run programs from the extracted archive. Each archive’s README lists the systems it was tested on. AlmaLinux 8 is the glibc 2.28 baseline; installing packages does not make an older system compatible.

Experimental macOS and Windows downloads

Releases also publish experimental compiler archives for macOS on Apple silicon and for 64-bit Windows:

ArchiveRequirements
plenty-VERSION-aarch64-apple-darwin.tar.gzmacOS 11 or later on Apple silicon; the Xcode Command Line Tools (xcode-select --install) for linking
plenty-VERSION-x86_64-pc-windows-msvc.zipWindows 10, Windows Server 2016, or later on x86-64; Visual Studio Build Tools with the “Desktop development with C++” workload and the Windows SDK, which also supply the MSVC runtime the compiler uses

Both contain bin/plenty, bin/plentyc (.exe on Windows), the README, licenses and notices. Both include the dataframe engine and bin/dataframe-notices/: bin/libplenty_dataframe_v1.dylib on macOS, bin/plenty_dataframe_v1.dll on Windows. Linear algebra remains unavailable on these two platforms. The release workflow builds each archive natively, runs the compiler/runtime unit tests and platform-boundary checks, then extracts it on a macOS 15 or Windows Server 2025 runner with Rust and Cargo removed from PATH, and compiles and runs a small program. Both additionally exercise a dataframe Parquet query and relocatable application distribution. Projects, most standard-library services such as SQLite and TLS, and the diagnostics for a missing linker are not yet tested there, and several OS services are incomplete; see platform support.

The executables are not notarized or code-signed for distribution. The macOS executables carry the linker’s ad-hoc signature; a browser download is quarantined, so run xattr -dr com.apple.quarantine on the extracted directory, or download with curl, which does not quarantine files. Windows may show a SmartScreen warning. The archive’s README gives installation and troubleshooting steps for each system.

Install and use

cd ~/Downloads
sha256sum -c plenty-VERSION-x86_64-unknown-linux-gnu.tar.gz.sha256
tar -xzf plenty-VERSION-x86_64-unknown-linux-gnu.tar.gz
export PATH="$HOME/Downloads/plenty-VERSION-x86_64-unknown-linux-gnu/bin:$PATH"
plenty --version
plenty --compile hello.plenty -o hello
./hello

Add the export line to your shell’s startup file to keep it. The directory can be moved or renamed, and paths may contain spaces; update PATH to match. Plenty writes nothing inside it: the compiled-function cache lives in ~/.cache/plenty, and temporary files in TMPDIR.

plenty --version prints the compiler’s version and target: plenty X.Y.Z (x86_64-unknown-linux-gnu), with X.Y.Z replaced by the installed version (including any pre-release suffix).

Upgrade and switch versions

Each release extracts to its own versioned directory, so several can be installed side by side. The first plenty on PATH is the one that runs: point PATH at another release’s bin/ to switch, and check with plenty --version or command -v plenty. Uninstall by deleting the directory.

Troubleshooting

SymptomCause and fix
failed to invoke linker driver `cc`, mentioning gcc or clangNo C compiler on PATH. Install the prerequisites, or pass --linker clang or another cc-compatible driver
linker driver `cc` failed naming Scrt1.o or crt1.o, -lc, -lgcc_s, or -lpthreadThe C library development files are missing; install the prerequisites (libc6-dev on Debian and Ubuntu)
GLIBC_2.xx not found when starting plentyThe system is older than glibc 2.28

plenty --check FILE needs no linker.

Programs you build

Generated programs link against the C library of the machine that compiles them, through its cc. A program built on a new distribution can need a newer glibc than Plenty itself and fail on an older system. To deploy to older systems, compile there, or in a container of the oldest supported distribution, as the release itself is built.

Maintainers: see releasing Plenty.

Neovim support

The repository includes a runtime plugin in editors/nvim. It detects *.plenty files, highlights source syntax using the active colorscheme, sets four-space soft tabs, supplies # comment formatting, and configures the language server for Neovim’s built-in LSP client.

Add this to init.lua, replacing the path with your Plenty checkout:

vim.opt.runtimepath:prepend('/absolute/path/to/plenty/editors/nvim')
vim.cmd('filetype plugin on')
vim.cmd('syntax enable')

Restart Neovim and open examples/classes.plenty. Check :set filetype? if highlighting is missing; it should report plenty. The plugin directory’s README also describes native package installation and the headless test commands. Extension detection takes precedence over generic content detection, including for source beginning with comments. An already-open buffer can be corrected with :setlocal filetype=plenty after the runtime path is available.

Highlighting includes keywords, primitive and collection types, function declarations, annotations such as @copy, numbers and suffixes, strings, byte literals and escapes, operators, and comments. Single- and double-quoted strings stay on one line; triple-quoted strings span lines. See source syntax. Highlighting uses regular expressions and does not perform semantic checks. Reserved words can be colored even when their construct is not implemented.

Language server

With Neovim 0.11 or later, editors/nvim/lsp/plenty.lua configures the server. Enable it after the runtime path above:

vim.lsp.enable('plenty')
-- If `plenty` is not on PATH:
-- vim.lsp.config('plenty', { cmd = { '/path/to/plenty', 'lsp' } })

vim.api.nvim_create_autocmd('LspAttach', {
  callback = function(event)
    local client = vim.lsp.get_client_by_id(event.data.client_id)
    if not client or client.name ~= 'plenty' then
      return
    end
    vim.lsp.inlay_hint.enable(true, { bufnr = event.buf })
    vim.keymap.set('n', 'gd', vim.lsp.buf.definition, { buffer = event.buf })
    vim.lsp.completion.enable(true, client.id, event.buf, { autotrigger = true })
  end,
})

K shows the hover. gd goes to the definition, into a read-only copy of a std module when the definition is there; CTRL-] does the same through Neovim’s default LSP tag function. Errors appear as diagnostics; ]d and [d move between them. Completion triggers after . and with CTRL-X CTRL-O. With nvim-cmp instead of the built-in completion, leave out the vim.lsp.completion.enable line and add the nvim_lsp source.

Neovim 0.10 has no vim.lsp.config; start the server from a FileType autocommand instead:

vim.api.nvim_create_autocmd('FileType', {
  pattern = 'plenty',
  callback = function()
    vim.lsp.start({
      name = 'plenty',
      cmd = { 'plenty', 'lsp' },
      root_dir = vim.fs.root(0, { 'plenty.toml', '.git' }),
    })
  end,
})

Inferred type hints

The LspAttach example enables inferred types inline for Plenty buffers. Neovim 0.10 or later provides vim.lsp.inlay_hint.enable. With the 0.10 startup example, add the same LspAttach callback but omit the vim.lsp.completion.enable line (that completion API requires 0.11). To enable or disable hints in an already-attached buffer, run:

:lua vim.lsp.inlay_hint.enable(true, { bufnr = 0 })
:lua vim.lsp.inlay_hint.enable(false, { bufnr = 0 })

A binding such as count = 42 displays an inferred : i64 after count. These labels do not change the file. An explicit count: i64 = 42 has no redundant hint. If hints are missing, check :checkhealth vim.lsp for an attached Plenty server, ensure the server binary includes inlay-hint support, and resolve check errors. See the hint coverage and limits. No editor-wide setting is changed by the server.

Formatting, automatic block indentation, and Tree-sitter are not provided.

Language server

plenty lsp serves the Language Server Protocol on standard input and output. It checks files with the compiler’s own front end, so its errors and names match plenty check and plentyc --check. Neovim is the supported editor; see Neovim support for its setup.

Features

RequestBehavior
DiagnosticsThe file’s check errors and warnings, published when it is opened, saved, or edited (after a 150 ms pause)
HoverThe declaration as written and its docstring, as Markdown
Go to definitionThe declaration’s name, in the same file, another module, a dependency, or std
Inlay hintsInferred types after unannotated simple local binding names, when the current check succeeds
CompletionNames in scope and keywords; fields, methods, enum variants, and module members after .

Hover and definition work on functions, methods, classes, enums, enum variants, protocols, fields, constants, type aliases, imported names, modules in an import, locals, and parameters. A hover shows a function’s full header, including type parameters, the receiver (self, &mut self), defaults, and the return type, and pub when it is written. Hover on a local or parameter shows its type as the checker inferred it. A method call resolves through its receiver’s type, so p.dist() opens the method of p’s class. Completion items carry the declaration and the docstring’s first line.

A docstring is the string that begins a function body; see signatures and calls. Classes, enums, and fields have no docstrings.

Inferred types on code lines

Enable inlay hints in your editor to see, for example, : i64 after the name in count = 42. These are editor-rendered annotations, not source edits; you can keep writing inferred bindings without inserting type annotations. Other language servers provide inferred type annotations through this same LSP 3.17 inlay-hint request. The editor requests hints for a document range and decides how to display them. See Neovim setup to enable them.

Plenty supplies type hints for new simple local bindings, including mut, using the checker’s type spelling as hover does. Explicit annotations, reassignments, parameters, tuple destructuring, loop and pattern bindings do not get hints. Compiler-internal and producer-specific types (such as local pipelines and capturing closures) are omitted. An uninstantiated generic body has no checked hints; when instantiations disagree on a binding’s type, it is omitted. Hints are withheld while the current check has errors, rather than guessing or displaying types and positions from an older version of the source.

The server advertises inlayHintProvider with no resolve support and returns complete string labels of kind Type, without text edits or extra padding. Hint positions are immediately after the binding name, within the requested half-open range, using the negotiated UTF-8, UTF-16 (default), or UTF-32 position encoding. Hint requests reuse the document’s analysis; a pending edit is checked first, just as for hover. No parameter-name hints are provided.

How a file is checked

The server checks the file being edited as the entry of a compilation, with the unsaved text of every open file in place of its saved text. A file that defines main is checked as an application, and any other file as a library module, as plentyc --check-module does. Inside a package (a plenty.toml above the file’s src/), imports resolve through the package graph without fetching or changing plenty.lock; run plenty fetch once for Git dependencies. Outside a package, imports resolve from the file’s directory.

Checking recovers from an error in one function: the function keeps its signature, the other functions are still checked, and each error is published. Errors that stop a whole module, such as a syntax error, an unresolved import, or an invalid class declaration, are published alone; while one is present, hover, definition, and completion use the names found by the file’s last check that got further.

Definitions in an embedded std module open as a read-only copy of its source, written to std-sources/VERSION/std/ below $PLENTY_CACHE_DIR, $XDG_CACHE_HOME/plenty, or ~/.cache/plenty. The copies are ordinary files, so any editor can open them, and they are rewritten when the compiler’s copy differs.

Limits

  • Names inside a generic function or method are resolved only where it is instantiated with concrete types; a method called on a protocol-bounded parameter opens the method of the class it was instantiated with, not the protocol’s requirement. A method of a generic class is instantiated only where it is used. An uninstantiated generic body resolves its module-level names and locals only.
  • Builtin functions, methods of built-in types such as list and str, and Ok, Err, Some, and Nothing have no declaration to show.
  • Variant names in case patterns are not resolved.
  • Locals offered by completion are those bound in the enclosing function above the cursor, by line.
  • An error in a module other than the open file is shown on the file’s first line, with the other file’s position in the message.
  • Each check reads the whole program again; there is no incremental reuse between checks. A program importing std.http.server, std.json, and five other large std modules checks in about 0.2 s with a release build.
  • Find references, document symbols, and signature help are not implemented.
  • A message body that is not valid JSON is answered with a -32700 parse error and the session continues. A header without a valid Content-Length leaves the stream unframed, so the server exits as if the input had ended.
  • After shutdown, the server answers any other request with a -32600 invalid-request error and ignores notifications until exit.

Other editors

Any client that starts a server with a command line and speaks LSP over stdio can use plenty lsp for files with the .plenty extension. No VS Code extension is provided; a generic LSP client extension configured with the command plenty lsp works.

Compiler commands

These commands compile single files. plentyc accepts exactly these commands; plenty accepts them too, alongside the project commands.

CommandBehavior
plenty FILEReuse a cached executable or compile/link, run privately, then remove the temporary directory
plenty --check FILECheck an application without emitting or executing code
plenty --check-module FILECheck a module and imports without requiring main
plenty --compile FILE -o OUTProduce a native executable (dynamic dependencies remain)
plenty --compile FILE --dist DIRPublish a new application distribution with only reached engines and their notices
plenty --emit-object FILE -o OUTEmit one application object, including imported modules; requires main
plenty --dump STAGES FILE -o DIRWrite each function’s IR and machine code as text; --dump-std and --dump-filter TEXT select functions
plenty --fuzz FILE -o OUTProduce a coverage-instrumented libFuzzer executable that runs main once per input; --libfuzzer PATH selects libFuzzer
plenty --shared-library FILE --library-name NAME -o OUTBuild a shared C library with generated header and interface
plenty --static-library FILE --library-name NAME -o OUTBuild a static C library with generated header and interface
plenty --extract-interface LIBRARY --library-name NAME -o OUTExtract embedded source metadata without executing native code
plenty --verify-interface LIBRARY INTERFACEVerify a generated contract and compatibility symbol
plenty --runtime-interface FILE --library-name NAME -o OUT.plentyiGenerate a typed runtime loader from checked export source, without loading a library or invoking native tools
plenty --emit-runtime DIRExtract libplenty_runtime.a, link-args.txt, and target.txt
plenty --print-targetPrint the packaged runtime’s target and CPU baseline
plenty --version (-V)Print the compiler’s version and target: plenty X.Y.Z (x86_64-unknown-linux-gnu), where X.Y.Z is the installed version

Checking modes print warnings on stderr; they do not change the exit status.

--check-all-methods with --check or --check-module also checks every method of every instance the program creates of a generic class or enum it declares, including methods nothing uses, which are otherwise checked only when used. Types from std keep per-use checking. Other modes reject it: compiling would emit those methods, and a reused cached object would skip their check.

Code generation is optimized by default, including plentyc, object and shared library output, fuzz targets and dumps. --fast-compile skips speed optimizations for shorter edit-run cycles in large projects; it preserves all language checks. --release remains a compatibility alias for the optimized default. See profiles.

--unchecked MODE selects what plain unchecked: suites outside std do with their arithmetic and indexing preconditions: trap, the default, checks them; assume omits the checks, so a violation is undefined behavior. Assumed list, slice-view and Bytes indices must be nonnegative and in range; bounds checks and negative-index normalization are omitted. It applies to compiling modes, including plenty build, run, test and scripts; checking modes reject it.

--test-hooks with FILE or --compile enables failure injection in an ordinary executable. It does not arm failures automatically. Default builds have no hooks; library, object-only, fuzz, and checking modes reject this flag.

--compile and --shared-library output is stripped of its symbol table and debug sections; --no-strip keeps them.

Compiled functions are reused from the compile cache, and so is a whole program’s object when none of its sources changed. --no-cache compiles without it, --no-program-cache reuses functions but not programs or executables, --cache-stats reports its hits and misses after compiling, and plenty cache stats and plenty cache clean inspect and empty it.

With no arguments, plenty displays help. --module-root DIR selects the source root for file commands. --package-graph GRAPH instead resolves imports through a package graph file for FILE, --check, --check-module, and --compile. Library builds do not require main; their exported contracts follow the C interface rules.

Compile-and-run inherits standard streams, environment, and working directory. It propagates the child’s exit code; Unix signal termination becomes 128 plus the signal number. Temporary objects and runtime files are removed on success and failure. On supported Linux toolchains, single-file runs also reuse a cached executable by default, avoiding linking when its inputs match. Other runs link normally. Unix scripts can use #!/usr/bin/env plenty as their first line and be made executable with chmod +x FILE; hashbang launches use the same caches.

For argument forwarding, optional embedded manifests and isolated dependency resolution, use plenty script, with the Unix shebang #!/usr/bin/env -S plenty script. The compiler shortcuts above remain unchanged.

Warnings

A warning reports code the checker accepts but that has a simpler correct form. Warnings never reject a program and never change an exit status.

Checking reports them: plenty --check, plenty --check-module, plenty check, and the language server. Compiling and running do not, because a program cache hit skips the frontend that finds them. Run a check to see a program’s warnings.

The command line prints each warning on stderr after the check passes, sorted by file and position:

main.plenty:3:14: warning: `copy(original)?` is redundant: `str` copies on assignment and when passed or returned; write `original` [redundant-copy]

The bracketed name is the lint’s stable identifier. The language server publishes the same text with warning severity and the name as the diagnostic code. A check that fails reports only its errors.

There is no option to disable a warning or silence one site. Every warning has a replacement that keeps the program’s behavior; apply it.

std code and the bodies of generic functions and generic class or enum methods report no warnings: a generic body is checked per instance, and code that is redundant for one type argument can be needed for another.

Lints

NameReports
redundant-copycopy(x)? when the type of x already copies

redundant-copy

A value whose type copies is copied by plain use: assigning, passing, or returning it. copy(x)? then produces the same value as x, with a Result and an AllocError path that cannot add anything. The lint uses the checker’s own copy rule for the instantiated type, so str, numbers, bool, bytes, copyable tuples and sums, and @copy classes such as Point[i64] are reported, and a list or ordinary class is not.

It reports copy(x)? only when writing x instead keeps the type, evaluates the same expression once, and moves nothing. x must be:

  • a local that is not a reference or view;
  • a field path from such a local, such as record.name; a field read through a reference, such as self.name in a self method, is left alone;
  • a list, dictionary, or tuple element, such as words[i];
  • a string, number, boolean, or bytes literal.

For a name or field path the message spells the replacement. These are left alone:

  • copy(x) used as a Result, for example matched or stored, rather than propagated with ?;
  • a reference argument such as copy(r)? for r: &str, or copy(&x)?; *r reads a copyable value through a reference;
  • a call or other computed argument, such as copy(f())?;
  • copy(x)? of a type that moves, which makes an independent value.
def main() -> Result[(), AllocError | IoError]:
    original = "hello"
    copied = original
    print(original, copied)?
    Ok(())
hello hello

Removing a copy(...)? can leave a function’s AllocError unused. The function still checks; narrow its declared error type if nothing else needs it.

Single-file scripts

plenty script compiles a standalone application and runs it without creating a package. It requires the normal native compiler/linker environment and an explicit main, not top-level execution.

#!/usr/bin/env -S plenty script

def main() -> Result[(), IoError]:
    print("Hello, Plenty!")?
    Ok(())
Hello, Plenty!

Save it as hello.plenty and run:

plenty script hello.plenty 123
chmod +x hello.plenty
./hello.plenty 123

Arguments and modes

plenty script [OPTIONS] [--] FILE [ARGS...]
plenty script --check hello.plenty
plenty script --compile -o ./hello hello.plenty

Runner options precede FILE. Everything after it is passed unchanged to the application, including flags, empty strings and a literal --. -- before FILE allows filenames beginning with -. Thus plenty script --help displays runner help, while ./hello.plenty --help asks the application for help.

On Unix, args() starts with the invoked script path, including a symlink launcher’s spelling; args_bytes() preserves native argument bytes without UTF-8 conversion. Retained binaries have normal executable argument zero. Streams, environment and working directory are inherited; imports do not change the working directory. Child exit codes and Unix signal status follow ordinary compile-and-run.

--check and --compile -o OUT resolve the same dependencies and preserve source diagnostics, but reject program arguments and never run the application. Scripts optimize by default; --fast-compile skips optimization for shorter edit-run cycles. --release remains a compatibility alias for the default. --unchecked MODE selects the unchecked arithmetic mode. --offline, --locked, --verbose, --no-strip and --no-cache retain their ordinary meanings. --cache-stats reports artifact reuse. Build status goes to stderr; a fresh cached run is quiet unless reporting was requested.

Optional embedded manifest

A script may have one ---plenty line, TOML metadata, and one closing --- line before its declarations. The optional shebang must be the first line; blank lines may precede the header. Delimiters are unindented lines with no extra whitespace. Duplicate, misplaced and unterminated headers are errors. CRLF is supported. Imported modules cannot contain script manifests. plenty FILE and plentyc remain compiler interfaces: an embedded header directs their users here rather than fetching dependencies.

For example, given an existing library package at ../util:

#!/usr/bin/env -S plenty script
---plenty
[package]
name = "helper"

[dependencies]
util = { path = "../util", version = "^0.1.0" }
---
from util import answer

def main() -> Result[(), IoError]:
    print(answer())?
    Ok(())

The package manifest dependency schema applies, including Git tag/rev, subdir, and version requirements. No registry or version-only lookup is introduced. manifest-version defaults to 1; [package] is optional, with version 0.0.0 and a name derived from the lowercase filename stem. Invalid name characters become _; names needing an initial identifier character or colliding with reserved std receive a script_ prefix. Explicit names use package validation. Description, authors, license and minimum plenty-version are supported. Unknown keys, workspaces and explicit library/binary targets are rejected. There is exactly one target: the original script. Neighboring targets and enclosing workspaces are never discovered.

The physical script directory is both the module root and the base of relative dependencies. A symlink launcher uses resources beside its target. Sibling modules are supported, with normal reserved-name, alias-conflict and cycle checks. A script with sibling modules or path dependencies is not self-contained when copied. Diagnostics retain original filenames, lines and body byte positions. Editors recognize the header; dependency completion uses already cached resolution without fetching. A script run with only an explicit durable lock may need an ordinary cached-lock run before editor dependency resolution is available.

Storage and locks

Generated dependency state lives under scripts/state/HASH/ in PLENTY_CACHE_DIR, or $XDG_CACHE_HOME/plenty, or ~/.cache/plenty. HASH covers the canonical script path, not its basename. Unix directories are created private to the user. Nothing is generated beside the source by default, so read-only script directories work. Per-script file locks serialize resolution; locks are released before running the application. Successful object and executable entries are published atomically; runs execute private copies, never a shared writable binary. Failed builds do not run an earlier executable.

Dependencies use a cached plenty.lock, retained across body edits and reconciled when dependencies change. --locked rejects missing or stale locks; --offline uses cached dependency sources only. A dependency-free first run needs no lock and works with both flags. Artifact caching can be disabled without disabling the persistent dependency state needed by these flags.

For a durable, shareable lock:

plenty script --lockfile ./hello.lock hello.plenty
plenty script --locked --offline --lockfile ./hello.lock hello.plenty

The option path is relative to the caller and must be outside the Plenty cache so cleanup cannot remove it; local source paths inside the lock are relative to the physical script directory. Cached locks are not portable reproducibility guarantees: clearing the cache or moving a script can resolve a moved tag differently. Full revisions do not pin mutable tags in transitive dependencies; path dependencies remain live trees.

plenty cache stats includes script dependency state, and plenty cache clean removes it as well as compiled artifacts. Cleanup takes each namespace’s lock; small coordination lock files remain so concurrent launches keep using the same lock. Automatic artifact-size trimming does not evict dependency state. Durable locks outside managed cache state are not removed. Executable reuse has the existing Linux toolchain eligibility limits; other toolchains link normally. Unix shebang/argument behavior is tested on Linux; non-Unix script argument zero is the generated executable name.

This is not a sandbox or interpreter, and does not provide a script test command. For a multi-target application, portable package locks, or package test discovery, use packages. See the design rationale for alternatives.

Packages and project commands

A package is a directory containing plenty.toml. Project commands search the current directory and its parents for the nearest manifest, then for an enclosing workspace. The compiler receives a fixed package graph; imports never fetch or resolve packages.

Commands

CommandBehavior
plenty new PATH [--lib] [--name NAME]Create PATH with a manifest, src/main.plenty (or src/lib.plenty), and .gitignore
plenty init [--lib] [--name NAME]Create a package in the current directory; refuses to replace a manifest or source
plenty check [--locked] [--offline] [--check-all-methods]Check the library (without requiring main) and every executable, and report unchecked sources; --check-all-methods also checks unused methods of the package’s own generic instances, not those of std or dependency types; prints each package’s warnings once
plenty build [--fast-compile] [--no-strip] [--locked] [--offline]Check the library and compile every executable to target/PROFILE/NAME
plenty run [--fast-compile] [--no-strip] [--locked] [--offline] [--bin NAME] [-- ARGS...]Build one executable and run it with ARGS
plenty test [--fast-compile] [--locked] [--offline] [--no-capture] [--seed N] [--list] [--jobs N] [--timeout SECONDS] [FILTER...]Compile and run the package’s tests
plenty add [ALIAS] SOURCE [--version REQ] [--offline]Add a dependency to the manifest and lock it
plenty fetch [--locked] [--offline]Resolve, download, and lock every dependency without compiling
plenty update [PACKAGE...] [--offline]Move Git dependencies to the commits their tags and revisions name now
plenty graph [--locked] [--offline]Print the package graph file for plentyc --package-graph
plenty cleanRemove target/
plenty lspServe the language server on stdin and stdout

--locked fails instead of creating or changing plenty.lock. --offline never runs git fetch; it uses only the package cache in .pkg/. Executables are stripped of their symbol table and debug sections in both profiles; --no-strip keeps them.

build --test-hooks and run --test-hooks enable failure injection in project executables. Switching this flag rebuilds the executable. These instrumented builds retain symbols, like plenty test, and are intended for integration tests.

The package name defaults to the directory name and must be a lowercase identifier other than std. Status lines go to standard error.

plenty run inherits standard streams, environment, and working directory, and exits with the program’s status; Unix signal termination becomes 128 plus the signal number. With several executables, --bin NAME is required.

Any other first argument is a compiler command: plenty FILE still compiles and runs one file. A file named like a project command needs a path, such as plenty ./run. The plentyc binary provides only the compiler commands.

Check coverage

A target’s check covers the files its import graph reaches: its entry, the package modules it imports, and the dependency libraries those import. plenty check checks every target, then reports each .plenty file below the package’s src/ that no target reached. Such a file was not parsed or type-checked, so it may contain errors:

  Checking tools v1.0.0
warning: src/util/orphan.plenty is not imported by any target of `tools`; it was not checked
   Checked tools v1.0.0: library and 2 executables; 4 of 5 files in src/

The warning does not change the exit status. The summary names the targets checked and counts the files below src/ that they reached. Test files are not targets: plenty test checks tests/ when it compiles them, and plenty check ignores that directory. Every .plenty file below src/ is package source, which build reuse also fingerprints; keep fixtures, generated samples and other intentionally unbuilt .plenty files in another directory, such as tests/fixtures/ or fixtures/. Files without the .plenty extension are never reported.

Tests

Each file directly in tests/ is a test file; files in subdirectories are helper modules. A test file defines either main or test_ functions, not both, and defining neither is an error:

  • A file with main is one test, named by the file stem, that passes when the program exits with status zero.
  • Each module-level function whose name starts with test_ is a test named FILE::FUNCTION. It takes no parameters and returns () or Result[(), E] written out, not through an alias; any other signature is an error. It passes when it returns normally or returns Ok. A returned Err is written to standard error as error: NAME returned ERROR, except Failure.Unspecified and std.test.TestError.Assertion: the first carries nothing to show, and the second follows an explanation std.test already wrote.

Plenty has no attributes or reflection, so plenty test finds test functions by parsing each file, then compiles the file with a generated main that calls the function named by its one argument. Every test runs in its own process, so a trap, abort, or leaked state fails or affects only that test. The generated main exits 0 when the test passes and 3 when it returns Err; a trap exits 1.

A test file imports the package’s library under the package name, the package’s dependencies under their aliases, and its own helpers under tests/. Because those names are dependency aliases, a test file or helper cannot be named after the package or one of its dependencies.

plenty test compiles every selected test file to target/PROFILE/tests/PACKAGE/STEM before running any. These executables also link the hooks behind std.test failure injection, which wrap the C library’s allocation and file calls; plenty build and plenty run never link them. Each test runs with the package directory as its working directory, empty standard input, and TMPDIR set to a new empty directory that the runner removes, with everything in it, when the test ends. That removal also covers scratch directories of a test that traps or times out, which skip destructors. The per-test directories live in one plenty-test-run- directory under the runner’s own temporary directory.

Tests run in parallel, at most one per available CPU unless --jobs N says otherwise. Results are printed in discovery order whatever order the tests finish in, so repeated runs print the same lines. Tests that share files in the package directory must use --jobs 1 or their own names.

Each test’s standard output and error are captured together and printed under failures: only if it fails; --no-capture passes them through instead, and parallel tests’ output then interleaves. The command prints one line per test and a summary, and exits nonzero if any test fails:

running 2 tests
test area::test_square ... ok
test area::test_negative ... FAILED (returned Err)

failures:

---- area::test_negative output ----
assertion failed: left == right
  left: -9
 right: 9

test result: FAILED. 1 passed; 1 failed
OptionEffect
FILTER...Run only the tests whose names contain one of the filters; the summary counts the rest as filtered out
--listPrint the selected test names, one per line, without compiling
--no-captureLet test output through as it is written
--seed NSet PLENTY_TEST_SEED=N for every test; std.test seeds its generator from it
--jobs NRun at most N tests at once; N is at least 1
--timeout SECONDSKill a test still running after SECONDS (at least 1) and fail it as timed out after SECONDSs; processes it started are not killed
--fast-compile, --locked, --offlineAs for plenty build

A failing test’s line gives the reason: returned Err, exit status N (a trap is status 1), terminated by signal N, or timed out after Ns. Without --timeout a test can run forever.

Profiles

ProfileSelected byOutputCode generation
optimizeddefaulttarget/optimized/Cranelift opt_level=speed and optimized call-site passes
fast-compile--fast-compiletarget/fast-compile/Cranelift opt_level=none; skips optimized call-site passes

Profiles change code generation only. Overflow, bounds, ownership, and every other language check behave identically; neither profile adds debug information. Use --fast-compile for shorter edit-run cycles in large projects. Measure runtime performance with the optimized default. Compiled functions, program objects and executables remain cached separately by profile.

plentyc FILE and plentyc --compile FILE -o OUT also optimize by default; --fast-compile selects the same unoptimized code generation for single files. --release is accepted as a compatibility alias for the optimized default. These are Plenty profiles, independent of Cargo’s profiles for building the compiler.

Build reuse

build, run, and test skip a target whose output exists and whose inputs are unchanged since its last successful build; the package’s status line then reads Fresh instead of Compiling. build reuses a library check the same way; check always checks again. The inputs are the content of every file below each graph package’s src/ (not only the imported ones), the package graph, the entry file, the target kind, the profile and linker options, and the compiler executable (path, size, and modification time; the compiler embeds std and the runtime). Fingerprints live in target/PROFILE/.fingerprint/. A build forgets the old fingerprint before it starts, so a failed or interrupted build is never reused. Reuse is per target: a stale target is recompiled whole.

Manifest

Language editions are not supported; edition is an unknown manifest key. The editions proposal explores per-package source compatibility independently of plenty-version and manifest-version.

manifest-version = 1

[package]
name = "app"
version = "0.1.0"
description = "optional"
authors = ["optional"]
license = "optional"
plenty-version = "0.1.0"

[lib]
path = "src/lib.plenty"

[[bin]]
name = "app"
path = "src/main.plenty"

[dependencies]
util = { path = "../util", version = "^0.1.0" }

[dependencies.audio]
git = "https://example.com/audio.git"
tag = "v1.2.0"
subdir = "packages/audio"

manifest-version and [package] with name and version are required. version and plenty-version are MAJOR.MINOR.PATCH. plenty-version is the minimum toolchain version; an older toolchain rejects the package. The numbers above are sample package versions and compatibility requirements, not the current compiler version. plenty new and plenty init start the new package at 0.1.0 and set plenty-version from the compiler that creates it. Unknown keys and tables are errors, so a misspelled setting is never ignored. The manifest is a subset of TOML 1.0: basic and literal strings, decimal integers, booleans, arrays, tables, inline tables, and arrays of tables. Dotted keys, dates, floats, and multi-line strings are errors.

Targets default to src/lib.plenty (the library) and src/main.plenty (an executable named after the package) when those files exist. [lib] and [[bin]] replace the defaults; their paths are .plenty files below src/. A package needs at least one target. A library entry may define main; it is an ordinary function there.

Dependencies

Each [dependencies] key is an alias: the only name under which that package’s code imports the dependency’s library. import util binds the library module, and import util.shapes reads src/shapes.plenty in the dependency. Aliases are lowercase identifiers. std cannot be an alias, and an alias that matches a local top-level module or directory is an error.

path is relative to the declaring manifest and names a live directory containing plenty.toml. A git dependency is described below.

version is an optional requirement on the dependency’s version, checked after resolution; it never selects a version. X.Y.Z and ^X.Y.Z accept versions at least X.Y.Z with the same leftmost nonzero component (^0.2.1 accepts 0.2.5 but not 0.3.0), =X.Y.Z accepts only that version, and >=X.Y.Z accepts that version or later. A mismatch names both packages:

error: package `app` requires `util` ^2.0.0, but /work/util is version 1.4.0

Resolution reads every manifest before compiling. A graph contains one package per name: two dependency paths reaching the same directory share one package and its types, whatever their aliases; two different directories with the same package name are an error that names both paths. Package cycles are errors. Dependencies are not transitive: a package imports only what it declares.

Each package’s sources live in its own src/ directory and cannot import files of another package except through a declared alias. The same module path in two packages names two modules, so Point in two packages is two distinct types. Diagnostics name a dependency’s declarations as PACKAGE::MODULE.Name, such as util::lib.Point; the root package’s names are unqualified.

Git dependencies

[dependencies]
tz = { git = "https://github.com/plentylang/tz.git", tag = "v0.3.0" }
codec = { git = "https://example.com/media.git", rev = "4f2a9c1", subdir = "codec" }

A git dependency needs exactly one of tag (a tag name) or rev (7 to 40 lowercase hexadecimal digits of a commit). subdir is the package’s directory inside the repository, relative and without ... Plenty runs the system git program; any URL it accepts works, including file:// and SSH. Credentials come from Git’s own configuration; Plenty never prompts.

The first resolution fetches every branch and tag of the repository into a bare repository under .pkg/git/db/, records the commit that the tag or revision names in plenty.lock, and extracts that commit’s tree into .pkg/git/checkouts/REPO/COMMIT/. Later commands build the locked commit, even if the tag has moved upstream; they fetch only when the cache lacks the commit, as after deleting .pkg/. Each repository is fetched at most once per command. Snapshots are extracted beside their final path and renamed into place, so an interrupted fetch never leaves a partial snapshot. .pkg/ is generated: the .gitignore that plenty new writes excludes it.

plenty update resolves every tag and revision again and rewrites the lockfile; plenty update NAME... resolves only the named packages. It prints each changed commit.

A path dependency inside a Git package names another package of the same commit, and cannot leave the repository. A graph contains one commit per package: if two dependents reach the same repository directory at different commits, resolution fails and names both chains:

error: conflicting requirements for package `geo` from https://example.com/mono.git:
  app requires tag=v2 (commit c25fe24762e9)
  app -> shapes requires commit 421b7a9fd85a through a path in the same repository
a graph may contain one commit per package
SituationBehavior
--offline and the repository or commit is not cachedError naming the URL; nothing is fetched
--locked and the lockfile has no commit for a requirementError before any fetch
The locked commit is gone upstream and not cachedError suggesting plenty update
A locked commit is not 40 lowercase hexadecimal digitsError: the lockfile is malformed; delete it to resolve again
The tag or revision does not existError naming the URL and tag or revision
git is not installedError saying Git dependencies need git on PATH

Adding dependencies

plenty add appends one entry to the [dependencies] table of the package containing the current directory, keeping the rest of the file unchanged, then resolves and locks the graph. If resolution fails, the manifest is restored.

plenty add --path ../util
plenty add tz --git https://github.com/plentylang/tz.git --tag v0.3.0 --version 0.3.0
plenty add codec --git https://example.com/media.git --rev 4f2a9c1 --subdir codec

--path is relative to the current directory and is written relative to the manifest. The alias defaults to the dependency’s package name; adding an alias that already exists is an error.

Workspaces

A workspace is a directory whose plenty.toml lists member packages:

manifest-version = 1

[workspace]
members = ["app", "util"]

Members are directories below the workspace, without ... The same manifest may also contain [package], which makes the workspace directory a member too. Members share one plenty.lock, one .pkg/, and one target/ in the workspace directory, and depend on each other through ordinary path dependencies.

A command run in the workspace directory acts on every member: check and build process each in manifest order, run chooses among all their executables (whose names must be unique), and test prefixes each test name with PACKAGE/. A command run inside a member acts on that member only, with the shared lockfile and outputs. plenty add and plenty graph need a member.

Lockfile

Every project command writes plenty.lock beside the root manifest (or in the workspace directory) when it is missing or out of date. It lists each package’s name, version, source, and dependencies as alias/package pairs: the root package first, then the others sorted by name. A source is root, path+RELATIVE from the lockfile’s directory, or git+URL with the subdir, the full commit, and the selectors (such as tag=v1.2.0) that resolved to it. It contains no absolute paths, so a checkout can move. Writes replace the file atomically. --locked fails instead of creating or changing it. A lockfile that cannot be read is an error; deleting it resolves every dependency again.

Package graph files

plenty graph prints the resolved graph of the current package as TOML, which plentyc --package-graph FILE reads in place of --module-root:

graph-version = 1

[[package]]
identity = "app"
root = "/work/app/src"
dependencies = { util = "util" }

[[package]]
identity = "util"
root = "/work/util/src"
library = "/work/util/src/lib.plenty"
dependencies = {}

The first package contains the entry file. identity is unique and qualifies the package’s declarations in diagnostics; dependencies maps each alias to an identity whose package has a library. Relative paths are relative to the file. --package-graph works with FILE, --check, --check-module, and --compile. It lets another build system compile a package without plenty.toml, and makes the compiler’s input visible when debugging resolution.

Application exit and error reports

The native wrapper calls the application’s main once. Unit and Ok(()) produce status zero, i32 and Ok(i32) supply the status, and Err produces one. The operating system may truncate statuses; small nonnegative values are portable. Entrypoint signatures are defined in the language contract.

A returned Err produces one report, identically for compile-and-run and a compiled executable. The wrapper flushes standard output, writes one line to standard error, destroys the error payload once, and returns one. Intermediate ? propagation prints nothing.

The line begins error: main returned . When the concrete error type satisfies std.fmt.Formattable, its borrowed format method supplies the message. An anonymous sum takes this path only when every member satisfies the protocol. Otherwise the payload is rendered as str.repr would render it. Errors needing an explanation add one after a colon: IoError.System(code) uses the OS description, payload-free IoError variants use fixed descriptions, and Failure states that it retains no original details. Choose a concrete error type when the report should identify the cause.

Reporting is best effort. On the structural path, if allocation fails or the type cannot be rendered (for example it contains a generator or recursive data), the wrapper writes the allocation-free fallback error: main returned Err. Standard-error write failure, including a closed descriptor or a pipe without a reader, is ignored. Every path still destroys the payload and returns one.

Human-readable reporting allows 16,384 bytes for the prefix, message and final newline, with a maximum rendering depth of 64. A formatting or budget failure appends [report unavailable] and a newline, outside that byte budget. The already-written prefix remains; the formatter is never called again and its error is not reported recursively. A terminal stderr failure stops output without attempting the suffix. These writes suppress SIGPIPE and need no heap allocation, though a user formatter may allocate. Reports stream in pieces and can interleave with other writers; they are not atomic records. User-supplied text is emitted unchanged, so it may itself contain newlines.

Context[E] uses human-readable reporting when its cause supports it, printing the operation/path followed by the cause once. Other contexts retain structural reporting. Reporting does not change which types may be returned in Err or the behavior of intermediate propagation.

Compilation/checking errors execute no program code. Runtime errors can leave effects already performed. Traps and stack overflow do not unwind.

Native linking and targets

Native emission implements x86_64-unknown-linux-gnu and aarch64-unknown-linux-gnu. ARM deployment qualification is described in Linux AArch64 development. --target TRIPLE requires an exact match between the requested target, packaged runtime, and Cranelift ISA before emission. Unsupported explicit targets fail before source loading. This flag does not enable cross-compilation. Checking is independent of native emission unless a target is explicitly requested.

--linker PATH selects a cc-compatible driver; the default is cc on PATH. Repeat --link-arg ARG to pass individual arguments after program/runtime inputs and before the runtime’s native dependencies. Static-library builds use --archiver PATH, defaulting to ar; it must support deterministic rcsD mode.

With no --linker or --link-arg override, Plenty prefers mold, then lld, then the driver’s default linker. It looks for executable ld.mold or ld.lld on PATH and checks that cc -fuse-ld=NAME -Wl,--version succeeds before selecting one. An unavailable or rejected candidate falls through to the next choice. A failure linking the actual program is reported without retrying with a different linker.

Discovery runs once per compiler process; set PATH and the driver’s environment before starting it. The probe’s result is also recorded in the compile cache directory, under DIR/linker-probe/, so later processes skip the probe. A record’s key covers PATH, COMPILER_PATH, and the resolved path, size and modification time of cc and of each ld.mold and ld.lld found on PATH: installing, removing, upgrading or touching any of them causes a fresh probe. The directory keeps the 64 newest records. With the cache off (PLENTY_CACHE=off, --no-cache), each process probes again. A linker that cc finds elsewhere than PATH, such as its own installation directory, is not part of the key; after changing one, remove DIR/linker-probe/.

PLENTY_LINKER_THREADS=N limits the automatically selected linker to N threads: it adds -Wl,--thread-count=N for mold and -Wl,--threads=N for lld. Unset or empty keeps the linker’s default, and a value that is not a positive integer fails the link. It does not apply to a linker selected with --link-arg or found through COMPILER_PATH. mold uses every core by default, which is fastest for one link; when many links run at once, as in a parallel test run, each link’s threads compete with the others, and one thread per link finished 400 links at 8 at a time in about 60% of the time.

Pass --verbose to report the effective driver command. --linker cc opts out of automatic selection and uses the driver’s configured default; to pin a particular linker, use e.g. --link-arg -fuse-ld=bfd or --link-arg -fuse-ld=mold. Any explicit --link-arg, even -lm, disables automatic selection. These options apply to both compiler binaries, including shared libraries and fuzz executables. Object emission does not probe a linker. In the Rust API, CompileOptions::auto_linker = false opts out; non-default drivers and nonempty link_args also disable discovery. verbose reports the command. The caller must explicitly disable auto_linker when selecting cc itself, since assigning that default value cannot record an explicit override.

Symbols and debug information

Executables and shared libraries are linked with -Wl,--strip-all, which drops the symbol table (.symtab, .strtab) and every .debug_* section. Generated code carries no debug information; these sections come from the Rust runtime archive and its precompiled standard library, and were about three quarters of a hello-world executable:

ProgramUnstrippedStripped
empty main2,727,008 bytes302,280 bytes
print("hello")3,289,728 bytes789,776 bytes

Measured on x86-64 with mold. Stripping leaves the loaded segments unchanged (size reports the same text, data and bss), so it affects file size, not memory use or speed.

Nothing reads these sections at run time. Trap reports find the failed check through the compiler’s site and trap-instruction tables, which the linker resolves as relocations, not through symbols; the runtime prints no backtraces. A shared library keeps its dynamic symbol table (.dynsym), so its exports and --verify-interface are unaffected, and its embedded .plenty.interface.* metadata is not a debug section.

--no-strip keeps both, for --compile and --shared-library; plenty build and plenty run accept it too. For single-file compilation with plenty or plentyc, --fast-compile and --no-strip may appear in either order before or after --compile FILE, including between the mode and filename. Use --no-strip to inspect an executable with nm, perf, or a debugger, or to split the debug information into a separate file:

plenty --compile --no-strip app.plenty -o app
objcopy --only-keep-debug app app.debug
objcopy --strip-all --add-gnu-debuglink=app.debug app

--fuzz targets, plenty test executables, and explicit --test-hooks builds are never stripped: they run and are debugged where they are built. Static libraries are archives of unlinked objects, so the final link decides; --emit-object likewise leaves stripping to the external link (see below). In the Rust API, CompileOptions::strip is true by default.

Stripping is part of the linker’s deterministic output and does not change the reproducibility contract below; package build fingerprints include it.

Size budgets

tests/test_executable_size.rs fails when an empty program or a hello program grows past its stripped-file or .text budget. The measurements and current limits live beside BUDGETS in that file. x86-64 and ARM64 have separate limits, as do ordinary runtimes and the runtime-checks feature’s allocation accounting. Compare both sizes: linker layout and segment alignment can increase the file without increasing its code. ARM files with a separate read-only LOAD segment after executable code and 64 KiB segment alignment use a separately measured file-size budget. Compact layouts keep their smaller file limits, and both layouts keep the same .text limits.

A failure preserves the measured executable, source, and layout.txt in the directory printed by the test. The report records sizes, section offsets, segment alignment, ELF producer comments, runtime-checks mode, Git revision, and available toolchain versions. Missing version commands are recorded rather than preventing artifact retention. Set PLENTY_SIZE_ARTIFACT_DIR to a dedicated directory to retain successful measurements too; each sample gets its own subdirectory. CI uploads these samples as executable-size-ARCH-shard-N artifacts with seven-day retention, so comparisons can use the actual runner ELFs. Failed Linux release builds also upload their samples as release-size-evidence-TARGET artifacts with fourteen-day retention. Local failures and explicitly requested samples remain until removed. Tool versions queried during the test describe the current environment; ELF producer comments can identify the actual linker (mold and lld record their versions there).

The same file checks, for each std module with runtime native code, that hello links none of that code, that importing any std module links none of it, and that a program calling the module does link it. Its NATIVE_MODULES table names each module, a program, and the symbols that prove the code is linked: the module’s entry symbols and, for a vendored library, part of its symbol names. A new native wrapper adds a row. Raising a budget is a reviewed change that records the new measurement beside BUDGETS.

Reproducible artifacts

Generated code uses the original x86-64/SSE2 baseline or generic ARMv8-A, regardless of the compiler machine’s CPU extensions. ARM does not require optional LSE, SVE or crypto instructions. --print-target reports the native triple and CPU baseline. Reproducibility applies within one target and fixed toolchain; artifacts from different architectures have different bytes. With the same compiler binary, source tree, module contents, and options, application objects and extracted runtime artifacts are independent of source directory, output name, time, user, and code-generation worker count. Static archives use deterministic member timestamps, owner IDs, and modes.

Executables and shared libraries additionally depend on the linker driver, its options, startup objects, and native libraries. Automatic selection makes the available linkers and driver environment inputs too: pin the linker for reproducible builds across machines. Package build fingerprints include the automatically selected linker name. Different versions of the same tool still require a clean rebuild after changing the native toolchain. Determinism is checked with the locally selected toolchain; other linker implementations are separate inputs. A program that reaches a native engine also depends on the engine’s directory, which ordinary output records in RUNPATH. Distribution output instead uses only $ORIGIN and copies the selected engines and their notices. Rebuilding the compiler also depends on rustc and its precompiled standard library. Runtime compilation remaps project, output, and sysroot source paths; it cannot rewrite paths already compiled into the Rust standard library. Changing the compiler or any of these toolchain inputs can change output bytes.

Paths and arguments are passed directly, without a shell, expansion, or word splitting. There is no CC environment override. The linker driver must accept object/archive inputs, Unix native flags, and -o OUT; GCC and Clang do. Drivers with different conventions need a wrapper. Choosing a driver does not expand target support. Failures report its identity, status, and diagnostics.

External build systems

--emit-object requires no linker and exports plenty_main. Link exactly one application object, followed by the runtime archive extracted from the same compiler build and its native arguments, one per line in link-args.txt. The first argument is -Wl,--gc-sections. The object puts each function and data object in its own section, so the linker drops every function main cannot reach, including the unused functions of imported std modules, and then the runtime code only they call, such as the std.crypto hashes, the std.compress codecs and the std.tls stack. The runtime supplies C main. link-args.txt lists only what linking requires, so it does not strip; add -Wl,--strip-all (or -s) to match plenty --compile. Neither object emission nor runtime extraction needs Cargo or rustc on PATH.

Ordinary Plenty functions, including pub functions, use private symbols and calling conventions. Application object output does not expose a C library API or provide separate Plenty module compilation; use a library build for exports.

The Rust API exposes CompileOptions, executable compilation with options, compile_source_to_object, compile_file_to_object, and emit_runtime. Runtime extraction returns archive and argument paths/values for build-system integration.

Native engines

An engine is a large native library behind a std module that ships outside the runtime archive, as a separate shared library. Only programs whose retained code reaches the module link it. std.dataframe (Polars) and std.linalg (faer, issue #490) are engines today; images (issue #476) is another candidate.

When a module is an engine

Code in the runtime archive costs nothing to programs that do not reach it: the linker drops it with --gc-sections (see linking), provided only the module’s own entry symbols reach it (issue #501). It still grows the compiler, which embeds the archive, and every compiler build compiles it. So a native module stays in the runtime archive unless one of these holds:

  1. Size. A program reaching one ordinary operation of the module grows by more than 2 MB, measured as the stripped executable minus the stripped print("hello") executable.
  2. Dependencies. Its crates need a native toolchain beyond rustc and cc, a system library outside the release baseline, or a build that adds more than a minute to a clean release build of the compiler.
  3. Release cadence. It must be rebuilt or upgraded independently of the compiler, for example to follow a fast-moving third-party release.

Measured on 2026-10-26 with stripped release builds:

CodeSize
print("hello") executable789,584 bytes
plus a std.tls client configuration2,075,096 bytes (+1.3 MB)
runtime archive embedded in the compiler15.1 MiB
Polars dataframe engine, libplenty_dataframe_v1.so89,553,952 bytes

std.tls, the largest runtime module, stays in the archive; the dataframe engine qualifies on size and build time. A new wrapper’s issue or PR records the same measurement before choosing.

Shared libraries, not static archives

Engines are shared libraries. A separately supplied static archive, passed to the linker only when reached, was considered and rejected:

  • Linking. An engine written in Rust carries its own copy of Rust’s standard library, allocator shims and panic runtime. The runtime archive already defines these and installs Plenty’s global allocator, so two Rust static archives in one link either collide or must be built as one crate graph, which is the runtime archive again. A shared engine keeps its copy private and exports only its facade symbols.
  • Executable size. A static engine adds the code each program reaches. Large engines dispatch internally (query planners, codec tables), so a program reaching one operation retains most of the engine, in every executable. A shared engine adds only a DT_NEEDED entry and the imported symbols, and one copy serves every program on the host.
  • Deployment. The cost of a shared engine: the application ships the library beside the executable (distribution output automates this).
  • Reproducibility. Under the reproducibility contract an executable that reaches an engine also depends on the engine’s directory, which ordinary output records in RUNPATH (distribution output uses only $ORIGIN), and on the facade’s symbols, but not on the engine’s bytes. A static engine would make every executable depend on the engine library’s bytes, which would then need the runtime’s deterministic build too.

Revisit static archives only for an engine with no Rust standard library or allocator of its own.

The engine contract

A second engine follows std.dataframe in each of these.

PartRule
Facadestdlib/std/NAME.plentyi, a trusted interface whose extern def declarations name the engine’s C symbols. Plenty code in the facade is ordinary std code.
SymbolsEvery export starts with a prefix unique to the engine and its ABI generation, such as pldf_v1_. The library exports nothing else.
ABI generationlibplenty_NAME_vN.so, also its SONAME. Adding an operation adds symbols and keeps N. Changing or removing a symbol, a layout or an ownership rule starts N+1, with a new prefix and file name, so existing executables keep loading vN.
ReachabilityThe compiler follows the object’s relocations from its roots, as --gc-sections does. An unused import, unused helper or unused facade function needs no engine and adds no DT_NEEDED. Windows executable emission omits unreachable user helpers before COFF import resolution; explicit objects, libraries and dumps retain user definitions (emission policy).
Discovery at compile timeCompileOptions field, then PLENTY_NAME_ENGINE, then libplenty_NAME_vN.so beside the compiler executable. Ordinary output rejects : in the engine directory (an ELF RUNPATH separator). Build fingerprints include the selected path, size and modification time.
Discovery at run timeRUNPATH of $ORIGIN, then the engine’s directory at link time for ordinary output; distribution output uses only $ORIGIN. The loader’s LD_LIBRARY_PATH and system configuration also apply.
Missing engineLinking fails, naming the module, the path tried, how to build or install the engine, and the environment variable. Type-checking never needs the engine.
Mismatched engineAn older engine lacking a symbol fails at link time (undefined reference to PREFIX...) or at load time (symbol lookup error). Release packaging rejects an engine whose exports do not cover the facade’s declarations. A different N is a different file and is never substituted.
ErrorsEach call returns an error code; the engine catches panics at the C boundary and reports them as an error. Handles are freed by the engine that allocated them. Engine allocations are outside Plenty’s allocation accounting; the module’s reference page states whether allocation failure is recoverable.
BuildA separate Cargo workspace, plenty-NAME/, with its own Cargo.lock, built by a script outside the compiler build, for the selected x86-64/SSE2 or ARMv8-A target and the release glibc.
ReleaseThe matching compiler archive bundles the target’s engines in bin/ beside NAME-notices/, with engine.json (Plenty version, source commit, target, ABI generation, library checksum and platform minimum), Plenty and dependency licenses, and Rust standard-library notices. --dist validates metadata and the copied-library checksum and includes the notices; ordinary compilation does not read engine.json.
TestsThe ordinary suite checks that unreached uses link no engine and that a reached use without one reports the missing engine. A separate CI job builds the engine and runs the real-engine tests, gated by PLENTY_NAME_TESTS=1.

macOS dataframe engine

The experimental macOS ARM64 archive bundles libplenty_dataframe_v1.dylib and dataframe-notices/. Discovery uses the same option/environment precedence, then the dylib beside the compiler. The dylib has versioned identity @rpath/libplenty_dataframe_v1.dylib; applications use @loader_path to find their sibling copy. Ordinary compilation copies the dylib beside its output; distribution output includes validated metadata and notices. Engine builds target Apple M1 and macOS 11. Packaging audits the deployment minimum, dependencies, exports and ad-hoc signature. The linear algebra engine remains Linux-only.

Windows dataframe engine

The experimental Windows x86-64 archive bundles plenty_dataframe_v1.dll and dataframe-notices/. Discovery uses the same option/environment precedence, then the DLL beside the compiler. MSVC lib.exe generates an import library from the DLL’s C exports. Ordinary compilation copies the DLL beside the output; distribution output includes its validated notices. Runtime lookup follows Windows loader rules rather than ELF RUNPATH. The linear algebra engine remains Linux-only.

Adding a second engine

The compiler side is a table: an entry in ENGINES (src/toolchain.rs) gives the facade module, library, symbol prefix, environment variable and install hint, and drives reachability, link arguments, the facade’s .plentyi resolution and build fingerprints. The CompileOptions override is still a named field per engine.

The shared build and packaging entry points are scripts/build_native_engine.py and scripts/release/package-engine.py, each taking an engine name. scripts/build_dataframe_engine.py remains compatible. Packaging checks facade exports and the engine SONAME, generates component-specific dependency notices, and installs the library with its NAME-notices/ directory, in the compiler archive’s bin/ for releases. The native-engine CI matrix builds and tests each engine separately; engine tests never make normal compiler builds depend on either engine workspace.

Application distributions

Use explicit distribution output when shipping an application:

plenty --compile analysis.plenty --dist dist/analysis
./dist/analysis/analysis

plentyc accepts the same command. The executable is named after the source file’s stem plus the native executable suffix (analysis.plenty becomes analysis.exe on Windows and analysis on Unix), not the destination directory. Dots in the stem are preserved: analysis.v2.plenty becomes analysis.v2.exe on Windows. Paths may contain spaces. --fast-compile, --no-strip, --module-root and --package-graph work as for ordinary executable compilation. --dist requires --compile FILE and rejects -o/--output, extra --link-arg arguments and other output modes. A custom --linker must still produce the required lookup policy for the native platform.

The destination must not exist, even as an empty directory, file or dangling symlink. Choose a new name for each build. Plenty stages the output beside the destination and publishes the complete directory without replacing an existing path, including one created concurrently. Compilation, linking, missing notices or publication failures leave no partial destination; temporary staging is removed on normal error exits. Parent directories may be created. Abrupt process termination can leave a hidden .plenty-dist-* staging directory, not a published bundle.

Contents and engine selection

A program without reachable native-engine calls produces only its executable. Otherwise Plenty copies only the engines selected for that link, using the ordinary engine discovery and reachability rules. Unused imports and unreachable helpers do not add engines. Each used engine adds:

dist/analysis/
  analysis
  libplenty_NAME_v1.so
  NAME-notices/
    engine.json
    LICENSE-MIT
    LICENSE-APACHE
    THIRD-PARTY-NOTICES.md
    licenses/...

NAME is dataframe or linalg; a program may need both. Executable names that collide with engine or notices names are rejected. Application assets, application licenses and non-engine dynamic dependencies are not collected automatically. Add your own required files before distributing the directory.

The selected engine needs its packaged notices. Supported input layouts are:

  • A library beside NAME-notices/, which holds engine.json, license files and licenses/. Linux release archives install both engines this way in bin/, and the compiler selects them without configuration.
  • lib/libplenty_NAME_v1.so with the same files in the parent of lib/. Set PLENTY_NAME_ENGINE to the library’s absolute path.

Plenty checks manifest format 1, compiler release version, native target, ABI generation and library name; requires source-commit and minimum-glibc provenance; and verifies the copied library’s SHA-256 and SONAME. It links that staged copy, so the bundled bytes are the bytes selected for linking. Missing files, empty license directories, symlinks within notices, malformed metadata and mismatches fail explicitly. No notices are fabricated or silently omitted.

This is consistency checking, not signature verification or proof that a local engine was built from the compiler’s source revision. Use trusted, matching release artifacts or build and package from the same checkout. Ordinary --compile FILE -o OUT does not require or validate engine metadata.

macOS dataframe distributions

On macOS ARM64, the same command produces analysis, libplenty_dataframe_v1.dylib and dataframe-notices/. The executable links the dylib’s versioned @rpath identity and searches only @loader_path for the bundled engine. Ordinary compilation also copies the dylib beside its output, using the same lookup policy. Neither form retains the original engine path.

The manifest requires minimum_macos instead of minimum_glibc. The compiler checks the copied library’s SHA-256, ARM64 Mach-O kind, versioned dylib identity, signature presence and absence of build-machine dependencies/search paths. Release packaging additionally verifies signatures with codesign, deployment minimums and facade exports. Applications can require a newer OS than the engine: compile and test on the oldest system you intend to support. macOS archives bundle the dataframe engine only and remain unnotarized.

Windows dataframe distributions

On Windows x86-64, the same command produces analysis.exe, plenty_dataframe_v1.dll and dataframe-notices/. The DLL uses Windows loader lookup from the application directory. The compiler derives an import library with MSVC lib.exe; that build artifact is not needed by end users. Metadata requires minimum_windows instead of minimum_glibc; the copied library must be x86-64 PE and match its SHA-256. There is no ELF SONAME or RUNPATH on Windows. The Windows archive bundles only the dataframe engine.

Package a local engine

For local development, use just build for the compiler and both engines, or just engine linalg / just engine dataframe for one engine. These recipes install the library and matching NAME-notices/ in target/release/, ready for --dist. They record the engine’s actual glibc requirements without claiming the portable release baseline; see the local build workflow.

A bare library installed by build_native_engine.py is enough for ordinary compilation, but not distribution output. Add local notices to it with:

python3 -B scripts/release/package-engine.py linalg \
  --engine target/release/libplenty_linalg_v1.so --out-dir target/release --local-install

For a portable engine, package a baseline-compatible build from the same checkout with the release packager, which the release build also uses to install engines in the archive’s bin/ (substitute dataframe and its library for the Polars engine):

python3 -B scripts/release/package-engine.py linalg \
  --engine /path/to/libplenty_linalg_v1.so --out-dir target/engine-bundle
export PLENTY_LINALG_ENGINE="$PWD/target/engine-bundle/libplenty_linalg_v1.so"
plenty --compile analysis.plenty --dist dist/analysis

The packager needs the Rust standard-library license texts and engine dependency sources to generate notices, and audits facade exports, SONAME, native dependencies and the glibc 2.28 baseline. A library built on a newer host may fail that audit. Build in the pinned release environment instead; the existing bash scripts/release/local-check.sh --skip-smoke builds and packages locally with Docker, without publishing. See local release checks. Do not relabel another release’s metadata to bypass checks.

Relocation and platform limits

On Linux, bundled executables use only $ORIGIN (the executable’s own directory) for engine RUNPATH, never the original engine directory. A sibling lib/ is not searched. Ordinary builds retain $ORIGIN followed by the build-time engine directory. PLENTY_NAME_ENGINE affects compilation only; loader configuration, LD_LIBRARY_PATH and LD_PRELOAD still apply at run time. Keep those paths trusted.

Move the whole bundle, run it from another working directory without loader overrides, and test that removing an engine makes loading fail even on the build machine. End users need no Plenty, Rust or C compiler, but still need compatible host C libraries. This is not static linking or OS-independent packaging: native emission uses the installed compiler’s native target. Building on a newer system can raise the executable’s glibc requirement. Build and test on the oldest system you support; engine metadata alone cannot establish portability.

Library artifacts and publication

Library builds produce a binary, C header, source interface, and native link arguments. Export ownership and ABI rules belong in the C interface contract.

Embedded metadata can be extracted without loading or executing the binary. Explicit interface verification checks the contract and its compatibility symbol; see embedded contracts.

For embedding applications, read_library_interfaces(path) returns all LibraryInterface entries sorted by library name, including exact source and target; extract_library_interface(path, name, output) selects one for output. verify_library_interface(binary, interface) performs explicit verification without executing a linker or library code.

Publication guarantees

The compiler stages a library’s files in a private directory beside the requested output, on the destination filesystem. It checks destinations before building and before publication. Each must be absent or an ordinary file; symlinks, directories, and paths overwriting loaded source or imported interfaces are rejected.

Compilation, native tool, argument, and staging failures preserve previous files. Publication replaces files by individual atomic renames and keeps backups until the complete set is installed. An installation error rolls back replacements and removes new outputs. If rollback fails, the diagnostic identifies the retained recovery directory containing backups.

Publication is not a crash-atomic transaction across files. Build systems must exclude concurrent writers and coordinate readers. A crash or power loss can leave a mixed set; fingerprints and verification help detect it.

Interface extraction uses the same mechanism for its single output. It rejects symlink destinations and cannot overwrite the input binary. Native tools receive paths as individual arguments, including paths with spaces or commas.

Fuzzing

plenty --fuzz FILE -o OUT builds a libFuzzer executable from a program. libFuzzer runs the program’s main once per input, watches which code each input reaches, and keeps the inputs that reach new code to mutate further. Inputs that trap are saved as findings.

plenty --fuzz main.plenty -o fuzzer
./fuzzer -max_total_time=60 corpus/
./fuzzer crash-1060186d20321afe84b2d22eeca505a577c686c6   # replay one input

Fuzz targets optimize by default. --fast-compile before --fuzz skips optimization when compilation latency matters more than fuzz throughput. --libfuzzer PATH names libFuzzer’s archive. Discovery asks clang -print-file-name=libclang_rt.fuzzer_no_main.a first, then the native target’s suffixed name (libclang_rt.fuzzer_no_main-aarch64.a or libclang_rt.fuzzer_no_main-x86_64.a). It accepts only a successful query naming an existing absolute file. LLVM’s compiler-rt supplies the archive: install compiler-rt on Fedora, or clang and libclang-rt-dev on Debian/Ubuntu. The build fails with a message when neither name resolves. When clang ships a separate libclang_rt.fuzzer_interceptors archive beside the selected runtime, Plenty links its matching generic or target-suffixed companion so libFuzzer can learn the strings compared by the program. It never substitutes another target’s companion. --linker and --link-arg apply as for --compile. The libFuzzer flags are LLVM’s; -help=1 lists them.

Writing a target

The program reads the input through two C functions that the fuzz driver defines. Declare them in a trusted module beside the program, for example fuzz_input.plentyi:

pub extern def size() -> u64 = "fuzz_input_size"
pub extern def byte(at: u64) -> u8 = "fuzz_input_byte"

byte returns 0 past the end. A target is an ordinary program that imports this module, rebuilds the data it needs from the bytes, and calls the code under test:

import fuzz_input
import std.bytes

def main() -> Result[(), std.bytes.BytesError]:
    mut data = std.bytes.empty()?
    mut at = 0u64
    while at < fuzz_input.size():
        data.push(fuzz_input.byte(at))?
        at += 1u64
    # parse `data` here
    Ok(())

A finding is a fatal runtime error such as an index out of range or unwrap of an Err, stack exhaustion on the thread running inputs (see the driver), or main ending with a nonzero status, including Err. libFuzzer reports the first as “fuzz target exited” and the others as “deadly signal”, saves the input, and exits with status 77. To check a property, return Err when it fails.

Each input runs the whole program, as the runtime’s main would, in the same process, so state outside the program, such as files it writes, persists between inputs. A target that serves a network protocol should feed the input through an in-memory stream rather than a socket; fuzz/http_server/main.plenty does this for std.http.server.

Instrumentation

Every block of every function, including the standard library’s, increments its own byte counter when it runs. The counters and a table of their functions are laid out as clang’s -fsanitize-coverage=inline-8bit-counters,pc-table lays them out, in sections __sancov_cntrs and __sancov_pcs; the driver passes both to libFuzzer. Fuzz targets keep their symbol table and debug sections, unlike --compile output. cov: in libFuzzer’s output counts the blocks reached, and ft: the features, which also distinguish how often a block ran.

Blocks are counted, not edges: a branch whose arms meet without running a block of their own adds nothing. Integer comparisons are not traced. String and byte-string equality reach memcmp, and libFuzzer’s interceptors, linked from libclang_rt.fuzzer_interceptors.a beside the --libfuzzer archive when it exists, report the strings compared, which libFuzzer then tries in inputs. A dictionary (-dict=FILE) of the format’s keywords helps with the rest.

Instrumented functions are cached like any other; their entries are separate from those of ordinary builds. Fuzz builds keep the counters and table of every function, so --gc-sections drops nothing they compile. Ordinary builds contain neither the counters nor the driver.

The driver

plenty-runtime/native/fuzz_driver.c is compiled with the compiler and linked only into fuzz builds, with -Wl,--wrap=main: the C library starts the driver, which registers the counters and calls libFuzzer’s LLVMFuzzerRunDriver, and each input calls the runtime’s main. libFuzzer is C++, so the link adds -lstdc++.

Parallel runs use -jobs=N -workers=N, independent processes sharing the corpus directory. With the interceptors linked, libFuzzer’s -fork mode crashes its own parent process within seconds (clang 22.1.8), reporting a “deadly signal” with an empty input; it is not a finding.

On Linux, libFuzzer’s RSS limit uses getrusage(RUSAGE_SELF).ru_maxrss, the process’s peak resident memory. Linux preserves this peak across exec, so a target launched directly by a large process can report an OOM before doing meaningful work. The integration tests first exec a small Python interpreter, then spawn the target from that interpreter’s small address space. An exec alone does not reset the peak. The tests retain a 2048 MiB per-target limit and report resource-limit failures (oom-* / timeout-*) separately from expected crash-* findings with exit status 77. The HTTP fuzz runner already spawns its target from Python. See LLVM’s peak RSS implementation and resource-limit handling.

libFuzzer’s handlers for SIGSEGV and SIGBUS, which save the input, run on the faulting thread’s stack. So that they still run when a program has exhausted its stack, the driver gives the thread running inputs a 256 KiB alternate stack and moves the handlers onto it. The runtime’s own stack-overflow message is not printed in fuzz builds. Only the thread running inputs has the alternate stack. A stack overflow on a thread the program spawns kills the process without saving the input.

Without AddressSanitizer libFuzzer has no leak checking or stack traces. Plenty code needs neither for memory safety; a trap prints its source position before libFuzzer’s report.

The target is the installed compiler’s GNU/Linux architecture (see linking). ARM libFuzzer/tool discovery and sanitizers require separate qualification; a successful ordinary ARM build does not prove their availability.

Compiler and runtime architecture

source → lexer/parser → name and alias resolution → signatures → checking
                                                                  ↓
                                                        typed operation IR
                                                                  ↓
                                                     Cranelift → object → linker
                                                                           ↑
                                                           embedded Rust runtime

The opt-in compiler phase profiler reports lowering setup/body/finalization, ownership analysis subphases, and native compilation/code collection/context cleanup. Benchmark workloads can be selected independently; profiling is never activated by the feature alone.

These pages describe mechanisms. Source rules live in the language contract, and operation guarantees live beside the built-in APIs.

Frontend and checking

frontend.rs handles name resolution, local inference, mutability, and lowering to typed operations. Tokens and the lexer (frontend/lexer.rs), the syntax tree (frontend/syntax.rs) and the core parser (frontend/parser.rs) read no lowering state; feature modules add their own parsing beside their lowering. Type references retain source positions. Aliases resolve through an iterative, cached dependency walk before signatures are collected; alias-only cycles are rejected without recursive traversal of long chains.

Nominal definitions reserve identities before resolving fields. Weak stored edges and table-owning handles represent recursive type graphs without compiler memory cycles. Concrete type facts and native metadata use finite graph walks. Cloning a structural type shares its already-owned child storage. When a field escapes a nominal table, cloning promotes its weak edges and copies only the structural paths that contain them. This is a single pass through the structural type, without following nominal table links; stored definitions still keep weak edges so sharing cannot create a table ownership cycle. Callable and boxed callable signatures follow the same rule: a signature is shared unless an input or the output holds a weak edge. A complete class or enum caches its facts; a collection or box combines its contents’ facts instead of walking again. Aliases normalize to Ty values and generate no runtime operations. Generic instance arguments distinguish Type(Ty) from typed constant values; values never become storage types. The constant evaluator also serves value arguments during type resolution, before ordinary module-constant folding. Quantity constant output types resolve lazily with cycle detection. The Bounded checker validates each concrete interval and lowers admitted literals and checked conversions to private single-field class construction.

The compiler’s own hash tables (src/hash.rs) use FxHash rather than std’s SipHash. Their keys are names and types from the program being compiled, so flood resistance buys nothing, and SipHash was about 11% of checking instructions (issue #302). Iteration order stays unspecified.

Module loading canonicalizes qualified names and checks imported modules together before producing one application object. Import traversal is bounded to depth 128 and 4,096 modules. This is not separate module compilation.

A module is lexed one top-level item at a time. The parser holds the tokens of the item it is parsing and of the next one, so a module’s tokens never all exist beside its syntax tree (issue #54). A lexical error anywhere in a module is still reported before any syntax error in it.

The whole syntax tree is live when lowering starts, so its layout sets much of checking’s peak memory. Blocks, call arguments and parameters are collected on parser scratch stacks and stored at their exact length. Every node holds a 40-byte token that names its module by an interned file number, not a reference-counted path (issue #422).

op.rs defines backend-neutral structured operations and an independent checker. Its compile-time operand stack is private compiler machinery. CompiledFn carries a signature, documentation, body, and typed local slots. Unit produces zero stack values. Loads and stores address slots; exhaustive branches produce conditional values.

The checker collects every signature on one thread, then checks function bodies on all available cores: a body reads only the signature table. It reports the first error in program order whichever body failed first, so diagnostics do not depend on the core count. Operations, types, and signatures are shared between threads without copying.

Name resolution of a module’s functions, and of its classes’ fields and methods, runs on all cores when the module has at least 256 functions or 64 classes: each reads only the module’s scope. Frame layout and generator operation validation after lowering also run on all cores. Both report the first error in source or program order, as a serial pass would. Threads share generator, closure, and nominal layout caches. A cached layout records how deep its frames and environments nest, and is reused only where it cannot hide the nesting limit of 64; otherwise the type is measured again. Each function’s layout error is therefore the same whichever thread measured a type first (issue #126).

Parsing, the rest of name resolution, type resolution, class expansion, generic preparation, signatures, and lowering bodies to operations run on one thread (issue #59). Type resolution can create generic data instances in a table shared by every module, and lowering interns strings, records generic instances, and infers returned fields in tables shared by every body. Generic preparation resolves each concrete function’s parameter and return types once; registering the function’s signature reuses them.

Three ways to run these phases on several threads were measured and not adopted. Each gained little and raised peak memory, because each thread’s allocator keeps its own free memory:

  • Parsing one module in pieces: for 5,000 units, parsing took 0.09 s instead of 0.21 s, lowering took 0.06 s longer, and peak memory rose from 417 MB to 538 MB.
  • Parsing imported modules in parallel, a level of imports at a time: for a program whose imports load 21 standard modules (12,549 lines), parsing took 9 ms instead of 23 ms and lowering 5 ms longer; the check was no faster, and peak memory rose from 28 MB to 35 MB.
  • Lowering bodies speculatively on all cores and keeping them in queue order: checking 5,000 units was about 9% faster and object emission no faster, and peak memory rose 12–14%.

Measuring compilation gives the time each phase takes.

Control flow and access

Return, TailCall, Break, and Continue terminate paths. The checker joins only continuing branches, validates transfers against signatures or loop entry stacks, and rejects operations after guaranteed exits. Loops lower to headers, bodies, and exits; the for increment precedes each continue.

An if statement lowers to one operation that holds its elif arms as a list, each with its own condition, so no pass recurses once per arm and a chain costs linear time. Every pass does recurse once per level of block or expression nesting, as do the clone and drop of the syntax tree. The parser therefore bounds that depth to 64 and later passes rely on it instead of checking again.

Ownership checking tracks definite slot availability through branches and loop backedges. Borrow checking flattens typed access facts into a separate source-level CFG with branch successors, early exits, loop edges, and stable binding-place IDs. Each CFG node stores up to two successors inline; only matches with more than two arms allocate a separate successor slice. Edges retain source branch order, including duplicate successors when empty arms join the same continuation. Loan liveness reaches a fixed point before native lowering. Functions without loan facts skip this analysis. It never runs on Cranelift IR and needs no whole-program alias analysis or per-call inspection of function bodies.

Native lowering still consumes structured operations. Cranelift SSA variables represent slots; stores define variables and sealed joins reconcile paths. Only slots whose address is taken (&x, &mut x) or whose values have inline storage live in the function’s stack frame; the frame holds those slots alone. Augmented assignment of a local (x += 1) takes no address, and starting a thread adds no slot: task storage has its own frame space and a worker reaches parent slots only through borrowed arguments. Generators keep every slot in memory. Source spans and marker operations carry diagnostic provenance without changing move or loan rules. A generic instance also records the use that first required it, so that lowering and ownership errors in its body are reported at that use.

Std ceiling checking

frontend/ceilings.rs checks fixed upper bounds without introducing value generics or another integer type. Embedded std source can append __max(1024) to a parameter or field declared Positive[i64] or NonNegative[i64]. The decimal non-negative bound is metadata on its TypeRef; resolved types, layout, equality and callable signatures are unchanged. The marker is rejected in user declarations and is hidden in public editor signatures and the generated std signature index.

Direct function, method and generated class-constructor calls check their supplied arguments before named/default wrappers replace expressions with local names. Defaults are checked at their declaration. Direct field assignments and replace(owner.field, value) use the same check, including through borrowed or boxed owners. The module-constant evaluator handles literals, module constants (including integer-domain and Quantity constants), and supported constant operators/casts. Constant folding runs after type resolution so quantities retain normalized dimensions and storage types during evaluation. Ordinary local bindings, arbitrary function calls and explicit fallible domain conversions are not constant evaluation. Indirect callable invocations and writes through a separately borrowed field do not retain this metadata. Existing std runtime guards cover values that reach operations through those paths; this mechanism does not claim a proof-carrying bounded integer type.

The annotations cover the fixed ceilings enforced by the std limit-domain migration: format/JSON/TOML/XML/template/regex nesting, fs/tempfile cleanup depth and attempts, sketch construction limits, unicode scalar limits, JWT token size, HKDF output size, image dimensions, actor heaps and procfs read sizes. Decoded sketch max_capacity has no ceiling. General bounded integers and lower bounds above one remain B96.

Embedding APIs

Source-string checking and compilation accept Plenty source and reject imports. File APIs (check_file, check_module_file, and file compilation) resolve imports from an explicit optional root. Checking performs no execution or native emission.

Editor queries

ide::analyze, behind plenty lsp, runs the same load, resolution, and lowering with a per-thread recorder switched on (src/frontend/ide.rs). Module loading records modules and imports; name resolution records declarations, module-level names, and locals with their binding positions; lowering records what needs types: methods resolved through the receiver, fields, enum variants, and the types of locals. An ordinary compilation records nothing. Expressions keep only the position of their first token, so member expressions also carry the position of the member name. In this mode an error in one function’s resolution or lowering is recorded and checking moves to the next function. See the language server.

Specialization and native emission

Generic declarations produce cached concrete specializations. Names, types, and operations are resolved before Cranelift receives IR; runtime metadata describes known storage and operations, not dynamic inference. Value parameters enter the same instance keys as typed integer or boolean constants, distinct from type arguments. Equal resolved constants share an instance even when spelled through different module constants; different values consume separate instances. Substitution turns body reads into typed literals and adds no runtime parameter.

Placeholder identities and queued concrete signatures break recursive checking dependencies. Each generic function or method allows 16,384 instances. A dependency chain of newly required instances may be 256 steps longer than the number of generic functions and methods declared plus generator consumers specialized. A program has finitely many generic declarations, so expansion without end must grow the instances of one of them without end. A chain in which no declaration repeats is no longer than that number, so only repeats can exceed the depth limit: a finite chain through distinct generic functions, however long, does not reach it (issue #435). Independent calls start at depth one, even when they specialize the same template with different arguments. A new instance inherits its first requiring caller’s depth plus one; lifted closures and generated call wrappers preserve that depth. Repeated calls, ordinary recursion, and alias-equivalent arguments reuse cached specializations without spending any limit. The depth check guards expansion, rather than counting the unrelated generics in a library. It is a conservative implementation bound, not a proof of infinite recursion. The per-template limit bounds expansion that branches within the depth limit, such as f[T] requiring both f[list[T]] and f[Box[T]]. A separate budget of 262,144 instances per compilation bounds the resources of expansion that drags other generic functions along, and reports a resource-budget diagnostic (issue #428).

Generic data and generator consumers follow the same scheme (issue #430). Each generic class or enum allows 16,384 instances, with definitions nested at most 64 deep, and a compilation allows 65,536 generic data instances; Option instances do not count. Each function with a Generator parameter allows 16,384 concrete frame specializations, which share the dependency depth limit of generic functions, and a compilation allows 65,536. Type nesting is still bounded separately.

A generic class or enum instance records its methods without lowering them (issue #515). Lowering a body queues each instance method it names through a call, function address, callable adapter, worker, or destructor, which is the set native reachability follows. A protocol bound queues the methods it matched, and __del__ is queued with the instance. Methods never queued are neither checked nor emitted, so they create no instances, strings, or cache entries. --check-all-methods, accepted only by check modes, queues every method still unqueued of a type declared outside std and dependencies once the queue empties, repeating while those bodies create instances (issue #517).

Before native preparation, emission removes unreachable definitions from the embedded standard library (issue #400). It keeps every user definition, C export and generator as a root, then follows direct/tail calls, function addresses, closure and boxed-callable adapters, thread/executor workers and sort key adapters through nested operations. A standard class’s __del__ is retained when a retained body arms the class after construction (issue #506). Release runs the hook only for an armed instance, so this covers values dropped through fields, containers, payloads, captures and queues; a descriptor of a class that no retained body arms has no hook. Standard-library provenance comes from the embedded-module loader, not a filename or symbol prefix. Generated foreign-call wrappers preserve that provenance too (issue #688): an unused standard-library engine import must not leave native references for the COFF linker to resolve. Windows executable builds use a narrower root set for their temporary COFF object (issue #690): entry operations and C exports, followed through the same function references and concrete generator types in live signatures and locals. Unreachable user helpers and generators are omitted together with the native imports only they need. COFF resolves undefined symbols even in sections its linker will discard, so section garbage collection alone cannot do this. A reached foreign call keeps its real import; no fallback definition is generated. All bodies still receive frontend and ownership checks. This applies in both fast-compile and optimized executable builds, so an unreachable user function has no executable symbol or breakpoint location on Windows. Explicit --emit-object, library objects and --dump preserve their existing user-definition and generator roots for inspection and external linking. External COFF consumers of such an object must provide its native imports, including those in unused helpers.

Retained definitions keep their declaration order. All frontend and ownership checks run before this step, including checks of bodies subsequently omitted. This saves native work and object space; it does not avoid parsing or lowering imported modules. See the edit-build measurements.

Each function’s trap records are merged into one read-only source-site table. On AArch64, functions after the table’s first record address their own base through a local symbol alias into that same table. Their GOT relocations have zero addends: an ARM GOT relocation selects a GOT slot, so a placement-time offset on that relocation would not add to the pointer loaded from the slot. Aliases are finalized before object emission, preserving the same source records for cold, cached, and cache-disabled compilation without runtime pointer fixups.

Native emission builds and compiles each retained function on any available core. The main thread places finished functions in the object in declaration order, preserving byte-for-byte output across core counts. There is no worker-count flag. Tests compare one, two, and eight workers. A function compiled before, by this or any other program, is taken from the compiled-function cache instead; the object is the same either way.

For ordinary user functions, each worker reuses the previous compilation’s cleared Cranelift Function buffers as well as its builder and compilation contexts. Function names, signatures, instructions, source locations and symbol references are rebuilt; only storage survives. Both cache hits and misses clear the function before reuse. Storage lasts until the worker finishes, and a large function can leave larger buffers for later small functions. Generator bodies and generated adapters still construct fresh IR. See the allocation measurements.

Workers claim one job at a time to balance functions of different sizes, but send completed results in batches of up to 32 jobs. This reduces result-channel wakeups on warm-cache builds. A partial batch is sent when a worker finishes or stops after an error; the main thread still places each job in declaration order. Batching can delay placement until the worker fills its batch and retains up to 31 additional completed jobs per worker before sending them.

With the cache enabled, a result keeps a shared owner of the cache’s immutable machine-code bytes until placement, avoiding a separate allocation and copy for each job. The object builder still copies those bytes into its own sections. Relocations remain specific to each function and are resolved independently. This adds no cache retention: decoded stencils already live until the compilation ends. With caching disabled, each result owns its machine-code buffer.

The plenty and plentyc executables allocate with mimalloc (version 2, without transparent huge pages) instead of glibc malloc (issue #315). Workers free operations the main thread allocated and grow IR buffers at a high rate; glibc serializes such frees and growth on per-arena locks, so threads slept on each other. Emitting 1,000 units from a warm cache on 16 hardware threads made about 86,000 voluntary context switches with glibc and 480 with mimalloc. The compiler library does not choose an allocator, so tests and other programs that call it in-process keep their own.

A command-line check (--check, --check-module, and --package-graph --check) leaks the checked program instead of freeing it, since the process exits next (issue #321). On the 1,000-unit benchmark this saves 2.0% of instructions and about 4% of cycles; peak memory is unchanged. Only the single-file compiler interface enables it, through CheckOptions::skip_teardown: plenty lsp, project commands, tests and library callers free every program. Other allocation strategies were measured on 2026-10-20 and not adopted:

  • Leaking the operation tables and object at the end of emission saved 0.02% of --emit-object instructions: workers already free each function’s operations once its code is built.
  • A never-freeing bump allocator, the upper bound for arenas, saved 10.5% of checking instructions and 2.7% of emission instructions, but took 30–45% more task time from page faults, and raised peak memory from 88 MB to 354 MB for checking and from 97 MB to 1.9 GB for emission.
  • The allocator itself takes 8% of checking cycles and 6% of emission cycles. The largest sources in checking are lexer tokens and type clones in the frontend, tracked by issue #302; most emission allocation is inside Cranelift, whose per-function register allocator context Plenty cannot reuse without patching it (issue #153).

A body can need symbols that are declared on first use: type descriptors, its trap-site table, foreign imports. Their order decides the object’s bytes, so a worker building IR does not declare them. It names each in a placeholder namespace, and the main thread declares them in declaration order as it places the function, then points the function’s relocations at them. If any function fails, emission reports the error of the first failing function in that order.

Match dispatch

Integer literal and lowered enum-tag dispatch uses a jump table when at least 256 distinct keys form one contiguous range (issue #395). Other sets retain source-order comparison chains. This conservative threshold follows the dense-match measurements; small tables regressed for some input distributions.

The lowerer sorts keys with their original arm blocks, checks contiguity and rejects duplicates or an early wildcard before selecting a table. It subtracts the minimum key with wrapping arithmetic at the scrutinee’s width, then checks the unsigned index against the last valid entry before reducing to the 32-bit index Cranelift’s br_table takes. This preserves negative ranges, ranges crossing zero, unsigned maxima, and full 256-value byte ranges without letting distant 64-bit values alias an entry. Storage is proportional to the number of cases, never an unbounded sparse key span.

The default edge remains the wildcard or the defensive invalid-tag trap. Only dispatch changes: arm bodies, payload loads, ownership cleanup, source-order payload subpatterns, joins and tail exits use their existing lowering paths. No source syntax, allocation behavior or target CPU requirement changes.

A str match whose arms before the fallback are at least 16 distinct literals searches one table instead of comparing each literal (issue #565). Codegen sorts the literals by byte length and then bytes, keeping the first source arm of any repeated literal, and emits a read-only data object of n + 1 32-bit offsets followed by the bytes. plenty_str_match binary-searches it and returns the entry index, or n for no match; br_table maps the index to the arm block, with n going to the wildcard. The scrutinee is released once before the branch. Other str matches keep their plenty_str_eq chain; see the string match measurements.

A match on a standard sum whose arms test only tags 0 and 1, with any other arm a wildcard, reads a one-bit tag field. It tests that bit and branches, with no compare against each tag and no invalid-tag trap (issue #757).

When the scrutinee of a sum match is a local held in a register, each tag arm records the local’s tag for its body:

  • An arm that rebuilds the matched variant around its own payload, in a sum of the same width, reuses the matched value instead of splitting and re-tagging it. case Ok(value): Ok(value) in a generic helper that changes only the error type is the common case.
  • In the arm of a managed sum’s field-free variant 0, such as Nothing, the local holds zero, so lowering writes the shared empty slot and the local’s later release emits nothing.

Splitting a sum built from two words, or widened from one, reuses those words instead of emitting isplit, which Cranelift’s optimizer does not look through, so a tag shifted out and back in folds away.

Symbol references

Code is compiled position-independent (Cranelift’s is_pic), so the object links into a PIE or a shared library. How code refers to a symbol depends on where the symbol is defined (issue #372):

TargetReferenceRelocation
Program function, runtime helpercall rel32R_X86_64_PLT32
Data the object defines locally (literals, trap-site tables, type metadata, coverage counters)lea sym(%rip)R_X86_64_PC32
Trusted C declaration, memmove, memsetload from the GOT, indirect callR_X86_64_GOTPCREL
Address of a program functionload from the GOTR_X86_64_GOTPCREL

Runtime helpers are marked colocated: the runtime archive is linked into every executable and shared library made from the object, and a shared library’s version script makes the runtime’s symbols local, so the linker resolves the call directly, without a PLT entry. A C library may be shared, so its functions stay behind the GOT.

Under is_pic, Cranelift loads every symbol address from the GOT, even a colocated one, and the object’s plain R_X86_64_GOTPCREL relocations are not relaxed by the linker. After compiling a function, the compiler therefore rewrites each GOT load of colocated data into a lea with a PC-relative relocation (src/codegen/relax.rs), the rewrite a linker applies to R_X86_64_REX_GOTPCRELX. Only data no other module can preempt is colocated. The rewrite depends only on the code and the function’s IR, so it applies equally to cached code, and the cache key, which covers the IR, still identifies the result. Function addresses, taken for function values, closures, generators, and runtime callbacks, are not rewritten.

On examples/http_server_bench.plenty with the standard library (779 functions, release), this removed 42,389 of 42,760 GOT loads and 35,220 of 35,590 indirect calls, and reduced machine code from 1,888,952 to 1,730,245 bytes (8.4%).

The worker count is the number of processors the process may run on. Restricting its CPU affinity, for example with taskset, restricts the workers.

A function’s operations are released once its IR is built, and the object is written to its file one section at a time. Emission therefore holds neither every function’s operations nor a second copy of the object (issue #54).

Cranelift’s verifier checks compiler-generated IR, not source programs. It runs in debug compiler builds, including tests; release builds omit it.

The compiler uses Cranelift’s backtracking register allocator. Its time grows with the square of a function’s branch arguments, the values its branches pass to block parameters: two for each variable assigned in a conditional arm, for example. A function with more than 2,048 branch arguments is compiled with Cranelift’s single-pass allocator instead, whose time is linear (issue #56). The largest function in the test suite has 156. Generated code reaches the limit: about a thousand conditional assignments, two thousand match arms, or a thousand-element collection literal in one function. In measurement such a function’s machine code was 60% to 135% larger and took 1.1 to 3.6 times the CPU cycles to run. Applied to whole programs, the single-pass allocator reduced emission CPU time but increased code size and slowed native workloads, so no other function uses it. The choice depends on the function alone, so it is the same for every core count. tests/test_large_functions.rs runs functions over the limit.

Identical code folding

A local function whose placed code equals an earlier function’s shares that function’s code instead of getting its own copy. Code is equal when its bytes, relocations, requested symbols, trap-site records and checks are equal; calls compare by callee after folding, so a caller of folded functions folds too when its callees were placed first. Quantity instances that differ only in dimension are the common case, since dimensions are erased: sum[Length] and sum[Time] in std.units compile to one body. An instance that formats a value or builds a collection keeps its own body, because its type descriptor names the dimension.

A folded function is defined empty, and once the object module is finished its symbol is moved to the shared code, so every call and table entry reaches it. On Mach-O the moved symbol is marked N_ALT_ENTRY, keeping the code one linker atom. Exported functions and coverage builds keep every body. So does every function used as a value: function values compare code addresses, so two distinct functions never compare equal. Other functions may still fold into one. In --dump asm, a folded function lists ; shares the code of SYMBOL instead of instructions.

Call-site substitution

Before any body is lowered, src/codegen/small_calls.rs replaces two kinds of call with the callee’s operations (issue #363):

  • Constants. A read of a module const becomes a literal of the constant’s declared type, so width, sign and float bits are those of the declaration. An integer domain constant such as Positive[i64] becomes its literal and the domain’s constructor call, as a literal argument lowers. This happens in every build, and the constant has no function in the object.
  • Leaf predicates, in optimized builds only. A function whose parameters and result are numbers or bool, with no locals, and whose body is at most 12 operations drawn from parameter reads, literals, constants, comparisons, not, and, or, and matches on bool or integer literals. Nothing in such a body can trap, call, allocate, or touch memory. The caller stores the arguments in new local slots and runs the body; calls to one predicate in a body share its slots. The definition is still emitted, since a function value may name it.

Arithmetic, which can overflow, any call, and recursion keep the call. Generator bodies keep predicate calls, since their frame layout is fixed by then, as does plenty_main.

The substituted operations report no source position, so a later trap or call in the caller reports the same position and call chain as before. They become part of the caller’s IR, so the compiled-function cache key changes when a constant’s value or a predicate’s body does.

Call-site inlining

In optimized builds, src/codegen/inline_calls.rs lowers some calls by lowering the callee’s typed body in the caller (issues #548 and #722). The body keeps its own locals, source position, trap sites and cleanup, so traps and ? failures behave as the call would; only its final return becomes a jump. A callee is inlined when it is not recursive, not a generator, has no tail call and defines no function, and its operations, counting the callees inlined into it, stay within a bound:

CalleeBound
A class’s new or __init__64 operations
Any function with one direct call site160 operations
Any other function16 operations, or 24 while its call sites times its operations stay within 256

The larger shared bound (issue #770) covers bounds-checked accessors such as Bytes.get and Bytes.set and short arithmetic helpers called in hot loops, whose call would otherwise also keep the loop around it from being versioned. The product bound keeps a helper called from dozens of places, as in the regular-expression parser, a call, so code size and compile time grow little.

A tail call is inlined only when the caller has no managed locals or has just one tail-call site; otherwise it retains the shared tail-exit cleanup, keeping code growth linear in the number of owned locals and exits. C string adapter failures also return through the inlined callee’s exit, so the caller can handle the error.

An inlined function’s own body is built only if placed code or data still refers to it, for example a call from a generator, a function value, a protocol table or a thread adapter; exported functions are always built. Otherwise the copy at its call site is its only code, so inlining a function called once costs no extra compile time. The symbol of a body not built stays in the object as a single trap instruction, so code outside Plenty that calls it, such as C linked against the object, stops with SIGILL rather than running into the next function. --dump builds keep every body.

Last-use moves

After substitution, src/codegen/last_use.rs turns the last read of a local into a move (issue #388). A read of a local counts a new owner, and the local’s own owner is released later, when it is reassigned or the function returns. When no later operation can read the local, the read takes its value and leaves an empty slot. A match on an owned parameter and a payload binding read once are the common cases.

An empty slot holds zero, and the release of a value known to be zero emits nothing (issue #391). Every write of an empty slot in a function uses the same one-word and two-word zero constants from its entry block, so where all paths into a join leave a slot empty, the slot’s value there is that constant and its later release emits nothing. A slot moved from on only some paths keeps its release, which tests for zero before calling the runtime. In user code the test also skips the call-chain bookkeeping that names the release site, so an empty slot costs a compare and branch.

A local whose address is taken lives in a stack slot, and lowering reads it back with a load, so it cannot see the zero there. Before compiling a function, src/codegen/empty_slots.rs finds stack slots whose address is only loaded from or stored to, and tracks which of their bytes hold zero on every path; a call does not change them. A zero test or one-word release that reads such bytes is removed (issue #549). A constructed value whose fields have all been moved out is the common case.

A backward liveness pass over each function finds these reads. Loops are solved to a fixed point; break, continue, early returns and the error path of ? are followed. Only managed copyable types qualify: they have no destructor, so ending the last owner earlier changes no output. Generator bodies are skipped. A borrowed local moves only at a read after which none of its loans is live, by the loan liveness of the ownership checker in src/ownership.rs (issue #755). Reading a key after its get_mut lookup has ended moves it into insert; a read followed by a use of a reference to the local still copies. A local with any borrow that records no loan is never moved. A one-word read that feeds a collection query keeps the borrowed read from issue #379, which costs less than a move.

When a match reads the tag of a local it has just copied, the copy’s counting is removed, so testing an owned scrutinee costs no runtime call. The pass runs after the ownership checker, so diagnostics are unchanged.

Constants in loops

In optimized builds Cranelift rematerializes an integer or float constant in each block that uses it rather than keeping it in a register across a loop: this saves a register for the loop’s lifetime at the cost of one instruction per use. On x86-64 a constant that is the right operand of a float operation is read from the constant pool. A left float operand, or a 64-bit integer constant in a wrapping_add, is built with movabs (plus movq for a float) each time it is used (issue #370).

Lowering therefore moves a non-NaN constant to the right of float + and *, which gives the same bits: only the choice between two NaN operands depends on order. 2.0 * x and x * 2.0 become x + x, which rounds, overflows, and propagates signed zero, infinities and NaN identically. No other float operation is rewritten; there is no reassociation, fusion, or fast-math mode.

A non-zero constant on the left of - or /, which cannot move, is emitted as lane 0 of a vconst vector constant (issue #380). Cranelift does not rematerialize a vconst, and x86-64 loads it from the constant pool with one movupd, so 1.5 - x reads its constant without a general-purpose register. The operation and its operand bits are unchanged. Zero keeps its f64const, which lowers to a register-clearing xorpd, and a constant whose other operand is also constant is left for Cranelift to fold. In a throughput-bound loop with four c - x terms (taskset, perf stat, 10⁹ iterations, Ryzen 7 7840HS) the change lowers cycles from 8.8 to 5.4 billion, matching the same loop with the constants passed as arguments; in a loop bounded by a dependency chain the movabs was off the chain and the cycles do not change.

x.fabs() and x.copysign(y) are bit masks rather than runtime calls: band with the magnitude mask, and bor of that with band(y, sign mask). Each mask is lane 0 of a vconst, loaded from the constant pool with one movupd; Cranelift’s own fabs and fcopysign build their masks with movabs + movq. Since Cranelift 0.137 the x86-64 backend keeps both operands of a scalar float band, bor or bxor in registers, because a memory operand reads 128 bits, so an f64const mask would also be built with movabs (issue #439). Like Rust’s abs and copysign, they keep a NaN payload. A constant receiver or sign argument is folded into the masks while lowering.

Float negation -x is bxor with the pooled sign mask rather than Cranelift’s fneg, whose x86-64 lowering builds the mask with movabs + movq at each use (issue #385). Both flip only the sign bit, NaN payload included. Lowering folds a constant operand and -(-x); Cranelift’s other fneg rewrites, such as (-x) * (-y) to x * y, no longer apply. In a throughput-bound loop with four s + -(x * a) terms the change lowers cycles from 7.7 to 4.4 billion and instructions per iteration from 30 to 22.

Integer constants are left as Cranelift places them. In the examples/cpu_codegen_audit mix kernel the per-iteration movabs is off the loop’s dependency chain: passing the constant as an argument instead removes the instruction but not a cycle (7.05 billion cycles either way for 10⁹ iterations). Removing the rematerialization would need a Cranelift change.

Float rounding

x.floor(), x.ceil(), x.trunc() and x.round() are inline SSE2 sequences, equal bit for bit to Rust’s floor, ceil, trunc and round_ties_even (issue #386). Generated code targets the SSE2 baseline, where Cranelift’s floor, ceil, trunc and nearest are library calls rather than SSE4.1 roundsd.

For a magnitude a below 2⁵² (2²³ for f32), (a + 2⁵²) - 2⁵² is a rounded to a whole number, ties to even, because numbers in [2⁵², 2⁵³) are spaced 1 apart. round uses that directly. trunc steps back one when the rounded magnitude exceeds a; floor and ceil apply the sign of x and step one down or up when the rounded value lies past x. The step is a vector compare (cmppd) whose lane mask selects the constant 1, so no branch depends on the fraction. ORing in the sign bit of x gives -0.0 for a negative x whose result is zero. A larger magnitude or an infinity is already whole and is returned unchanged, behind the one branch; a NaN passes through the arithmetic, which quiets it and keeps its sign and payload. A constant receiver is folded while lowering.

The calls went through plenty_f64_math to Rust’s libm, which rounds with integer operations and data-dependent branches, and forced live float registers onto the stack. With --release, taskset on one core, perf stat, Ryzen 7 7840HS, best of five:

Loop (2×10⁸ iterations)call (G cycles)inline (G cycles)
Two floor terms, steady step4.0–4.91.8–2.0
One floor of a random value5.22.0
One round of a random value6.81.8
Dependency chain y - y.floor()7.35.0
Dependency chain y - y.round()10.43.4

ceil and trunc match floor. A chain whose rounding the call’s branches predict exactly, such as floor of values always in [0, 1), can be faster as a call, because the prediction takes the call off the chain. Each use adds about fifteen Cranelift instructions to compile: a file of 2,000 one-line functions with four rounding calls each takes 55% (release) to 85% (debug) more compiler instructions.

Loop layout

A loop header whose condition ends in a compare branches to the exit on the complemented compare, so the exit is the branch’s first successor (issue #767, lower_loop_copy in src/codegen/collections.rs). Cranelift orders blocks by a depth-first search that visits the first successor first, then emits them in reverse postorder. With the body as the first successor, every loop body lands after all code reachable from its exit, at the end of the enclosing loop. A value used after an inner loop and live across it then sits in a register through the out-of-line body, while the header keeps it on the stack: regalloc2 reloads it at the start of each pass. With the exit first, the body follows its header. In fannkuch-redux this removes reloads from three hot inner loops.

A condition that is not a compare, such as the join of a short-circuit and, keeps the body as the first successor: inverting it would cost a compare and a second branch on each pass.

Loops are also rotated into do-while form (issue #726). The header becomes a guard that tests the condition once and branches to the exit or the body. The body ends in a latch that tests the condition again and branches back to the body or to the exit, so a pass ends in one conditional branch instead of a jump to the header and the header’s branch. continue goes to the latch. Each copy of a versioned loop is rotated on its own; the guard keeps the exit-first branch above.

The condition is lowered twice, so rotation is limited to conditions whose second copy adds no observable work: reads of scalar, bool and view locals, arithmetic, compares, casts, not, short-circuit and/or, and a view’s length and items (rotatable). Both copies carry the condition’s source position, so a failing index or overflow reports the same error and location from either copy, and they evaluate in the same order. A condition with a call, a list’s length, a collection method or a managed value keeps the header test.

A loop’s own index facts (the lo < hi relation and in-range counters in src/codegen/induction.rs) describe values read in the body. Neither copy of the condition records them, since the guard’s values also reach the exit when the loop does not run (issue #816). An unrotated loop records none: its body reads the header’s values, which reach the exit too.

A body that never falls through and has no continue gets no latch.

Rotation removes one instruction per pass from tight loops: the reverse kernel’s swap loop matches C’s eight instructions. In loops that make calls, regalloc2 can place a split’s moves on the latch’s back edge instead; fasta gains about 0.7% instructions this way.

Values carried to shared exits

A function’s ? failures, returns and tail calls share their cleanup and release code: each exit jumps into a chain of shared blocks, passing the values it carries (src/codegen/exits.rs). regalloc2 merges a block parameter with the arguments passed to it when their live ranges do not overlap. Parameters that every exit passes to join each exit’s values, and the values those share registers with, into one bundle spanning the function. regalloc2 splits such a bundle a few times, then cuts it into minimal pieces whose spill choice ignores loop depth, so a loop’s counter or total ended up stored and reloaded on every pass.

In a function with a loop, counting loops of callees inlined into it, an exit therefore stores its carried values in a stack slot and jumps to a reload block, which loads them and enters the shared blocks (issues #771 and #811). The loads address the slot from a different base than the stores, so Cranelift’s alias analysis cannot replace a lone exit’s loads with the stored values and reform the merge. A loop-free function passes parameters: it has no hot loop to protect, and a hot return path, such as a constructor called in a loop, would pay a store and a load. break and continue pass parameters too. Staging returns removes 4.4% of n-body’s instructions and 1.6% of fannkuch-redux’s.

Loop counter checks

Integer arithmetic keeps its overflow and division checks, and Cranelift does not remove them, since it does not track value ranges. The lowerer drops two kinds that a while loop’s own condition makes impossible (issue #369, src/codegen/induction.rs). Both apply to a counter kept in a CLIF variable whose condition compares it with another local or a literal, as in while i < n or while i >= 0:

  • Step overflow. The first write to the counter in the body, if it is a top-level i = i + k or i = i - k with a literal k, runs at most once per pass, with the condition still true. i < n puts i at least one below the type’s maximum, so i + 1 cannot overflow; a literal bound also admits larger steps that stay in range, and i > n admits i - 1. Such a step is a plain add. Later writes, steps inside nested loops, and steps such as i <= n with a variable n keep their check. When the bound is also a local, its own first step is proven the same way, so in while lo < hi both lo += 1 and hi -= 1 are plain (issue #720).
  • Signed division. A signed counter that enters the loop holding a nonnegative literal, and whose every write in the loop adds a nonnegative literal, is never negative, since each add is checked or proven. // and % of such a counter by a nonnegative literal or counter use unsigned division, with no floor correction or MIN // -1 check. The zero-divisor check stays.

A for loop over a range(start, stop) call with step 1 and an i64 or u64 element gets both facts from the range instead of a condition: every value of the loop variable i is below stop, so a top-level i + 1 in the body is a plain add, and a nonnegative start keeps i nonnegative. The lowerer accepts the loop only when its hidden counters start from that range’s computed length, start, and step, and nothing in the body writes them except the loop’s own step before continue. Reversed ranges, other steps, narrower elements, and range values in locals keep every check.

Counters in memory, borrowed locals and generator frames, are not analyzed. for loops over ranges step with wrapping arithmetic. A range over an integer type narrower than i64, or one with a positive literal step and a start known nonnegative, cannot have more than i64.MAX elements, so its length needs no overflow check (issue #721).

Short circuits in conditions

a or b lowers to a match on a whose arms push True or evaluate b; the arms meet in a join block that takes the bool as a parameter. Cranelift does not thread branches through block parameters, so an if, while, match, or enclosing and/or that consumes the result would test the join parameter again, even on edges where an arm passed a constant.

Before Cranelift runs, src/codegen/threading.rs retargets each edge into a block that only passes its parameters on or branches on them, when no other block uses its values and it has no instruction that traps, touches memory, or calls. An edge whose arguments decide the block’s brif goes straight to the chosen target; a jump whose argument is the unknown condition, or its not, becomes that brif, with the targets swapped for not (issue #377). Only branches move, so operands still run in order, only when needed, with their traps and ? propagation. A short-circuit result stored in a variable and read after the condition keeps its join block.

Measuring compilation

A local run of compiler-benchmark used 40,000 functions, three requested runs, and disabled caches. The machine and run date were not recorded, so these numbers are not independently reproducible:

CompilerBuild + run (µs/function)
Clang 22.1.8172
Plenty 0.2.4233
Zig 0.16.0269
GCC 16.2.1367
Go 1.26.10440

Plenty’s build took approximately 9.3 seconds, with 176 MB peak RSS. Generated-code optimization was off; only the non-generic Plenty workload was implemented. All four adapter tests passed, including native-result verification.

examples/compile_bench.rs generates units containing a protocol, class, enum, and three functions. It reports median checking time, optionally full AOT/linking, and a fourfold-input scaling ratio. Functions and methods independently declare precise arithmetic errors, use explicit successful Ok returns and propagate fallible callees without wrapping each operator. The current template generates 40,003/160,003 lines for 1000/4000 units; see its functional validation and controlled comparison. Historical measurements retain their original source shapes. tests/test_compile_scaling.rs guards against superlinear checking of classes, functions, protocols, enum variants, and match cases.

For time and allocations per compiler phase in one run, build with the profile feature; see Profiling the compiler.

For the 2026-10-10 release measurement on an AMD Ryzen 7 7840HS, x86_64 Linux, 16 hardware threads: 4,000 units / 156,003 lines checked in 1.61 s and 16,000 units / 624,003 lines in 6.69 s. Parallel emission and omitting release-build verification reduced full AOT of 1,000 units / 39,003 lines from 3.25 s to 0.87 s, and object emission of 5,000 units / 195,003 lines from 16.9 s to 4.35 s (2.6 s checking). With one core the latter took 13.2 s, isolating verifier removal. These are workload measurements, not general latency guarantees.

The 2026-10-12 release measurement used the same machine restricted to its eight physical cores, and 5,000 units / 195,003 lines. Checking function bodies on all cores reduced the independent check from 0.48 s to 0.09 s and --check from 1.81 s to 1.44 s (five runs each). Object emission then spends, in seconds:

PhaseThreadsTime
Parsing and name resolution10.31
Type resolution, class expansion, generic preparation, signatures10.35
Lowering bodies to operations10.41
Layout, operation validation, rest of the frontend10.13
Independent check80.09
Tail-call preparation; declaring functions, strings, adapters10.19
Building IR and compiling it82.1
Serializing and writing the object10.22

On one core, building IR takes 0.95 s and compiling it 13.1 s. On eight, the compile step holds every core: the main thread spends 0.05 s defining compiled functions and otherwise builds IR or compiles.

With all 16 hardware threads the same emission took 9.9 s, against 4.2 s on the eight cores. The instruction count was the same; above about ten busy threads this machine ran at 1.1 GHz instead of 3.9 GHz. Measure on a fixed set of cores, for example taskset -c 0,2,4,6,8,10,12,14. The 2026-10-20 measurement below did not reproduce the slowdown.

The 2026-10-15 release measurement used the same eight cores while other builds loaded the machine, with paired runs of the compiler before and after building IR on worker threads. Object emission of 5,000 units took 6.0 to 6.5 s before and 5.7 to 5.9 s after (three runs each); 20,000 units took 25.8 to 26.7 s and 23.2 to 24.4 s (two each). CPU time and peak memory were unchanged. The per-function trap-site tables added 0.4 MB of read-only data to the 5,000-unit object (66.6 MB) and 0.2% to a linked 1,000-unit program’s code and data. The remaining serial work is the frontend, about 1.9 s of the 5,000-unit emission.

The 2026-10-16 release measurement used the same eight cores, again beside other builds, with paired runs before and after resolving names, validating frames, preparing tail calls, and finding thread and executor adapters on all cores. For 5,000 units these phases took 48, 19, 35, and 20 ms instead of 99, 85, 47, and 35 ms. --check took 1.77 to 2.02 s before and 1.62 to 1.84 s after, and object emission 6.1 to 7.9 s before and 6.0 to 6.3 s after (five runs each). Peak memory was unchanged. On 1,000 units the difference was within noise.

The 2026-10-20 release measurement compared glibc malloc with mimalloc in paired, alternating runs of --emit-object (no linking), with a load average of 4 to 7 from other builds. Medians of three runs for 5,000 units, wall time with user and system CPU time in seconds:

CacheCoresglibcmimalloc
Off8 physical2.26 (7.7 + 0.41)1.98 (7.2 + 0.37)
Off16 hardware threads2.07 (10.7 + 0.61)1.74 (9.9 + 0.45)
Warm8 physical2.11 (3.7 + 1.55)1.42 (2.4 + 0.49)
Warm16 hardware threads2.13 (4.8 + 3.16)1.39 (2.9 + 0.57)

With mimalloc, voluntary context switches fell from 556,000 to 1,200 for the warm 16-thread build, and peak memory was 375 MB against 388 to 414 MB. For 1,000 units the warm build took 0.26 s instead of 0.37 s on 16 threads, and peak memory rose from 92 to 110 MB. Sixteen threads were as fast as eight or faster, so the worker count stays one per available processor. In this run the clock stayed near 4 GHz on 16 threads; sixteen threads executed the same instructions as eight in 31% more cycles, from sharing cores. Parsing, type resolution, class expansion, generic preparation, signatures, and lowering remain on one thread, about 0.75 s of the 5,000-unit frontend, as do declaring functions (0.1 s) and writing the object.

The 2026-10-17 release measurement used eight logical processors on four cores (taskset -c 0-7), beside other builds, before and after resolving each declaration once: class templates are copied without method bodies, concrete types are defined without copying their declarations, and signatures are registered from the types generic preparation resolved. Checking 5,000 units spent, in milliseconds (medians of nine runs each):

PhaseThreadsBeforeAfter
Parsing1279279
Name resolution1 and all5353
Type resolution110786
Public API check and class expansion16364
Generic preparation1140149
Signature registration18528
Lowering bodies to operations1465462
Frame validationall2425
Independent checkall128134
Freeing the program1121124

--check took 1.51 s before and 1.46 s after (eight runs each); 1,000 units took 0.30 s and 0.28 s (ten runs each), and a program whose imports load 21 standard modules 119 ms and 112 ms (twenty runs each). Peak memory fell from 406 MB to 404 MB at 5,000 units. Object emission of 5,000 units took 5.25 s and 5.21 s (medians of six runs each), within noise.

Running the declaration phases, from type resolution to signature registration, on several threads could save about 0.1 s of their 0.33 s at most. Type resolution creates nominal identities and generic instances in program order, and registration fills the signature table; only resolving signatures and expanding classes read the tables without changing them, and they would need thread-safe type tables. The parallel designs measured for parsing and lowering are in Frontend and checking. Freeing the program after the independent check costs 0.12 s, which a process about to exit could leave to the operating system.

Report build mode, machine, input size, repetitions, and included phases with new results. Measure checking, IR construction, native emission, runtime extraction, and linking separately when assessing compiler changes.

Value representations

Classes, tuples, and enum payloads keep fields inline in their owner’s storage. Native slots contain ordinary value bits plus any required inline payload bytes. Moves and copies relocate private addresses into the destination; addresses never extend the lifetime of departed frames. Immutable metadata describes field layout, ownership, relocation, equality, and formatting. Concrete layout facts are cached.

Runtime record traversal advances a running slot offset, including each field’s inline payload and any class cleanup state word. A complete pass takes one width lookup per field, without allocating an offset table. Hashing, equality, ordering, formatting, copying, relocation, and retain/release use this traversal. Copy failure cleanup walks the completed prefix backwards with a decreasing offset. This bounds offset work per pass; nested payload work and repeated visits while queued destruction resumes remain separate costs. Strings, the most common heap field, are freed during the pass and cause no resumption.

A class with exactly one field and no __del__ is represented as that field: same slot, same bytes, no record and no internal address. Constructing it, reading or moving its field, copying it, and releasing it are the field’s own operations; Wrap(file) costs nothing beyond file. Its runtime descriptor carries the field’s layout under the class’s name, so formatting still prints Wrap(x=1) and columnar lists still see one column. A one-field class with __del__ needs the cleanup state word and keeps the record layout, as do classes with two or more fields. The flat class layout proposal records why the record holds an address and the layout contract C interop depends on.

A field whose type is a class with a record (after looking through one-field wrappers) is stored flat when its owner is a class record, a tuple, or an enum variant with two or more fields: its record bytes sit at a fixed offset in the owner’s record, with no slot word of its own. Reading, borrowing, matching or writing owner.inner.x adds constant offsets to the owner’s record address; moving the owner copies its bytes with no address fixes for flat fields, so a tuple or variant of such classes moves as plain bytes. An all-zero flat record is an empty field, so moving a field out zeroes its bytes and releasing an empty field does nothing. The rule has these exceptions, which keep the slot word followed by the record:

  • top-level class values (locals, list elements, boxes, captures), because runtime operations and sum payloads receive one 128-bit value;
  • the payload of a single-field variant, such as Some(point) or Ok(point), which is that one 128-bit value with the sum’s tags stacked in its high word: the record follows it, and moving the sum fixes only that one address, as moving a bare class value does;
  • field-less classes without __del__, which have no record bytes to place.

A reference to a class value names the 16 bytes before its record whether or not a slot word is stored there: readers use address + 16 without loading the word, writers copy the record bytes, and writing an empty value zeroes them. A __del__ hook receives such a reference and must not read the word. The runtime release walk clears a finished flat record so a resumed walk skips it.

Compiled code avoids the runtime walk when the type makes it unnecessary. A payload with no internal addresses, such as a range, a flat record, or a closure environment without inline captures, is copied as plain bytes; a sum whose payloads are all such is copied by testing its tag. An all-zero value is an empty ownership slot, such as a moved-from local, and its release makes no runtime call. These paths keep high-frequency helpers, such as error mappers called per byte, out of runtime operation calls.

Record copies and clears of up to 192 bytes are emitted inline as 16-byte vector loads and stores rather than memmove and memset calls. A local moved straight into a call argument, with only operand pushes between the move and the call, is passed at its own address instead of a fresh copy: the callee copies its arguments on entry, so the caller’s storage is not read again. Inlined constructors, generator producers and portable SIMD operations keep argument addresses, so moves into them still copy.

A class with a record also has a plan in its runtime descriptor: the word offsets of the strings it owns, including those of flat class fields, whether those strings are all it owns, and whether its bytes hold any address into themselves. When the strings are all it owns (no __del__ at any depth, no other managed field), the runtime releases the record by freeing those strings directly instead of walking its fields; freeing a string runs no user code, so the order is unobservable. A record holding no internal addresses moves without relocation: only its owner’s value word changes. Collection rows use both, and a list whose row class owns nothing skips its per-row release entirely.

References use two native words: an address and packed inline-sum tag-offset information. They point into typed local, frame, field, or collection storage; source loans determine when those locations remain usable. Function values are code addresses without capture environments. Returned-reference analysis caches precise field-origin summaries when available and otherwise uses conservative owner projections.

Scalar references read and write only the scalar’s declared bytes, whether they address a packed list element, a local slot, or a projected enum payload. Slot padding is not part of the scalar value; runtime equality, hashing and formatting ignore it. Projected writes leave ancestor tags unchanged.

Generator resumption reserves space for the complete inline yield layout after the runtime result word. The shared runtime-call scratch area grows for larger records and nested payloads; yielding a record does not impose a fixed-size limit.

Lists

Dense lists pack primitive numeric and boolean elements into 1-, 2-, 4- or 8-byte rows with natural alignment. Other rows contain a 16-byte value slot and its inline payload, with no unused dictionary value slot. The runtime keeps a 16-byte-aligned buffer, rounding allocation sizes up to whole words; its exposed length counts initialized bytes. Borrowed scalar indexing uses the concrete element width to calculate row counts and addresses. General borrowed iteration obtains the runtime row stride. Native strided FFI views use that same stride, and signed reads extend their declared bytes without a post-call repair pass. List storage costs describe the remaining space used by sum and record elements.

Strings

A str is one 64-bit word. Heap and literal pointers address a 32-byte prefix: a 16-byte managed header, byte length, and scalar count, followed by UTF-8 bytes without a terminator. Literal storage is immortal and eight-byte aligned.

A set low bit marks an inline string. Its low byte stores length << 1 | 1; the remaining seven bytes store UTF-8. These values own no heap allocation and retain/release ignores them. Byte equality, hashing, and printing handle both representations identically. Source-visible text behavior belongs in text operations.

A &substr is two words, like a &[u8]: the address of its first UTF-8 byte and its byte count. It has no prefix, so it is not a &str. A view of an inline string addresses the bytes in the borrowed string’s slot, which its loan keeps in place.

Standard sums

Option and Result use a 128-bit value: a 64-bit terminal payload and a 64-bit path of binary tags, outermost first. Nested sums add bits without boxing, bounded by the 64-level type limit. The payload is scalar bits, an owned heap handle, or a private address of inline data. No numeric sentinel or null pointer is reserved as a source value. LoadError uses three terminal tag bits for seven variants.

Scalar/pointer sums use 16-byte addressable slots. Inline payload bytes follow the value word; slots grow only when the payload requires it. Native results use caller-owned storage, and relocation preserves sum tags. Runtime aggregate helpers receive aligned slot pointers instead of relying on a C u128 calling convention. A one-word operand is stored as its word and a zero word, so no two-word zero stays live across the call (issue #757). These layouts are private and are not the public FFI ABI.

Ranges

Ranges contain 32 bytes of bounds, step, and length. Stored ranges use 48-byte slots: ordinary value storage plus inline range data. Collection relocation and frame/result moves repair embedded addresses. Range creation and return need no separate owner allocation; see ranges.

Memory and destruction

Each live managed operand, local, field, and capture has an owner. Moving an affine value clears its source slot. Loading a copyable inline value copies its storage and retains contained strings. Stores stage the new value before releasing the old owner. Source cleanup order is defined by destruction, not reference counts.

Runtime objects share {atomic u64 refs, destroy_callback}. Heap objects begin with one reference; literals use an immortal count. A Shared[T] handle is a box whose count is shared by its source-level owners. Native helpers borrow inputs and return owned managed results, including retained projections and builder aliases. Buffers have explicit owners; type metadata is immutable program data.

A class instance owns its fields in a record inside its owner’s slot; the slot’s value word is the record’s address, which moves relocate. A class field, tuple component or multi-field variant field of class type is stored flat inside its owner’s record, so moving the owner copies bytes without relocating that field; an all-zero flat record marks a moved-out or released field. A one-field class without __del__ has no record: the instance is its field, so moving it or moving the field out copies the field’s bits and releases nothing. See value representations.

Retain uses relaxed atomic updates. Final release acquires preceding releases before destruction; the final thread owns the destruction queue entry. This protects ownership counts, not concurrent mutable payload access or foreign resource transfer. Source permissions still come from borrow and worker checks.

Sole-owner release

Release first loads the count. An immortal count returns. A count of 1 skips the locked decrement: an acquire fence follows the load and the object is destroyed, leaving the dead count at 1 until the destruction queue reuses the word. Other counts take the release decrement.

This is sound because the releaser consumes a counted reference it owns, so the count is at least 1 and other counted references or borrows account for the rest:

  • Every count operation by another thread comes from a counted reference or a borrow of one. A borrow of the releaser’s own reference ends before the release, through a scoped-worker join or a job or channel handoff, which orders that thread’s retains before the load. By coherence the load then reads their increments or a later value, so it cannot read a stale 1.
  • When the load reads 1, no other counted reference exists, and no new one can appear: creating one needs the releaser’s reference, which it owns exclusively. A retain racing with this release would already race with destruction under an atomic decrement, so borrow and worker checks exclude it.
  • The 1 was written by the initial store or by another owner’s release decrement. The acquire fence synchronizes with that decrement, so the other owner’s payload use happens before destruction, as under the atomic path.
  • Shared[T] handles each own a count, so two live handles keep it at 2 or more. Borrowed contents are released only when the last handle destroys the box or under a mutex.
  • The destruction queue writes its link only after the release has decided to destroy, as before. Immortal literals keep u64::MAX and are never written, so read-only literal data stays valid. Copies into another actor heap check the count only for the immortal value.

Retain has no such path. It also runs through shared borrows, which carry no count: a scoped worker and its parent copying one string, or two jobs reading the same Shared[list[str]], can both read 1, and a plain store of 2 would lose an increment. Retain therefore always uses an atomic update. Compiled code changes counts only by calling plenty_retain and plenty_release.

An iterative queue destroys owned graphs without recursive native-stack growth. A record, list, or box being destroyed hands each heap child to the queue and resumes its walk after that child is gone, which keeps cleanup in declaration order. A string owns nothing and runs no user code, so the walk frees it in place instead. A class whose fields own only strings, at any depth and without __del__, is released in one step from the string offsets its descriptor records; see value representations. Inline generator cleanup uses the count handoff but finishes synchronously while its enclosing storage is live. Private helper aliases do not authorize mutable source aliases. Traps end the process without unwinding language scopes.

Box destruction

When a box’s content releases without running code, the free order of its children cannot be observed. Its descriptor then carries a compiled release of the content slot, and the runtime calls that function instead of walking the content’s descriptor. The compiler builds one for each boxed type that is:

  • a string, collection, box, or another value released with one plenty_release;
  • a sum such as Option[Box[T]] in which one variant owns a single such value; or
  • a class without __del__ whose fields are of those kinds or own nothing.

The function reads each owned child at its fixed offset and passes it to plenty_release. During destruction that call only queues the child, so deep trees still drop without growing the native stack. Content whose release runs a destructor keeps the ordered walk described above.

Box(value) for content with inline storage, such as a class record, asks the runtime only for the allocation. Compiled code then copies the content into the box’s slot. If the allocation fails, the runtime releases the value the box would have owned. The slot is not zeroed first, just as a local’s stack slot is not: the copy writes the value word and the payload that word names, such as the current variant’s fields of a sum, and every reader of the box reads the value word first and then only that payload. Bytes past a smaller sum variant stay unwritten and are never read.

Allocator provenance

Runtime objects and container buffers both come from the global allocator, and an object allocation carries no prefix before its header: with only one allocator, freeing needs no record of which one created an object. A box is a 16-byte header, its descriptor word and a padding word, then its content slot. Public allocator selection and scoped allocator lifetimes are unsupported. Adding them (backlog B13) must record provenance again without enlarging objects of the global allocator; a 16-byte prefix made each binary-trees node a 112-byte malloc chunk rather than 96 (issue #814).

Checked buffer growth and partial-construction cleanup implement the recoverable allocation contract. The compiler’s runtime-checks feature enables the embedded runtime’s allocation-checks instrumentation for native integration tests. Allocation-free regions count successful allocations, including those freed inside the region; failed attempts do not count. Failure injection is separate. See runtime validation.

Collection storage and costs

collection.rs supplies static operation signatures. Typed loops and builders lower to structured operations; Cranelift calls a fixed runtime ABI using 128-bit slots and immutable element metadata. No per-element compilation or dynamic type inference occurs.

Lists use dense storage with constant-time indexing. Primitive scalar rows use their declared widths; other rows use a value slot plus their inline payload. The buffer is a byte prefix of an aligned Vec<u128> allocation. Inline records, tuples, and enum payloads occupy contiguous rows; buffer moves relocate private addresses. Mutation grows buffers in place. Explicit copy duplicates owned contents. Literals and comprehensions build in amortized linear time under ordinary hash distribution; operation contracts retain their documented allocation failures.

Hash storage

Dictionary/set hash buckets point to stable row slots. A separate fallibly allocated buffer uses two machine words per reserved slot (16 bytes on the native target) for insertion-order links or the dead-row free list. Occupancy does not depend on payload bits. Removal unlinks a row and repairs its probe cluster by backward shifts, without moving payloads or scanning the whole table.

Each bucket is one 64-bit word. For a table of 2^s buckets, its low s bits hold the row slot and the bits above hold the low 64 - s bits of the key’s hash, so storing hashes costs no memory. A probe compares keys only where those bits match its hash. Growth rebuilds the table from stored bits without reading or hashing keys, and removal finds a shifted bucket’s home from them; both fall back to rehashing keys only above 2^32 buckets (issue #761). Load stays at most one half, so the slot field always fits.

Stored bits also serve another table at least as long, which needs no more of them (issue #817). A copy whose rows and table length match its source copies the bucket array. Set algebra probes each operand with the other’s stored bits and indexes its result from them, update reuses the consumed source’s bits, and intersection_update and difference_update repair the index in place. Where a target table is shorter than the source’s, each key is hashed once; a copied dictionary with removed rows rehashes its keys.

New keys reuse free slots at the logical tail; replacement preserves position. Live-row cursors make traversal, snapshots, copying, formatting, and destruction follow insertion order without visiting dead rows. Capacity follows peak occupancy and explicit reservation, and persists until destruction. Hashes use SipHash-1-3 with a process-wide secret key obtained from OS entropy. The key is shared across worker threads; dictionary insertion order remains independent of bucket placement. See the hashing contract for initialization, failure behavior, and complexity limits.

Top-level str, bytes, integer and bool keys hash their contents directly. Tuple keys recursively hash scalar fields, string bytes, and byte-literal contents using their static metadata. Inline storage addresses never participate in the hash, so copied keys, relocated rows, lookup temporaries, and newly constructed equal strings all retain the same hash. Tuple hashing and equality allocate nothing.

std.deque builds a ring buffer in Plenty over a private list of optional payloads. End removal replaces a slot with Nothing and transfers its previous owner. Growth doubles the slot count and initializes a replacement buffer before moving live entries; list removal behavior is unchanged.

std.collections keeps each binary heap in a private list and restores heap order with swap, so sifting never copies or drops a payload.

std.bytes does not use collection slots. Its private owner holds a contiguous one-byte-per-element buffer, reached only through range-checked runtime operations that never expose an address. See std.bytes.

Reservation obtains bucket, link, and row capacity before publishing indices or transferring owners. Failure preserves contents and order, though capacity may have grown. Relocation ignores dead rows. A popped inline payload stays readable until mutation or receiver release; codegen snapshots it before either.

Narrow native helpers

Integer, float and bool element reads and writes through a list place are inline machine code, with no runtime call (issue #509). This applies to any list place: a local, a field (self.xs[i], a.b.xs[i] = v), a borrowed receiver, or an inner row (grid[i][j] += 1), in any loop. Codegen loads the list’s first-row address, stored byte count and popped-row flag from the owner at the offsets in plenty-runtime/src/list_layout.rs, which the runtime asserts against its repr(C) Collection and Entries; a scalar row is 1, 2, 4 or 8 bytes according to its type. One unsigned comparison accepts an index in 0..len; any other index takes a cold path that adds the length and checks again, so a nonnegative index needs no normalization (issue #715). A constant index, or one known nonnegative, takes the single comparison only, and both paths join on the row address (issue #720). An access to the same rows, length and index values as a check earlier in the block, or in the chain of check and trap-guard blocks leading to it, reuses that check’s address, since it cannot fail if the first passed (issue #721). It then loads or stores the declared bytes. Reads preserve payload bits before narrowing or bitcasting, and writes store the same bits a write through an element reference would. An index out of bounds calls plenty_trap_list_index, which reports the index as written, the length and the site, as the generic path does. The failure block stores the index and length to the function’s scratch slot and passes its address: as register arguments, they would pin the loop’s values to the call’s argument registers and add moves to every iteration. The length of a borrowed scalar list is read the same way. The list’s loan keeps the rows in place for each access. Outside the loop caches below, codegen does not cache the address across source operations, so growth between accesses is observed.

A for or while loop caches the length and first-row address of a named scalar list, or of a scalar-list field of a named class value (self.items[i]), defined before it (issues #429, #715 and #717). The frontend emits the header reads before the loop and lowers xs[i] in the loop to ListScalarGetView or ListScalarSetView, which take the cached values as operands. A list of any other element type gets the same cache for the row references that field access and element references take (bodies[i].x, &mut bodies[i]): ElementRefView takes the cached values and computes the row address inline (issue #739). An immutable shared reference cannot change during the loop. For an owned list or owner, an exclusive reference, or a retargetable binding, the frontend checks the finished loop’s operations against a list that cannot resize or replace any list (arithmetic, control flow, loan bookkeeping, ?, scalar row access, row references, length queries, view construction, calls, and no write to the binding). A direct call is allowed when none of its arguments can resize the list: Plenty has no globals, so a callee reaches a list only through its arguments. Shared borrows and &mut passed to a slice view cannot resize it, nor can &mut to an unrelated owned local. The loop falls back for a &mut list or class argument, including a method receiver, that reaches the cached list or its owner or that is borrowed from a reference parameter; for a by-value argument that holds references; for indirect and closure calls; and for any other operation. The fallback rewrites those accesses to ListScalarGetRef, ListScalarSetRef and ElementRef, which read the header at each access. The cache is held in ordinary locals rather than in loads marked readonly: Cranelift treats a readonly load as valid for the whole function, which a later append would violate.

The length of a borrowed list of any element type is read from the header inline: the stored byte count divided by the constant row size, a shift when it is a power of two, minus the popped-row flag. The length of an owned temporary uses plenty_list_len, and indexing an owned temporary scalar list or iterating one by value uses plenty_list_scalar_get, bypassing the argument block of a runtime operation. These helpers take a live owner pointer and an i64 index. Named receivers are borrowed without temporary retain/release; temporary owners are released after obtaining the result.

Text and bytes queries that return a scalar (len, byte_len, isascii, indexing, ord) call one narrow helper each, such as plenty_text_len or plenty_bytes_item; a bytes literal index calls plenty_byte_str_item. plenty_text_ord returns the code point, or u64::MAX when the text is not one scalar, and codegen builds the Result inline. Results, bounds traps, and sites match the generic operations’. A query of a borrowed place reads through the reference. A query of an owned local or full-expression temporary reads the value in its slot: codegen removes the retain the load added and skips the matching release, since nothing runs between the load and the helper call.

startswith, endswith, find, rfind, and count call plenty_text_starts_with and its siblings with both texts in registers; plenty_text_find and plenty_text_rfind return u64::MAX for no match, and codegen builds the Option inline. These searches, in, ==, <, and the runtime operations whose operands are all texts or scalars (replace, upper, string concatenation, and so on) read in place each text operand that its source still holds: a copy loaded from a local or read through a reference, or a string literal. Codegen tracks these copies across the operand pushes that run no code and release nothing (loads, literals, references, last-use moves, and stack swaps); it removes each copy’s retain and skips its release. Any other op, including a conditional or call that computes an operand, ends the tracking.

Runtime operations that only read their operands read every such copy in place, whatever its type: dictionary get, get_mut, and indexing; in on a list, dictionary, set, range, or text; list find, rfind, and count; set subset, superset, and disjoint tests; and Bytes reads, writes, and scans that do not resize the buffer. In counts.get_mut(&key) the key is not counted, nor is a dictionary read through a shared reference, and a Bytes method’s read of self._owner counts nothing. These operations run no Plenty code (keys hash and compare structurally) and release or replace no stored value. Operations that can, such as pop, discard, and inserts, keep the copy’s count, because a stored value they release may be the copy’s source.

Inserts store their operands without adjusting counts (issue #762). Every operand codegen passes is owned, whether its read moved it or counted a copy, so insert, add, append, and a get_or_insert miss make that count the collection’s, and a failed insert releases it once. A get_or_insert hit keeps the stored entry: the runtime releases affine operands, and codegen tests a hit flag in the result and releases copyable ones inline. A mutation of a receiver through &mut (counts.insert(k, v) on a parameter or local, pop, discard, clear, set updates, reverse, sort, and so on) reads the collection’s owner in place under the exclusive loan, so the receiver is neither retained nor released. clear, reverse, sort, swap, and the set updates return unit rather than an alias of the receiver, so they make no count change either (issue #813). Literals, comprehensions, and collect() do the same: each element moves into the insert from a hidden local, and the collection under construction is loaded from its owner directly before the insert, so a counted element costs no retain or release (issue #826). intersection_update and difference_update also read their set argument in place when its source still holds it (issue #825). That source can be a &set parameter, a local, or a temporary parked in an expression local because its keys have __del__. The argument stays under its shared loan, and a removed member’s __del__ reaches only that member, since keys cannot hold collections. The receiver must be a local, a &mut parameter, or a field of one. An element receiver such as grid[0] still counts the argument (issue #831).

An element reference (&xs[i], &mut xs[i], &mut grid[i][j], and the implicit one under bodies[i].x) of any element type checks the index inline like scalar access and computes the row’s address, rows + index * row_bytes, where row_bytes is the element’s constant row size (issue #739). An index out of bounds calls plenty_trap_list_index with the same message as other list indexing. That address is the reference; reads and writes through it are direct loads and stores. Dictionary element references still use a runtime operation.

A borrowed list loop (for x in &xs, for x in &mut xs, reversed(&xs), or a loop over a list reference) iterates through the reference without copying the list’s owner. Before the first iteration it reads the length and the first row’s address from the header and uses the element’s constant row size as the distance between rows. Each element reference is then rows + index * row_bytes, the slot a generic element reference names, so a scalar read is one load with no call. The loop holds the list’s loan from its first condition to its last body use; no mutation can grow, shrink, or move the rows meanwhile, which is what makes the cached length and address valid. Element values keep reference semantics: writes through &mut land in the list, and an element can be passed on or returned under the list’s loan.

std.bytes element reads and writes and Bytes.len use plenty_bytes_item, plenty_bytes_set_item, and plenty_bytes_len. Bytes.push calls plenty_bytes_push with its storage owner read through the exclusive field reference, so appending a byte neither retains nor releases the owner and reaches the allocator only when the buffer grows. Text len, byte_len, isascii, scalar indexing, and the forward and reverse iteration cursors use plenty_text_* helpers. A text or byte owner reached through a place (a local, parameter, field, or row) is read through a shared reference under the caller’s loan, so the query neither retains nor releases it; other receivers are released after the call. A scalar text result is always inline. Indexing traps report the same site and message as the generic path.

Managed reads and some nested projections retain the generic ownership path. String length is cached, UTF-8 scalar indexing scans boundaries, and text iteration uses a byte cursor. Compiler-generated builder/iterator locals count toward the 256-slot limit. examples/collection_bench.py and examples/collection-bench.md provide reproducible workloads and measured costs.

Generator frames and closure environments

Each generator producer/specialization has a concrete frame layout, resume function, and immutable descriptor. A frame owns parameters and locals in typed slots, plus slot metadata, continuation state, resume callback, and reentrancy guard. Its internal resume ABI is (frame, out_slot) -> ready; yields transfer owned values, completion clears slots, and destruction releases remaining captures.

Ordinary slots use 16 bytes; inline payloads add their complete storage. Standard sums reserve the largest possible payload, and frames can contain other frames. Constructors and returning functions use caller-owned output storage. Moving nested frames repairs addresses; cleanup completes before surrounding storage expires.

Layout caches are per concrete producer. Recursive inline layouts are rejected, nesting is limited to 64 levels, native offsets fit signed 32-bit sizes, and consumer specialization is limited to 256 per compilation. Unresolved factory returns are rechecked with inferred context so all paths agree on concrete storage.

Yield requires an empty residual operand stack. Lowering adds resume-dispatch edges to continuation blocks; all locals are frame-backed, so SSA values need not survive invocations. Dispatch currently uses a linear comparison chain. There is no interpreter, native-stack suspension, or eager yield collection. Iteration wraps resume results in inline Option values.

Closures

Capture environments are concrete inline layouts with nominal producer identities. Consumers specialize for their actual layout; matching signatures do not erase identity or introduce boxing. Owned factories return into caller storage. Nested environment moves repair addresses, including environments stored in generator frames or collection rows.

Reusable invocation borrows an environment; consuming invocation transfers it. Borrowed capture loans remain source-level facts through environment moves and generic consumers. Captures are released in reverse capture order; an environment whose captures need no cleanup, such as a capture-less error mapper, is not retained or released at all. Cached facts and visited-type traversal avoid repeatedly expanding shared capture/type graphs. Nesting is limited to 64 levels and storage must fit the native stack-layout limit. The callable contract defines permitted captures, storage, and invocation; the generator contract defines suspension and lifetime restrictions.

Arithmetic boundaries and callbacks

Every named function and closure has an independent declared return contract. The frontend’s CheckedFunction operation gives arithmetic failures a shared exit through that public Result; it does not wrap successful source returns. An explicit checked: expression instead constructs its own local Result. unchecked: lowers its lexical suite’s operators with trap guards, which validate its no-failure precondition. Nested function definitions start their own recoverable boundary in either mode. See function arithmetic contracts.

There is no private callback error channel, CheckedUnwrap operation, or checked-block lifetime restriction. Ordinary calls, generic callback consumers, and substituted pipeline callbacks preserve the callback’s declared Result and its normal ownership rules. Consumers must handle fallible callbacks explicitly; a caller’s arithmetic mode never changes a callee’s ABI. Serial integer pipeline sum() also has a public overflow Result, rather than inheriting the consumer’s local mode.

Runtime key sorting uses one caller-owned first-error cell per invocation. A failed key evaluation saves its error, releases earlier keys, and returns an equal comparison. Later comparisons skip user callbacks. The sorter finishes its ownership-safe row moves, then its caller forwards the saved error. This preserves its bounded sorting complexity and allocation behavior without unwinding across Rust frames; a failure does not roll back row order.

Tail-call ABI

Native Plenty functions use Cranelift’s Tail convention. Validation and lowering share a checked representation plan. Approved transfers emit return_call or return_call_indirect; disagreement at lowering is an internal compiler error. Unsupported transfers become ordinary calls before lowering. Direct source-tail calls in a recursion cycle instead fail checking when no safe plan exists; see the function contract.

RepresentationNative transfer
Scalar/heap-owner arguments and exact-compatible resultSupported
Exact-compatible inline result, including ranges, records, sums, concrete framesSupported through incoming result storage
Owned inline argument, including range, record, closure, or frameUnsupported: its bytes must outlive frame removal
Reference argumentSupported only when ForwardsReferences proves every passed reference originates in a reference parameter
Indirect Callable callSame result, owned-input, and reference checks
Conversion, wrapping, propagation, or incompatible result areaOrdinary call; remaining work follows return

Concrete closure invocation uses its environment adapter and does not perform a native tail transfer. ABI capabilities do not expand allowed callable signatures.

Storage and cleanup

An inline result uses caller-supplied storage. The plan checks native scalar representations, payload capacity, alignment, and component offsets; exact source types ensure compatible nested relocation metadata. A transfer forwards the incoming hidden pointer unchanged. Final return relocates the result there before local cleanup. Source code cannot borrow this hidden result area.

Arguments are staged before caller-owned locals are released. Different argument counts, registers, and stack argument sizes are handled by the calling convention. Owned inline arguments currently pass addresses into caller storage. Cranelift’s StructArgument can copy bytes but cannot relocate Plenty’s nested addresses, so a raw copy cannot safely remove the frame. No hidden heap allocation extends it. Ordinary fallback keeps the frame alive and uses staged snapshots; unborrowed locals still receive the source-defined pre-call cleanup.

Compile cache

Every compile used to lower every function a program reaches, including all of an imported std module, from IR to machine code. Most of that code is the same from one program to the next, and Plenty’s objects are reproducible (native emission): the same input, compiler, and target give the same bytes. The compiler therefore keeps a local, content-addressed cache of compiled functions and reuses an entry whenever a function it is compiling has one.

Reusing functions still runs the frontend and builds every function’s IR: with every function cached, these were 45% of a warm build of a large program (issue #304). The cache therefore also keeps each program’s object, which a build whose sources have not changed uses without running the frontend or code generation. Single-file runs also keep linked executables, avoiding repeated linking.

What is cached

The unit is one compiled function: its machine code, alignment, and relocations, as Cranelift serializes them. Generic instantiations are functions of their own, so they are cached like any other.

Cranelift compiles a function’s stencil: the IR without the identities of the external symbols it names. The IR refers to a callee, a type descriptor, or a string literal by an index into a small table belonging to the function; the table is applied to the compiled code afterwards, and the placing thread resolves the function’s requested symbols in the object. A cached entry holds the code for the stencil, with relocations against those indices. On a hit the compiler still builds the function’s IR, so the table and the requested symbols are this program’s, and applies them to the cached code. Two programs whose functions have the same stencil but print different types, call different functions, or sit at different source positions therefore share one entry and each get their own symbols.

The key

An entry’s name is a SHA-256 of:

  • Cranelift’s key for the function: the stencil, the target triple, and every ISA flag, which includes the optimization level, the register allocator (chosen per function, see emission), the verifier setting, and the frame and probe settings;
  • the cache format version;
  • the compiler’s version and Cranelift’s version.

The compiled code for a stencil depends on nothing else: the compiler’s own work ends with the IR, which the key covers, and what it does with the result (rewriting GOT loads of local data, see symbol references, then placing bytes and relocations in the object) is the same whether they came from Cranelift or the cache. Entries hold Cranelift’s output before that rewrite. The compiler’s build identity, such as a Git hash, is deliberately not part of the key: it would empty the cache at every commit without excluding any stale entry. A change to Plenty’s code generation changes the IR it builds, and so the key.

Every damaged or stale entry is a miss, never an error: an entry carries a checksum of its payload, and Cranelift rejects entries written by another version of itself.

Plenty buffers the key’s Hash byte stream before feeding it to SHA-256. The byte stream and serialized entries remain compatible with the pinned Cranelift version (0.137.0); unit tests compare buffered keys to Cranelift’s keys and exercise entry exchange in both directions. Dependency upgrades must retain these checks: the upstream key implementation is not a stable serialization format.

Within one compilation, each distinct key has a lazily initialized memory entry. Concurrent requests for that key share its disk read, decoding, or compilation; unrelated keys can initialize in parallel. The memory entry keeps only machine bytes, alignment, and unresolved relocations. Relocations are resolved using each current function’s symbol table and function ID; source locations and other parameterized Cranelift metadata are not reused. Cranelift’s optional cache-check mode continues through its original path, including recompilation and comparison of full metadata.

This saves duplicate decoding at the cost of retaining all distinct machine code and relocation buffers until the compilation’s cache is dropped. There is no separate memory eviction limit. In-memory reuse counts as a hit, but adds no disk bytes read; the reported time saved is still an estimate.

Program objects

A build of an application (plenty FILE, --compile, --emit-object, plenty build, plenty run, and plenty test) looks up its program before loading any module. An entry holds the object the build wrote and the inputs it read. Library builds, --check, and legacy-syntax programs do not use program entries.

The entry’s name is a SHA-256 of:

  • the cache format version, the compiler’s and Cranelift’s versions, and the compiler’s identity: its executable’s path, size and modification time;
  • a digest of every standard module’s text, which the compiler embeds;
  • the target, the optimization level, and whether the build has plenty test hooks or fuzzing coverage, and the executable reachability policy (Windows temporary objects omit unreachable user helpers; inspectable objects retain them);
  • the request: the canonical entry file and the package graph (each package’s canonical root, identity, library and dependencies), and the main that plenty test appends; or, for a source compiled from memory, its text.

Unlike a function’s stencil, the frontend’s output depends on the compiler’s whole code, so the key names the compiler binary rather than its version. A rebuilt compiler has a new modification time, so every build after a compiler change, including a change to std, misses. The binary is not hashed: that would cost about as much as the warm build it saves.

The name does not cover the imported modules, which a build finds only by parsing. The entry records them instead: each source file read, with a SHA-256 of its text, the package whose root it resolves against, and where each of its imports resolved. A file read by include_str or include_bytes is recorded the same way, with no imports. A lookup uses the entry only if every recorded file still has that text and every recorded import still resolves to the same file or standard module. Resolving an import again repeats every existence check the loader made, so a new file that would make an import ambiguous, or change which file it names, is noticed though the program never read it. The package checks of a build, such as reserved names and overlapping roots, run again too. Anything else (a missing, damaged, or undecodable entry, an edited file, a changed resolution) is a miss: the build runs and its object replaces the entry. Each request therefore keeps one entry, its latest build’s.

The first build of a request stores only an empty entry, a marker that it was built; a later build of it stores its object. So the third build of an unchanged program is the first hit, and after an edit, the first rebuild stores and the next one hits. A program built only once, such as each temporary program of the test suite, costs a few bytes rather than its object. Without markers, one run of the test suite stored 3,870 objects (1.3 GB, beside 194 MB of compiled functions); with them it stores 438 objects and 2,925 markers (96 MB).

plenty script records its successful object on the first build, so its first unchanged rerun can skip both compilation and linking.

The frontend and code generation are a function of these inputs: no phase reads the environment, the clock, or any file other than the modules it loads, and the object is the same for any number of cores. A program hit therefore writes the bytes a build would. Only a successful build stores an entry, so a program hit never hides a diagnostic. Linking still runs on an object hit unless a single-file run also hits the executable cache.

Every edit to a module the program reads misses, including a change to a function body that no other module’s code depends on: the frontend checks a program’s modules together (see frontend), so there is no smaller unit to keep. See later phases.

--no-program-cache (CompileOptions::program_cache = false) keeps reusing compiled functions but neither uses nor records program entries. compile_bench uses it, so its builds measure the frontend and code generation. With --cache-stats, the line begins program hit or program miss when the build consulted the program cache; a hit reports no function hits or misses.

Single-file executables

plenty FILE and plentyc FILE, including Unix hashbang launches such as #!/usr/bin/env plenty, cache their linked executable by default. plenty script, including #!/usr/bin/env -S plenty script, uses the same executable reuse contract. No extra flag is needed. The first successful link stores an entry; an unchanged run can reuse it immediately. The object/function caches still run before this lookup, so this avoids linking, not the object cache’s initial warm-up.

The executable key covers the object bytes, compiler/runtime identity, working directory, complete environment, effective link arguments, and the selected C driver and linker’s identities. The linker writes a dependency file naming its native inputs, including startup objects, libraries and linker scripts. Entries record those inputs’ resolved paths, sizes, timestamps and file identities, plus library/tool search directories (including absent ones), GCC specs and collect2. Replacing a native input or adding a library to a watched search directory invalidates the entry. This uses filesystem metadata, not content hashing of system libraries; it is not a defense against deliberate filesystem tampering. Environment changes conservatively miss even when they would only affect the running program.

A hit verifies the checksummed entry, validates native inputs, and writes a private executable with owner-only execute permissions in the run’s temporary directory. It does not execute the shared cache file: concurrent replacement, eviction, or a cache filesystem mounted noexec cannot disrupt a running program. Publication uses the cache’s temporary-file/rename protocol. Failed links never publish an entry; missing, damaged or unwritable entries fall back to linking. The program always runs again; output and runtime data are not cached. Streams, working directory, environment and exit status are unchanged.

The bundled runtime archive is extracted only when linking runs, after this lookup misses or is bypassed. Its bytes are already covered by the compiler identity, so a hit needs no temporary runtime archive. The object and private executable are still restored, and toolchain/native-input checks still run. See the warm startup measurements for the attributed cost and reproducible comparison (issue #699).

Executable reuse currently supports Linux’s default GCC-based cc driver (a native ELF binary whose resolved name identifies GCC) with mold, lld or GNU ld when the selected linker supports --dependency-file. Before looking up an executable, Plenty asks GCC to resolve the selected linker and requires it to be a native ELF binary. It also rejects non-native private linker candidates and collect2 helpers in GCC’s search directories. A shell wrapper cannot serve cache hits merely by forwarding --help or --version: its external configuration and delegated tools are not depfile inputs (issue #676). Clang, wrapper drivers and wrapper linkers bypass executable reuse and link normally because their implicit inputs are not tracked. Explicit drivers/link arguments, non-Linux hosts, test-hook and fuzz builds, and redirected/injected toolchains (COMPILER_PATH, GCC_EXEC_PREFIX, LIBRARY_PATH, LD_PRELOAD, LD_AUDIT) retain ordinary linking. Explicit --compile outputs and project plenty run do not use this layer. --no-cache, PLENTY_CACHE=off and --no-program-cache disable it. --cache-stats adds an executable cache: hit or miss line when consulted; --verbose prints a link command only when linking actually runs.

Location and controls

SettingEffect
PLENTY_CACHE_DIR=DIRUse DIR. The default is $XDG_CACHE_HOME/plenty, else ~/.cache/plenty
PLENTY_CACHE=off, --no-cacheCompile without the cache; the flag applies to one compiler command
--no-program-cacheReuse compiled functions, but not program objects or executables
PLENTY_CACHE_LIMIT=SIZEKeep at most SIZE bytes (512M, 2G, or a byte count); the default is 1 GiB
--cache-statsPrint whether the program hit, function hits and misses, bytes read and written, and an estimate of the time saved
plenty cache statsPrint the directory, its entry count (functions, programs and executables) and size, and the limit
plenty cache cleanRemove every entry

The same directory also holds the automatic linker probe’s results under DIR/linker-probe/ (Native linking); they are not cache entries, and plenty cache clean leaves them.

Function entries live under DIR/fn/, program entries under DIR/program/, and executable entries under DIR/executable/, sharded by the first byte of the hash. Each is written to a temporary name in its shard and renamed, so compilers running at once in several processes and threads share one directory without a lock and never read a partial entry. Two of them missing the same function write the same bytes; two building one program at once each write a complete entry, and the later rename wins.

A compile that wrote entries trims the directory to the limit when no sweep ran in the last hour (DIR/sweep records the last one), removing the least recently used entries of any kind first, by the later of each file’s access and modification time. Linux updates access times lazily, so recency is coarse, and no hit writes anything. A program entry with an object is about the size of that object (1.7 MB for a program that imports std.http.server).

If the directory cannot be created or written, the compiler prints one warning and compiles without the cache for the rest of the process; a read-only directory still serves hits. A program entry that cannot be written is skipped without a warning, so a read-only directory that serves every function does not warn on each build. The cache changes no diagnostic: code generation never reaches a program with errors, and only successful builds store programs.

The test suite compiles thousands of programs; the workspace’s .cargo/config.toml points PLENTY_CACHE_DIR at target/plenty-cache, so every test run of a worktree shares one cache and none touches the user’s ~/.cache. Tests that exercise the cache use directories of their own.

Trust

The cache holds machine code that the compiler will link into programs. A local directory is trusted like the compiler binary itself: anyone who can write it can already replace the compiler. The checksum detects damage, not tampering. Sharing a cache between users, or over a network, needs the trust model described below, which this phase does not implement.

Later phases

These are design notes, not implemented behavior.

The bounded module-cache investigation records current edit profiles, a scalar native-artifact spike, and a runnable counterexample to signature-only invalidation. It changes no cache behavior.

Per-module artifacts. Program objects make a no-change rebuild nearly free, but any edit runs the whole frontend again, std modules included. A per-module cache would keep, for each module:

  • its interface: the resolved public declarations other modules can see (function signatures, class, enum and protocol definitions with their type parameters and layouts, aliases, exports, and access rules), including any fact that lowering infers from a body and a caller relies on, such as the fields a function returns;
  • its body: the lowered, checked operations of its non-generic functions, with its string literals as a table of its own, and the generic functions and data it defines, in the resolved form that instantiation needs;
  • the generic instances it requests, with their type and value arguments.

Its key would be a SHA-256 of the module’s text, the compiler’s identity and settings, and the interface hashes of its direct imports. A body edit then rebuilds only the edited module; dependents rebuild only when an interface they import changes. An instance of a generic depends on the generic’s body, not just its interface, so an instance would be keyed by the defining module’s body hash as well. Native call-site substitution also creates body dependencies that invalidation must cover. A program build would combine the modules’ bodies and instances in a fixed order, deduplicating instances and renumbering strings, so the object stays byte-identical to an uncached build.

The frontend cannot produce these yet: it checks all modules as one program. Type resolution creates generic data instances in a table shared by every module, lowering interns strings and records instances and returned fields in tables shared by every body, and signatures are collected into one whole-program table (see frontend). The operations and types also have no serialized form; nominal types are graphs with weak edges. The work is: give lowering a per-module view built from imported interfaces, make instance and string tables per module and merge them afterwards, serialize operations and types, and keep program-order diagnostics.

No smaller artifact pays for itself (issue #317). Loading a stored tree allocates every node, as building it does, so a copy of a phase’s result bounds what loading it costs:

Phase output (release compiler, system allocator)Building itCopying it
Syntax trees of the 1,000-unit program63 ms to parse40 ms
Operations of the 1,000-unit program (7,001 functions)109 ms to lower15 ms
Operations of a program importing std.http.server (605 functions)19 ms to lower0.5 ms

Stored syntax trees would save at most a third of parsing, about 5% of a warm build after an edit. Name resolution rewrites those trees in place and costs less than a copy of them (20 ms for the 1,000 units split into ten modules). Only lowered operations are much cheaper to load than to build, so a module artifact must hold them, and that needs the lowering work above. A program that is one module, such as compile_bench’s, would need the same artifacts per function.

For std, lowering only the functions a program reaches would cut work in every build, cold or warm: in a program that imports std.http.server and prints, the calls in its operations reach 64 of the 222 std functions (drop hooks and thread, executor and sort adapters, not counted, reach more), and in examples/http_server_bench.plenty 155 of 230.

Native emission now omits unreachable standard definitions after checking (issue #400; see emission). This reduces object size and native work without a cache redesign. It does not implement selective frontend lowering or module artifacts. The module-edit probe checks unchanged, body, interface and generic-body edits against uncached objects and provides fresh-process time and peak-memory measurements for further work.

Remote and shared cache. The same keys over HTTP to a content-addressed store would serve CI and teams. It must be optional and opt-in, readable by default and written only by trusted builders such as CI, perhaps through the Bazel Remote Execution API’s cache protocol or a minimal GET/PUT. Hashes are not a trust boundary: an entry is machine code that will run. Trust only configured servers, and verify entries by recompiling a sample in CI.

Measurements

Release compiler, eight cores, warm page cache (issue #144):

ProgramNo cacheCold cacheWarm cache
import std.http.server and print (576 functions)0.14 s wall, 0.67 s CPU, 65 MB0.15 s, 0.75 s, 68 MB0.06 s, 0.14 s, 24 MB
compile_bench 1,000 units (7,002 functions, 16 MB of entries)0.86 s wall, 4.6 s CPU, 102 MB0.80 s, 4.1 s, 103 MB0.47 s, 1.5 s, 94 MB

A one-line edit to the 1,000-unit program misses one function. A cold run already has hits: functions of one program with the same stencil share an entry.

A program hit (issue #317; plenty --compile, release compiler, function cache warm in both cases, paired runs on a loaded host):

ProgramFunction cache onlyProgram hit
compile_bench 1,000 units as a file3.59 G instructions, 1.72 s CPU, 0.46 s wall0.22 G, 0.24 s, 0.08 s
import std.http.server and print0.75 G, 0.40 s, 0.13 s0.19 G, 0.23 s, 0.07 s

What remains on a hit is mostly linking. A build after an edit misses; it also hashes the sources and writes the entry, about 1% more instructions for the 1,000-unit program.

A build that misses its program entry, as after an edit, runs the whole frontend even with every function cached (--emit-object, profile build, eight cores):

ProgramBuildParsingName resolutionLoweringObject emission
1,000 units in ten modules267 ms41 ms20 ms103 ms90 ms
import std.http.server and print51 ms9 ms3 ms17 ms19 ms

With plenty --compile, these builds take 3.16 G and 0.62 G instructions, and 3.08 G for the 1,000 units as one module.

Inspecting generated code

plenty --dump STAGES FILE -o DIR writes what the compiler produced for each function as plain text, for reading, grep, and diff. It compiles the program as --emit-object would, with no object written and nothing linked.

plenty --dump all app.plenty -o dump/
plenty --fast-compile --dump clif-opt,asm --dump-filter parse app.plenty -o dump/
plenty --dump asm --dump-std app.plenty -o - | less
OptionEffect
STAGESall, or a comma list of ops, clif, clif-opt, asm
-o DIRWrite index.txt and one STAGE.txt per stage into DIR, creating it; stage files from an earlier dump that this one does not write are removed
-o -Print the files to standard output, each after a #### NAME line
--fast-compileDump unoptimized code; the default dump uses the optimized profile
--dump-stdInclude retained standard-library functions; unreachable standard definitions are omitted before native emission. By default only the program’s own modules are dumped
--dump-filter TEXTDump only functions whose name or symbol contains TEXT
--module-root DIRResolve imports as for the other file commands

Stages

StageContents
opsThe typed operation IR the frontend hands to code generation, after call-site substitution: each local slot’s type, then one operation per line, with nested bodies indented
clifCranelift IR as lowered from the operations, before Cranelift’s passes
clif-optCranelift IR after its passes, as compiled to machine code
asmFinal machine code with byte offsets: x86-64 AT&T syntax or AArch64 instructions

With --fast-compile, Cranelift runs at opt_level=none, so clif-opt shows legalized rather than optimized IR; the native emission page describes what optimization does. Operations are printed with the compiler’s own debug form and change with it; they are for comparison, not parsing.

Files

Every stage file holds one section per function, in the order the compiler places functions in the object, which does not depend on the worker count:

For example, an x86-64 dump of a function with explicit unchecked: arithmetic can contain:

==== twice  symbol=__plenty_fn_twice  kind=function  source=app.plenty:1
     0:  push    %rbp
     1:  mov     %rsp,%rbp
     4:  mov     %rdi,%rax
    # line 2
     7:  imulq   0x20
     e:  jo      0x0000000000000019
    14:  mov     %rbp,%rsp
    17:  pop     %rbp
    18:  ret
    19:  ud2  # plenty_trap_overflow
    1b:  .byte   0x00,0x00,0x00,0x00,0x00,0x28,0x00,0x00,0x00,0x00,0x00,0x00,0x00

The header names the function as source shows it (Point.length, first[i64], main.<closure@26:11> for a closure at line 26, column 11 of main), its object symbol, its kind, and where it is defined. Kinds are function, drop and resume (a class’s destructor and a generator’s resume function, keyed NAME@drop and NAME@resume), entry (plenty_main), and adapter (thread, executor and sort trampolines). Keys are unique; a repeated name gets a #N suffix.

  • clif and clif-opt mark each change of source line with ; line N, and name each external function and data symbol on its declaration line.
  • A check’s trap in an executable names the runtime helper its trap table entry reaches: trap user4 ; plenty_trap_div_zero in CLIF, ud2 # plenty_trap_overflow in x86 asm, or an AArch64 udf with the same helper comment.
  • asm marks source lines with # line N, names the symbol each relocated operand refers to, and shows the constants Cranelift places after the code as .byte rows. x86 RIP-relative addresses are offsets within the function. AArch64 offsets advance by four bytes per instruction, and PC-relative operands retain the decoder’s displacement notation. Unknown ARM encodings appear as .inst words, preserving their bytes. A function that shares another’s code shows ; shares the code of SYMBOL instead.
  • Data symbols carry what they hold: a string literal’s text, a type’s metadata, or the object’s trap-site table, with the function’s record count and its offset into the table (.Ldata1 (trap sites: 1)+44).

index.txt starts with the compiler version, target, optimization level, stages, scope, and filter, then lists one row per dumped function, largest machine code first:

#  bytes    ops   clif    opt kind     lib  source                           symbol key
      56      6      7      7 function user app.plenty:1                     __plenty_fn_twice twice

ops counts operations, including nested ones, and clif and opt count Cranelift instructions before and after its passes; adapters have no operations and show -. lib is user or std.

Guarantees and limits

  • The output is deterministic: the same compiler and source produce the same text for any worker count and cache state. Source files appear relative to the module root, so the checkout’s location does not matter.
  • --dump bypasses the compile cache and the program cache: every function is compiled and nothing is stored. Source line markers are part of the function’s cache key, so ordinary builds never carry them; they do not change the machine code.
  • Line markers carry a line, not a column. Code Cranelift moves or merges keeps the line of the operation it came from.
  • Offsets and data symbol numbers shift when unrelated code changes, which adds noise to a diff of asm between revisions; diff clif-opt or compare the index first.
  • Dumps use the compiler’s native target. ARM bytes are decoded as A64 words, including zero-valued UDF traps; they never pass through the x86 decoder. The platform contract applies to compilation generally.

The compiler phase profiler measures where compile time goes; --dump shows what that time produced.

Runtime packaging

plenty-runtime is a Rust workspace crate using the standard library. Its vendored dependencies include the RustCrypto crates behind std.crypto, zlib-rs behind std.compress, rustls and its ring provider behind std.tls, SQLite’s C amalgamation behind std.db.sqlite, and siphasher for collection hashing. Its private C ABI accepts fixed-width scalars and pointers to typed storage. Generated code supplies valid initialized fields, bounded immutable metadata, and checked ownership transfers; Rust owns buffers and narrowly scoped unsafe operations handle raw storage, packed pointers, counts, and callbacks.

The compiler build invokes the selected rustc directly for Cargo’s target, avoiding recursive Cargo invocation and build-lock contention. It builds optimized ThinLTO, abort-on-panic static archives and embeds them with native link requirements. Program compilation extracts the archive and links through a cc driver. Runtime source and the Rust toolchain are not needed by an installed compiler.

Direct native dependency builds use the pinned cc crate through plenty-runtime/native/cc_build.rs for target compiler and archive selection. SQLite and Brotli use native C options; ring selects its upstream assembly for the target OS and architecture, including pregenerated COFF objects on Windows x86-64. The GNU archive-symbol parser and panic-symbol redirects are Linux-only. The standalone build-tool and source-selection tests do not by themselves establish a working native compiler port.

Standard library Plenty sources are embedded in the compiler as well. std.math is parsed and checked as an ordinary module, with a stable toolchain identity; it is never loaded from the user’s source root. Its algorithms and error policy remain Plenty code, while numeric methods supply IEEE/platform primitives. See the standard library and the math contract.

The application archive supplies C main, calling plenty_main. A separate library archive omits both startup and that dependency; shared-library export maps hide runtime symbols. Both archives are built with the compiler.

Native emission validates the native target triple, Cranelift’s ISA and pointer type, and the packaged runtime target. Current layouts depend on 64-bit pointers, 128-bit slots, little-endian metadata, the native C ABI and fixed descriptor offsets. Cranelift names both Linux C conventions SystemV; on AArch64 this implements AAPCS64, including aligned u128 register pairs. Ordinary Plenty functions keep the private Tail convention, with native adapters at foreign boundaries. Layout assertions and native tests protect these boundaries; selecting a linker does not validate a new platform.

Target identities

The compiler’s private src/target.rs module owns the admitted target table: triple, architecture, OS, object format, pointer width, byte order and CPU baseline. Frontend target facts and binary compatibility checks consume that description. Tool discovery and command execution remain in toolchain; target facts do not own compiler state or runtime OS services.

Cargo’s HOST runs the build script; Cargo’s TARGET identifies the compiler executable’s host. Those may differ when cross-building the compiler. Native Plenty compilation requires the compiler host, packaged runtime and requested program target to match. That check preserves cross-building the compiler without offering cross-compilation through an installed Plenty compiler. The target table includes both GNU/Linux targets plus experimental aarch64-apple-darwin and x86_64-pc-windows-msvc paths. Experimental emission uses AppleAarch64 or WindowsFastcall at C boundaries, Mach-O or COFF objects, and native linker syntax. Mach-O interface records have aligned length frames so combined metadata sections retain each library. These paths are being qualified in native CI; they are not new release guarantees. See platform support.

Runtime OS boundaries

The runtime keeps its compiler-facing ABI independent of native handles. threads stores the native thread identity in one opaque word and delegates creation and joining to Unix pthread or Windows HANDLE implementations. Cancellation tokens keep their wake resource behind the native module; Linux uses eventfd. Other systems retain the existing unavailable-readiness-wake result, while ordinary token waiting uses the shared condition variable.

The generated mutex fast path remains the same atomic state protocol. Its contended wait and wake operations use Linux futexes, Windows address waits, or public pthread primitives on macOS. The macOS implementation shares static parking storage and broadcasts on wake, avoiding private Darwin ulock APIs and per-mutex allocations. Unrelated contended mutexes can wake together; performance under heavy contention still needs native measurement.

Signal register access is selected by OS and architecture. Darwin’s machine context is a pointer to a separate record; Linux’s is inline. The POSIX trap and stack-overflow handlers use native errno access and signal conventions. macOS stack exhaustion can arrive as SIGBUS or SIGSEGV, and its alternate stack reservation accounts for Darwin’s larger minimum. Existing host handlers retain their signals. Windows uses vectored exception handlers and a per-thread stack guarantee. Only registered checked-trap addresses are redirected; unrelated exceptions continue through the host chain. Library threads do not install these handlers.

Resident-library loading also has native backends. POSIX uses libc’s native loader flags; Windows uses UTF-16 paths, native symbol lookup and module pinning. The Windows byte-path boundary accepts UTF-8 and rejects invalid byte sequences; it does not yet represent filenames containing unpaired UTF-16 surrogates. Closing a lookup lease preserves the resident-code contract.

These boundaries and experimental output paths do not establish macOS or Windows standard-library parity or release support. The native qualification workflow tests synchronization separately so failures elsewhere in the runtime do not prevent exercising these primitives. Native Windows exception subprocess tests pass in CI. Object and linker support, actor recovery, OS services and end-to-end Plenty execution remain qualification requirements in the portability plan.

Entry symbols

A standard module’s native code is reachable only through its own entry symbols. Generated code calls each runtime operation N through its own exported function plenty_op_<N>(arguments, descriptor, site); no dispatcher shared between modules takes an operation number. --gc-sections keeps only the entries a program calls and the code they reach, so a program that never calls std.net, std.process, std.crypto, std.compress, std.tls or std.db.sqlite links none of their runtime code. A dispatcher shared by several modules would keep every module it reaches (issue #501). hello world calls two entries. BUDGETS in tests/test_executable_size.rs bounds the size of hello world and an empty program, and NATIVE_MODULES there checks this for every module with native code; a new module adds a row there (see size budgets).

The argument block holds three 16-byte inputs at offsets 0, 16 and 32, the result at 48, and further operands at 64 and 80. The compiler always initializes the first three input words, using zero for omitted inputs. It initializes offsets 64 and 80 only for actual fourth and fifth operands; the result and unused scratch words may be uninitialized. Runtime handlers must not read those spare words, even if an optimizer would discard the value.

The compiler and runtime share the scratch-operand count in plenty-runtime/src/collection_opcodes.rs. Code generation checks its actual native operand count against that contract, including hidden operands such as a column field index. The runtime reads only the declared scratch inputs and supplies zero for unused elements of its local operand array. Adding a native operation with more than three operands requires updating this shared contract. No extra caller stores initialize spare words. Each entry inlines its constant operation number and handler, so the operand count and handler selection fold away rather than adding a runtime arity lookup.

plenty_control, plenty_executor and plenty_channel keep an operation number. Each serves one facility (cancellation tokens, executors and channels respectively), so a program that does not use it does not reach it. Narrow helpers such as plenty_list_len and plenty_text_find take their operands in registers; see narrow native helpers.

Boxed callable owners reuse the heap box prefix: a destruction header, concrete content descriptor at byte 16, shared-call adapter address at byte 24, and typed content slot at byte 32. Runtime operation 128 consumes an environment, allocates its box, and returns Result; failure drops the concrete content. Erased descriptors use kind J for managed ownership and opaque formatting, while the heap prefix retains the concrete box descriptor for relocation and cleanup.

Protocol objects reuse this prefix with a static method-table address at byte 24 and descriptor kind j. Each table entry is an adapter receiving a reference to the same concrete payload slot. Mutable signatures preserve exclusive loans in typed IR. Symbol requests intern tables by their ordered adapters, with native function relocations resolved in deterministic emission order. Reachability includes every adapter and its concrete method. Concrete descriptors retain relocation and destructor behavior independently of the erased public type.

Stack overflow

An executable exhausting its stack reports error: stack overflow on standard error and aborts with SIGABRT (shell status 134). This applies to its main, scoped-worker, and executor-worker threads. It does not unwind destructors or context exits. Libraries leave signal handling to their host and register no alternate signal stack on threads they create.

Executable startup installs a SIGSEGV handler unless one already exists. Each thread reserves an alternate signal stack in its own entry frame: 16 KiB on x86-64 and 64 KiB on AArch64, which has larger saved register contexts. It removes the stack when Plenty code returns, with no heap allocation or mapping. Startup checks the runtime page size and kernel minimum signal-stack requirement before installing the handler; an inadequate reserved stack leaves the default signal behavior in place. ARM qualification currently covers 4 KiB pages. Generated frames larger than a page probe each page. The handler recognizes a data fault below the entry frame and no more than one page below the faulting stack pointer. AArch64 loop probes leave SP unchanged, so the compiler also registers their exact store addresses in the read-only trap table. At those addresses only, the handler checks the negative probe offset in x16 against the frame size in x17 and uses the resulting probe address as the stack frontier. Other faults retain the default action. Foreign code skipping stack pages is outside this detection. Stack sizes remain the system’s limits, and core-file policy follows the system.

Rust panics

A Rust panic in the runtime is a runtime defect, not a Plenty error. An executable reports it as error: internal runtime panic at FILE:LINE:COLUMN: MESSAGE on standard error and aborts with SIGABRT, without unwinding or a backtrace. A failed Rust allocation reports error: memory allocation of N bytes failed and aborts, unless an actor handler contains it as a trap or it happens inside a reserved arena call, which returns an error instead: see RSA allocation. Plenty’s own traps and errors are unaffected.

std’s panic handler would link its panic hook and backtrace symbolizer into every executable, even with abort-on-panic. The application archive’s link arguments therefore redirect std’s rust_begin_unwind and __rust_alloc_error_handler to the runtime’s handlers with -Wl,--defsym, and --gc-sections drops std’s versions. Measured on 2026-10-26, hello world’s .text is 69 KB with the redirect and 271 KB without it. Libraries keep std’s handlers, since their host links them.

Trap sites

An operation that can trap passes its runtime helper a pointer to its source position. Positions are sixteen-byte records (line, column, the offset to a length-prefixed file name, and an embedded-standard-library flag) in one read-only table per object. Each function numbers its own records, deduplicated by position and provenance, so its IR is built on any thread with the offsets known; the thread placing the function appends them to the object’s table and carries the function’s base offset in its relocation, and file names follow all the records, each named once. The module loader supplies provenance; a filename that resembles a standard module cannot acquire it. File names are relative to the source root, which keeps objects independent of the source directory. The table is a local symbol with no relocations, so each Plenty library in a program keeps its own.

Inline checks (integer overflow, division by zero, shift counts, float-to-integer casts, unwrap) branch to a cold trap block; checks of one kind at one position share the block. In an executable the block is a single x86 ud2 or AArch64 udf, with its own Cranelift trap code, and the object holds a trap table mapping each such address to the check’s kind and site record. Its addresses are 32-bit offsets the linker resolves, so a PIE applies no relocations to it and its pages stay unread until a check fails. plenty_main registers the table, and startup installs a SIGILL handler. It uses the alternate signal stack by default; if a custom SIGILL action was already installed, its SA_ONSTACK and SA_RESTART choices and signal mask are retained. Unrelated signals invoke that saved action without removing Plenty’s checked-trap handler. On a kernel-raised SIGILL at a listed check address the handler resumes the thread in the check’s helper with the site as its argument, so the report, exit status and thread behaviour match a direct call. x86 restores a call-compatible stack; AArch64 restores PC, aligned SP and argument registers for a native C helper entry. Any other SIGILL goes to the previous action. The table lists only functions reachable from exported functions and destructors, so linker garbage collection still removes the rest. A function has 247 trap codes; later checks, and every check in a library or --fuzz build, compute the pointer in the cold block and call the helper. Either way the passing path gains nothing. A check whose condition the compiler computes from constants and finds false, such as the divisor’s zero test in x // 10 or the count test in x >> 13, emits no branch or trap call, in debug and release builds. A check that may fire, or always fires, keeps both. Collection helpers that check an index, key, range parameter, or separator take it as an argument: one address computation per call. An operation entry writes its result at a fixed offset in the argument block, which frees an argument for the site. Iteration reads and operations that cannot trap pass null. The runtime reads the record only when the trap fires and formats the report directly to standard error.

For objects containing embedded std functions, user calls into std save and restore a thread-local pointer to the nearest user call site, except calls to std accessors that only read a field, such as NonNegative[i64].get, which cannot trap or run other code. User code also saves and restores it around generator advancement, collection operations that can run callbacks, and cleanup that can run compiled code: a class __del__, a generator, or a closure, channel or executor, which may own either. Releasing strings, byte storage, and records, boxes or collections of only those runs the runtime alone, which neither traps nor reads the context, so those releases skip it (issue #750). Std code saves and restores it only around indirect calls and direct calls to functions that may change it: a function may change the context when it tail-calls std from user code, makes an indirect tail call, calls such a function in tail position, or is a generator or a function without Plenty source. The runtime restores the context itself after each destructor, generator resumption, and sort-key comparison it runs, so no runtime helper changes it. Direct user-to-user calls need no context operations. Tail calls transfer the pointer without retaining the discarded frame. A std trap prints the original site and a caller note; ordinary user traps ignore this context. Site records are immutable, so the context needs no allocation. Worker threads start without a caller, and executor jobs clear the context between jobs. The pointer names a site, not a stack trace.

A null site prints the message alone. Stack overflow, the token front end, and runtime contract failures such as a library object from another loaded instance have no site.

Validation

Native tests and compile-time layout assertions check public helper signatures and private layouts. Miri exercises runtime storage with exposed-provenance semantics for packed pointers; it cannot execute generated Cranelift code. runtime-checks enables allocation counting and failure injection. Allocation-free markers count successful allocations on the calling thread and keep a separate baseline for each thread; failed attempts do not count, and failure injection is also thread-local. Live-heap checkpoints count bytes across the process, so workers must be idle or joined and any retained worker storage counts toward the checkpoint’s allowance. All workers and their results must be cleaned up before the final allocation-balance check. A relocated compiler test rejects per-program C/Rust runtime compilation. The runtime contributor guide lists validation commands.

Concurrent runtime mechanisms

Worker eligibility is a closed-world compiler check over finite type graphs, reachable helpers, concrete callbacks, and destructors. Visited identities bound recursive walks. Atomically counted owners can cross certified boundaries; counts alone do not permit mutable sharing. The concurrency contracts define allowed effects, owned jobs, blocking, and shutdown.

Each native primitive (CollectionOp) reports an effect from one exhaustive match in src/effects.rs; the certification walk rejects a worker that can reach a denied effect and names the call. Primitives without a native contract, such as foreign calls, are rejected separately. See native effects on worker threads.

Tasks and queues

Scoped tasks pin argument/result storage in the parent’s frame. Generated C adapters invoke typed workers and publish results there. POSIX join must complete before reclaiming storage; unexpected join failure terminates the process to avoid a worker using departed storage. Task bookkeeping does not allocate a Plenty owner; native thread stacks and OS bookkeeping still use resources.

Channels allocate a pinned core/ring whose slots reserve complete message layouts. Endpoint values reference counted headers. One mutex protects predicates and disconnect flags; separate condition variables serve readers and writers. Disconnect wakes all waiters. Receive relocates values before slot reuse, and user destructors run outside locks. nowait can still contend on a mutex.

Executors keep pinned worker records and a bounded ring. Partial startup failure joins started workers. Each accepted job has one fallibly allocated cell for its concrete environment and result; queue/worker and future owners use separate atomic counts. Queue and job state have separate locks. User callbacks/destructors run outside them; pool destruction drains and joins.

Cancellation and selection

Tokens combine an atomic cancellation flag with a mutex/condition-variable handshake for waiter registration. Timed waits track a monotonic elapsed budget, so wakeups do not restart it and huge millisecond values need no overflowing absolute deadline.

Receive selection registers pinned stack waiter nodes in participating queues and uses a per-call condition variable. Queue-to-waiter lock ordering synchronizes notification; selection transfers a value under its queue lock and removes all registrations on every return. It needs no global selection lock or per-wait heap allocation.

Atomics

std.atomic operations compile inline, with no runtime call. A method borrows its receiver’s place, so a local atomic lives in the function’s frame and every thread addresses the same slot. The payload is the slot’s low word, naturally aligned because slots are at least 8-byte aligned. Each access is a Cranelift atomic instruction (atomic_load, atomic_store, atomic_rmw, atomic_cas, fence) of the element’s exact width, never a plain load the optimizer could merge. Checked fetch_add/fetch_sub loop on atomic_cas and trap before storing. Cranelift’s atomics are all sequentially consistent, so each requested Ordering is validated at compile time and then strengthened. On x86-64 a load is a mov, a store and a fence add mfence, and read-modify-write operations use lock-prefixed instructions or a lock cmpxchg loop.

Mutexes

std.sync.Mutex[T] is the generic class in stdlib/std/sync.plenty: an Atomic[u32] state word and the value, stored inline, so moves, destruction, and worker eligibility are ordinary class behavior. Its fields are private; lock() and into_inner() are compiler intrinsics. A lock block stores the mutex reference in a hidden guard local and roots the value’s exclusive loan there, while the mutex’s own shared loan stays live until the block’s exit. The ownership checker treats the guard as live from the lock to the unlock and rejects a join reached while a live guard’s T matches a Mutex[T] the worker’s inputs can reach. Each pool submission records the Mutex[T] types its job input can reach, beside the sender message types, in the function’s submission list and in the future’s local provenance; result() and each shutdown run the same guard scan against them. Reachability walks through Shared[...] contents.

Shared owners

Shared[T] reuses the Box[T] runtime record and its type metadata kind: a counted header, the content type, and a 32-byte-aligned content slot. The frontend emits BoxOp::Share (the box allocation entry point) and BoxOp::SharedRef, a shared-only projection to the content slot; it never emits the box’s move-out or mutable projection for a handle. share() is the ordinary retain of a loaded owner. Construction rejects contents that reach an executor or future, or that reach the same Shared[T] through a Mutex or a channel endpoint.

Locking follows mutex 2 of Drepper’s “Futexes Are Tricky”: state 0 is unlocked, 1 locked, and 2 locked with possible waiters. Lock is an inline atomic_cas from 0 to 1. On failure, plenty_mutex_lock_slow exchanges in 2 and, until the exchange returns 0, sleeps with FUTEX_WAIT_PRIVATE on value 2; EINTR and EAGAIN retry the exchange. Unlock is an inline atomic_rmw exchange to 0, and calls plenty_mutex_wake (FUTEX_WAKE_PRIVATE of one waiter) only when it replaced 2. The runtime issues the futex system call directly. Nothing allocates; a held lock is never abandoned by unwinding, because traps terminate the process.

Native interfaces and metadata

Trusted interface declarations lower to host C calls, keeping Rust collection and enum layouts behind private runtime helpers. Scalar output adapters use call-scoped native storage; UTF-8 adapters lend bytes and length, and C-string adapters create terminated temporary buffers. Owned exports use native boxes with generated matching destruction. The interop contract defines what foreign code must preserve; these adapters cannot enforce foreign behavior.

Generated export interfaces and headers describe names, ABI types, ownership, and error conventions. Library artifacts embed interface metadata and a SHA-256 compatibility symbol. Bounded compile-time extraction reads metadata without loading the binary; explicit verification compares its declared contract.

Typed runtime loaders cache resolved symbol addresses and check exact metadata and object-instance identity. Runtime primitives use eager local GNU/Linux loading and RTLD_NODELETE. Closing a lookup lease releases lookup ownership while resolved code/static state remain resident until process exit. Dynamic loading executes library constructors; it is not sandboxed metadata inspection. Loader-owned memory and constructors are outside Plenty’s allocator accounting.

Support index

The reference describes the supported subset, not Python’s full API. Detailed restrictions live with their contracts.

AreaContract and principal limits
Source and modulesDeclarations, imports, visibility, and entrypoints; compile-time constant expressions, static assertions, and included files; if const target conditionals over a closed set of target facts; one source root per package, no import cycles or re-exports
Packagesplenty.toml, path and Git dependencies pinned in plenty.lock, version requirements, workspaces, plenty add/fetch/update, per-target build reuse, optimized/fast-compile profiles, plenty test with per-process test functions run in parallel under optional timeouts, and plentyc --package-graph; no registry or version selection
Types and dataTypes, including byte literals of type bytes; fixed layouts and recursive data through heap boundaries; read-only std.bytes operations and stream writes take a &[u8] view of bytes, Bytes or list[u8] without copying
ExpressionsOperators, assignment, and control flow; blocks and expressions nest to 64 levels; f-strings with any expression as a field and compile-time Python format specs, whose nested fields supply only fill, alignment, width, and precision; copying slices xs[a:b:c]? of lists, text, bytes, tuples, and clists; local slice views &xs[a:b] of lists and bytes, typed &[T], and of text, typed &substr, with &xs, &s, and literals converting to the whole view where a view is expected; no slice assignment
OwnershipMoves, checked borrows, returned references including Option[&T] and Result[&T, E], and cleanup; general stored references unsupported
FunctionsNamed functions and concrete closures; named arguments and fresh per-call defaults for static calls, with callable/builtin/protocol restrictions; explicit fallible shared-call erasure
GenericsConcrete specialization and structural protocols, including integer/bool value parameters and Bounded, and requirements with their own protocol-constrained type parameters; methods of a generic class or enum instance are checked and compiled only when used; explicit boxed protocol dispatch with shared/mutable receivers
DimensionsCompile-time dimensions: nominal bases, normalized algebra, exact roots and Dimension generic arguments; Quantity construction, numeric/domain operators, conversions, coherent-unit formatting, module constants and scoped alias diagnostic notes. Storage-type native ABI with release instruction evidence
Errors and resourcesSums, propagation, allocation, and context scopes; traps do not unwind; explicit std.error result mapping and contexts
IterationLoops, comprehensions, and native generators; local serial pipelines with borrowed sources, fused adapters including chain, flat_map, and callback-free filter_value, reductions, next(p), lending as &mut Iter[T], and parallel pipelines on a pool with .par(&pool) over owned or borrowed lists and ranges; no reference yield or public borrow across suspension
Built-insNumeric, text, and collection operations; explicitly fallible allocating operations; columnar lists for primitive-field classes with borrowed scalar pipelines
Standard libraryEmbedded std.math, std.path, std.fs, std.deque, and std.bytes; typed errors, Linux paths/metadata and walking, owned double-ended queues, and compact byte buffers. std.time adds checked durations, monotonic deadlines, and wall timestamps. std.io adds partial-progress byte streams, retry helpers, and fixed-capacity buffering over files, process streams, and memory. std.encoding adds strict hex/base64 codecs and explicit-endian integer packing. std.url adds strict RFC 3986 URL parsing, component-aware percent encoding, and query pairs. std.collections adds binary heaps. std.complex adds f64 complex numbers with arithmetic operators. std.cache.lru adds fixed-capacity least-recently-used caches. std.csv adds bounded streaming CSV records and writers. std.random adds caller-owned seeded generators and std.secrets OS-backed secret bytes and tokens. std.crypto adds digests, HMAC, BLAKE3 hashing/key derivation, constant-time comparison, and AEAD, Ed25519, X25519, and HKDF. std.compress adds bounded gzip, zlib, and DEFLATE streams. std.archive adds TAR and ZIP reading, writing, and confined extraction. std.regex adds linear-time regular expressions with explicit resource limits. std.db.sqlite adds SQLite databases through typed prepared statements and owned transactions. std.json adds strict, bounded JSON: a resumable event decoder, a validating encoder, and a value tree. std.log adds leveled structured events with explicit, bounded sinks. std.toml adds strict, bounded TOML 1.0 parsing and writing over a value tree. std.env reads variables and the working directory, and std.process spawns owned children from argument vectors without a shell, with pipes, bounded output collection, signals, and explicit reaping. std.ready adds readiness waits on one handle or many (selectors) under a deadline and a cancellation token that wakes them directly. std.actor runs supervised actors over std.ready on x86-64 and AArch64 GNU/Linux, each with a limited heap that a trap discards, bounded mailboxes of copied messages, timers, request-reply and opt-in restart. std.signal delivers SIGTERM and SIGINT as an event, so a server can drain when its supervisor stops it. std.net adds TCP and UDP sockets, IP address parsing, and cancellable name lookup under the same deadlines. std.tls adds verified TLS client and server streams over those sockets, with client certificates, ALPN, SNI certificate selection, revocation lists, and selector-driven use. std.http adds bounded HTTP/1.1 message framing with streamed bodies and strict rejection of ambiguous framing. std.http.server serves it, and HTTPS through std.tls, with typed routes, bounded workers and admission, idle connections parked on a selector, per-phase deadlines, and graceful shutdown; its qualification lists the production gates it passes. std.http.client sends requests over pooled connections with deadlines, bounded redirects, an explicit proxy, and streamed uploads, shares one pool between threads through Shared, and std.http.https adds verified https with CONNECT tunnels. std.http.body, std.http.cookie and std.http.proxy add bounded forms, JSON and multipart uploads, validated cookies, and trusted-proxy client addresses for handlers. std.http.files serves static files confined to a root directory; cookies can be signed. std.test adds assertions that report their caller’s position, fixtures, fake clocks and streams, and allocation and file failure injection for plenty test. std.diff adds Myers and patience differences over lists, bytes, and text tokens, with unified-diff output
I/OText/byte streams, raw Unix paths, and owned UTF-8 files; direct file iteration unsupported
ConcurrencyThreads, channels, pools, futures, and parallel operations, with atomics and mutexes shared through scoped-thread borrows or atomically counted Shared[T] owners; certified worker effects, serial ordinary loops; no weak handles
InteroperabilityTrusted C imports, exports, metadata, and runtime loading
Editor supportLanguage server with diagnostics, hover, go to definition, and completion, configured for Neovim; generic bodies resolve per instantiation
Native toolchainCommands and artifacts, and per-function IR and assembly dumps; native x86_64-unknown-linux-gnu and aarch64-unknown-linux-gnu, linking with the matching host cc; release requirements and ARM qualification distinguish implementation from tested deployment environments

The ARM path uses generic ARMv8-A instructions, scalar std.simd, Linux ELF, and the AAPCS64 C ABI. It is a release platform: the release workflow builds the compiler and both optional engines in the glibc 2.28 image on native ARM runners, runs the full suite there, and smoke-tests the archives in AlmaLinux 8 and Debian 13. Kernels older than the one tested, page sizes other than 4 KiB and other distributions are not tested; see the ARM development notes. x86-64 retains its SSE2 baseline. Selecting --target or a linker does not turn an installed compiler into a cross-compiler.

Native emission for aarch64-apple-darwin and x86_64-pc-windows-msvc is experimental. It includes target ABI, Mach-O/COFF objects and native linker invocation. Release packaging includes the dataframe dylib on macOS ARM64 and the dataframe DLL on Windows x86-64, with their metadata and notices. The linear algebra engine remains Linux-only. Native release/portability workflows check Parquet queries and application distributions after relocation. Native CI and the release workflow pass compiler/runtime builds and unit tests, platform-boundary checks, and compilation and execution of a generated callback/result program on both operating systems. Runtime and standard-library capabilities remain incomplete, including actor recovery and several OS services, and these checks do not establish standard-library parity; see experimental macOS and Windows downloads.

There is no interpreter, REPL, JIT, async/await, inheritance, dynamic attributes, or exception unwinding. Future work and priorities remain in the backlog.

Runtime contributor guide

plenty-runtime is the dependency-free Rust workspace crate behind generated native code. The book’s compiler/runtime architecture pages describe packaging, representations, memory, and synchronization. This guide records boundary obligations and validation for runtime contributors.

Boundary invariants

  • Helpers use the host C calling convention with fixed-width scalars and typed storage pointers. Rust Vec and enum layouts never cross that ABI.
  • Generated code supplies initialized fields, bounded immutable descriptors, valid pointer lifetimes, and checked ownership transfers. Internal zero slots mean moved or uninitialized storage, never source-level nullable references.
  • Borrowing helpers return owned results. Consuming helpers explicitly transfer messages, jobs, or results; count changes alone do not confer mutable access.
  • Moves of inline payloads repair their embedded addresses. Inline frame cleanup completes before owner-provided storage expires. User callbacks and destructors run outside synchronization locks and outstanding mutable runtime views.
  • Allocation uses checked layouts and guards for partial construction. Buffers retain matching allocation/deallocation provenance. Traps and Rust panics terminate rather than unwind through generated frames.
  • Layout assertions and native integration tests protect compiler/runtime agreement. An internal layout is not a public foreign ABI.

Validation

cargo test --workspace --all-features
cargo clippy --workspace --all-targets --all-features -- -D warnings
MIRIFLAGS=-Zmiri-permissive-provenance cargo +nightly miri test -p plenty-runtime

cargo test -p plenty-runtime tests the crate independently. Its Cargo library omits the process entrypoint; embedded application builds enable it separately.

runtime-checks enables allocation counting and failure injection in the compiler’s archive. Allocation-free regions count every successful allocation, zeroed allocation, and reallocation on the calling thread, including temporary allocations already freed. Failed attempts, injected or real, obtain no memory and do not count, so failure injection can run inside a region. Each thread has its own region baseline, so workers can use overlapping regions. Failure injection is also local to the calling thread and covers allocation, zeroed allocation, and reallocation without allocating inside the instrumentation. Native tests check cleanup, retained source state, and retries after failure. Normal compiler builds omit these checkpoints. The runtime crate’s allocation-checks feature enables its counting allocator and gated ABI unit tests. --all-features selects both; workspace --features runtime-checks alone does not enable the runtime crate’s feature.

The __test_small_live_heap__ marker and final allocation-balance check count live bytes across the process, including storage transferred between threads. At a live-heap checkpoint, workers must be idle or joined and any worker-owned heap storage must fit within the 4096-byte allowance. Join all workers and drop their results before the final allocation-balance check.

Miri checks layouts, metadata, ownership, relocation, and reentrant destruction. Packed pointer slots require exposed-provenance semantics; this does not establish strict-provenance correctness. Native tests remain necessary because Miri cannot execute Cranelift-generated machine code.

For instrumented archives, PLENTY_RUNTIME_RUSTFLAGS supplies whitespace-separated rustc flags. Compiler RUSTFLAGS are not copied implicitly. A nightly address sanitizer build can use -Zsanitizer=address -Clto=off with an ASan-capable linker wrapper. Record the flags and tested native boundary with validation results.

Comparisons

These pages compare Plenty with other languages, one page per language. They describe implemented Plenty behavior and link to the reference pages that define it. Plans and rejected alternatives are in the backlog and proposals.

  • Rust: ownership, borrowing and what unsafe permits.

Rust

Plenty’s ownership model is Rust-influenced: values have one owner, borrows are checked, and destruction is deterministic. See ownership, borrowing, and lifetimes. This page records where the two languages differ.

Unsafe code

Plenty has no unsafe blocks. Rust’s unsafe permits five operations that safe Rust rejects; Plenty code can perform none of them, except calling C through trusted declarations.

unsafe Rust permitsPlenty
Dereferencing raw pointers (*const T, *mut T), with pointer arithmetic, null or dangling pointers, and aliased &mutNot permitted. * dereferences only checked references (&T, &mut T). The only raw pointers are opaque foreign handles, which cannot be dereferenced, converted to integers or offset.
Calling unsafe functions: C functions, transmute, get_unchecked, manual allocation, uninitialized memoryC functions are declared in .plentyi modules, whose author vouches for the C side. Bit reinterpretation is limited to four std-only integer/float intrinsics used by std.encoding. The other operations do not exist.
Reading and writing mutable staticsNot permitted. There are no global variables; module scope holds only constants.
Implementing unsafe traits such as Send and SyncNot permitted. Worker eligibility is a closed-world compiler check of stored types, reachable bodies and native effects; code cannot assert it.
Reading union fieldsNo unions. Enums and anonymous sums are tagged and checked when matched.

Rust’s inline assembly (asm!) also requires unsafe. Plenty has no inline assembly.

Where unchecked code lives

Code that Rust would write inside unsafe lives outside Plenty source:

  • trusted .plentyi declarations, where a wrong declaration or a faulty C library can violate memory safety;
  • the Rust runtime, whose scoped unsafe code handles raw storage, packed pointers, counts and callbacks;
  • compiler intrinsics used by the standard library.

A defect in one of these, or in the compiler, can corrupt memory. Plenty code cannot.

Consequences

  • Buffer overruns. List, text, range and Bytes indexing is bounds-checked, and an out-of-range index stops the program with index out of bounds: index I, length N (traps). No option disables these checks. Slices clamp their bounds instead of trapping. Integer overflow traps outside checked:, so an overflowed length cannot size a buffer.
  • References. A reference is created from existing storage, such as a variable, a list element or a slice, and cannot outlive it. Creating one from an index goes through the bounds check.
  • Limits. A custom allocator, an intrusive linked list, an unchecked fast path or memory-mapped device access cannot be written in Plenty. Write it in C or Rust and call it through C interoperability, or add it to the runtime.

Supplementary examples

These examples complement the core learning path. Use them to look up a particular operation once you understand ownership, borrowing, and recoverable errors.

See collections and iteration, owned files, and text stream positions for the corresponding contracts and limits.

The syntax cheat sheet summarizes everyday syntax on one page and lists the Python habits Plenty rejects.

The finder experiment tests these facilities against a realistic CLI task and records where the implemented surface still blocks it.

Linked data structures compares owned chains and trees with arena IDs, including fallible insertion and allocation-free rotation.

Syntax cheat sheet

A compact summary of everyday Plenty, for readers who already know Python. Every program on this page runs as a test, so the forms shown are current. Each section links to the reference, which remains authoritative. The tutorial teaches the same material in order, and the signature index lists the standard library.

Builds optimize by default. Use plenty --fast-compile FILE, plenty script --fast-compile FILE, or plenty run --fast-compile for shorter edit-run cycles in large projects; runtime checks stay enabled. Package output is in target/optimized/ by default or target/fast-compile/ with that flag.

Programs and printing

def main() -> Result[(), AllocError | IoError]:
    # print returns Result[(), IoError]; `?` propagates the error.
    print("hello", 42)?
    # Collection literals allocate, so they return Result[_, AllocError].
    names = ["Ada", "Grace"]?
    print(len(names), names)?
    Ok(())
hello 42
2 ["Ada", "Grace"]

Executable statements live in functions; module scope holds only declarations, imports, const values, and compile-time assert checks. main returns (), i32, Result[(), E], or Result[i32, E]. A Result cannot be silently discarded: use ?, match, or drop(result). See entrypoint, source files, and branches.

Bindings and numbers

import std.math as math

def main() -> Result[(), IoError | ParseError | OverflowError[i64] | ZeroDivisionError]:
    mut count = 0
    count += 1
    small = 7u8
    ratio: f64 = 1.5
    total = f64(count) + ratio
    print(count, small, total)?
    print(7 // 2, -7 // 2, 7 % 2, 7.0 / 2.0)?
    base = 2
    print(base.pow(10), (3).pow(2), math.abs(-4)?, math.max(3, 9))?
    print(i64.MAX, u8(200), i64.parse(" 12 ")?)?
    Ok(())
1 7 2.5
3 -4 1 3.5
1024 9 4 9
9223372036854775807 200 12
  • Bindings are immutable; mut declares a mutable one, once.
  • There is no int: unsuffixed integers are i64, floats f64. Suffixes: 7u8, 1.5f32. Integers may be hexadecimal, octal, or binary (0xFFu8, 0o755, 0b1010) and use _ separators (1_000_000).
  • Types never widen implicitly; cast with f64(n), u8(n) (traps if out of range), T.try_from(n), or T.wrapping_from(n). T.parse(text) returns Result[T, ParseError].
  • // floors and % is integer-only. Integer arithmetic returns errors from the function: include OverflowError[T] and, for plain divisors, ZeroDivisionError in its Result signature. Still write Ok(...) on success; operators themselves return ordinary numbers. No **: use x.pow(n).
  • abs, min, max, sum come from std.math.

Use checked: for local recovery instead of returning arithmetic errors from the function. Its last line is the Ok value:

def scaled(n: i64, d: i64) -> Result[i64, OverflowError[i64] | ZeroDivisionError]:
    checked:
        n * 1000 // d

def main() -> Result[(), IoError]:
    print(scaled(7, 2).unwrap(), scaled(7, 0).is_err(), scaled(i64.MAX, 1).is_err())?
    Ok(())
3500 True True
  • Positive[T] (> 0) and NonNegative[T] (>= 0) accept signed integer T. Literals convert where expected; computed values use Positive.try_from(n)? or NonNegative.try_from(n)?. Read with .get(); widen a Positive with NonZero.from(p) or NonNegative.from(p). See integer domains.

  • Bounded[T, lo, hi] accepts the eight integer types and inclusive compile-time bounds. Literals convert where expected; an alias such as type Percent = Bounded[u8, 0, 100] provides Percent.try_from(n)? for runtime inputs. Read .get() for arithmetic. Value parameters use const N: i64 or [T: IntType, const N: T].

  • Each function and closure has its own declared arithmetic contract, independent of its definition or call site. There is no private callback channel. Handle a callee’s Result normally.

  • unchecked: asserts that its arithmetic cannot fail and yields an ordinary suite value, not a Result. A violation traps by default and is undefined behavior with --unchecked assume; unchecked trap: always traps. Use wrapping operations for modular arithmetic. Keep opt-outs small and document their bounds.

  • ?, return, break, and continue inside it act on the function or loop.

  • Float arithmetic stays IEEE: overflow may produce infinity and division by zero may produce infinity or NaN. For recoverable zero division, x.div_checked(y) is Result[F, ZeroDivisionError].

See numbers and assignment.

Text

def main() -> Result[(), AllocError | IoError]:
    name = "Ada"
    greeting = ("Hello, " + name)?
    print(greeting)?
    print(f"{name} has {len(name)} letters; pi is {f64.PI:.2f}"?)?
    words = "a,b,,c".split(",")?
    print(words, "-".join(&words)?, "  x ".strip()?)?
    print("hello".find("l"), "hello".startswith("he"), "ll" in "hello")?
    print(name[0], name[-1], "hello"[1:3]?, str.from(42)?)?
    Ok(())
Hello, Ada
Ada has 3 letters; pi is 3.14
["a", "b", "", "c"] a-b--c x
Option[i64].Some(2) True True
A a el 42

str is immutable. Operations that build text return Result[str, AllocError]; ? binds tighter than +, so write (a + b)?. An f-string has type Result[str, AllocError], or Result[str, AllocError | FormatError] when a field runs a display or debug method or a spec is computed. Escapes are \n, \t, \r, \0, \\, quotes, and \u{H...} ("e\u{301}"); there is no \x or \N{...}. len counts Unicode scalars; text.byte_len() counts bytes. A slice such as name[1:]? or name[::-1]? copies, so it needs ?. See text, building text, and format strings.

Collections

def main() -> Result[(), AllocError | IoError | OverflowError[i64] | ZeroDivisionError]:
    mut items = [3, 1, 2]?
    items.append(4)?
    items.sort()
    print(items, items[0], items[-1], items.get(10))?
    print(items[1:3]?, items[::-1]?, items.pop(), 2 in items)?

    mut counts: dict[str, i64] = {}?
    for word in "a b a".split()?:
        *counts.get_or_insert(word, 0)? += 1
    counts.insert("b", 5)?
    match counts.get_mut("a"):
        case Some(count):
            *count += 10
        case Nothing:
            pass
    print(counts, counts.get("z"))?
    for word, count in counts.items():
        print(word, *count)?
    print(len(counts.keys()), "a" in counts.keys())?

    mut seen = {1, 2}?
    seen.add(2)?
    squares = [n * n for n in range(5) if n % 2 == 0]?
    print(len(seen), squares)?
    Ok(())
[1, 2, 3, 4] 1 4 Option[i64].Nothing
[2, 3] [4, 3, 2, 1] Option[i64].Some(4) True
{"a": 12, "b": 5} Option[i64].Nothing
a 12
b 5
2 True
2 [0, 4, 16]

Displays, comprehensions, append, insert, and add can fail to allocate, so they return Result. Empty displays need a type: xs: list[i64] = []?. Indexing a missing element or key traps; get returns an Option. Dictionaries have no indexed assignment: insert adds or replaces, get_mut returns Option[&mut V] to change a value in place, and get_or_insert(k, default)? returns &mut V after inserting a missing key. for v in (&mut d).values(): and for k, v in (&mut d).items(): change every value in place. See collections, lists, and dictionaries.

Tuples

def div_mod(a: i64, b: i64) -> Result[(i64, i64), OverflowError[i64] | ZeroDivisionError]:
    Ok((a // b, a % b))

def main() -> Result[(), AllocError | IoError | OverflowError[i64] | ZeroDivisionError]:
    pair = (42, "cm")
    single: (i64,) = (7,)
    print(pair, pair[0], pair[-1], single[0])?
    quotient, remainder = div_mod(7, 2)?
    print(quotient, remainder, (1, "b") < (1, "c"))?
    mut point = (1, 2)
    x = &mut point[0]
    *x += 10
    print(point)?
    for n, unit in [(1, "kg"), (2, "m")]?:
        print(n, unit)?
    Ok(())
(42, "cm") 42 cm 7
3 1 True
(11, 2)
1 kg
2 m

Tuples allocate nothing. Indices are integer literals; negative ones count from the end, so pair[-1] is pair[1]. There is no pair[0] = value, so assign through &mut pair[0]. Moving an owned component out needs whole-tuple unpacking, including nested targets such as (x, y), label = value. Starred targets are unsupported. match accepts tuple patterns such as case (Some(a), Some(b)):. See tuples and assignment to several targets.

Control flow and matching

def classify(n: i64) -> str:
    match n:
        case 0:
            "zero"
        case 1 | 2 | 3:
            "small"
        case _:
            "many"

def main() -> Result[(), IoError | OverflowError[i64] | ZeroDivisionError]:
    for i in range(0, 10, 4):
        print(i, classify(i))?
    mut n = 4
    while n > 0:
        n -= 1
        if n == 2:
            continue
        elif n == 0:
            break
        print("n", n)?
    parity = "even" if n % 2 == 0 else "odd"
    print(parity)?
    for index, letter in enumerate("hi"):
        print(index, letter)?
    Ok(())
0 zero
4 many
8 many
n 3
n 1
even
0 h
1 i

Block if and match can also be the whole value of a binding, a single-place assignment, or return; see conditional values. A suite’s final expression is its value, so return is only needed for early exits. Conditions must be bool; there is no truthiness. Chained comparisons and loop else are rejected. match works on enums, classes, tuples, integers, str, and bool, must be exhaustive, and has no guards. Tuple patterns nest within enum/class payloads and each other. See matching and loops.

Functions and closures

def scale(value: i64, factor: i64 = 2) -> Result[i64, OverflowError[i64]]:
    Ok(value * factor)

def increment(value: i64) -> Result[i64, OverflowError[i64]]:
    Ok(value + 1)

def apply[F: Callable[[i64], Result[i64, OverflowError[i64]]]](f: &F, value: i64) -> Result[i64, OverflowError[i64]]:
    f(value)

def main() -> Result[(), IoError | OverflowError[i64]]:
    print(scale(5)?, scale(5, factor=10)?)?
    offset = 100
    shift = def [offset](value: i64) -> Result[i64, OverflowError[i64]]:
        Ok(value + offset)
    print(apply(&increment, 1)?, apply(&shift, 1)?)?
    print(apply(&(def(value: i64) -> Result[i64, OverflowError[i64]]: Ok(value * 3)), 2)?)?
    Ok(())
10 50
2 101
6

Every parameter and result has a type. Anonymous functions are def(params) -> T:; lambda is rejected. Captures are explicit: def [x, mut y, &z, &mut w](...). Accept callbacks as F: Callable[...] with &F, which takes named functions and closures; use &mut F for a closure that changes a mut capture. See signatures, function values, and closures.

Classes and enums

For primitive-field classes, clist[T]() creates allocation-free empty columnar storage. items.append(row) returns Result[(), AppendError[T]]; items[i].field reads a scalar, items.copy_row(i) returns Option[T], and items.column(T.field) is a borrowed local pipeline yielding scalar values. Whole-row and packed-field references are unsupported.

@copy
class Point:
    x: f64
    y: f64

class Counter:
    name: str
    hits: i64

    def bump(self: &mut Counter) -> Result[(), OverflowError[i64]]:
        self.hits += 1
        Ok(())

    def report(self) -> i64:
        self.hits

enum Shape:
    Dot
    Circle(f64)
    Rect(f64, f64)

    def area(self) -> f64:
        match self:
            case Shape.Dot:
                0.0
            case Shape.Circle(radius):
                f64.PI * *radius * *radius
            case Shape.Rect(width, height):
                *width * *height

def unused(counter: &Counter) -> bool:
    match counter:
        case Counter(_, 0):
            True
        case Counter(name=_, hits=_):
            False

def main() -> Result[(), IoError | OverflowError[i64]]:
    mut counter = Counter("clicks", 0)
    counter.bump()?
    counter.bump()?
    print(counter.report(), counter)?
    print(Shape.Rect(2.0, 3.0).area(), Shape.Dot)?
    origin = Point(0.0, 0.0)
    mut shifted = origin
    shifted.x = 1.5
    print(origin, shifted)?
    print(unused(&counter))?
    Ok(())
2 Counter(name="clicks", hits=2)
6.0 Shape.Dot
Point(x=0.0, y=0.0) Point(x=1.5, y=0.0)
False

Classes have typed fields and a generated positional constructor; there is no inheritance. An explicit __init__ returns () for a plain class result or Result[(), E] for precisely Result[Class, E], with explicit Ok(()) on success. Construction adds no allocation error; initializer operations and parameter defaults must declare or map their own errors. Bare self is a shared borrow; mutating methods declare self: &mut Class. Explicit self: Class transfers an owned receiver into the method, just like a by-value function parameter; the caller cannot reuse an affine receiver. @copy before class makes instances copy like integers; every field must copy, and the class cannot define __del__. A generic @copy class copies when its type arguments copy, and T: CopyType requires an argument that copies (copy classes). @newtype on a one-field class forwards the field’s operators between two values of that class (newtype forwarding). A field can move out of an owned binding or receiver, after which the object cannot be used as a whole until that field is assigned again; see moving fields. Variants are always qualified (Shape.Dot), and matching a borrowed enum binds references, so payloads are dereferenced with *. case Counter(name, hits=0) takes a class apart, written like its constructor, with every field named once (class patterns). See classes, enums, and borrowed matching.

Classes and enums can define arithmetic hooks such as __add__(self, other: &T) -> T, __rmul__, and __neg__. Operators borrow their operands and dispatch statically. Fallible hooks return Result: write (a + b)?. In-place hooks such as __iadd__(self: &mut T, other: &T) -> () mutate an existing destination; there is no automatic += fallback. Equality remains structural. Other __name__ methods, such as __eq__, __str__ and __len__, are rejected: write display/debug for formatting and named methods for the rest.

Compile-time dimensions

Use dimension for a base or a derived dimension, and D: Dimension for an argument that carries no runtime value:

dimension Length("m")
dimension Time("s")
dimension Velocity = Length / Time
const metre: Quantity[f64, Length] = Quantity[f64, Length](1.0)

def main() -> Result[(), IoError]:
    distance = 12.0 * metre
    elapsed = Quantity[f64, Time](4.0)
    speed: Quantity[f64, Velocity] = distance / elapsed
    print(speed)?
    Ok(())
3.0 m/s

Products and quotients normalize; Pow[A, n] and Root[A, n] require integer exponents and exact roots. One is dimensionless. A dimension cannot be a value type. Quantity[T, D] checks dimensions in arithmetic and .get() reads its stored number. For floats, .to(unit) divides by a quantity of the same type. std.units provides SI dimensions, units such as u.km and constants; declare your own only for other bases. See dimensions and quantities.

Option, Result, and errors

def parse_pair(text: str) -> Result[(i64, i64), AllocError | ParseError]:
    parts = text.split(",")?
    if len(parts) != 2:
        return Err(ParseError.Invalid)
    Ok((i64.parse(parts[0])?, i64.parse(parts[1])?))

def first_even(values: &list[i64]) -> Option[i64]:
    two: NonZero[i64] = 2
    for value in values:
        if *value % two == 0:
            return Some(*value)
    Nothing

def main() -> Result[(), AllocError | IoError | ParseError]:
    print(parse_pair("3,4")?)?
    match parse_pair("3"):
        case Ok(pair):
            print(pair)?
        case Err(error):
            print("bad input:", error)?
    values = [1, 3, 6]?
    print(first_even(&values), first_even(&values).unwrap_or(0))?
    print(first_even(&values).ok_or(ParseError.Invalid)?)?
    Ok(())
(3, 4)
bad input: ParseError.Invalid
Option[i64].Some(6) 6
6

There is no None and no exceptions: absence is Option[T] (Some(x)/Nothing), failure is Result[T, E] (Ok(x)/Err(e)). ? returns early on Nothing or Err; the function’s error type must equal the source’s or contain it in an A | B union. ? never converts an Option into a Result: use ok_or(error)?. To wrap an error, map it explicitly: print(line).map_error(AppError.Io)?. unwrap() traps. print shows the full enum name, as in Option[i64].Some(6). See propagation and anonymous sums.

Ownership and borrowing

def total(values: &list[i64]) -> Result[i64, OverflowError[i64]]:
    mut sum = 0
    for value in values:
        sum += *value
    Ok(sum)

def double_all(values: &mut list[i64]) -> Result[(), OverflowError[i64]]:
    for value in values:
        *value *= 2
    Ok(())

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    mut values = [1, 2, 3]?
    print(total(&values)?, total(values)?)?
    double_all(&mut values)?
    backup = copy(values)?
    moved = values
    print(moved, backup)?
    label = "team"
    other = label
    print(label, other)?
    Ok(())
6 6
[2, 4, 6] [2, 4, 6]
team team

Numbers, bool, str, bytes, ranges, and named function values copy. A tuple, Option, Result, enum, or anonymous sum (A | B) copies when all of its parts copy. A class copies only when it is declared @copy, and a generic one only when its type arguments also copy. Every other value moves: collections, boxes, closures that capture, generators, and other class instances. After moved = values, values is unusable. str assignment keeps both names usable without copy; immutable storage may be shared. copy(x)? explicitly duplicates eligible owned mutable contents. An eligible ordinary class does not need @copy for this explicit operation; nested Shared handles retain the same content. Custom destruction, recursive data, generators and captured closures restrict copying. &x lends shared access and &mut x exclusive access; a &T parameter borrows an owner implicitly, but &mut must be written. Iterating &list yields &T, so dereference with *. References cannot be stored in classes, enums, tuples, or collections. See moves and copies and borrowing.

Generators

def countdown(start: i64) -> Generator[i64]:
    mut n = start
    while n > 0:
        yield n
        # The loop proves 1 <= n <= i64.MAX, so decrement cannot overflow.
        unchecked:
            n -= 1

def main() -> Result[(), AllocError | IoError]:
    for n in countdown(3):
        print(n)?
    print(list(countdown(4))?, countdown(5).any(def(n: &i64) -> bool: *n == 2))?
    Ok(())
3
2
1
[4, 3, 2, 1] True

A function containing yield returns Generator[T]; its body cannot use ?. Generators have the pipeline methods, such as any(predicate). See generators.

Local pipelines

Local transformations can use iter(source).map(callback).filter(predicate) with a loop or a terminal such as fold(initial, callback) or count(). Borrowed sources and callback captures stay borrowed until the pipeline is consumed. Generators expose the same fluent methods directly. An unfinished pipeline stays local; .generator() explicitly transfers owned sources and owned captures into one resumable frame that can be passed or returned. See local iterator pipelines. enumerate and zip produce pair items: mapping takes two parameters, predicates borrow both, and folds take the accumulator followed by the pair’s components. Collect owned pairs into an expected dict[K, V] with collect(). a.chain(b) continues with a second source of the same item type; flat_map(f) emits the items of the owned source f returns for each item. filter_value(v) keeps items equal to v (True by default) without a callback; borrowed items accept v or &v.

def add_square(total: i64, value: i64) -> Result[i64, OverflowError[i64]]:
    Ok(total + value * value)

def add_result(total: i64, value: Result[i64, OverflowError[i64]]) -> Result[i64, OverflowError[i64]]:
    Ok(total + value?)

def squares(stop: i64) -> Generator[Result[i64, OverflowError[i64]]]:
    iter(range(stop)).map(def(n: i64) -> Result[i64, OverflowError[i64]]: Ok(n * n)).generator()

def main() -> Result[(), IoError | OverflowError[i64]]:
    values = iter(range(5))
    print(values.try_fold(0, add_square)?)?
    print(squares(4).try_fold(0, add_result)?)?
    Ok(())
30
14

Python habits that Plenty rejects

PythonPlenty
Statements at module levelPut them in main
int, floati64, f64, or another width
NoneNothing in an Option[T]
try/except, raiseReturn Result; handle with ? or match
xs = [1, 2]xs = [1, 2]?
print(x) as a statementprint(x)?
"a" + b("a" + b)?, or an f-string with ?
x ** nx.pow(n)
range(a, b, step) with a computed steprange(a, b, NonZero.try_from(step)?); a literal step converts
abs(x), min, max, sum(xs)Integer std.math.abs(x)?, min, max, sum(&xs, 0)?; float abs_float / sum_float
if items:if len(items) > 0:
a < b < ca < b and b < c
xs[1:3], s[1:3]xs[1:3]?, s[1:3]?: a slice allocates a copy; &xs[1:3] borrows a list view, and &s[1:3] a &substr, in place
lambda x: x + 1def(x: i64) -> Result[i64, OverflowError[i64]]: Ok(x + 1) inside brackets
map(f, xs), filter(p, xs), reduce(f, xs, init)iter(&xs).filter(p).map(f), .fold(init, f); no import
any(flags), all(p(x) for x in xs)iter(&flags).any(), iter(&xs).all(p)
d[k] = vd.insert(k, v)?, which adds or replaces
d[k] += 1*d.get_or_insert(k, 0)? += 1, or *v += 1 on Some(v) from d.get_mut(k)
case x if x > 0:Test inside the arm
class B(A):Composition or a protocol
Global variablesPass values as parameters; const holds a number, quantity, bool, str, or bytes computed at compile time
y = xs, then using xsy = copy(xs)?, or borrow with &xs
A name first bound inside an if, loop, or match arm, used after itBind name = if condition: (or name = match value:), or use a if cond else b; the binding ends with its block. Only if const arms open no scope
x is Nonex.is_none(), or match on Some/Nothing
"-" * 3"-".repeat(3)?
del xs[i]xs.pop(i), which returns an Option
assert x in a functionReturn an error: if not x: return Err(...)
xs.sort(reverse=True)xs.sort(True)

Three of the most common rejections:

def main() -> ():
    print("hello")
    return
cannot implicitly discard a Result
def main() -> ():
    x = 2 ** 3
exponentiation with `**` is not supported; use `base.pow(exponent)`
def main() -> Result[(), AllocError | IoError]:
    values = [1, 2]?
    moved = values
    print(values)?
    Ok(())
use of moved binding `values`

Run plenty --check FILE to type-check without building.

Finder feasibility experiment

The repository’s examples/finder/ attempts a small fd-style search tool using the current language. Its README records the assessment, commands, validation, and remaining limits. main.plenty and matching.plenty implement a working filter for supplied paths, with literal basename search, */? wildcards, full-path matching, hidden-component filtering, and CLI errors.

cargo build --offline
target/debug/plenty --compile examples/finder/main.plenty -o /tmp/plenty-finder
printf 'src/main.plenty\nnotes.txt\n' | /tmp/plenty-finder --paths --glob '*.plenty'
python3 -B examples/finder/check.py -v

The program processes one UTF-8 path per line. It borrows strings while matching, avoids allocating basename slices, and preserves concrete I/O errors through Result[i32, IoError | OverflowError[i64]]. ? in a glob matches one Unicode scalar. Matching is case-sensitive; omitted patterns match all nonempty visible paths.

Without --paths, the finder searches [PATTERN [ROOT]], defaulting to .. It uses the embedded std.fs walker over native enumeration, prunes hidden directories before descent, and never follows discovered directory links. Directories, regular files, symlinks and other types are eligible output. An explicit root can itself resolve through a link. Traversal errors stop with the attempted path, operation and concrete IoError. Python supplies test fixtures and an independent traversal oracle, not the application’s candidate paths. The capability probes now check working directory and metadata APIs.

Use --null for lossless Unix paths: args_bytes() preserves raw patterns, input_bytes(0u8) reads NUL records, and write_stdout_bytes emits exact names followed by NUL. CR, LF, and invalid UTF-8 are preserved. Matching borrows list[u8]; in this mode ? matches one byte, so UTF-8 é takes two ? tokens. The default text mode retains its Unicode-scalar and line semantics.

Issue #61’s transport acceptance cases are covered by a real Unix fixture with spaces, tabs, CR/LF, and invalid UTF-8, plus raw argument patterns and a byte wildcard oracle. Native traversal preserves those names through Path, joining and filesystem lookup. Broader compact byte buffers and portable path/library APIs remain within issue #49.

See the current standard stream and argument, binary transport, paths, directory enumeration, walking, text, and foreign text adapter contracts.

Choose a representation for linked data

Start with the ownership and operations the application needs:

NeedStarting representationImportant cost or constraint
Stack or sequential datalist[T]Contiguous storage; growth can allocate.
Queue with two endsstd.dequeRing buffer; amortized constant-time insertion and constant-time removal.
Tree with uniquely owned childrenClasses/enums with boxes or child listsSubtrees move as owners; boxes allocate per node.
Parent/back links, shared targets, or cyclesList owning nodes plus IDsIDs need validation; removal/reuse needs an identity policy.
Frequent rewiring at known nodesLinked owners or arena linksFinding the node may dominate; measure allocation and locality costs.

These are starting points, not performance rankings. The current box runtime has a 32-byte header before the payload, in addition to allocator overhead and alignment. Treat this as an implementation detail, not a stable ABI or proof that one representation wins. See memory architecture.

Boxes allow direct field access and transfer without explicit dereferences in many contexts. They do not permit two owners of the same node. Current borrowing rules also reject stored references and simultaneous mutable loans into distinct indices of one list. Sequential updates such as nodes[a].next = Some(b) work; the IDs themselves borrow nothing.

Automatic destruction of recursive owners uses bounded native stack space and does not allocate. Recursive user traversal and destructor hooks do not inherit that guarantee. A descendant reference cursor handles a chain; a non-consuming general tree traversal needs a supported worklist representation. Arena IDs are one option when a list of borrowed nodes is unavailable.

An intrusive collection embeds its links in the application object. It can support multiple memberships and, in pointer-based designs, may need stable addresses and externally managed lifetimes. Arena-owned objects with integer links cover some of these uses, but do not promise stable physical addresses. The design assessment discusses this boundary and the candidate standard-library abstraction.

Push, pop, and walk an owned chain

Each node owns its successor. Insertion allocates before transferring the old head so a failed allocation preserves the caller’s chain. Pop transfers the successor out before destroying the removed node.

pub class Node:
    pub value: i64
    pub next: Option[Box[Node]]

pub class Linked:
    pub head: Option[Box[Node]]

pub def push(xs: &mut Linked, value: i64) -> Result[(), AllocError]:
    mut node = Box(Node(value, Nothing))?
    node.next = replace(xs.head, Nothing)
    xs.head = Some(node)
    Ok(())

pub def pop(xs: &mut Linked) -> Option[i64]:
    match replace(xs.head, Nothing):
        case Some(node):
            mut removed: Node = node
            xs.head = replace(removed.next, Nothing)
            Some(removed.value)
        case Nothing:
            Nothing

pub def bump(xs: &mut Linked) -> Result[(), OverflowError[i64]]:
    match &mut xs.head:
        case Some(head):
            mut cursor: &mut Node = head
            while True:
                cursor.value += 1
                match &mut cursor.next:
                    case Some(next):
                        cursor = next
                    case Nothing:
                        break
        case Nothing:
            pass
    Ok(())
from linked import Linked, push, pop, bump

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    mut xs = Linked(Nothing)
    for value in range(1, 4):
        push(&mut xs, value)?
    bump(&mut xs)?
    print(pop(&mut xs))?
    print(pop(&mut xs))?
    print(pop(&mut xs))?
    print(pop(&mut xs))?
    Ok(())
Option[i64].Some(4)
Option[i64].Some(3)
Option[i64].Some(2)
Option[i64].Nothing

cursor advances by borrowing from its current referent. It does not create stored references or require a recursive call. Pop and this traversal allocate nothing. bump propagates overflow and retains increments already performed; it does not roll back the chain. Each successful push allocates one box; its failure destroys the incoming value, not the existing chain.

This is a teaching module with public links, not a packaged collection that enforces an abstract API. Use a contiguous list for an ordinary stack unless the workload benefits from separate nodes. See representation choices.

Rotate a tree without allocating new boxes

A right rotation moves the left child into the root position and transfers its right subtree to the old root’s left. Keep both nodes boxed: converting a box to a Tree value would free that box and require allocation to box it again.

pub class Tree:
    pub value: i64
    pub left: Option[Box[Tree]]
    pub right: Option[Box[Tree]]

pub def rotate_right(root: Box[Tree]) -> Box[Tree]:
    mut old = root
    match replace(old.left, Nothing):
        case Some(left):
            mut pivot = left
            old.left = replace(pivot.right, Nothing)
            pivot.right = Some(old)
            pivot
        case Nothing:
            old
from tree import Tree, rotate_right

def main() -> Result[(), AllocError | IoError]:
    left = Box(Tree(1, Nothing, Nothing))?
    root = Box(Tree(2, Some(left), Nothing))?
    rotated = rotate_right(root)
    print(rotated.value)?
    match &rotated.right:
        case Some(right):
            print(right.value)?
        case Nothing:
            print("no right child")?
    Ok(())
1
2

The rotation takes constant time and allocates nothing. If there is no left child, it returns the original owner unchanged. It preserves in-order node order; it does not by itself balance a tree or validate search-tree ordering.

Removing a subtree similarly uses replace(root.left, Nothing). The returned owner can be reattached or dropped. Parent references cannot be stored in these nodes; use arena IDs when parent navigation is required.

Build balanced search trees

A search tree keeps keys ordered while its shape changes. These examples implement two balanced sets directly in Plenty:

  • A B-tree with minimum degree 2: each node holds up to three keys and four children, also called a 2–3–4 tree.
  • A left-leaning red-black tree: each node holds one key, two child IDs, and a color bit.

Both implement insertion, membership, deletion, length, and sorted traversal. They store distinct i64 keys, including the full signed range. Duplicate insertion and missing-key removal return False without changing the tree. These are educational implementations, not std APIs or benchmark results. Use a built-in set[T] when ordering is unnecessary.

Run the implementations

From a repository checkout:

plenty book/src/design/examples/data-structures/balanced-trees/main.plenty
plenty book/src/design/examples/data-structures/balanced-trees/checks.plenty

Or save the B-tree module, red-black module, and demo in the same directory, then run plenty main.plenty. Save checks.plenty there to run the invariant and model checks as well.

The included source is the actual demo compiled by the integration tests:

import btree
import red_black

def main() -> Result[(), AllocError | IoError]:
    mut b = btree.create()?
    mut r = red_black.create()?
    for key in [40, 10, 70, 20, 60, 30, 50]?:
        drop(btree.insert(&mut b, key)?)
        drop(red_black.insert(&mut r, key)?)
    print(btree.ordered(&b)?)?
    print(red_black.ordered(&r)?)?
    print(btree.insert(&mut b, 40)?, red_black.insert(&mut r, 40)?)?
    print(btree.has(&b, 30), red_black.has(&r, 99))?
    print(btree.remove(&mut b, 40), red_black.remove(&mut r, 40))?
    print(btree.remove(&mut b, 99), red_black.remove(&mut r, 99))?
    print(btree.ordered(&b)?)?
    print(red_black.ordered(&r)?)?
    print(btree.valid(&b)?, red_black.valid(&r)?)?
    Ok(())

Expected output:

[10, 20, 30, 40, 50, 60, 70]
[10, 20, 30, 40, 50, 60, 70]
False False
True False
True True
False False
[10, 20, 30, 50, 60, 70]
[10, 20, 30, 50, 60, 70]
True True

Shared interface and ownership

Both modules expose the same operations:

OperationResultAllocation
create()Result[Tree, AllocError]Empty owner creation can fail.
insert(&mut tree, key)Result[bool, AllocError]May grow storage; True means newly inserted.
has(&tree, key)boolNone.
remove(&mut tree, key)boolNone; True means removed.
length(&tree)i64None.
ordered(&tree)Result[list[i64], AllocError]Allocates an owned sorted result.
valid(&tree)Result[bool, AllocError]Allocates visitation marks for diagnostics.

A Tree exclusively owns a private list of nodes. Links are private indices, not references or owning edges. -1 means no child; it is never used as a list index by a valid operation. Updating links in separate statements avoids simultaneous mutable loans into the same list. Neither implementation needs stored references, parent pointers, unsafe operations, or a foreign library.

Deleted nodes join an internal free list. Reusing them needs no generation counter because node IDs never leave the module. Surviving IDs remain valid when the backing list grows. Storage is retained for reuse rather than shrunk; dropping the tree releases all of it. Neither type is a copyable value.

Structural index/count arithmetic uses small unchecked: regions with bounds maintained by these private representations, not caller-supplied numeric values. The B-tree additionally caps its key count at i64.MAX: inserting a new key at that representational capacity returns AllocError.CapacityOverflow before allocation or mutation. Duplicate insertion still returns Ok(False) there. One key per live node bounds the red-black tree’s count by its node-list length.

Validate more than sorted output

checks.plenty compares both trees against a sorted-list model after every operation and runs each implementation’s structural validator. It checks:

  • All 14,400 pairs of insertion and deletion permutations for five keys, including i64’s minimum and maximum; repeated insertions, repeated removals, empty trees, and refill after complete deletion.
  • Four 257-key scenarios using ascending, descending, and alternating-extreme orders, including root growth and contraction, followed by slot reuse.
  • 16,000 deterministic mixed operations across four seeds, followed by draining every remaining key.

The validators check strict ordering, key counts, reachability, duplicate or cyclic links, and separation of reachable nodes from the free list. The B-tree also checks occupancy and equal leaf depth; the red-black tree checks its color and black-height invariants.

The Rust integration harness additionally sweeps allocation-failure points before insertion, checks retry after failure, and verifies deletion and reuse with allocation disabled:

cargo test --locked --features runtime-checks --test integration test_data_structure_patterns::balanced_tree

These tests are regression evidence, not a proof over every possible history. The validators assume storage created by the module, not hostile serialized input. Recursive traversal and validation use native stack proportional to tree height; ordinary balanced operations keep that height logarithmic.

Deliberate limits

There are no generic comparators, key/value maps, range cursors, rank/select, persistence, concurrency, disk pages, or externally reusable handles. The B-tree’s tiny fixed degree makes split/merge cases readable; its separately allocated per-node key and child lists are not a cache-tuned storage design. Growing either backing list can move many rows, so insertion is not a strict worst-case logarithmic-time operation in this representation.

See representation choices and the linked-list lesson for the ownership trade-offs behind index-based links.

Split, borrow, and merge in a B-tree

The complete B-tree implementation is a minimum-degree-2 B-tree, or 2–3–4 tree. Its nonempty root and every other live node contain one to three keys. An internal node with n keys has n + 1 children; all leaves have the same depth. Unlike a B+ tree, keys can live in internal nodes, and leaves have no sibling chain.

Use the balanced-tree demo and checks to run it.

Storage

Each node owns two fixed-length lists: three key slots and four child-ID slots. n selects the live key prefix. Unused key slots may retain old scalar values; unused child slots in live nodes are -1. A leaf has no child IDs. The root, size, free-list head, and node storage belong to one private Tree owner.

The free pool retains deleted nodes and their two lists. Taking a slot resets its child IDs; releasing a slot adds it to the pool without moving other nodes. No public operation exposes a node ID.

Insert: split before descending

insert first searches for a duplicate. For a new key, it measures the current height and prepares enough spare nodes for every possible split on the descent. A full root needs two slots: a new root and a right sibling. Each remaining level needs at most one sibling. With height counted in node levels, reserving height + 1 spare nodes covers the empty tree and both cases.

Preparation is the only fallible mutation phase. It allocates detached empty nodes and links successful allocations into the free pool. Failure can retain extra spare storage, but does not change the root, reachable shape, keys, or size. A retry can reuse those spares. Once preparation succeeds, rewiring, key shifts, and taking spare nodes allocate nothing.

For a full child [a, b, c], split:

  1. Leaves [a] in the original node.
  2. Moves [c] and the appropriate children into a spare sibling.
  3. Moves b into the parent, shifting parent keys and child IDs to make room.

Splitting a full root first ensures the parent always has room. Splitting each full child before descending means insertion reaches a nonfull leaf, where an in-place key shift creates the final slot.

Delete: do not descend into a one-key node

remove first confirms membership, so a missing-key removal changes nothing. The internal erase helper therefore assumes it will find its key.

At a leaf, shift keys left and decrement n. At an internal matching key:

  • If the left child has at least two keys, replace the key with its predecessor and delete that predecessor from the left subtree.
  • Otherwise, if the right child has at least two keys, use its successor.
  • Otherwise, merge the two one-key children with the separating key, then delete within the merged three-key node.

When descending toward a key, a one-key child needs help first:

  • Borrow from the left: move the parent’s separator into the child’s first slot, move the sibling’s largest key into the parent, and transfer the sibling’s last child if these are internal nodes.
  • Borrow from the right: perform the symmetric transfer using the sibling’s smallest key and first child.
  • Merge: if neither sibling can lend, join two one-key children around a parent separator. Return the unused sibling slot to the free pool.

This preserves the minimum occupancy without a separate upward underflow pass. If the root becomes empty, its only child becomes the root; deleting the final leaf key makes the tree empty. The old root slot is released in both cases.

Costs and validation

At fixed degree, search and deletion take O(log n) time and deletion allocates nothing. Insertion performs O(log n) tree work and prepares at most O(log n) spare nodes; occasional backing-list growth costs O(n), giving amortized O(log n) insertion rather than a worst-case bound. ordered visits every key in O(n) time and allocates its result. Recursive deletion/traversal uses O(log n) native stack space for trees maintained by these operations.

Storage follows the historical peak plus the split reserve, not just the current key count. valid allocates and scans visitation marks for all retained slots, so its cost follows retained storage. It verifies occupancy, ordered bounds, equal leaf depth, active child counts, total size, and free-pool membership.

The test driver exercises both borrow directions, predecessor/successor replacement, merges, root changes, and reuse through permutation, ordered, and mixed histories. These are tests of a teaching implementation, not performance qualification for a database or filesystem.

Acknowledgements

  • Cormen, Leiserson, Rivest, and Stein, Introduction to Algorithms: the published B-tree split and top-down borrow/merge algorithms. The Plenty implementation is original; no external source code or data is bundled.

Rotate and recolor a red-black tree

The complete red-black implementation uses left-leaning red-black (LLRB) links. It is a red-black tree with an extra orientation rule, not the parent-pointer variant with symmetric insertion and double-black deletion cases. Use the demo and checks to run it alongside the B-tree.

Invariants

Each node has one i64 key, two child IDs, and a red bit describing its incoming link. Missing children are black. After each public operation:

  • The root is black.
  • Keys obey strict binary-search ordering.
  • No red link leans right.
  • A red node has no red child.
  • Every path from a node to a missing child has the same number of black links.

A black node connected to a left red child represents a logical three-node containing two keys. Collapsing these red links gives a balanced 2–3 tree. The degree-2 B-tree instead allows three keys in a node; the two examples are related balanced representations, not identical encodings.

Ownership and rotations

The tree owns a list of private scalar-field nodes. Child links are indices, so a rotation updates existing nodes in place without allocating boxes or storing references. Deleted slots join a private free list and are reset before reuse.

A left rotation promotes the right child, transfers its left subtree to the old root’s right, and makes the old root its left child. The promoted node inherits the old root’s color; the old root becomes red. A right rotation is symmetric. These changes preserve key order but are only part of balancing.

flip toggles a node’s color and both child colors. Unlike setting colors to constants, toggling supports both splitting and joining logical nodes. Internal callers ensure both children exist before invoking it.

Insert

After rejecting a duplicate, insert allocates or reuses one detached red node before changing any tree links or colors. Failed growth therefore preserves the existing set, shape, and size.

The recursive put links that node at its binary-search position, then calls balance while returning toward the root. The repair sequence is:

  1. Rotate left for a red right link.
  2. Rotate right for two consecutive left red links.
  3. Flip colors when both children are red.

The shared repair helper also handles deletion’s transient shapes. The public operation finally paints the root black. Once the detached node exists, this phase allocates nothing.

Delete

remove first confirms membership, so its internal descent can assume the key exists. If both root children are black, it temporarily paints the root red to permit a color transfer without reducing the final black height.

Before descending into a black child whose left child is also black, move_red_left or move_red_right transfers a red link toward that child. Color flips merge logical nodes temporarily; rotations can borrow from a sibling. These are top-down preparations, not an upward double-black repair.

When deleting a matching internal key, replace it with the smallest key in its right subtree, then use delete_min to remove that successor node. A matching terminal node is released to the free list. balance repairs links on return, and the remaining root is painted black. Deletion and all its rotations and color changes allocate nothing.

Costs and validation

Balancing bounds height by O(log n). Membership and removal take O(log n) time; recursive update and traversal use O(log n) native stack space. Insertion performs logarithmic tree work, but growing the backing list can move O(n) rows, so the representation provides amortized O(log n) insertion, not a strict worst-case bound. Sorted collection visits O(n) keys and allocates an owned list.

Free slots retain storage up to the historical peak. valid allocates and scans visitation marks across that retained storage. It independently checks ordering, black root, no right red link, no consecutive red links, equal black height, size, reachability, and free-list separation.

See the model and invariant tests for permutations, mixed operations, allocation failures, and allocation-free removal and reuse. This implementation is intentionally a set of integer keys, without generic comparators, ranks, or external node handles.

Acknowledgements

  • Robert Sedgewick and Kevin Wayne, Algorithms: Balanced Search Trees: the published left-leaning representation, rotations, color flips, and top-down deletion algorithm. The Plenty implementation is original; no external source code or data is bundled.

Retrieve strings with a trie

A trie, or retrieval tree, shares common prefixes between keys. Each edge here represents one Unicode scalar. The path c → a → r represents "car"; a terminal marker distinguishes that stored key from a prefix that merely leads to "cart". The permanent root represents the empty string.

This example implements a set of strings directly in Plenty, including exact lookup, insertion, pruning deletion, prefix enumeration, and longest-prefix matching. It uses private node IDs rather than stored references. It is an educational implementation, not a new std API.

Run the example

From a repository checkout:

plenty book/src/design/examples/data-structures/trie/main.plenty
plenty book/src/design/examples/data-structures/trie/checks.plenty

Or save the implementation, demo, and test driver in the same directory, then run plenty main.plenty or plenty checks.plenty. These are the canonical source files compiled by the Rust regression tests. The demo below includes its source rather than maintaining another copy:

import trie

def main() -> Result[(), AllocError | IoError]:
    mut words = trie.create()?
    for key in ["", "car", "cart", "cat", "dog", "café", "cafe\u{301}", "猫", "猫咪"]?:
        drop(trie.insert(&mut words, key)?)
    print(trie.ordered(&words)?)?
    print(trie.has(&words, "car"), trie.has(&words, "ca"))?
    print(trie.has_prefix(&words, "ca"), trie.has_prefix(&words, "z"))?
    print(trie.with_prefix(&words, "car")?)?
    print(trie.with_prefix(&words, "猫")?)?
    print(trie.longest_prefix(&words, "cartwheel"))?
    print(trie.longest_prefix(&words, "unknown"))?
    print(trie.insert(&mut words, "car")?)?
    print(trie.remove(&mut words, "car"), trie.has(&words, "cart"))?
    print(trie.with_prefix(&words, "car")?)?
    print(trie.remove(&mut words, ""))?
    print(trie.longest_prefix(&words, "unknown"))?
    print(trie.valid(&words)?)?
    Ok(())

Expected output (the visually similar café spellings are distinct keys):

["", "café", "café", "car", "cart", "cat", "dog", "猫", "猫咪"]
True False
True False
["car", "cart"]
["猫", "猫咪"]
Option[str].Some("cart")
Option[str].Some("")
False
True True
["cart"]
True
Option[str].Nothing
True

Interface

OperationResultMeaning
create()Result[Trie, AllocError]Create an empty set with a permanent root.
length(&tree)i64Count stored keys, including the empty string if present.
insert(&mut tree, key)Result[bool, AllocError]True for a new key; duplicates return False.
has(&tree, key)boolTest for a complete stored key, not just a path.
remove(&mut tree, key)boolDelete a key and prune unused suffix nodes; missing keys return False.
has_prefix(&tree, prefix)boolWhether any stored key starts with the prefix.
with_prefix(&tree, prefix)Result[list[str], AllocError]All matching keys in scalar-lexicographic order, including the prefix itself if stored.
ordered(&tree)Result[list[str], AllocError]All keys; equivalent to with_prefix(&tree, "").
longest_prefix(&tree, text)Option[str]The longest stored key that prefixes the text.
valid(&tree)Result[bool, AllocError]Check structure, terminals, pruning, and free-list membership.

All key, prefix, and text parameters borrow &str; the implementation keeps immutable string handles only when needed. Exact lookup, prefix existence, longest-prefix matching, and deletion allocate nothing. Insertion can allocate node storage; enumeration allocates its result list; validation allocates visitation marks and a worklist.

For an empty set, has_prefix(&tree, "") is False, enumeration returns an empty list, and longest_prefix returns Nothing. If "" is stored, longest-prefix matching returns at least Some(""), even when no nonempty key matches. Removing a key does not remove its longer descendants or shorter stored prefixes.

Unicode and ownership

Matching is exact and case-sensitive. There is no normalization, case folding, locale collation, or grapheme-cluster segmentation. "é" and "e\u{301}" follow different paths. A combining mark is its own edge; an emoji outside the basic multilingual plane is one scalar, not several byte edges. Embedded NUL and the maximum Unicode scalar are accepted like other valid string content.

Terminal nodes retain full keys as immutable str handles, so enumeration need not rebuild each string from its path. Returned strings remain valid after deletion, slot reuse, or dropping the trie. They share immutable storage rather than copying key bytes; an inserted heap-backed key can therefore keep that storage alive. See text storage and iteration.

Validation

The model driver compares the trie with a sorted-list set and independent prefix/longest-prefix scans. It checks 14,400 pairs of five-key insertion/deletion permutations, duplicates, missing removals, complete pruning and refill, plus 8,000 mixed operations over Unicode, case-sensitive, empty, NUL-containing, and prefix-related keys. A high-fanout case inserts 256 distinct scalar edges at both root and interior levels in reverse order, then alternates first/middle/last sibling deletion. A 10,000-scalar key checks returned strings after the original key, query, and trie owners are destroyed, with same-sized allocation pressure before comparison against independently rebuilt text.

The integration tests additionally sweep insertion and prefix-result allocation failures, retry failed insertion, disable allocation during lookup/pruning/reuse, and exercise a 100,000-scalar key under a 256 KiB native stack on Linux:

cargo test --locked --features runtime-checks --test integration test_data_structure_patterns::trie

Read trie storage, updates, and costs for the implementation and its limits. Compare the balanced search trees when ordering alone, rather than shared-prefix retrieval, is the main requirement.

Trie storage, updates, and costs

The runnable trie uses a private list[Node]. Each node contains a one-scalar edge label, a parent ID, first-child and next-sibling IDs, an optional stored key, and free-list bookkeeping. The root occupies slot zero and has no edge label. -1 means no link, never a valid index to dereference.

A parent’s children form a singly linked list ordered by scalar label. To find an edge, scan siblings until its label matches or exceeds the desired scalar. This keeps storage and rewiring simple, but costs a linear scan in the parent’s fanout; it is not the constant-time edge lookup sometimes assumed for tries.

One owner controls all nodes. IDs borrow nothing, and each mutation updates one field at a time. Backing-list growth can move rows without changing IDs. Parent links support upward pruning and iterative traversal without stored references or a stack of borrowed nodes. No node ID escapes the module, so slot reuse needs no public generation or cross-collection identity scheme.

Insert without damaging the trie on allocation failure

Insertion makes two streaming passes over the input scalars:

  1. Follow existing edges until the first missing edge, then count the remaining missing suffix nodes. An existing terminal returns False immediately.
  2. After storage preparation succeeds, walk again, insert the missing edges in sibling order, and set the final node’s stored key.

Preparation appends empty nodes to a detached free pool until it has enough slots for the entire missing suffix. Failure may retain spare capacity and free nodes, but cannot change reachable links, terminals, or key count. A retry can reuse successful partial preparation. Taking slots, rewiring, and copying an immutable string handle allocate nothing once preparation succeeds.

Inserting the empty key, or marking an already-existing prefix node terminal, needs no new nodes. No repeated key[index] scanning is used: string iteration visits Unicode scalars directly.

Remove and prune without allocating

Locate the terminal; missing paths and nonterminal prefixes return False. Clear its stored key, then walk upward while the current node is a nonterminal leaf. Unlink that node from its parent’s child list, clear its fields, and put its slot on the free list. Stop at the permanent root or a node still needed by another key.

For example, removing "car" leaves the path for "cart". Removing "cart" afterward may prune the r and t suffix, but must preserve the shared ca path when "cat" is stored. Removing "" changes only the root’s terminal. No backing-list entry is removed or shifted.

Prefix and longest-prefix traversal

Prefix enumeration first locates the prefix node. It then performs iterative preorder over that subtree: emit the terminal before visiting ordered children, use sibling links to advance, and parent links to backtrack. Stop before following the starting node’s sibling. This produces lexicographically ordered keys without sorting or rebuilding text, and without crossing into unrelated prefixes. The empty prefix starts at the root and visits the whole trie.

Longest-prefix matching follows the query’s scalar edges while remembering the last terminal encountered. It returns that terminal’s stored string handle, not a newly allocated slice of the query. The root terminal naturally handles an empty-string fallback.

All production operations are iterative. Trie depth follows key length, so the logarithmic-height stack bound of a balanced search tree would not be sufficient here. The validator also uses an explicit heap worklist, not recursion.

Complexity and retained memory

Let m be the input’s scalar length and d the maximum fanout encountered. Membership, prefix existence, longest-prefix lookup, and the tree work of insertion/removal are bounded by O(m × d), plus streaming the input. Pruning also scans affected sibling lists. A missing-suffix insert may prepare O(m) slots; occasional backing-list growth can move all retained rows, so it is not strictly bounded by key length on each call.

After locating a prefix, enumeration takes O(v + k) work for v visited nodes and k emitted handles, plus result-list allocation. Its traversal state uses constant native stack and no auxiliary traversal list. Output is unbounded: callers asking for the empty prefix collect the whole set.

Storage follows the historical peak number of scalar-prefix nodes, including unused slots from partial preparation, rather than the current number of keys. Deletion releases stored string handles and clears labels in pruned nodes but retains node capacity for reuse. Terminal keys also retain their string storage; this example is not a compressed radix trie or a minimal dictionary automaton.

valid allocates marks/workspace over retained storage. It checks sorted unique sibling labels, valid live IDs, parent consistency, reachability without cycles or duplicate edges, no unpruned nonterminal leaves, terminal paths and key count, and free/live separation. It then searches each stored key to verify its path, so its cost includes all those searches; it is not a constant-time health check or a parser for hostile serialized state.

Deliberate limits

This is a set, not a generic key/value map. There are no result limits, lazy cursors, wildcard/fuzzy search, compressed edges, persistence, concurrency, or normalization policy. A sorted child vector, dictionary, or radix representation could trade allocation and implementation complexity for faster high-fanout queries; measure a workload before choosing one. No new standard-library contract or language feature is needed for this example.

Collection operation examples

Use these examples when a program needs a particular collection operation. Start with Collections, ownership, and borrowing for ownership and borrowing.

Handle collection allocation failures

list[T]() creates an empty list and returns Result[list[T], AllocError]. Sets and dictionaries work the same way: set[T]() and dict[K, V]().

If you know how many entries you need, use with_capacity(n). The collection starts empty and has space for at least n entries, including any hash table. Here construction and insertion both propagate errors to the caller:

def answers() -> Result[list[i64], AllocError]:
    mut values = list[i64].with_capacity(2)?
    values.append(21)?
    values.append(42)?
    Ok(values)

def main() -> Result[(), IoError]:
    match answers():
        case Ok(values):
            print(values)?
        case Err(AllocError.OutOfMemory):
            print("not enough memory")?
        case Err(AllocError.CapacityOverflow):
            print("requested capacity is too large")?
    Ok(())
[21, 42]

On failure, construction reclaims any memory it already obtained. Type aliases work too: after type Numbers = list[i64], you can write Numbers().

append for a list, add for a set, and insert for a dictionary each return Result[(), AllocError]. reserve(n) reserves space for at least n additional entries; it is available on all three collection types and returns the same result type.

def add_answers(values: &mut list[i64]) -> Result[(), AllocError]:
    values.reserve(2)?
    values.append(21)?
    values.append(42)?
    Ok(())

def main() -> Result[(), AllocError | IoError]:
    mut values: list[i64] = []?
    match add_answers(&mut values):
        case Ok(done):
            print(values)?
        case Err(AllocError.OutOfMemory):
            print("not enough memory")?
        case Err(AllocError.CapacityOverflow):
            print("requested capacity is too large")?
    Ok(())
[21, 42]

AllocError has two variants, neither of which allocates. OutOfMemory means the allocator rejected a request. CapacityOverflow means the requested size cannot be represented; a negative reservation is also a capacity error. You can handle a failure without terminating the program:

def main() -> Result[(), AllocError | IoError]:
    mut values = [1, 2]?
    match values.reserve(-1):
        case Ok(done):
            print("reserved")?
        case Err(error):
            print(error)?
    print(values)?
    Ok(())
AllocError.CapacityOverflow
[1, 2]

On failure the collection keeps its existing contents. An insertion consumes its arguments even when it fails, so an owned argument is cleaned up rather than returned to you. Reserve first if you want to keep an item until storage is ready. Successful reservation covers collection storage, including a dictionary or set’s hash table; constructing the elements themselves may still allocate.

These methods require a mutable receiver. Dictionary insert(key, value) replaces an existing value or adds a new entry. Adding an existing set element or replacing a dictionary entry needs no storage growth.

Handle allocating arguments too: copy([1, 2]?)? checks both construction and copying. Explicit .unwrap() chooses to terminate if a result is an error.

Empty a collection and reuse its storage

Lists, dictionaries, and sets have clear() -> (). The method requires mutable access, drops the contents, and retains capacity for later insertions:

def main() -> Result[(), AllocError | IoError]:
    mut items = [10, 20]?
    items.clear()
    print(items)?
    items.append(30)?
    print(items)?
    mut scores = {"Ada": 10}?
    scores.clear()
    print(scores)?
    Ok(())
[]
[30]
{}

Clearing itself allocates no storage. Owned elements are destroyed before the method returns, in list order or dictionary insertion order; custom destructors may have their own effects and allocations. Clearing an empty collection is fine.

Collection literals

Collection literals directly return Result[collection, AllocError]:

def build() -> Result[list[list[i64]], AllocError]:
    [[1, 2]?, [3]?]

def main() -> Result[(), AllocError | IoError]:
    print({"answer": 42}?)?
    print(build()?)?
    Ok(())
{"answer": 42}
[[1, 2], [3]]

Construction stops at its first allocation failure, drops the partial collection, and skips later entries. The nested ? operations extract each inner list and propagate its failure to build. Empty displays need context: values: Result[list[i64], AllocError] = [], or values: list[i64] = []? inside a function returning Result[..., AllocError].

Byte keys and iterable conversion

Byte literals and tuples of hashable values can be dictionary keys or set members. Call-style list constructors can collect text scalars or literal bytes:

def main() -> Result[(), AllocError | IoError]:
    names = {b"GIF8": "gif"}?
    seen = {(b"tag", 1), (b"tag", 1)}?
    print(names[b"GIF8"], len(seen))?
    print(list("aé")?)?
    print(list[u8](b"AB")?)?
    Ok(())
gif 1
["a", "é"]
[65, 66]

The .from(...) methods accept collections, ranges, and generators, but not text or bytes. See collection conversion for the supported constructor forms.

Copy without losing the source on failure

copy(value) returns Result[T, AllocError]. It observes the source, so your original value remains usable whether copying succeeds or fails. Use ? to keep the success path short:

def extended(source: &list[i64]) -> Result[list[i64], AllocError]:
    mut result = copy(source)?
    result.append(30)?
    Ok(result)

def main() -> Result[(), AllocError | IoError]:
    original = [10, 20]?
    print(extended(&original)?)?
    print(original)?
    Ok(())
[10, 20, 30]
[10, 20]

This also works for nested collections, ordinary classes, and enum payloads. If copying a later field or element fails, the partial copy is cleaned up and the original remains unchanged. Scalars and immutable values, including strings, need no allocation to copy; their existing immutable storage can be shared.

Generators and classes with custom cleanup cannot be copied, including when nested in another value. The operation covers duplication itself. In copy([1, 2]?)?, the first ? handles construction and the second handles copying.

Take dictionary snapshots with recoverable allocation

keys() and values() build new lists in dictionary insertion order, returning Result[list[T], AllocError]:

def names(scores: &dict[str, i64]) -> Result[list[str], AllocError]:
    result = scores.keys_snapshot()?
    Ok(result)

def main() -> Result[(), AllocError | IoError]:
    mut scores = {"Ada": 10, "Bea": 20}?
    saved = scores.values_snapshot()?
    scores.insert("Ada", 30)?
    print(names(&scores)?)?
    print(saved)?
    print(scores)?
    Ok(())
["Ada", "Bea"]
[10, 20]
{"Ada": 30, "Bea": 20}

For numbers, strings, and other immutable values, the dictionary remains usable after either outcome. String storage is shared, so the new list does not copy string contents. The snapshot remains valid after the source changes or leaves scope. An empty dictionary produces Ok([]) if its new list header can be allocated.

For owned values such as lists or classes, values() requires an owned temporary. Request duplication explicitly to keep the original:

def rows(data: &dict[str, list[i64]]) -> Result[list[list[i64]], AllocError]:
    copy(data)?.values_snapshot()

def main() -> Result[(), AllocError | IoError]:
    data = {"first": [1, 2]?, "second": [3]?}?
    print(rows(&data)?)?
    print(data)?
    Ok(())
[[1, 2], [3]]
{"first": [1, 2], "second": [3]}

Calling make_dictionary().values_snapshot() instead transfers owned values from that temporary into the list. If allocation fails, the temporary and its values are cleaned up. keys_snapshot() never takes the dictionary’s values, so it works on borrowed dictionaries regardless of their value type.

Both operations reserve all list storage before retaining or transferring any elements. They report OutOfMemory or CapacityOverflow through AllocError.

Update a dictionary by transferring another dictionary

destination.update(source) returns Result[(), AllocError], replacing existing values and adding new entries. Existing keys keep their positions; new keys appear in source insertion order:

def merge(destination: &mut dict[str, i64], source: dict[str, i64]) -> Result[(), AllocError]:
    destination.update(source)

def main() -> Result[(), AllocError | IoError]:
    mut scores = {"Ada": 10, "Bea": 20}?
    changes = {"Bea": 25, "Cam": 30, "Ada": 15}?
    merge(&mut scores, changes)?
    print(scores)?
    Ok(())
{"Ada": 15, "Bea": 25, "Cam": 30}

The source is consumed on both outcomes. On allocation failure, the destination stays unchanged and source owners are dropped. Use destination.update(copy(source)?) to keep the source. Storage for every new entry is reserved before any value is replaced. Old destination values are dropped during successful replacement; their destructors keep their own effects. Updating only existing keys needs no new runtime storage. Only same-typed dictionaries are accepted; pair iterables and keyword arguments are deferred.

Extend a list by transferring another list

items.extend(other) returns Result[(), AllocError]. It moves all elements from another list of the same type after reserving space:

def combined() -> Result[list[i64], AllocError]:
    mut items = list[i64]()?
    more = [10, 20, 30]?
    items.extend(more)?
    Ok(items)

def main() -> Result[(), AllocError | IoError]:
    print(combined()?)?
    Ok(())
[10, 20, 30]

The source is consumed even when reservation fails; its elements are then cleaned up and the destination stays unchanged. To retain a source, explicitly write items.extend(copy(source)?). References and general iterables are not accepted as sources yet. The destination needs mutable access. Existing capacity is reused, and moving elements does not clone or allocate their contents. The source literal also returns an allocation result; its ? propagates that failure before extend runs.

Search and count list elements

For lists of integers, floats, booleans, or strings, these queries borrow their inputs and allocate nothing:

def main() -> Result[(), AllocError | IoError]:
    names = ["Ada", "Lin", "Ada"]?
    print(names.count("Ada"))?
    print(names.find("Ada"))?
    print(names.rfind("Ada"))?
    print(names.find("missing"))?
    Ok(())
2
Option[i64].Some(0)
Option[i64].Some(2)
Option[i64].Nothing

Searches return optional zero-based positions. The query must match the element type exactly. Float comparisons follow IEEE rules: NaN never matches, and positive and negative zero match each other. Searching lists of aggregates is not yet supported.

Take a list slice

A slice items[start:stop:step] returns a new list inside a Result. Start is included and stop is excluded. Negative bounds count from the end, omitted bounds run to the ends, a negative step walks backwards, and bounds outside the list clamp to its ends:

def middle(items: &list[i64]) -> Result[list[i64], AllocError]:
    items[1:-1]

def main() -> Result[(), AllocError | IoError]:
    items = [10, 20, 30, 40]?
    print(middle(&items)?)?
    print(items[-100:100]?)?
    print(items[3:1]?)?
    print(items[::-2]?)?
    print(items)?
    Ok(())
[20, 30]
[10, 20, 30, 40]
[]
[40, 20]
[10, 20, 30, 40]

Bounds must be i64, and the step is a NonZero[i64]. Even an empty result can fail to allocate its list header; use ? or match to handle AllocError.

The slice copies each selected element with copy(), so nested lists are duplicated and the source keeps its elements. Element types that cannot be copied, such as std.bytes.Bytes, are rejected. For those, call items.slice(start, stop) on an owned temporary, such as a list returned by a function: it transfers the selected elements and drops the rest. Use copy(items)?.slice(start, stop) only when the elements can be copied anyway. If allocation fails, the temporary is cleaned up in full; a borrowed source remains unchanged.

Preserve different error causes

Declare a sum of the errors an operation can return. Each ? injects its original error into that sum, preserving the variant and its payload:

type BuildError = ParseError | AllocError | IoError | OverflowError[u8]

def work(text: str) -> Result[list[u8], BuildError]:
    n = u8.parse(text)?               # ParseError
    values = [n, n + 1u8]?           # AllocError and OverflowError[u8]
    print("built values")?          # IoError
    Ok(values)

def main() -> Result[(), AllocError | IoError]:
    print(str.repr(work("41"))?)?
    match work("invalid"):
        case Ok(values):
            print(values)?
        case Err(error):
            match error:
                case cause: ParseError:
                    print(cause == ParseError.Invalid)?
                case cause: AllocError:
                    print(cause)?
                case cause: IoError:
                    print(cause)?
                case cause: OverflowError[u8]:
                    print(cause)?
    Ok(())

built values
Result[list[u8], AllocError | IoError | OverflowError[u8] | ParseError].Ok([41, 42])
True

The alias names the combination; it does not wrap the errors or allocate. Matching a member gives the original concrete value. ? also widens an error sum into a larger declared sum without dropping its cause. Normal scope cleanup runs on the error path.

A main returning Err reports the error on standard error and exits with status one. Use ? for propagation and match for recovery. .unwrap() is an explicit choice to terminate immediately on Err or Nothing, without normal scope cleanup. Neither sums nor ? catch runtime traps or turn Option.Nothing into an error.

Builtin Failure discards the original cause. It is a last resort, for when no precise error can be written; these errors combine without it. The error tutorial describes its behavior.

Choosing numeric types in expressions

Type arguments go before the call: range[u8](8). A collection annotation or function return type can also guide a directly written range comprehension.

def squares() -> Result[list[u8], AllocError | OverflowError[u8] | ZeroDivisionError]:
    [n * n for n in range(8) if n % 2 == 0]

def main() -> Result[(), AllocError | IoError | OverflowError[u8] | OverflowError[u16] | ZeroDivisionError]:
    print(squares()?)?
    print([n * n for n in range[u8](8) if n % 2 == 0]?)?
    small: list[u16] = [n + 1 for n in range(3)]?
    print(small)?
    print(list(range[u8](5, 0, -2))?)?
    fraction: f32 = 3.5
    print(fraction * 2)?
    Ok(())
[0, 4, 16, 36]
[0, 4, 16, 36]
[1, 2, 3]
[5, 3, 1]
7.0

Context guides unsuffixed literals, never changes an existing value’s type. x: u8 = 256 fails, as does assigning an i64 binding to a u8 binding. Explicit suffixes remain authoritative. A range’s bounds must fit its element type; range[u8](256) therefore fails even though its stop is exclusive.

Inference stays within straightforward expressions. It does not infer a range type backward through filters, arbitrary calls, or a previously stored range. Use explicit range[T] in those cases. There is no silent numeric widening or narrowing.

Find and validate collection values

Option[T] has two variants: Some(T) and Nothing. Some, Nothing, Ok, and Err are built in and need no type prefix. Nothing belongs to one concrete option type; it is not a universal null.

def first_positive(values: list[i64]) -> Option[i64]:
    for value in values:
        if value > 0:
            return Some(value)
    Nothing

def main() -> Result[(), AllocError | IoError]:
    for values in [[-1, 0]?, [-1, 42]?]?:
        match first_positive(values):
            case Some(value):
                print(value)?
            case Nothing:
                print("not found")?
    Ok(())
not found
42

Use Result[T, E] when the absent result has an explanation. Its variants are Ok(T) and Err(E). This example intentionally maps arithmetic errors to explanatory strings; a numeric API can instead preserve the builtin error types. The return signature supplies both types:

def divide(left: i64, right: i64) -> Result[i64, str]:
    if right == 0:
        return Err("division by zero")
    result = checked:
        left // right
    match result:
        case Ok(value):
            Ok(value)
        case Err(OverflowError(_)):
            Err("quotient out of range")
        case Err(ZeroDivisionError()):
            Err("division by zero")

def main() -> Result[(), AllocError | IoError]:
    for result in [divide(8, 2), divide(8, 0)]?:
        match result:
            case Ok(value):
                print(value)?
            case Err(message):
                print(message)?
    Ok(())
4
division by zero

Validate text in a collection

For an operation that can fail but has no success data, return Result[(), E] and construct success with Ok(()):

def validate(name: str) -> Result[(), str]:
    if len(name) == 0:
        return Err("name is empty")
    Ok(())

def main() -> Result[(), AllocError | IoError]:
    for result in [validate("Plenty"), validate("")]?:
        match result:
            case Ok(_):
                print("valid")?
            case Err(message):
                print(message)?
    Ok(())
valid
name is empty

Unit is a real success payload, distinct from the absence represented by Nothing. Option[()] also works. Standalone unit bindings and unit parameters remain unsupported. User-defined enum variants still use their enum’s prefix, even if a variant happens to be named Ok or Some.

Program entrypoints and native compilation

Running and linking

Plenty compiles ahead of time. Running a file compiles a temporary native executable, runs it, and removes it afterward. Both running and compiling require a cc-compatible linker driver. The default is cc on PATH; add --linker clang or --linker /path/to/driver to select another. Repeat --link-arg ARG for extra native link arguments. Checking needs no linker. The Rust runtime is packaged with Plenty; compiling Plenty programs needs neither cargo nor rustc.

Entrypoints and exit status

Execution starts by calling main once. Every binary application must declare main with no parameters. It can return (), i32, Result[(), E], or Result[i32, E]. Ok(()) produces exit status zero and Ok(status) uses that status. An Err is reported on standard error, then dropped, and the program exits with status one. For example, a main that returns the error from reading a missing file prints error: main returned IoError.System(2): No such file or directory. The error is printed once, when main returns it, not at each ? on the way. The () form finishes successfully with exit status zero. The i32 form returns a process exit status: zero means success, and nonzero means failure. Use an i32 literal such as 0i32 or 1i32; return annotations also guide unsuffixed literals.

Keep executable statements inside functions. Module scope contains def, class, enum, and type declarations, plus imports. Bindings inside main are local to it. The modules part explains imports and pub visibility.

Owned locals are dropped before the program exits, including when main returns a nonzero status. Operating systems limit the range of observable exit statuses; use small nonnegative codes for portable command-line programs.

You can also choose the successful process status explicitly:

def main() -> Result[i32, IoError]:
    print("ready")?
    Ok(0i32)
ready

Calling a function at module scope is an error, even if main is also declared:

def main() -> ():
    print("ready").unwrap()

main()
executable statements are not allowed at module scope

Keeping an executable or linking externally

Use --compile when you want to keep the executable and run it repeatedly without compiling again. The executable does not need Plenty installed.

For an external build system, use plenty --emit-object hello.plenty -o hello.o and plenty --emit-runtime runtime. Link the object with runtime/libplenty_runtime.a followed by the arguments in runtime/link-args.txt. These two commands do not invoke a linker. They emit a complete application, including its imports; ordinary pub functions are not C exports. Native compilation currently supports x86_64-unknown-linux-gnu. Use plenty --print-target to inspect the packaged target. An explicit --target TRIPLE must match it; choosing a linker does not enable cross-compilation.

Reverse a list in place

reverse() mutates a list through a mut binding or &mut reference and returns (). It allocates nothing and moves no elements out of the list:

def flip(items: &mut list[i64]) -> ():
    items.reverse()

def main() -> Result[(), AllocError | IoError]:
    mut items = [10, 20, 30]?
    flip(&mut items)
    print(items)?
    items.reverse()
    print(items)?
    Ok(())
[30, 20, 10]
[10, 20, 30]

Owned elements such as classes and nested lists work too. Reversing does not copy or destroy them; eventual list cleanup follows their new order. To keep the original order separately, explicitly copy the list first.

Sort a list and exchange elements

sort() orders a list in place without allocating. Tuples sort by their first component, so sorting (start, stop) spans puts overlapping spans next to each other. Merging them is then one pass:

def merge(spans: &mut list[(i64, i64)]) -> Result[list[(i64, i64)], AllocError]:
    spans.sort()
    mut merged: list[(i64, i64)] = []?
    for i in range(len(spans)):
        start, stop = spans[i]
        if len(merged) > 0 and start <= merged[-1][1]:
            if stop > merged[-1][1]:
                merged[-1] = (merged[-1][0], stop)
        else:
            merged.append((start, stop))?
    Ok(merged)

def main() -> Result[(), AllocError | IoError]:
    mut spans = [(8, 10), (1, 3), (15, 18), (2, 6)]?
    print(merge(&mut spans)?)?
    print(spans)?
    Ok(())
[(1, 6), (8, 10), (15, 18)]
[(1, 3), (2, 6), (8, 10), (15, 18)]

swap(i, j) exchanges two elements of any type in place. Here it moves the row with the largest leading value to the top, as elimination with pivoting does, without copying a row:

def pivot(rows: &mut list[list[i64]]) -> ():
    mut best = 0
    for i in range(1, len(rows)):
        if rows[i][0] > rows[best][0]:
            best = i
    rows.swap(0, best)

def main() -> Result[(), AllocError | IoError]:
    mut rows = [[1, 2]?, [7, 8]?, [4, 5]?]?
    pivot(&mut rows)
    print(rows)?
    Ok(())
[[7, 8], [1, 2], [4, 5]]

Copyable elements can also be exchanged by assigning several targets: items[i], items[j] = items[j], items[i].

Combine sets without consuming them

union returns an independent set inside a Result. Both inputs are borrowed:

def merged(a: &set[i64], b: &set[i64]) -> Result[set[i64], AllocError]:
    a.union(b)

def main() -> Result[(), AllocError | IoError]:
    a = {1, 2}?
    b = {2, 3}?
    match merged(&a, &b):
        case Ok(values):
            print(len(values))?
            print(1 in values and 3 in values)?
        case Err(error):
            print(error)?
    print(len(a))?
    Ok(())
3
True
2

An allocation failure preserves the inputs. Set iteration order is unspecified.

Find common members with intersection:

def main() -> Result[(), AllocError | IoError]:
    match {1, 2}?.intersection({2, 3}?):
        case Ok(common):
            print(len(common))?
            print(2 in common)?
        case Err(error):
            print(error)?
    Ok(())
1
True

Use difference to remove another set’s members from a new result:

def main() -> Result[(), AllocError | IoError]:
    wanted = {1, 2, 3}?
    completed = {2, 3, 4}?
    match wanted.difference(completed):
        case Ok(remaining):
            print(len(remaining))?
            print(1 in remaining)?
        case Err(error):
            print(error)?
    print(len(wanted))?
    Ok(())
1
True
3

symmetric_difference keeps members found in exactly one of the two inputs:

def main() -> Result[(), AllocError | IoError]:
    match {1, 2}?.symmetric_difference({2, 3}?):
        case Ok(changed):
            print(len(changed))?
            print(1 in changed and 3 in changed)?
            print(2 in changed)?
        case Err(error):
            print(error)?
    Ok(())
2
True
False

Compare sets

Set relationships borrow both sets and allocate nothing:

def main() -> Result[(), AllocError | IoError]:
    needed = {"read"}?
    available = {"read", "write"}?
    print(needed.issubset(available))?
    print(available.issuperset(needed))?
    print(needed.isdisjoint({"write"}?))?
    print(len(needed))?
    Ok(())
True
True
True
1

Both operands must have the same set type. Empty sets are subsets of every set and disjoint from every set.

Filter a set in place

intersection_update retains common members in a mutable set. It borrows the other set, returns unit, and reuses existing storage without allocating:

def main() -> Result[(), AllocError | IoError]:
    mut selected = {1, 2, 3}?
    allowed = {2, 3, 4}?
    selected.intersection_update(allowed)
    print(len(selected))?
    print(1 in selected)?
    print(2 in selected and 3 in selected)?
    print(len(allowed))?
    Ok(())
2
False
True
3

The source must be a different set: it stays borrowed while the destination is mutated. Use intersection when you want an independent result instead.

Use difference_update to remove the other set’s members in place:

def main() -> Result[(), AllocError | IoError]:
    mut pending = {1, 2, 3}?
    done = {2, 3, 4}?
    pending.difference_update(done)
    print(len(pending))?
    print(1 in pending)?
    print(len(done))?
    Ok(())
1
True
3

This also allocates nothing and keeps the source usable. To remove every member without needing another set, use clear().

Remove set members

Use values.discard(value) to remove a member from a mutable set. It returns True if the member was present and removed, or False if it was absent:

def main() -> Result[(), AllocError | IoError]:
    mut names = {"Ada", "Bea"}?
    name = "Ada"
    print(names.discard(name))?
    print(names.discard(name))?
    print(name)?
    print("Bea" in names)?
    print(len(names))?
    names.add(name)?
    print(len(names))?
    Ok(())
True
False
Ada
True
1
2

The argument is observed, so name remains usable. Discarding a stored zero, False, or empty string still returns True. Removal allocates nothing and preserves capacity for reuse. Like dictionary removal, it has expected constant cost for well-distributed keys, apart from hashing and cleanup; heavy hash collisions can still slow it down. Use an exclusive reference when removing members through a function parameter.

Update a set by transferring another set

Sets also support update(source) -> Result[(), AllocError]. Members already present are kept, and missing members transfer from the source:

def main() -> Result[(), AllocError | IoError]:
    mut names = {"Ada", "Bea"}?
    more = {"Bea", "Cam"}?
    names.update(more)?
    print(len(names))?
    print("Ada" in names and "Bea" in names and "Cam" in names)?
    Ok(())
3
True

The source must be a set of the same type and is consumed on both outcomes. Use names.update(copy(source)?) to preserve it. Failed reservation leaves the destination unchanged and cleans up the consumed source. Duplicate-only and empty sources need no new runtime storage; existing capacity can cover new members too. Set iteration order remains unspecified.

Cleanup before a tail call

A function that ends with a call makes a tail call. Plenty evaluates the arguments, cleans up the function’s remaining values, and only then enters the called function:

class Resource:
    name: str

    def __del__(self) -> ():
        print(("release " + self.name).unwrap()).unwrap()

def finish(total: i64) -> Result[i64, IoError | OverflowError[i64]]:
    print("finish")?
    Ok(total)

def step(total: i64) -> Result[i64, IoError | OverflowError[i64]]:
    scratch = Resource("scratch")
    finish(total + 1)

def main() -> Result[(), AllocError | IoError | OverflowError[i64]]:
    print(step(41)?)?
    Ok(())
release scratch
finish
42

step and finish deliberately share a Result type, including the arithmetic error that can originate in step, so forwarding the result remains a tail call. scratch is released before finish runs. A value passed as an argument moves to the called function, which cleans it up instead. Tail recursion therefore keeps its constant stack use when each step owns a resource. A call that borrows one of the function’s values, such as inspect(&scratch) or a method call on it, cannot release that value first: it runs as an ordinary call and cleanup follows. A reference the function received as a parameter refers to a value outside the function, so passing it on is still a tail call.

The destructor’s printing is only an observation aid for this cleanup example; its explicit unwrap() terminates if that diagnostic output fails. Ordinary work, including finish, propagates its errors instead.

Repeat work with tail recursion

A tail-recursive function can repeat work without growing the call stack. Ordinary range loops are usually the first choice for repetition. This page explains recursion when a function naturally calls itself.

def sum_to(n: i64, total: i64) -> Result[i64, OverflowError[i64]]:
    if n <= 0:
        return Ok(total)
    sum_to(n - 1, total + n)

def main() -> Result[(), IoError | OverflowError[i64]]:
    print(sum_to(100, 0)?)?
    Ok(())
5050

The recursive call is the last operation on that path. Plenty reuses its call frame in native code. An explicit return sum_to(n - 1, total + n) works too. In contrast, 1 + recurse(...) still has addition to do after the call and is not a tail call.

Returning a range also works: the result’s storage belongs to the original caller and stays alive throughout the recursion.

def countdown(n: i64) -> Result[range[i64], OverflowError[i64]]:
    if n == 0:
        return Ok(range(2, 5))
    countdown(n - 1)

def main() -> Result[(), IoError | OverflowError[i64]]:
    print(countdown(1000)?)?
    Ok(())
range(2, 5, 1)

Passing an owned range, record, or generator as an argument still needs an ordinary call and consumes native stack space. A direct recursive tail call with such an argument is rejected as described below. Returning one alone does not impose that limit.

A direct recursive call in tail position must use a native tail transfer. This also applies to mutual recursion: first calls second, which calls back into first. Plenty reports an error when it cannot remove the caller’s frame safely. For example, the following call borrows a local that needs that frame to stay alive:

def walk(value: &i64, remaining: i64) -> Result[i64, OverflowError[i64]]:
    if remaining == 0:
        return Ok(*value)
    child = remaining - 1
    walk(child, remaining - 1)

def main() -> ():
    pass
recursive call to `walk` is in tail position but cannot be a tail call: the reference argument borrows local `child`

Pass a reference borrowed from the caller, carry owned scalar state, or use a loop when the next step needs local storage. Non-tail recursion still uses the native stack. Calls through function values are outside this direct-recursion diagnostic; using a callback does not promise bounded stack use.

Text and Unicode values in a collection

Strings store explicit lengths, so an embedded NUL does not end the text. Indexing returns one Unicode scalar as a str; negative indices count from the end. Each scalar fits inline in the string value, so indexing does not allocate. General text can share immutable storage.

def main() -> Result[(), AllocError | IoError]:
    text = "é\0😀"
    print(len(text))?
    print(text[-1])?
    print("\0" in text)?
    print([text]?)?
    Ok(())
3
😀
True
["é\0😀"]

Text and file examples

Use these examples to look up text methods and advanced file operations. The text I/O reference lists signatures and platform limits.

Text methods borrow their inputs. Queries that return scalars or Option allocate nothing; operations producing independent strings or collections report allocation failure in a Result.

Bounded stream reads

Read a bounded number of characters with read(count). The count is Unicode characters, not UTF-8 bytes. Zero consumes nothing; a negative count reads to EOF.

def demo() -> Result[(), IoError]:
    write_text("bounded.txt", "é🦀hello")?
    with open("bounded.txt")? as file:
        print(file.read(2)?)?
        print(file.read()?)?
    Ok(())
def main() -> Result[(), IoError]:
    demo()?
    Ok(())
é🦀
hello

readline(count) limits a line read in the same way. A long line can arrive in several pieces; a negative count reads the rest of the line.

def demo() -> Result[(), IoError]:
    write_text("lines.txt", "abcd\nnext")?
    with open("lines.txt")? as file:
        print(file.readline(2)?)?
        print(file.readline(2)?)?
        print(file.readline(2)? == "\n")?
    Ok(())
def main() -> Result[(), IoError]:
    demo()?
    Ok(())
ab
cd
True

Checking stream capabilities

Use readable() and writable() to inspect an open file through a shared reference. Closed files return an error; closed itself remains an infallible property.

def demo() -> Result[(), IoError]:
    with open("capabilities.txt", "w")? as file:
        print(file.readable()?)?
        print(file.writable()?)?
    Ok(())
def main() -> Result[(), IoError]:
    demo()?
    Ok(())
False
True

Observe file closing

Files close automatically when dropped. Call close() explicitly to observe a close error. A closed file remains a valid value without an open handle.

Run this in a scratch directory: it replaces plenty-handle.txt.

def inspect_file() -> Result[(), IoError]:
    mut file = open("plenty-handle.txt", "w")?
    print(file.closed)?
    file.close()?
    print(file.closed)?
    file.close()?
    Ok(())

def main() -> Result[(), IoError]:
    inspect_file()?
    Ok(())
False
True

Closing twice succeeds. Automatic close cannot report errors or promise durable writes.

Creating a file without overwriting

Use mode "x" when an existing path should be an error. This check happens atomically during creation, so there is no separate existence-check race.

def demo() -> Result[(), IoError]:
    with open("new.txt", "x")? as file:
        file.write("first version")?
    print(read_text("new.txt")?)?
    Ok(())
def main() -> Result[(), IoError]:
    demo()?
    Ok(())
first version

Add + to an open mode to enable both reading and writing. r+ preserves an existing file, w+ truncates or creates, x+ creates exclusively, and a+ starts at EOF and always appends writes. Writes overwrite bytes at the current position; they do not insert characters. Use care with multibyte text.

def demo() -> Result[(), IoError]:
    write_text("update.txt", "hello")?
    with open("update.txt", "r+")? as file:
        file.read(1)?
        file.write("a")?
    print(read_text("update.txt")?)?
    Ok(())
def main() -> Result[(), IoError]:
    demo()?
    Ok(())
hallo

Saving and restoring text positions

tell() returns a u64 position cookie. Pass it back to seek() on the same file to resume reading there, or use seek(0u64) to rewind. Do not calculate with these values: they include newline state and are not byte or character counts. Saved positions are only meaningful while the file contents remain unchanged.

def demo() -> Result[(), IoError]:
    write_text("positions.txt", "one\r\ntwo")?
    with open("positions.txt")? as file:
        file.readline()?
        saved = file.tell()?
        print(file.read()?)?
        file.seek(saved)?
        print(file.read()?)?
    Ok(())
def main() -> Result[(), IoError]:
    demo()?
    Ok(())
two
two

truncate(size) resizes a writable file in bytes and returns its new length. With no size, it truncates at the current physical position. Neither form moves the cursor. A text-position cookie is not a byte size. Cutting through a multibyte character makes the file invalid UTF-8; a later read reports that error.

def demo() -> Result[(), IoError]:
    write_text("short.txt", "keep rest")?
    with open("short.txt", "r+")? as file:
        file.read(4)?
        print(file.truncate()?)?
    print(read_text("short.txt")?)?
    Ok(())
def main() -> Result[(), IoError]:
    demo()?
    Ok(())
4
keep

Flush and synchronize writes

write(text) returns a character count and preserves exact bytes, including newlines. flush() flushes runtime buffers; files are currently unbuffered. Use sync() when you need to request that the OS synchronize file contents and metadata. Closing or flushing alone does not make that request.

Run this in a scratch directory: it replaces plenty-stream.txt.

def write_example() -> Result[(), IoError]:
    with open("plenty-stream.txt", "w")? as file:
        print(file.write("hello\n")?)?
        file.flush()?
        file.sync()?
        file.close()?
    with open("plenty-stream.txt", "a")? as file:
        file.write("goodbye\n")?
    Ok(())

def main() -> Result[(), IoError]:
    write_example()?
    Ok(())
6

Explicit close lets you propagate its error. Context exit then closes the already closed owner harmlessly. Writing can leave a prefix after an OS error, and append calls are not guaranteed to be atomic records across processes. Synchronization still follows your filesystem’s durability rules.

Reading a collection of lines

readlines() collects the remaining lines into an independent list[str], keeping their translated newlines. At EOF it returns an empty list. Failures release the partial list, but may have consumed input. For large or untrusted files, bounded readline(count) lets you control memory use instead.

def demo() -> Result[list[str], IoError]:
    write_text("collection.txt", "first\r\n\nlast")?
    with open("collection.txt")? as file:
        return file.readlines()
def main() -> Result[(), IoError]:
    print(demo()?)?
    Ok(())
["first\n", "\n", "last"]

writelines(lines) writes a list[str] without consuming it. It inserts no newlines: include them in the strings when wanted. The operation returns Result[(), IoError]; an error may leave a partially written file.

def demo() -> Result[(), AllocError | IoError]:
    lines = ["one\n", "two\n"]?
    with open("written.txt", "w")? as file:
        file.writelines(lines)?
    print(len(lines))?
    with open("written.txt")? as file:
        print(file.readlines()?)?
    Ok(())
def main() -> Result[(), AllocError | IoError]:
    demo()?
    Ok(())
2
["one\n", "two\n"]

Classify text without allocating

isascii() accepts only ASCII characters, including controls and NUL. isspace() checks for nonempty text containing only Unicode whitespace:

def main() -> Result[(), IoError]:
    print("hello".isascii())?
    print("é".isascii())?
    print("".isascii())?
    print(" \t\n".isspace())?
    print(" x ".isspace())?
    print("".isspace())?
    Ok(())
True
False
True
True
False
False

Both queries borrow their receiver and allocate nothing. Whitespace means the same Unicode White_Space characters removed by strip; NUL and zero-width space do not count.

Remove an exact prefix or suffix

These methods remove one complete match at the chosen end. They return an independent string inside a Result, leaving the original usable:

def title(name: &str) -> Result[str, AllocError]:
    name.removeprefix("draft-")?.removesuffix(".txt")

def main() -> Result[(), AllocError | IoError]:
    name = "draft-notes.txt"
    print(title(&name)?)?
    print("abab".removeprefix("ab")?)?
    print("notes.txt".removesuffix(".csv")?)?
    Ok(())
notes
ab
notes.txt

Empty patterns do nothing. Each method can fail to allocate even when the contents stay unchanged. Use strip for surrounding whitespace instead.

Check a text prefix or suffix

startswith and endswith borrow strings and return bool without allocating. Matches are literal and case-sensitive; an empty prefix or suffix always matches.

def main() -> Result[(), IoError]:
    name = "report.plenty"
    print(name.startswith("report"))?
    print(name.endswith(".plenty"))?
    print(name.endswith(".PLENTY"))?
    print("".startswith(""))?
    Ok(())
True
True
False
True

Each method takes one string, including a reference. Bounds and lists or tuples of alternative patterns are not supported yet.

Find a substring

find and rfind return the first or last match as Option[i64]. Unlike Python’s -1 sentinel, a missing match is Nothing. Positions count Unicode scalars:

def main() -> Result[(), IoError]:
    text = "é🙂é🙂"
    print(text.find("🙂"))?
    print(text.rfind("🙂"))?
    print(text.find("missing"))?
    print(text.rfind(""))?
    Ok(())
Option[i64].Some(1)
Option[i64].Some(3)
Option[i64].Nothing
Option[i64].Some(4)

Both operations borrow their inputs and allocate nothing. Use match or ? in an Option-returning function. An empty needle matches at the beginning for find and at the end for rfind. Each takes one literal pattern; optional bounds and regexes are not supported.

Count occurrences

text.count(needle) returns an i64 without allocating. Matches are literal and do not overlap. An empty needle counts Unicode scalar boundaries:

def main() -> Result[(), IoError]:
    print("banana".count("ana"))?
    print("aaaaa".count("aa"))?
    print("é🙂".count(""))?
    print("".count(""))?
    print("text".count("missing"))?
    Ok(())
1
2
3
1
0

The method borrows both strings and takes exactly one pattern argument.

Trim surrounding whitespace

strip(), lstrip(), and rstrip() remove Unicode whitespace from both ends, the left end, or the right end. Each returns Result[str, AllocError]:

def main() -> Result[(), AllocError | IoError]:
    text = "  é🙂  "
    print(str.repr(text.strip())?)?
    print(str.repr(text.lstrip())?)?
    print(str.repr(text.rstrip())?)?
    print(str.repr(" \t\n".strip())?)?
    Ok(())
Result[str, AllocError].Ok("é🙂")
Result[str, AllocError].Ok("é🙂  ")
Result[str, AllocError].Ok("  é🙂")
Result[str, AllocError].Ok("")

They borrow the source and allocate only the final independent string. Empty and unchanged results still need that allocation. Interior whitespace is kept; NUL and zero-width space are not trimmed. These methods use Unicode White_Space and take no arguments; custom character sets are not supported yet.

Repeat text

text.repeat(count) accepts an i64 and returns Result[str, AllocError]. Zero and negative counts produce an empty string:

def main() -> Result[(), AllocError | IoError]:
    print(str.repr("é🙂".repeat(3))?)?
    print(str.repr("x".repeat(0))?)?
    print(str.repr("x".repeat(-2))?)?
    print(str.repr("x".repeat(9223372036854775807))?)?
    Ok(())
Result[str, AllocError].Ok("é🙂é🙂é🙂")
Result[str, AllocError].Ok("")
Result[str, AllocError].Ok("")
Result[str, AllocError].Err(AllocError.CapacityOverflow)

The source stays usable. The runtime checks the complete size before allocating one independent output string; even empty and single-copy outputs can fail to allocate. String multiplication syntax is not supported yet.

Slice text by Unicode scalar position

A slice of a string returns Result[str, AllocError], with the same clamping rules as lists. Positions count Unicode scalar values, just like len and indexing:

def main() -> Result[(), AllocError | IoError]:
    text = "Aé🙂Z"
    print(str.repr(text[1:-1])?)?
    print(str.repr(text[-100:100])?)?
    print(str.repr(text[3:1])?)?
    print(str.repr(text[::-1])?)?
    print(str.repr(text.slice(1, -1))?)?
    print(text)?
    Ok(())
Result[str, AllocError].Ok("é🙂")
Result[str, AllocError].Ok("Aé🙂Z")
Result[str, AllocError].Ok("")
Result[str, AllocError].Ok("Z🙂éA")
Result[str, AllocError].Ok("é🙂")
Aé🙂Z

text.slice(start, stop) is the method form, with required bounds and no step. The result owns its UTF-8 bytes and outlives the source. The operation creates only the final string, including for an empty or whole-string slice, so even these cases can return Err(AllocError.OutOfMemory). Source strings are never modified. Combining marks count separately; positions are not grapheme clusters or byte offsets.

Replace literal text with recoverable allocation

text.replace(old, new) replaces every non-overlapping match, scanning left to right, and returns Result[str, AllocError]. It borrows all three strings:

def rename(text: &str) -> Result[str, AllocError]:
    updated = text.replace("Plenty", "plenty")?
    Ok(updated)

def main() -> Result[(), AllocError | IoError]:
    original = "Plenty is Plenty"
    print(rename(&original)?)?
    print(original)?
    print("aaaaa".replace("aa", "X")?)?
    print("Aé".replace("", "-")?)?
    print("banana".replace("na", "")?)?
    Ok(())
plenty is plenty
Plenty is Plenty
XXa
-A-é-
ba

An empty search string inserts the replacement before, between, and after Unicode scalars. An empty replacement removes matches. Matches are literal and inserted text is not searched again; regexes and a replacement-count limit are not supported. The result owns its bytes and survives destruction of every input.

The runtime checks the complete output size and allocates only the final string. Even a call with no matches, or an empty result, makes that allocation and can return OutOfMemory; unrepresentable sizes return CapacityOverflow. Failure leaves all inputs unchanged.

Build text with recoverable allocation

Use text.concat(other) to combine two strings. To combine a list or another iterable of strings, use separator.join(parts). Both return Result[str, AllocError]:

def greeting(names: &list[str]) -> Result[str, AllocError]:
    joined = ", ".join(names)?
    "Hello, ".concat(joined)?.concat("!")

def main() -> Result[(), AllocError | IoError]:
    names = ["Ada", "Bea"]?
    print(greeting(&names)?)?
    print(names)?
    print("-".join([]?)?)?
    Ok(())
Hello, Ada, Bea!
["Ada", "Bea"]

The methods borrow their inputs. Joining first calculates the final size and then allocates one output buffer; it does not build a succession of intermediate strings. An empty list gives an empty string, and a one-element list adds no separator. Empty pieces still count: joining ["a", "", "b"] with "-" gives "a--b".

UTF-8 characters and embedded \0 are preserved. join also accepts a set, a dictionary (its keys), a string (its scalars), or a generator of strings; see building text for each form’s allocations.

Look up a character without trapping

Use text.get(index) when the index might be out of range. It returns Option[str]: Some(character) when the index exists, Nothing otherwise.

def show_character(text: &str, index: i64) -> Result[(), IoError]:
    match text.get(index):
        case Some(value):
            print(value)?
        case Nothing:
            print("missing")?
    Ok(())

def main() -> Result[(), IoError]:
    text = "café🙂"
    show_character(&text, -1)?
    show_character(&text, 3)?
    show_character(&text, 5)?
    print(text)?
    Ok(())
🙂
é
missing
café🙂

Indices are i64 and count Unicode scalars, as with text[index]. Negative indices count backward from the end; -1 selects the last scalar. Empty strings and indices outside either end return Nothing. Neither form allocates: a character is a short string stored inline in the value, independent of the original, which remains available.

Ordinary text[index] terminates the program for a missing index. Indexing ASCII text takes constant time; other text is scanned from the start to reach the requested scalar, so use for character in text to traverse it.

Backlog

This is the single maintained list of remaining language, compiler, runtime, standard-library and tooling work. Entries are candidates, not promises about syntax or delivery dates. Row order is a catalogue, not a priority list.

Delivered milestones and retired entries live in Completed work. A B ID can appear there for its delivered subset and here for its remaining extensions; IDs are never renumbered or reused. The implementation status and reference describe what works today. Proposals record rationale and alternatives, not another work queue.

The experimental Windows x86-64 dataframe build and release packaging are recorded in completed work. The macOS ARM64 dataframe subset is recorded in completed work. Broader native OS qualification remains in B18.7.

Current position

B34’s bounded cleanup/review, B71’s typed HTTP acceptance batch, B76’s initial dataframe datatype expansion, the initial B39–B70 library slices and B17 package tooling have landed. Their delivery records are in Completed work. Preserve their installed-toolchain, CLI/data, JSON-service, mutual-TLS and HTTP qualification checks during further work.

Issue severity takes precedence over the batches below. Defects, reproductions, claims and implementation/review status belong in the shared issues.db. Nothing on this page assigns ownership or approves an API without a scoped agreement and acceptance checks.

Next batches

  • B76 — analytical workloads: decimal columns follow the std.decimal proposal after its core module ships. Use the pinned Polars engine’s Parquet capabilities; fixed-size arrays, dictionary semantics, nested casts and broader list operations need separate scope. Use the daily sales application and examples/sales_report/ as evidence: truly large or rejection-heavy workloads should guide multi-output streaming. Temporal/grouped as-of matching and time-bucketed/rolling aggregations need a separate workload before expanding the API.
  • B23/B76 — measured SIMD extensions: packed views, wider shapes, broader predicates, maps/folds, runtime ISA selection and parallel scans need bounded legality/performance evidence. Account for code placement using the retained placement research.
  • B71 — additive error ergonomics: reporting conveniences remain independent candidates, not prerequisites for typed HTTP composition.

Further library breadth, B11 stored-reference extensions and B72 state/linear-obligation experiments remain candidates requiring a concrete use case and an agreed contract. This sequence preserves the existing batch scope; it does not schedule all catalogue entries.

Deliberate deferrals

  • B19 — module caching and frontend scale: serial efficiency, streaming lowering/emission and per-module caching need measurements and a focused batch. The module-cache investigation rejected separately compiled modules and stored syntax trees. A lowered-module cache needs imported interfaces and invalidation by body-derived facts; its broad-import benchmark offered about 30 ms of frontend work in a 44 ms body-edit build. Revisit for frontend-bound std-heavy applications, using module-edit measurements as the baseline. Remote caching waits on a local per-module contract.
  • B22 — broader concurrency: richer mapping/effects, streaming results, rendezvous, larger or send-side selection remain deferred. Existing channel and executor checks do not establish deadlock freedom.
  • Automatic parallelization (B24), optional backends (B20), stored-reference relationships (B11), broader callback erasure (B32) and automatic deep recursive operations (B33) require their own contracts and acceptance checks.

Scope of follow-on work

B22 — task models. The stackful-task investigation and async/await exploration do not commit Plenty to either implementation. Before adoption, compare the same bounded TCP/HTTP workload: I/O cancellation/registration lifetimes, allocation and failed-start preservation, stack budgets, suspension/loan checks, memory and latency. Fixed stacks trade pinning/relocation for reservations and fatal unchecked exhaustion. Retaining native concurrency remains an option. Prefer library APIs over new syntax; require bounded queues/backpressure, failed-send ownership, joins on all exits, cooperative cancellation and partial-result cleanup. Scoped generic tasks need not wait for heterogeneous callback storage in B32.

B32 — additional callback representations. Evaluate registry/event-handler use cases and specify invocation, move/drop, allocation failure and capture lifetimes before adding erased modes. Escaping borrowed captures depend on B11; C ABI callbacks and foreign-library lifetimes remain B28.

Work items

Language, compiler and tooling

IDRemaining workScope / dependency
B11Broader borrowingStored/returned references and views; returning a borrow chosen among several reference parameters; symbolic-index collection loans; field-precise loans through mut reference bindings. Optional/fallible references: phase 2 (mut locals and reference-carrying parameters), nested patterns through the carried reference, match expressions producing these values, dict.get_ref and list.get_ref/get_mut. Preserve owner/variant stability and temporary lifetimes; measure precision against compile-time cost. Proposal; delivered subset, including the len call-borrowing fix for issue #683.
B12Practical I/O and iterationReusable iterator/view protocols, stored/returned slice and substring views (B11), portable path semantics (B18). For substrings: str and std text APIs accepting &substr, more view methods and cached scalar counts. Further adapters need use cases; pipeline-specific extensions are B75. Delivered subset.
B13Public allocator controlGlobal/default and per-container selection, allocator state/lifetimes and buffer provenance. Recoverable allocation is delivered; allocator design remains proposed. Bounded/inline-capacity containers depend on B83 value parameters and fixed arrays.
B14Typed error compositionFinish unchecked arithmetic modes: a plenty.toml setting and a stdlib precondition audit, so std can follow --unchecked assume. Broader Option/Result adapters beyond the delivered fallback/mapping helpers (issues #800/#820), typed composition of parsing/conversion errors and remaining checked shift/cast operations. Preserve precise errors and use Failure only as a last resort. Human-readable reporting extensions are B71. Checked-default core, assumed list/view/Bytes indexing (issue #764), coordinated repository/benchmark-template adoption and bounded validation are recorded in the delivered subset and validation record.
B16Inference and value conveniencesExpected-result inference, partial explicit type arguments, contextual literals across generic arguments; NonZero[T] module constants; saturating conversion; standalone unit bindings/parameters/storage. Require concrete use cases/contracts, not automatic extensions of today’s inference. Parsing/conversion error composition is B14. Delivered subset, including block-if statement values (issue #674) and tuple patterns/nested unpacking (issue #675).
B17Modules, tooling and packagingShared per-user Git cache; semver selection over tags; per-module incremental objects (B19); re-exports and import-cycle policy. LSP: find references, document symbols, signature help, case pattern variants and incremental module re-checking. Preserve deterministic lookup, standalone compilation and reproducibility. Release artifacts: four targets via scripts/release/target.sh, with manual dry runs of the release workflow. Further targets need runtime/linker/artifact tests (B18); std breadth is B38. Exploratory workflows are listed under tooling investigations. Delivered subset, including the completed script design and optimized build defaults.
B18Windows and additional targetsNative GNU/Linux AArch64 compiler/runtime implementation is present alongside x86-64: baseline ARMv8-A code generation, target-aware native dependencies, Linux bindings, UDF traps, stack handling and AAPCS64 actor recovery. Qualification includes host SIGILL coexistence (issue #588), first actor-heap ownership (issue #589), and native libFuzzer discovery (issue #591). Development and emulated qualification records the reproducible harness; B17 owns artifact delivery. Native ARM CI passes the compiler/runtime suites, both optional engines and Docker proxy tests. Linux ARM64 is a release platform: archives are built and tested on native ARM runners and smoke-tested on AlmaLinux 8 and Debian 13. The v0.8.9 artifact audit confirms glibc 2.28 compatibility for both Linux targets (issue #730). Remaining ARM gates: dedicated hardware concurrency/signal stress, minimum kernel and page-size qualification. Packaged linalg integration and tutorial tests pass under the ARM guest; that local artifact does not qualify the release libc floor. The updated cross-platform assessment, interface/source map and toolchain experiments guide ARM macOS and x86-64 Windows; Windows ARM64 remains deferred. Changing the linker alone does not add cross-compilation. The Cosmopolitan/APE proposal recommends a bounded x86-64 compatibility experiment and identical-artifact non-Linux qualification before deciding on a restricted profile or broader port.
B19Measured codegen and storageSee the remaining performance work; completed measurements and optimizations are in the performance record.
B20Optional backendsEvaluate the GCC proposal before adopting another backend/common IR. The Rust-source investigation includes a checked scalar emitter; ownership-bearing APIs remain hand-lowered probes and need emitted semantic parity. Cranelift stays the production backend. Rust dependencies are B17.
B22Native concurrency extensionsUser-facing scoped pool jobs borrowing the submitting scope; broader iterator/closure mapping, streaming results, richer certified effects/types, rendezvous and broader selection. Mutable, one-shot, borrowed, generator-containing and indirect map callbacks need separate scope. Preserve fallible submission and deterministic shutdown. Task-model gates are under follow-on scope. Delivered subset.
B23SIMD extensionsPacked storage/views, broader vector shapes, further predicates, total maps/modular folds, runtime ISA selection and parallel scans. Require separate evidence; hide alignment/tail/target bookkeeping, preserve checked arithmetic and ordered floating folds, and do not treat generic lists as packed arrays. Spikes/backend gates; delivered subset.
B24Automatic parallelizationDeferred beyond B34. Require evidence that effects, cleanup, allocation failures and result ordering remain correct; a thread-count setting alone is insufficient.
B27Expanded C representationsExport text/byte buffers and views, error sums of payload-free enums, payload-carrying errors, recoverable returned ownership and explicit class-method adapters. Keep allocator provenance, output initialization and error-path transfer visible. Owned export contract.
B28Library loading/lifecycleGenerate loaders from extracted contracts without original export source; improve third-party C binding tooling and error detail. Unloading, retained/foreign-thread callbacks, C record layout and header-assisted bindings need separate contracts. Current resident/thread-bound loaders.
B29Further generic constraintsProtocol bounds on data, generic methods satisfying protocols, combined bounds/composition, generic aliases and enum constructor inference. Check bodies against declared bounds so argument-independent errors stay at definitions; preserve explicit lookup and bounded specialization. Library-owned operations favors composition/capabilities; protocol access grants remain deferred. Protocol bodies records agreed default/final dispatch, member-specific grants and definition-time contract checking, not automatic injection; no implementation scheduled. Explicit Box[P] owners, shared/mutable dispatch and fixed-argument inference are delivered; see the contract. Delivered implementation and investigation records.
B32Broader callback storageMutable/consuming/borrowed/worker-bound erased modes and generator-containing environments in heap owners need their own contracts. Escaping borrowed storage depends on B11; see follow-on scope. Delivered subset.
B33Deep recursive operationsIterative fallible copy/format work storage and an explicit bounded-memory contract for structural equality/membership, without hidden allocation failure. Preserve cleanup, shared-string ownership and float/NaN behavior. Rationale.
B36Named-call extensionsFiner protocol receiver tracking, deliberate builtin argument names and structured initialization for large records. Splats, variadics and //* markers remain in the unpacking proposal. Delivered defaults/named calls.
B37Ordering extensionsCached/fallible sort keys, stable sorting and user-defined ordering without hidden allocation. The ordering proposal recommends __cmp__/Order for comparisons/equality and runtime sorting, not dictionary keys. Evaluate Timsort/Powersort with fallible scratch space; naming/migration remain decisions. Use measured runtime-sort bounds (issue #331). Until user ordering lands, __cmp__ and the __eq__/__lt__ family are compile errors (issue #613). Delivered subset.
B38Standard-library breadthSee library follow-ups and ergonomics. Further breadth requires a use case and scoped contract. Prefer Plenty unless native complexity/security justifies a wrapper; use the engine criteria for large native libraries. Delivered modules.

Tooling investigations

These belong to B17 and are not scheduled implementation commitments:

  • Compiler boundaries: structured diagnostics, scoped recorder/teardown policy, the op and syntax-layer splits and grouped pipeline state are delivered (issues #691–#695). A compilation session and further lowering-state extraction wait for a consumer. Module replay remains B19, not a wholesale frontend rewrite.

  • Single-file script manifests, isolated dependency resolution, durable locks and argument forwarding are delivered. Warm executable hits also skip runtime extraction (issue #699). Broader executable-cache toolchain coverage remains B17/B18; a script test command would require its own entry-point/discovery design.

  • plenty-manager: a Plenty-written tool in a separate plentylang/plenty-manager repository for compiler installation, explicit updates, side-by-side versions and project pins. Settle release authentication and qualify manager hosts before delivery.

  • Language editions: wait for a concrete incompatible change; prove mixed-edition compilation with shared types/runtime before edition or migration tooling.

  • Rust crate dependencies: three prototypes compare Rust ABI, generated C adapters and Rust-source lowering. The next gate is real Plenty project integration with checked foreign owners and a coherent runtime build; generated C adapters are the proposal’s recommendation.

  • Local semantic search: improve chunk boundaries and measure retrieval quality before deciding on committed index snapshots. Prefer Plenty for the interface and record language defects in issues.db.

Remaining performance work

B19 is evidence-driven; these are candidates, not an approved optimization batch. Use profiling, faster-linking evidence and the completed measurements as baselines.

  • Generated code: retain/release for text operands computed by calls or loaded in another block; dispatcher copies of sums with inline-record payloads; code size and compilation/link latency. Revisit match density/size policies only with application/CPU evidence. An opt-in CPU target such as x86-64-v2 requires a decision against B17’s fixed-baseline reproducibility contract.
  • Layout/cleanup: natural record-field packing beyond 16-byte slots, compact sum tags/niches, relocation/release plans for records holding ranges/sums/other managed fields, records over 192 bytes and tuple release plans. Measure inline copies beyond current runtime-copy paths; owned temporaries and nonscalar list rows still use runtime helpers. Flat-layout rationale. Box size (issue #814): a box’s class-record slot still keeps a value word that only addresses the record after it, and Option[Box[T]] keeps a tag word rather than a null niche; dropping both would take a binary-trees node from a 96-byte malloc chunk to 64 (C: 32). A size-class free list for fixed-size boxes is unmeasured.
  • Serial frontend efficiency: module-scope name hashing, builtin-sum interning and Ty/nominal-handle clone traffic (about 10% of check samples with allocation excluded; issue #696). The signature map and nominal fact/lock costs measured below 2% and are not candidates. Whole-program table/cache misses still increase per-function cost at scale (issue #57); interned names or per-function batch processing need measurements. Further type-representation changes follow the completed issue #302 investigations. Measured attribution records the profile; existing frontend function symbols and type-fact caches are not missing features.
  • Peak memory: lower and emit one function at a time instead of retaining full syntax/operation sets at phase boundaries (issue #54). IR construction’s measured 7% of emission CPU limits scaling only above about fourteen cores.
  • Caching/lowering: imported-interface lowering with explicit outputs and serialized operations/types; body-derived invalidation and per-function granularity for one-module programs (issue #317). Separately evaluate lowering only reachable std functions. A trusted opt-in remote cache follows a local per-module contract, not before it. Prove fresh-environment replay using explicit module outputs before persistence. See deferrals.
  • Upstream gates: regalloc2’s quadratic block-parameter handling needs an upstream report/fix before removing the 2,048-argument fallback. Reopen issue #425’s deferred arithmetic-check work only when Cranelift changes unfused overflow branches, frame setup or callee-save spills; do not treat closed investigations as active claims.

Stdlib-driven language ergonomics

IDRemaining workScope / dependency
B71Error presentationBenchmark-template migration after active comparisons, and separately scoped conveniences such as field-level redaction (for example URL userinfo, beyond shipped class/enum redaction). Case guards, residual-set inference, empty sums, arbitrary protocol forwarding and dynamic error objects remain deferred. Delivered composition/HTTP work.
B72State/ownership ergonomicsExplore typestate and runtime-state enums. Linear obligations, resolution scopes and integration boundaries do not approve implementation or blanket std conversion. Real HTTP integration of the owned-response experiment remains unexplored. Field moves through Box/Shared, per-element tuple tracking, partial destruction for __del__ classes and whole-object destructuring need contracts. Delivered subset.
B73Arenas/linked dataBefore std.arena, replace caller namespaces with a safe arena-identity contract covering copy/concurrency/exhaustion, review callbacks/movement cost, and qualify deletion/reuse, bad handles, affine cleanup, OOM preservation and tree/graph consumers. Keep linked lists/trees as examples until a workload justifies an API; the linked-list lesson covers owned chains and index-based bidirectional links (delivery), and balanced-tree examples demonstrate B-tree/red-black insert/delete and invariant checks (delivery); the Unicode trie adds iterative prefix retrieval and pruning (delivery). Direct disjoint loans/worklists relate to B11; no intrusive-pointer extension implied. Assessment/prototype.
B75Pipeline extensionsEarly stop for parallel any/all and integer range sources beyond range[i64]. Broader reusable iterator/view relationships are B12/B11. All four original stages are delivered; they are not outstanding tasks.
B76Columnar extensionsDecimal columns after the B86 core module; temporal/grouped/floating as-of keys, time-bucketed/rolling aggregations, fixed-size arrays, dictionary semantics, nested casts and broader list operations need workload evidence. Validate use cases before static-schema/zero-copy machinery. For clist: row views/method adaptation, field references, affine fields, chunks, statistics, tombstones/vacuum and interchange. Broader SIMD/parallel scans need legality/performance evidence (B23); borrowing B11/B12, pipelines B75. Delivered dataframe/clist subsets.
B77Derived serializationAccepted design, not implementation: JSON-shaped std.serial; hooks/compiler-expanded entry points; class @serial; TOML; docs. SerialError[E] retains format error/field path. Measure compile time/throughput before derive. Relates to B29/B51; reuse the closed annotation mechanism.
B85Actor extensionsExtend the recorded actor HTTP qualification beyond x86-64 GNU/Linux. Matched normal/fault comparisons are recorded; actor WebSocket/SSE adapters remain; WebSocket uses B94’s shared session and driver/upgrade contract; blocking-work offload proposes bounded native workers, independent job ownership and completion delivery. The budget/stack probe validates native suspension with heap/trap restoration; compiler checkpoints, shared-context ownership, bounded stack admission, safe suspension regions, overflow/platform qualification and fairness/overhead measurements remain. Multi-thread stages; sliding-window/all-for-one restart, parent restart notices and an actor-specific &mut self spawn diagnostic remain. macOS heaps wait on B18. Actor HTTP design, delivered actors/network/TLS/HTTP.
B80PlottingFinish the accepted core milestone: remaining scales, smoothing, broader facet controls and font-measured layout. Then broader geoms/PNG, dates, reflected records and a separate dataframe bridge. Delivered subset.
B83Inline arrays and value-generic extensionsValue parameters and Bounded are delivered. Remaining accepted design: inline array[T, N], allocation-free literals at generic parameters and per-instance assertions; then BoundedList (B13) and measured internal SIMD rewrite (B23). Derived sizes in signatures remain deferred.
B86Exact decimalsstd.decimal proposal: allocation-free 38-digit Decimal in Plenty with exact operators and explicit rounding. Stages: u64.widening_mul, std-only @arithmetic methods/hooks, __cmp__ (B37) and a ZeroTest bound for NonZero; core module; __key__ key policy (reconcile with the delivered structural-key contract) and a format-spec hook; Polars decimal columns (B76) and JSON/CSV interchange. Arbitrary precision only on evidence.
B87Std conventionsPublic std APIs follow Standard library conventions. Align the listed deviations as they are scheduled (booleans with B36, byte inputs with B39); file issues for the untracked ones: bounds: vs limits:, AllocError unions, predicate names and count() sizes, watch shapes. Delivered: ThreadPoolExecutor.cancel_pending() replaces shutdown(bool) (issue #631); every thread start fails with SpawnError, and owned starts return their job (issue #630); native hooks are std-only intrinsics taking &Bytes, so std.bytes has no public hook methods (issue #633).
B88Copy rule@copy on enums; undeclared enums affine after a migration that annotates every currently copyable enum. Generic enums use B84’s conditional copying and CopyType bound. Delivered: one copy rule in the reference, cheat sheet and skill, checked by tests/test_copy_rule.rs (issue #636). Applied copy/Shared guidance is delivered in issue #810. Proposal.
B89Formatting pathPhases 1 and 2 done (issue #615): print/f"{x}" use display; str.repr/!r/aggregate elements use debug; private types format in their module; wrong-signature display/debug is a compile error. Remaining: main’s report uses display; the shared formatter needed for it is delivered (issue #634). Proposal; issue #615.
B91Iteration sourcesAdapters only on iterators; iter() for every source; membership on std containers stays B49. Delivered: len method dispatch (issue #623); uniform local dictionary views (issue #624); find/reduce copy borrowed copyable items, and iterator methods on non-iterators suggest iter(...) (issues #621, #622). Proposal.
B93Callable formsDelivered: reusable callable forms in serial pipelines and sort_by_key, and &mut named functions (issues #617, #619, #620); constructor values, including generic classes and Some/Ok/Err (issue #618); lexical checked callbacks with scoped failure propagation (issue #614). Remaining: par callbacks for stored capture-free closures and Callable values, including class and variant constructors. Callable names stay (issue #24). Proposal.
B96Bounded integer adoptionThe Bounded[T, lo, hi] type, literal checks and checked runtime construction are delivered. Remaining: adopt it for std.http.Limits.max_line/max_head (>= 2) and fixed std ceilings. Existing ceiling metadata (issue #686) still checks static std inputs; dynamic values still trap under those APIs.

Units of measure

IDRemaining workScope / dependency
B97Units plot axis labelsstd.plot axis labels from quantity units. Deferred; design open, including choosing a display unit that matches the values’ scale (km rather than m for long distances). Candidate design: plot axis labels. Compiler dimensions, Quantity operations, formatting, constants, scoped diagnostics, native erasure, std.units with zero-copy wrap_list/strip_list, the std.units.catalogue of SP 811 units, and shared bodies for instances that differ only in dimension are delivered. Design: units of measure proposal. A Positive[f64] float domain is a separate follow-up.

Runtime-independent protocol support

IDRemaining workScope / dependency
B94Sans-I/O networking protocolsAgreed codec/session/driver direction, not implemented as a general convention. Validate with WebSocket: extract bounded framing/session/handshake logic; migrate the existing blocking/WSS driver without silent behavior changes; add the B85 actor driver with explicit upgrade ownership, entropy supply, timers, backpressure and shutdown. Follow the session contract and acceptance gates; retain existing regressions and add deterministic/fuzz, real TCP/TLS and actor resource/fairness evidence. No universal protocol interface, new scheduler or wholesale HTTP rewrite. Broader protocols require separate scope under B38.

Standard library follow-ups

Only outstanding extensions are listed here. Initial delivery records, issue/PR references and reference links live in Completed library work. B IDs keep their original scope; a closed initial issue does not approve a follow-up. All candidates still require scope and acceptance checks.

IDAreaRemaining candidates
B38Additional breadthstd.bigint (issue #481): Plenty implementation per the bigint proposal; needs u64 widening-multiply and add-with-carry builtins. XML namespace URI resolution and tree/query APIs; recursive filesystem watches/automatic reconciliation; accurate summation/norms, float decomposition/adjacent values and special functions (the numerics core proposal schedules summation, norms and special functions in its first milestone, with optimization, integration, ODEs, interpolation and FFT under std.numeric; it replaces issues #492 and #493); constant function calls and bytes concatenation. Linalg conditioning, least squares and sparse storage need separate scope. Image: GIF/LZW/frames, baseline/progressive JPEG/encoding and WebP/VP8/VP8L/RIFF, then 16-bit buffers, streaming decode, metadata, resampling filters and the measured 2–6× Pillow speed gap; implement in Plenty under the same Limits. Broader statistics need numerical contracts; sketches need mergeable quantiles/frequent items, stronger tail accuracy and workload-led xor/fuse tuning. System: cgroup accounting, CPU utilization sampling and filesystem free space.
B39BytesStored/returned views (B11); OS strings in std.path, std.env, std.process and std.cli are list[u8]. Text APIs accepting &substr are B12. Dataframe byte-view inputs delivered.
B40Binary streamsUnscheduled nonblocking descriptors, vectored I/O, peeking and erased streams.
B41Error contextNative-cause OS description text (B14/B71).
B43ProcessesNative process-readiness wakeups for deadline waits, file/child redirection, stderr merging, dropped-child reaping policy and process groups. Polling deadline/cancellation waits delivered.
B46LogsJSON-lines sink, sampling/rate limits, per-module thresholds and span context.
B47Text/UnicodeGrapheme-aware text wrapping and broader classification; keep the external unicode_text package’s Unicode version aligned with std.unicode. Delivered paragraph tools.
B48RegexBroader DFA/one-pass coverage (Unicode and assertions), remaining scalar scan lowering (issue #760); full Unicode case folding, script and general-category properties. Bounded ASCII accelerators delivered (issue #742).
B49Collections/iteratorsUser iterator protocol (B12, needs B11 stored references); in on user and std containers; len() method dispatch delivered (record, issue #623), Python __len__/__contains__ names remain reserved (issue #645); further collections need use cases.
B50Random/secretsJump/split streams, distributions/float ranges, weighted choice/element sampling, scratch-free secret integers and a zero-on-drop buffer.
B51JSONBulk scans outside string runs, pretty-printing, JSON Lines and derived codecs (B77).
B52TOMLIndependent conformance checks (toml-test/reference crate); evaluate TOML 1.1.
B53CSVLenient parsing/recovery, header mapping/typed columns, non-UTF-8 encodings, multibyte delimiters and throughput benchmarks.
B54EncodingsBulk native codec/packing throughput, streaming codecs and varints. IEEE float packing delivered (issue #656).
B55FilesystemAdvisory locking delivered (issue #774); explicit child-fd inheritance remains issue #777. Passwd fallback, XDG_RUNTIME_DIR and XDG_*_DIRS search lists.
B56Concurrency primitivesCondition variables, rwlocks and semaphores only with a concrete use case.
B60URLsRelative resolution, normalization/equivalence, IDNA and form encoding.
B63HTTP clientWebSocket permessage-deflate and proxy tunnels; cookie jars/content decoding were declined, not scheduled.
B64HTTP serverSSE client reader; waking SSE streams from channel messages rather than polling.
B65HTTP helpersStatic-file Range requests and directory indexes.
B66HTTP qualificationGenerated route dispatch without user match waits on enum methods or existential protocol values; preserve existing qualification suites and proxy fixture ownership (issue #719; delivered scope).
B67CryptoP-384/JWT ES384 and SHA-384 HMAC/JWT HS384; password KDFs need separate use-case/API review.
B68CompressionGzip header access, zstd and trailing-zero tolerance; archives remain B69.
B69ArchivesPAX decimal range correction delivered (issue #705). Remaining: policy-controlled link creation, mode/time restoration and streaming ZIP writes with data descriptors.
B70SQLiteRuntime allocator integration for accounting/failure injection, user-defined functions and backup API.
B78DiffsLine-first pass for char/word diffs and diffs in std.test failures (B57). Performance baselines belong to B19.
B79MarkdownGFM tables, strikethrough/task lists and parser throughput beyond the recorded ~1 MB/s.
B82GraphsWeak components, Prim/Floyd–Warshall, flow/matching, compaction, non-f64 weights and () type arguments.

Reconciled completed work

The delivery records and retired entries moved to Completed work. This heading remains as a link destination; completed-only B IDs no longer appear as work to select here.

Keeping this list useful

  • Add, prioritize or split future work here. Keep IDs stable and link design detail rather than copying queues into proposals, tutorials or reference pages.
  • Settle a candidate’s scope and acceptance checks before implementation. Table order does not approve APIs or require every extension.
  • Before syntax/builtins, state the user problem, why ordinary library code is insufficient, and ownership, allocation, compilation and teaching costs.
  • On completion, move the delivered scope to Completed work, update reference/status/runnable lessons, and leave only genuine outstanding extensions here with a link to their delivery record. Do not mark an entire mixed item done because its initial slice shipped.
  • Issues, claims and review state stay in issues.db; reference pages describe implemented behavior. Use the development workflow for ongoing testing, documentation and performance practices.
  • Historical proposals may contain old syntax/dependencies; introductory notices must say so. Future priorities belong here, not on completed-work pages.

A JIT and an interpreter remain out of scope, not waiting at the bottom of this backlog. Async/await remains unadopted and unscheduled under B22’s investigation gates.

B18 — macOS and Windows implementation sequence

The portability design records rationale and toolchain evidence; this section owns the work sequence. Target ARM64 macOS and x86-64 Windows, preserving both Linux architectures. Use isolated child worktrees based on the portability branch, with component ownership agreed before concurrent edits. No local Mac/Windows machine is assumed; provision CI execution before treating either port as tested.

Work packageDependency / owned componentsCompletion evidence
B18.1 — backend and toolchain decisionsStandalone probes; pinned Cranelift, Zig/cargo-zigbuild, C/Rust adapters; compare Windows MSVC and GNUReal Mach-O/COFF output and native CI execution of mixed ABI/Tail/large-frame fixtures; record SDK/CRT/deployment choice and unwind strategy. Signal restart qualification no longer assumes sleep-based reader scheduling (issue #689). Cross-link success alone is insufficient.
B18.2 — target and host separationB18.1 findings; toolchain, frontend facts, cache/runtime identityOne target description, explicit build-host/compiler-host/output-target identities; incompatible combinations rejected; existing Linux suites pass.
B18.3 — object and linking interfacesB18.2; object metadata/exports/source/trap/coverage records, drivers/distributionELF implementation preserved; Mach-O/COFF metadata, retention, exports and relocation round trips pass; no GNU options leak into other drivers.
B18.4 — dependency and runtime buildsB18.1–2; build.rs, native sources, test hooks, engine helperBoth runtime modes and native dependencies match target; Cargo and embedded builds agree; choose direct-build versus orchestrated Cargo using a measured prototype; no recursive Cargo lock contention.
B18.5 — runtime coreB18.3–4; memory, errors, entropy, threads, mutex/parking, signals/exceptions, actor recoveryMinimal native Plenty program, C callbacks, fallible allocation, secure randomness, stack protection and recovery tests pass on each OS; no success stubs or hidden allocation regressions.
B18.6 — OS contracts and servicesDesign decisions begin alongside B18.1; implementations depend on B18.5; paths/resources/files/process/readiness/watchLossless names, full process status, rooted filesystem security and resource-aware cancellation tests; explicit watch/shutdown capabilities; separate compatible portable contracts from Linux-only APIs.
B18.7 — library parityB18.6; networking/DNS/TLS/HTTP, SQLite/KV, crypto/compression, dataframe/linalgReal native execution, including both optional engines and CPU fallbacks; count skipped/early-return opt-ins separately; publish exact capability gaps before any preview.
B18.8 — developer workflowsB18.3–6; package CLI/cache/LSP/fuzz/export tools and testsExtracted installation handles spaces/Unicode, literal arguments, replacement races, diagnostics and library hosts; explicitly report unqualified fuzz/sanitizer paths.
B18.9 — release admissionB18.7–8; native CI, packaging, notices, minimum-version audits, signing policyOffline clean-machine compilation with documented prerequisites; tested target/engine artifacts, baseline OS/SDK/CRT evidence and signing/unsigned-preview policy. Preserve Linux ARM’s outstanding hardware/release gates. Experimental macOS ARM64 (.tar.gz, macOS 11+) and Windows x86-64 (.zip, Windows 10+, dynamic MSVC CRT) archives include their dataframe engines and notices, with native unit tests, executable/library audits and clean-runner Parquet/application-relocation smoke tests. macOS uses ad-hoc signatures without notarization; Windows remains unsigned. Native execution evidence still depends on CI, including qualification after the engine version-lookup fix (issue #687) and unused native-wrapper and executable-helper pruning (issues #688 and #690) and native distribution executable suffixes (issue #697). Remaining: library-parity smoke tests (B18.7), project workflows (B18.8), signing/notarization.

Current evidence: the native CI smoke suite passes on macOS ARM64, Windows x86-64 MSVC and Linux x86-64. The first B18.2 increment centralizes target facts and compiler-host/runtime validation; it does not implement new object formats or OS services. This is a partial B18.1 gate: large frames, unwind, Rust boundaries and runtime dependencies still need qualification. Both new OS targets are experimental; neither is release-qualified. The shared Apple archive-command fix addresses issue #600; native build-tool tests run before the OS port build and on both Linux architectures. The first B18.5 increment separates native mutex waiting and thread handles, hides cancellation wake storage, and adds Darwin signal-context and stack-handler support. Standalone native synchronization tests run before the complete compiler build. Windows exception dispatch and per-thread stack guarantees are implemented with standalone subprocess tests that pass on the native Windows runner. Native OS entropy fixes dictionary initialization on non-Linux hosts (issue #604); Windows no-replace bundle publication and portable graph fixtures address issues #605 and #606. The initial B18.3 increment enables experimental Mach-O/COFF application output, native C conventions and linker syntax, with framed Mach-O interface metadata. Native qualification now passes full compiler/runtime builds, unit tests, platform-boundary checks and generated-program callback/result probes on macOS ARM64 and Windows x86-64. Linux x86-64 and ARM64 CI also passes all integration shards, both optional engines and Docker proxy tests. Full new-OS standard-library parity and release admission remain gates; these results do not complete B18.5–9.

B18.1 and B18.6 design research can run together; B18.3 and B18.4 can proceed in separate worktrees after the common description settles. Keep independent OS implementations separate after their contracts are agreed. A failing native probe changes the design before target admission. User-facing cross-compilation, Intel macOS and Windows ARM64 remain separate decisions, not implied outcomes.

Completed work

Delivery history and retired backlog entries live here, separate from the backlog, which is the single maintained list of future work.

B IDs are stable. A record here means the described subset shipped or its investigation concluded; it does not mean every conceivable extension of that B item is implemented. Mixed items keep their remaining scope only in the backlog, linked from these records. A completed investigation may have rejected an implementation rather than shipped one.

These are concise delivery records, not a second reference or a future-work queue. The reference owns current behavior and limits; implementation status summarizes support. Local issue/PR numbers identify records in the shared issues.db, not GitHub records. Reproductions, claims and review state remain there.

Reconciliation history

The backlog’s full reconciliation on 2026-10-22 covered local main and merged local PRs through #477, reference/tutorial content and the shared issues register. Compiler/columnar follow-ups were checked on 2026-10-24 through #531. Later merges added the delivered subsets recorded on these pages. The split from the active backlog was made on 2026-10-31; it does not independently requalify every historical claim or declare all remaining extensions complete.

B34’s bounded cleanup/review, B71’s typed HTTP acceptance, B76’s initial numeric, binary and nested dataframe expansion, B17 package tooling and the initial B39–B70 library slices have landed. Their acceptance checks remain ongoing regression requirements, not unfinished milestones.

Maintaining the split

When work ships, update its reference, implementation status and runnable lessons. Add a short delivery record here with issue/PR evidence and a reference link. Remove the delivered scope from the active backlog; if an item still has extensions, retain its ID there and link both ways. Do not copy outstanding task lists into these pages or recycle retired B IDs.

Typed time and cancellable waits

Delivered B42.

Issue #635 aligns database busy timeouts, JWT validation, cookie lifetimes, HTTP dates and SSE retry hints with Duration and Timestamp. WebSocket, SSE, readiness and child deadline waits accept caller cancellation while preserving server shutdown behavior. std.sync provides typed channel, future, token and two-channel selection waits; explicit millisecond primitives remain as low-level compatibility APIs (B42).

Completed language and tooling work

Completed-work index · Remaining work

These records describe delivered subsets and concluded decisions. References own current contracts; extensions of a mixed B item stay in the backlog.

Package check coverage (issue #805)

plenty check now warns about each .plenty file below a package’s src/ that no target imports, and ends with a summary of the targets checked and the src/ files they reached. Check coverage documents the rule: fixtures and other unbuilt sources belong outside src/.

Coarse check timing reports

Local check reports retain full-suite compilation/execution timings and local CI step durations, including failures, as Markdown and JSON. CI reports summarize Actions jobs and steps automatically and support historical run comparisons, keeping parallel runner time separate from elapsed span.

Embedded-runtime dependency reuse (issue #829)

Embedded-runtime Rust dependency caching retains the 27 fixed vendored rlibs across runtime edits and feature-only rebuilds, with source/toolchain content keys and artifact validation. Test shards restore these dependencies independently of the compiler workspace cache. Native C compilation and both runtime archives still rebuild; this change does not eliminate the remaining build-script cost (tracked in local issue #832). A local probe reduced the vendored stage from 18.95 to 0.46 seconds; whole-build timing was inconclusive because the remaining stages varied. Hosted savings have not been measured.

Native-engine dependency caches (issue #827)

Optional-engine CI caches use separate compiler and engine target directories. Compiler cache cleanup can no longer traverse the engine target and remove its Polars or faer dependencies. A new engine cache key repopulates the corrected layout; actual hosted cache-hit savings still need measurement. Local engine-build defaults and installed-library paths are unchanged.

Linux release glibc verification (issue #730)

The v0.8.9 artifact audit confirms the published glibc 2.28 baseline for both Linux architectures, including both optional engines. It separates the ARM build tool’s 2.30 requirement from release requirements and records baseline execution and qualification limits.

Agent skill refresh

The Plenty agent skill now distinguishes trapping and assumed unchecked arithmetic, routes readers to boxed protocols, structural collection keys and quantities, and uses the signature index’s module navigation. Published skill assemblies include the language reference so those contracts remain accessible outside the compiler checkout. Issue #810 adds task-oriented capability discovery and a versioned gap-verification workflow, with executable typed partial-error and cancellable-shutdown examples. Applied examples distinguish string assignment, explicit class copying and nested Shared ownership, and package guidance covers module privacy, one-way imports, cycles, target/test coverage and current editor limits. Publication bundles the linked concurrency and editor setup contracts as well as the existing API pages. Issue #738 recalculates signature-index module ranges after publication transforms, so line-based reads select complete sections in the installed skill too.

Experiment arithmetic migration

Local issue #737 updates stored and generated experiment inputs for checked integer arithmetic. The fannkuch kernels propagate overflow errors; infallible Rust ABI comparisons retain explicit trapping suites. The experiment smoke check covers source inputs, expected rejections and generated fixtures without claiming to refresh historical performance measurements.

Optional-engine check build reuse (issue #828)

Engine CI checks and just ci dataframe linalg use runtime-checks consistently for module tests and runnable tutorial examples. The sales report uses the CLI already built for the integration tests. These checks reuse one feature configuration and avoid a separate sales-report compiler build. Allocation-failure checks remain enabled, and the dedicated default-feature build retains its separate coverage.

Range-trap fixture reuse (issue #830)

The I/O range and fixed-width encoding trap tests compile 18 reusable fixtures instead of 154 separate programs. Each invalid input still runs in its own child process and checks exit status and the operation-specific trap text. Argument dispatch selects among the original literal call sites, preserving constant-input coverage, empty-input checks and reader/transport variants. The fixtures retain optimized compilation and runtime checks.

B83 value parameters and B96 bounded integers

Value parameters specialize functions, methods, classes and enums on integer and boolean constants. Resolved values participate in nominal identity and structural inference; typed substitution makes them available in bodies without runtime fields. A const parameter can use a preceding T: IntType, supporting signed and unsigned bound widths.

Bounded[T, lo, hi] checks inclusive bounds at instantiation, admits in-range literals in expected contexts, and provides try_from returning RangeError for computed input and get for explicit extraction. It reuses private copy-class storage and ownership. Regression tests cover widths, boundaries, aliases, imports, privacy, generic functions/methods/enums, inference, body forwarding, expected literal contexts, copying and moves. The runnable tutorial teaches validation at input boundaries.

This slice uses Bounded as the first consumer rather than requiring arrays. Remaining B83/B96 scope stays in the backlog.

B97 compile-time dimensions

Dimension declarations and algebra provide nominal bases, transparent derived aliases, One, normalized products and quotients, integer powers, and exact roots. Dimensions are interned types without values and are admitted by the Dimension generic constraint. Generic input expressions remain structurally inferable; output/body expressions are checked per instance. Tests compare random expressions against a dense reference evaluator and cover exponent boundaries, imports, privacy and generic diagnostics.

The second stage of issue #701 adds Quantity core: coherent-value construction, get, float to/from, domain try_from, dimension-aware operators and compound assignments. Numeric widths, domain preservation/erasure and checked failures use existing arithmetic rules. Tests cover storage/result dimensions, rejection rules, evaluation order, collection/reference destinations and allocation denial.

The third stage adds coherent-unit display and numeric format specs, normalized quantity debug output in aggregates, and writer methods through std.fmt. Unit spellings are static metadata, with no per-value dimension tag. Tests cover numeric widths and domains, normalized symbols, conversions, nested specs, writer limits and allocation denial.

The fourth stage adds compile-time Quantity module constants: construction, quantity arithmetic, coherent-value extraction and float unit conversion. Dimensions and storage domains survive evaluation; overflow and invalid constant expressions are compile errors. Imports, forward references, static assertions and skipped branches follow the existing constant rules.

The fifth stage adds scoped dimension-alias notes to type mismatches, quantity operators, constant checks, generic inference and dimension errors. Notes select the first visible matching alias in declaration/import order, respecting import bindings, visibility and generic-parameter shadowing. Canonical type identity and runtime formatting remain unchanged; tests cover nested types, defining-module scope, editor checks and concurrent compilations.

The sixth stage completes issue #701’s native erasure: Quantity parameters and results use their storage type’s ABI, construction/projection are direct, and unit constants substitute folded values. The x86-64 release probe produces identical instructions to plain f32/f64 for addition, multiplication and conversion by a parameter or constant unit. Tests cover numeric bits across calls, references, aggregates, closures, generators and workers in both modes. Domain-backed quantity aliases also resolve safely before storage definitions are finalized.

std.units is embedded Plenty: the seven SI base dimensions, distinct derived aliases, prefixed SI and common exact units, the exact SI defining and CODATA 2022 constants, and dimension-aware sqrt, cbrt, hypot, sum, mean, abs, clamp and std.math wrappers. Celsius, Fahrenheit and std.time.Duration convert at the boundary; to_duration rejects negative, NaN and out-of-range times. Symbols print in declaration order within a module, so std.units quantities read kg*m/s^2. wrap_list and strip_list convert between list[f64] and list[Quantity[f64, D]] by replacing the list header’s type descriptor, with no copy or allocation.

Instances that differ only in dimension share one body: the object places identical code once, for any function not used as a value. Quantity[T, D](x) written with explicit type arguments, as generic bodies must, now retags its value as other constructions do, instead of calling the constructor; that made std.units generic bodies such as sum and hypot identical across dimensions.

std.units.catalogue adds every SI prefix on every SI unit and the other named units of NIST SP 811 appendix B, built from their definitions and tested against SP 811’s factors. It is a separate module so its 768 constants cost compile time only where imported. B97 retains plot axis labels from units.

Heterogeneous collections tutorial

The heterogeneous collections section follows classes and enums. Runnable lessons teach enum wrappers with shared methods and anonymous-sum lists, sets, and dictionary keys and values. The lessons show mixed keys, mixed values, and both together, while preserving distinct member identities. Separate typed collections remain an alternative for independent lookup domains. Lessons also cover custom-cleanup keys, transferring keys and borrowing views without duplicating cleanup obligations.

Unicode trie example

The runnable trie stores exact Unicode strings, including the empty key, with insertion, lookup, pruning deletion, sorted prefix enumeration, and longest-prefix matching. Private parent/sibling IDs support iterative operations and slot reuse. Model/invariant checks cover permutation pairs, Unicode mixed histories and 256-way sibling lists. Retained-result checks drop original key/query/trie owners before reuse pressure and comparison. Allocation-failure sweeps cover insertion and prefix results, and a Linux small-stack test handles a 100,000-scalar key. This is a teaching set, not a std API, compressed trie, or normalization policy. B73’s general arena qualification remains outstanding.

Balanced-tree examples

Executable B-tree and left-leaning red-black sets implement integer-key insertion, lookup, deletion, ordered traversal and private slot reuse directly in Plenty. A sorted-list model and independent structural validators check exhaustive five-key permutation pairs, multi-level trees and mixed histories. Runtime allocation-failure sweeps check preservation and retry; deletion and reuse are checked with allocation disabled. These are educational examples, not new std APIs or storage-engine performance qualification. B73’s general arena identity/deletion qualification remains outstanding.

Linked-list tutorial

The linked-list lesson demonstrates owned singly linked push/pop and mutable traversal, then index-based doubly linked append and bidirectional traversal. Executable examples preserve existing links on allocation failure. Deletion, slot reuse and public handle validation are explicitly outside this teaching implementation; B73 remains unchanged.

Documentation interpolation colours

The default Ayu documentation theme colours f-string interpolation expressions and their braces cyan, distinct from surrounding string text. Nested tokens retain their syntax colours; other themes are unchanged.

B11: Borrowing

Borrowed enum matching/payload loans, reference rebinding within the original place, precise first-reference return origins and disjoint literal-index loans landed (issues #1/#30/#41, local PRs #488/#490). Optional/fallible returned references (Option[&T]/Result[&T, E]) shipped as phase 1 (issue #520, local PR #622). dict.get_mut replaced dictionary indexed assignment (issue #580, local PR #696); heap and graph reference getters shipped (local PR #629).

Local dictionary key/value views (issue #396), list/byte slice views (issue #581), mutable Bytes loops (issue #594), proven disjoint slice/loop-field loans (issue #593) and scalar-indexed &substr substring views (issue #595) are implemented. Methods on owned temporaries can lend within the statement (issue #597), calls returning mutable references can be assigned through (issue #598), and method receivers projected from temporaries borrow in place (issue #599). len also normalizes explicit shared borrows of literals and owned temporaries, preserving full-expression cleanup without allocating for the borrow itself (issue #683).

See borrowing, returned references, optional references and slices/views. Outstanding relationships remain B11 in the backlog.

B49: Length method dispatch

len(value) calls an accessible ordinary shared len(self) -> i64 method on classes and enums, including standard Deque, Bytes, and heap containers (issue #623). Owners, references, and temporaries follow ordinary method borrowing and cleanup. Unsupported length and membership diagnostics name the actual operand type rather than claiming every iterable supports the operation. See the collection contracts. User-container membership and iterator protocols remain B49 in the backlog; Python __len__ and __contains__ names remain reserved.

B12: I/O and iteration

Text-file iteration, uniform borrowed iteration and borrowed temporary source lifetimes shipped (issues #30/#75). Allocation-free enumerate/zip/reversed sources and boolean reductions (issue #35) now compose through B75 pipelines; free any/all were removed (local PR #562). Binary buffering is B39/B40; lossless Unix arguments/binary I/O (issue #61), native Linux std.path and local dictionary views are delivered. The finder exercises NUL-separated input and output with --null; directory traversal/metadata shipped for issue #60.

See iteration and native paths. B12 in the backlog owns reusable/escaping views and portable paths.

B14: Errors

General anonymous sums preserve payloads, normalize membership and support injection/widening. Precise errors, ? and explicit success values replaced the former try/try_ allocation conventions; Failure is a last resort (issue #342), not the recommended default. Option.unwrap_or and ok_or shipped as part of issue #35. Sum fallback conveniences (issues #800/#820) add Result.unwrap_or, lazy unwrap_or_else on both sums, Option.ok_or_else, and map_or/map_or_else on both sums. Owned consuming callbacks preserve conditional evaluation and cleanup; returning a Result from both mapping branches handles a fallible fallback without special propagation. Both sums also provide message-bearing expect with caller-located traps. B71 records typed presentation and Result.map_error/variant mapper values (issue #556).

Arithmetic hooks shipped for classes/enums, including reflected and explicit in-place methods (issue #512). One-field @newtype forwarding preserves nominal typing and explicit-hook precedence (issue #555). checked: blocks return typed integer overflow/zero-division errors for arithmetic written in the block; float div_checked and nontrapping MIN % -1 accompanied them (issue #557). The lexical private callback channel from issue #614 (B93) is superseded by independent declared function/closure error contracts in the checked-default compiler core. That core also supports lexical unchecked: suites. Their build modes followed (local PR #924): --unchecked trap checks the preconditions by default, --unchecked assume omits the checks outside std, and unchecked trap: always traps; caches key on the mode. Issue #764 extends assumed suites to list, slice-view and Bytes index reads, writes and borrows: indices must be nonnegative and in range, and both bounds checks and negative normalization are omitted. Trapping scopes, other indexing and std keep their contracts. The private callback channel is removed. Initial std migration covers math helpers, bytes and time: integer reductions preserve typed overflow payloads, separate float helpers retain infallible IEEE values, serial pipeline sums return public results, and key sorting transports fallible callbacks’ ordinary errors. Encoding sizes, stream extents, formatting counters, filesystem budgets and entropy-source offsets now preserve their existing error contracts or use bounds-proven opt-outs. Filesystem walks reserve only newly appended frames (issue #702); an allocation-failure regression distinguishes failure before reservation from failure while scanning a popped sibling. Coordinated adoption of remaining std modules, tests/examples and documentation is not complete and remains B14.

Core HTTP decimal/chunk sizes, networking/readiness, bounded text operations, deterministic random helpers and testing streams are migrated. HTTP accepts the full representable Content-Length range without overflowing an ASCII-digit intermediate (issue #703); larger lengths remain protocol errors. HTTP and DNS reply-buffer extents report existing allocation-capacity errors before consuming another chunk. FakeStream.fail_after now returns a precise overflow result and preserves the previous schedule on failure. Counter exhaustion returns stream OutOfRange before mutation. Representation-limit probes exercise the actual private helpers without huge allocations, and RNG golden/reference sequences, text-wrap compatibility/allocation tests and testing-toolchain checks pass.

URL output sizing, process request/output extents and incremental HTTP decoder arithmetic are migrated. URL expansion checks its final representation before allocation; terminal scans stop at the input endpoint. Process argument encoding reserves only additional bytes (issue #704), with zero-allocation spare-capacity and failed-mutation tests in debug and optimized output. Existing allocation causes remain intact. The process and readiness suites pass; incremental decoder parity tests pass apart from the still-unmigrated actor integration.

Typed HTTP framing and client arithmetic now preserve existing typed transport causes and progress. Complete chunk wire extents are checked before delivery; representation failure is OutOfRange, separate from invalid transport counts. Boundary tests use an affine cause type without constructing huge buffers and verify allocation-free arithmetic in debug and optimized output. Client deadline saturation, bounded counters and callback independence are explicit. The I/O range integration and typed composition/prototype examples pass; those cross-module dependencies were resolved by the coordinated checkpoint below.

JSON position arithmetic now reports the existing InputTooLarge fault before advancing a failing byte or ASCII-run cursor; input/output budgets and partial output progress remain intact. Unicode normalization arithmetic has explicit scalar-ceiling, successful-append and positive-index bounds. Both public APIs are unchanged. The component’s 35 runtime-check tests pass on an isolated build target, including optimized/unoptimized boundaries and the Unicode normalization corpus; independent review found no defects. A separately reviewed TOML/regex checkpoint adds 40 passing component tests and preserves parse errors, document edit atomicity, regex work/memory limits and replacement error precedence. Public Match lengths now expose typed subtraction overflow; TOML location exhaustion uses InputTooLarge with unavailable zero coordinates at the valid offset. XML/CSV and broader adoption were not yet included at that checkpoint.

The coordinated numerical, transport, server and actor checkpoint is integrated with independent source reviews. Ordinary integer SIMD operations return lane-typed overflow results; floats and explicit wrapping operations retain their contracts. Complex powers cover the full signed exponent range; matrix construction/indexing and statistics keep their typed errors. Diagnostic logger, server-report and actor counters saturate under the user-approved policy. Exact streamed progress, allocation extents, crypto/DER/UUID and JWT time bounds retain existing errors rather than hiding overflow.

Actor identity generations never wrap: exhausted slots and clock records retire. Mailbox rebasing preserves every target-owned child-exit fence and prefix order; affine callback arithmetic errors reach the parent without leaking actor heaps. Combined targeted runtime-check validation reports 733 passes, two failures in still-unmigrated HTTP body/cookie dependencies and one pre-existing ignored test. Full crypto, JWT, HTTPS, standalone server/client and actor TLS tests now pass together. The optional native-engine linalg bodies were initially unrun; broad documentation examples, remaining modules and full measurements were not yet included at that checkpoint.

Reviewed CLI/progress arithmetic now preserves exact work counts and output-failure state; the countbytes example exposes its aggregate overflow contract. Dataframe extents retain allocation-capacity causes, and plot counts/layout/rendering have explicit bounds without changing numerical aggregates. All 48 plot tests pass; the initial 13 dataframe native-engine self-skips were later replaced by real engine execution: all 17 dataframe and nine linalg tests pass, including native relocation and allocation cleanup after fixture-only arithmetic fixes. HTTP body/cookie helpers preserve semantic TooLarge errors and multipart/upload ownership. Actor HTTP has equivalent bounded lingering state without clamping actual consumed bytes; its 26 TCP/TLS/control/error tests pass. Remaining HTTP core failures still reach websocket. A reviewed proxy/helper checkpoint restores all 19 helper tests, preserving descriptor/cancellation/RFC/bulk-scan assertions. XML/CSV arithmetic preserves exact positions and explicit pending-output flushing at representation exhaustion. Temporary/system/watch/SQLite wrappers and compression retain their domain errors and resource/progress contracts; real procfs/inotify/SQLite and gzip/Brotli interop tests run. Actor HTTP interop retains curl and all 600 corpus mutations. Optional sqlite-vec and Docker cases remain unrun at this checkpoint.

The final reviewed std arithmetic checkpoints cover archive, HTTP files/SSE/ WebSocket, MIME/template/markdown, sketches and diff/image. Public mutable diff lengths return overflow Results. File metadata above i64.MAX is rejected before conditional validators (issue #706); all 11 file adapter tests now pass after MIME integration. The combined final component selection has 143 scoped passes, followed by image 23, Markdown 10, math 23 and registry/index 2 passes. The index reports no remaining diagnosed module. New byte-ring PNG filter operations are explicitly wrapping; parser columns, numerical sketch totals and semantic output progress remain checked rather than saturated. Independent review prompted a Markdown source-comment correction and an image OOM fixture change to exercise conversion rather than copying; both were applied and validated. These are focused checkpoints, not full-suite/native/export/performance acceptance. Reviewed parallel integer sums now expose precise first-overflow Results after workers drain, preserving serial-prefix behavior and callbacks-once semantics. All 30 pipeline tests pass. The initial implementation retains O(n) numeric values and combines them serially; no performance improvement is claimed. Generated split/count helpers have source bounds and exact extent errors; enumeration keeps its explicit trapping boundary. Reviewed early/advanced lessons, reference and skill examples use public callback Results and explicit worker error handling. The reviewed generic deque, heap, graph and LRU arithmetic checkpoint preserves public declarations and exact allocation/ownership contracts; its 49 targeted tests pass. Full executable documentation now has all 20 tests passing, rather than the initial 3 / 17. Reviewed fallible tree/pipeline reducers preserve ordinary callback Results, typed worker-error ownership and deterministic selection; 24 parallel and 37 parallel-pipeline tests pass. Pair adapters reconstruct only the existing two-field split and retain affine field/error cleanup. Registry success alone did not cover these generic instantiations. Native export/FFI fixture adoption preserves scalar and fieldless error ABIs, C/C++ guard-page assertions and exact resource cleanup; its 23 component tests pass. Repository fixtures/examples, full native and all-features validation were not yet complete at that checkpoint. An earlier all-features baseline compiled successfully but reported 4,465 passes, 1,109 failures and eight ignores; 27 confirmed self-skips are included in the reported passes. That snapshot predates the generic/native follow-ups and is not a current pass. A newer complete gate at ff52bd7e compiled with 4,813 reported passes, 813 failures and eight ignores, including the same 27 optional self-skips. No failed outer name is new relative to the older baseline, and no std-source diagnostic remains. Reviewed borrowing, tail/stack, control/trap and flow/shared fixture batches subsequently pass all 568 parent-run cases, including 258 identical failed names from that snapshot. No fresh whole-suite pass is inferred from these scoped results. Reviewed concurrency and collection fixture batches add 183 and 428 passing parent-run cases with native worker/guard-page/ownership oracles retained. The pre-existing propagated numeric-literal context defect (issue #710) is repaired through a non-evaluating success-payload hint. A rebuilt main baseline rejects the original source; normal/release native regressions and the unchanged unsuffixed-literal callable oracle now pass. Parent checks cover all 109 callable and ten literal-form cases. One unoptimized range-generator oracle was precisely updated with explicit user approval to require its bounded helper and caller-provenance calls; this does not establish zero-call or performance equivalence.

Reviewed generalized initializer errors preserve exactly the declared E, including generic whole-result parameters, aliases, named defaults and constructor values. Construction adds no AllocError implicitly. Partial initialization and affine error/default ownership are tested in both native modes; all 114 constructor/ checked-default cases pass. Issue #709’s partial-self closure-capture guard is integrated, while destructor/context exits remain infallible and C-export error layouts are unchanged. Generator/serial-pipeline fixture adoption has 184 passing cases with suspension, fusion, allocation and drop oracles intact; its corrected imported-callback test separately proves inlining and isolated cold/warm/edit cache behavior. HTTP qualification retains real TCP/TLS/curl/corpus/installed-toolchain assertions, with 15 passing cases and precise aggregate-overflow context. These are scoped checkpoints, not a substitute for the red whole-workspace baseline. The historical af8aa15d gate had 5,375 reported passes, 259 failures and eight ignores. After the reviewed final fixture and example batches, the complete host gate at 2ef36dce passes 5,650 tests with zero failures and eight existing ignores. All 259 exact failed names now pass, along with sixteen new outer tests; all 27 known optional self-skips execute their bodies. Real native dataframe, linalg, tutorial engine, Docker, libFuzzer and Neovim execution is recorded. Existing external engines were not rebuilt; transient host-tool conditionals are not exhaustively traced. Two selected ignored trials pass separately. This green snapshot does not establish performance acceptance or validate the pending shared Unit checked-suite repair (issue #712). Reviewed shipped examples and structure companions pass 20 structure tests and all three exhaustive-example tests. The explicitly run sqlite-vec transaction/inference preservation trial passes with deterministic test inference, not a real embedding model. Local skill assembly passes 19 sync tests and 152-page/837-target link validation; no publication, agent trial or performance acceptance is inferred. A native adapter audit found and repaired inline sort-error ownership (issue #711): typed caller-owned storage now receives the payload before the comparator returns. Reviewed native range/managed-error and layout regressions pass in both modes without heap allocation; all 21 checked-default cases pass after parent integration. Native probes did not reproduce wrong output before the repair; the pre-repair structural regression distinguishes the lifetime/layout fix. The independently reviewed Unit checked-suite verifier repair (issue #712) is also integrated: Unit success preserves its entry type stack, while nonunit behavior remains unchanged. Five verifier tests and four native both-mode tests cover mutations, cleanup, propagation and floating division. A new comparison baseline fork contains the identical production repair, explicitly labelled “main + #712 fix”; final common-fix build and full-gate validation remain separate. Reviewed call-retention probes preserve their arithmetic and error oracles and require actual optimized native leaf calls, not source-policy inference. Independent final-pin untimed review accepts all 124 check/native-build configurations, 284 exact runtime cases and six optimized native call gates on the common-fix baseline/candidate pair. The full 1000/4000-unit corpora compile and execute their designated checksum entrypoints in all four modes and both generated profiles. All 24 cleanup outcomes pass; four C/I zero-allocation intervals have four genuinely allocating controls. These bounded calling-thread checks do not establish allocation totals, timing, size or whole-suite acceptance. Reviewed checkpoint continuation observes all 5,667 registered pairs across stages: 5,659 passes and eight normal ignores, including all 27 optional bodies. The separate uninterrupted post-#712 run completes both zero-test doctest packages but fails one Caddy readiness assertion: concurrent fixtures selected the same HTTP port. Its 5,658 passes, one failure and eight ignores remain a failed gate; issue #719 tracks a separate fixture fix, not an established arithmetic regression. A later user-approved isolated fork applies only that reviewed Rust fixture patch, leaving production compiler/runtime/std and native Plenty fixtures unchanged. Its uninterrupted host gate passes 5,669 tests with zero failures and eight normal ignores, including all 27 optional bodies and ten added fixture helpers; both actual zero-test doctest packages complete. Independent review accepts this bounded host qualification. Neither PR branch is merged or rewritten, and the earlier failed run is not relabelled. Missing wrapper classifications, sampled-process limits, older compatible engines and the disclosed non-test pre-clean observation gap remain explicit limits. Exploratory runtime/compiler/code smokes pass 16 runtime samples, 32 compiler samples and 12 code reports, retaining full corpus sizes, native oracles, optimized call gates and allocation-proof integrity. One sample without warmups is not performance acceptance. The first two-job profile attempt remains blocked before either producer by issue #731’s authenticated-system-tool alias defect. The reviewed repair preserves strict non-tool alias rejection and consumes the exact authenticated record bytes; 102 profile and 182 combined controller mocks pass. The new external driver passes independent review and 58 synthetic methods, then a fresh two-job smoke succeeds.

The full approved schedules complete serially: 2466 timed general jobs (366 warmups and 2100 samples), all 22 code reports and all 16 official profile jobs. Exact attributed requests and approximate binary-unit byte displays remain separate metrics. Independent evidence review finds no issues with collection, not a general speedup or whole-proposal acceptance. Existing optimized code is inspected in 20 representative native symbols: guard/result/cleanup/return structures are accounted for, including the C→I corpus’s extra widening/tag test and shared-cleanup differences. A bounded native/export audit establishes no new defect; neither review claims universal ABI coverage.

The compiler benchmark template now uses precise independent arithmetic Results, without changing the original workload, controls or five tests. Four new tests bring its example suite to nine passes, including actual 1000/4000-unit checking, native checksums and mutation/failure paths. Fifteen arithmetic integration tests, formatting and strict workspace/all-target Clippy pass. Frozen comparison inputs remain unchanged. See collected results and limits for distributions, attribution, source bridging and bounded code analysis. Separate pre-existing archive-decimal and PNG-framing limits were recorded as issues #705 and #707, outside these arithmetic checkpoints. The archive decimal boundary fix addresses #705.

Class match patterns shipped with issue #558. Outstanding conveniences stay B14/B71 in the backlog.

B16: Values and syntax

Block if/elif/else supports whole-RHS bindings, single-place assignments, and explicit returns with the same contextual types, cleanup and lifetime restrictions as match-valued statements (issue #674). Arbitrary expression nesting remains intentionally unsupported; see conditional values.

Tuple component borrows (issue #68), once-only numeric augmented-assignment index evaluation (issue #32), checked casts/explicit wrapping conversions (issues #39/#65), RangeError conversions and unsigned-negation checks (issues #66/#67) shipped. Copying slice syntax and NonZero[i64] range/slice steps shipped (issues #579/#584, local PR #698). Negative literal tuple indices are supported (issue #583); nested Shared handles gain owners in copies/slices (issue #582).

Hex/octal/binary integer literals and \u{...} string escapes shipped (issues #563/#564). @redacted classes/enums format as TypeName(@redacted) through automatic paths, including std secret-bearing types (issue #586). Bytes now prints a round-tripping byte literal (issue #596). See numbers, text escapes and redacted types. Recursive tuple match patterns and nested tuple unpacking in declarations, assignments, loops, comprehensions and generator-expression targets shipped (issue #675). Product coverage, affine cleanup and borrowed component loans use the existing pattern and tuple machinery. Guards, general aliases and starred targets remain unsupported; see matching and tuples.

Remaining inference/value candidates stay B16 in the backlog.

B17: Tooling and packaging

plenty script delivers executable shebang scripts, lossless Unix argument forwarding, strict optional embedded manifests, isolated path/Git resolution, cached and durable locks, and original-source diagnostics. Per-script private cache state and resolution locks keep generated files out of source directories; warm runs reuse objects and eligible Linux executables. Check/compile modes, editor header handling and cache inspection/cleanup accompany the runner. A script test command and broader executable-cache platforms remain outside this delivery. The completed proposal records the rationale (local PR #851).

File imports/visibility, reserved embedded std, the plentyc/plenty split, strict manifests, pinned Git/path dependencies, locks/offline mode, workspaces, project commands/tests, per-target build reuse and --package-graph shipped (issue #45, local PRs #124/#130). Native artifacts use a fixed baseline, deterministic archives and runtime path remapping (issue #58). Release archives and distribution smoke tests are established (issue #151, local PR #187); releasing owns the current workflow.

Default stripping/--no-strip (issue #500, local PR #575), removal of linked Rust panic/backtrace code (issue #503, local PR #576), executable size budgets (issue #502, local PR #580) and reachability-rooted destructors (issue #506, local PR #581) reduce unused native linkage. Release/no-strip option ordering is fixed (issue #590, local PR #732).

Single-file compile-and-run now caches linked executables by default on supported Linux toolchains, including hashbang launches. Native dependency and toolchain identities invalidate reuse; entries share cache inspection, cleanup and eviction. Linker wrappers, including GCC-private overrides, are excluded before executable lookup (issue #676). Wrapper regressions isolate GCC’s private search paths and verify linker selection, including the manylinux GCC toolchain that prefers its private ld over PATH (issue #679). Explicit build outputs, project runs, custom linkers, non-Linux hosts and instrumented builds still link normally. See the executable cache contract.

The language server provides diagnostics, hover, definition, completion and simple-local inferred type hints, with Neovim integration (issues #155/#554, local PR #196). Compile-time constants/static assertions, embedded data and if const target.* shipped (local PR #628). Module constant indexing and import/module diagnostics were improved (issues #562/#568/#587).

The local semantic-search trial has repository extraction, incremental synchronization/deletion detection and hybrid queries in Plenty; a pinned Python/FastEmbed subprocess performs inference. Packages and linking own current contracts. Outstanding tooling/investigation scope stays B17 in the backlog.

Engine-notices packaging reads the Plenty version directly from the root Cargo.toml with Python’s TOML parser, avoiding the nested Bash version lookup that failed in Windows release CI (issue #687). Regression fixtures cover release and prerelease versions, LF/CRLF manifests, unrelated version keys and a repository path with spaces/Unicode. These local fixture checks do not substitute for a native Windows release build.

Generated standard-library foreign wrappers retain their loader provenance for unused-definition pruning (issue #688). Bare unused std.dataframe/std.linalg imports emit no engine symbols in debug or optimized objects; required native calls retain ordinary engine discovery. This fixes the Windows release smoke failure after packaging; native Windows CI must still confirm the archive run. Windows executable emission also omits unreachable user helpers and generators from its temporary COFF object (issue #690), eliminating native imports only those bodies require. Explicit objects, library objects and dumps retain their inspection contract; missing reached imports remain linker errors. Actual COFF links cover the unused-helper case and a reached-call negative control, in both optimization modes. Native Windows execution remains a CI qualification requirement. See the emission policy.

Distribution output appends the native executable suffix to the source stem (issue #697), producing analysis.exe on Windows while preserving dotted stems and Unix names. Filename regression tests cover both suffix conventions; the existing native archive smoke test still checks dataframe distribution and relocation. Native Windows execution remains a CI qualification requirement.

B21: Documentation authoring

Executable Markdown checked by tests/test_tutorial.rs remains the lesson source; generated literate lessons were evaluated and not adopted (local PR #537). See the documentation design. This is a concluded decision, not an implementation waiting in the backlog.

B22: Native concurrency

Scoped native threads, structural eligibility/reachable effect checks, consuming jobs with failed-start ownership, bounded MPMC channels, executors, ordered fallible map/tree reduction, cooperative cancellation, timed waits and receive-pair selection complete the committed batch. Named workers accept owned arguments and map callbacks share eligible immutable reusable environments (issue #28). Sending-task join/shutdown guards (issues #12/#74), receiving-worker join checks (issue #108) and blocking future-result checks (issue #109) landed.

Atomics/mutexes can be shared through scoped loans or Shared, with cycle and lock-held wait checks (issue #326, local PR #366); HTTP client sharing followed (local PR #369). Parallel terminals drain before return and borrow list sources in place (issue #455, local PR #571).

See threads, channels, executors and parallel operations. Broader concurrency/task models stay B22 in the backlog.

B29: Generic checking

Callable bounds on data, use-site instantiation diagnostics and per-value argument blame (including aliases/projections) shipped (issues #40/#518/#519/#526, local PRs #620/#638). Generic class/enum methods are checked/emitted when used or required by a protocol/destructor (issue #515, local PR #616); --check-all-methods checks the program’s own instances, not dependency/std types (issue #517, local PR #621). See generic data. The library-owned operations investigation completed in issue #145 (local PR #177): existing owning wrappers and optional capabilities meet the assessed use cases; no language or stdlib change was approved. Future default/final methods and private-hook grants belong to the separate protocol-body proposal. Additional bounds/default-body contracts remain B29 in the backlog.

The boxed protocol investigation records a proposed Box[P] contract, compiler/runtime integration map and qualification criteria. Four original probes established the then-current rejection of protocol-valued boxes, concrete boxed method calls, and single-method erasure of class/enum instances through BoxedCallable. The subsequent implementation adds explicit Box[P](value)?, shared/mutable method tables over one concrete payload, fixed protocol argument inference, identical-protocol generic bounds, aggregate storage and descriptor-driven cleanup. Executable lessons cover heterogeneous lists and dictionary values. Borrowed outputs, consuming or generic requirements, copying, keys and native transfer remain unsupported. Shared contents and hidden executor/future, channel-endpoint or erased-callback payloads are conservatively rejected. See the implemented contract.

B32: Callback storage

Owned concrete reusable/consuming environments and worker eligibility work in ordinary aggregates. Explicit fallible BoxedCallable supports heterogeneous owned shared-call storage (issue #25). See the storage decision and boxed callable reference. Additional erased modes stay B32 in the backlog.

B36: Named arguments and defaults

Named functions/methods/constructors have static name binding, source-order argument evaluation, fresh per-call defaults, callee-local propagation and nonescaping default loans. See call rules and the completed design. Additional call forms stay B36 in the backlog.

B37: Ordering

Natural list sorting, swap and multi-target assignment (issue #31), lexicographic strings (issue #22), allocation-free unstable sort_by_key/fallible sorted (issue #64) and tuple relations/heap ordering (issue #111) shipped. Runtime introsort with heapsort fallback uses generated native key comparators and cached tuple field offsets (issue #331). See lists. Additional ordering contracts stay B37 in the backlog.

B72: State and ownership

HTTP assembly no longer needs placeholder heads, empty-string copies or dummy Results (issues #134–#136). Owned method receivers reuse by-value call rules; std.text.Builder.finish() adopts them. Field moves from owned locals/self are path-tracked with reassignment and cleanup rules (issue #534). The owned-response experiment validated retry ownership, partial-write reporting and exactly-once cleanup, without implementing real HTTP integration. See field moves. Remaining experiments/ownership extensions stay B72 in the backlog.

B75: Fluent pipelines

All four proposal stages shipped: local borrowed/owned fused plans, adapters/pair callbacks and fallible terminals; explicit owned .generator() conversion; chain/flat_map and callback-free filtering/reductions; resumable/lendable plans; and explicit chunked parallel terminals. std.iter was removed in favor of this surface (local PR #545). Generator methods resolve concrete producers (issue #401), known lengths reserve capacity, and affine cleanup/callback performance fixes landed (issues #398/#399/#452/#455).

Delivery evidence includes local PRs #540/#543/#546–#548/#551/#552/#554–#556/#560/#562; issue #371 is fixed. See pipelines and callback measurements. Only the remaining extensions are B75 in the backlog.

B84: Copy classes

Nongeneric @copy classes copy on ordinary use when every field copies and no __del__ is declared (issue #535, local PR #655). This also introduced the closed declaration-annotation mechanism. See copy classes. Generic @copy classes copy when their type arguments copy, and a field that moves for every argument is rejected at the class; the built-in CopyType bound requires a type argument that copies (local PR #785). Dictionary keys remain B74 in the backlog.

B90: Owner-to-view conversion

&xs, &mut xs, &s, reference bindings and b"..."/str literals convert to whole views at view types, and &Shared[T] converts to &T like Box (local PR #784). Byte-reading std APIs, including the prelude byte writers and every public read-only &Bytes input, take &[u8], and ByteSource is removed; &b"..."[a:b] views literal data and a shared view argument can view a call result (local PR #800). See slice views and the proposal.

Retired foundations

These original B IDs have no separate active initial-delivery task. A delivery may have later evolved; use linked references for today’s behavior, particularly dictionary writes and exact target support.

IDDelivered scope / decision
B01Linker selection via CLI/Rust options, cc-compatible drivers, verbose output, argument fidelity and deterministic native/mock tests; mold/lld preference (issue #314). Linking.
B02Application object output and embedded-runtime extraction without a linker on PATH, with external-driver tests. Commands.
B03Validation of the packaged x86-64 Linux GNU target, runtime triple and pointer layout before emission. This does not claim additional targets shipped. Runtime packaging.
B04Trusted .plentyi C scalar/opaque-pointer imports and native ABI fixtures. Declarations.
B05Ownership-aware pointer/UTF-8/C-string adapters, opaque handles, cleanup and partial/failing transfer fixtures. Adapters.
B06Static/shared C exports, scalar/Result adapters, factories/destruction, transfer contracts, embedded interfaces, SHA-256 guards and verification. Exports.
B07Typed resident runtime loaders, exact metadata/symbol checks, ownership-preserving adapters and allocation-free error tests. Loading.
B08Function values, explicit reusable/consuming closure environments, factories/inference, callable constraints, captured generator frames and lifecycle cleanup. Closures.
B09Initial generic classes/enums, IntType bounds, inference, generic methods, structural contracts, specialization limits/reuse and allocation/drop checks. Generics.
B10Box/collection-bounded self/mutual recursive source types and aliases, finite type graphs, and allocation-free cleanup of 100,000-node chains on a 256 KiB stack. Recursive data.
B15Qualified/nested/or/scalar patterns, inferred generic arguments, whole-scrutinee case _ as name and match-valued bindings/assignments/returns (issues #36/#114/#118). Omitting user enum qualifiers was declined. Matching.
B25Concrete inline generator frames, producer specialization and no-allocation construction/move/resume/drop tests. Resume scratch storage covers the full inline yield layout, including large records (issue #768). Generators.
B26Copyable inline ranges with allocation-free calls, indexing, membership and repeat traversal. Ranges.
B30Ordinary nested updates without enclosing copies/allocations, with saved-index/RHS ordering and alias/failure cleanup tests. Dictionary updates subsequently moved to insert/get_mut (issue #580). Assignment.
B31Callable-simplicity audit retained Callable/Closure, ordinary generic callback records, mismatch guidance and closure source locations (issue #24).
B34Bounded post-concurrency cleanup/review through local main 736fe80: ownership, diagnostics, inline storage, cleanup, typed reporting, collection conveniences, compiler scaling, reproducibility and docs. Later channel checks are recorded under B22.
B35Inline classes/tuples/enums with allocation-free storage construction, recursive heap boundaries and boxed C handles (issue #17). Fallible argument expressions/custom initializers retain their failures. Classes.

Other retired premises: deterministic runtime cleanup replaces leaked string/heap storage; strings already carry lengths and embedded NULs; the precompiled runtime is embedded, not rebuilt from C per program. Function type inference, unit type arguments in function specializations (issue #110), catch-all matching and the mdBook migration are delivered. Integer bitwise operations, checked shift counts and explicit wrapping arithmetic shipped for issue #33.

Outstanding work is not duplicated here.

Constructor function values

B93’s constructor subset (issue #618): class constructors and payload-bearing variants, including Some, Ok, and Err, are positional function values. Concrete expected callable results specialize generic constructors; explicit instances work without context. Pipelines accept constructor values, and shared or mutable callback arguments can borrow their code addresses. Fallible class initializers keep Result[Class, AllocError] as their callable result. See function values and remaining B93 scope.

B95: Integer domain types

Positive[T] and NonNegative[T] for signed integers now provide literal conversion, fallible run-time conversion, private construction, .get(), domain-preserving arithmetic with checked overflow, ordinary comparisons, copying, scalar formatting and hashing. Explicit NonZero.from(positive) and NonNegative.from(positive) widen without failure or allocation. Invalid element diagnostics suggest NonZero[uN] or uN only for unsigned integers, not floats or other unsupported types (issue #700).

See integer domains. Plain integer dividends accept NonZero[T] and Positive[T] divisors for //, % and augmented assignment, preserving floor semantics while omitting the zero guard (issue #664). A NonZero divisor retains signed division overflow handling; a Positive divisor also removes the impossible -1 overflow/remainder guard. Checked blocks infer only the remaining failures. The std migration is B95: Std limit domains.

B95: Positive concurrency counts

Builtin channel capacity, ThreadPoolExecutor worker/queue counts and parallel pipeline chunks now take Positive[i64] (issue #665). Literals are checked at compile time; computed counts use Positive.try_from once. Channel construction returns AllocError; PoolError retains only Allocation(AllocError) and Thread(ThreadError). Parallel chunks need no run-time positivity trap. Existing queue ownership, startup rollback, bounds and allocation-failure contracts are preserved.

See channels, executors and parallel chunks. The remaining std APIs are B95: Std limit domains.

B95: Std limit domains

Std parameters and Limits fields with a fixed lower bound of 0 or 1 take NonNegative[i64] or Positive[i64], completing migration groups A–C of the domain types proposal. This covers url, kv, markdown, json, toml, xml, regex, template, text, fmt, report, log, csv, io, process, fs, tempfile, system, archive, compress, image, stat, math, sketch, http (client, body, form, multipart, typed, websocket), jwt, crypto and the actor, actor networking, TLS and HTTP server entry points.

The now-impossible Err cases are gone: UrlError.InvalidLimit, BodyError.InvalidLimit, KvError.InvalidLimit, xml Fault.InvalidLimits, CsvError.OutOfRange, BytesError.OutOfRange, StartError.Limit and SystemError.InvalidInput among them. Fixed ceilings (json and xml depth 1024, regex nesting 1000, template and fmt depth 64, hkdf size 8160, actor heap limit, image dimensions) trap with a message naming the operation and argument. std.actor.net.connection returns AllocError. The conventions state the rule for new APIs.

std.http.Limits.max_line/max_head (>= 2) stay i64 until Bounded (B96). Fixed std ceilings now reject statically known out-of-range arguments (issue #686), including module-domain constants, named arguments, generated config constructors, direct field updates and replace of a field. Std-only metadata preserves the existing integer types and runtime traps for dynamic values; indirect calls and writes through separately borrowed fields keep those runtime checks. See the checking mechanism. Std first used private getter functions for constant limits; domain-typed module constants (issue #685) replaced them.

Local dictionary views

B91’s dictionary-view consistency slice (issue #624) makes values() and items() reusable shared local views with constant-time len and pipeline sources. Set observations, algebra, and filter mutations accept dictionary key views. Views keep their dictionary borrowed and remain local-only; mutable value/item iteration stays immediate. See dictionary views and the remaining B91 scope.

Issue #803 documents (&mut d).values() and (&mut d).items() as the way to update every value without a per-key get_mut. values_mut(), items_mut() and iter_mut(), indexed field or method writes, and writes through a shared value loop now name that form in their diagnostics.

Structural key diagnostics

Rejected dictionary keys, set members, key views and KeyType arguments now identify the first unsupported nested component in declaration order. Paths name class fields, enum variants and one-based payload/tuple positions; custom cleanup and anonymous sums have specific explanations. Key admission remains unchanged. See the generic constraint reference and B74 for remaining key design decisions.

Optimized builds by default (issue #723)

All compilation entry points now optimize generated code by default. The fast-compile profile trades runtime optimization for shorter compilation without changing language checks. Package output uses target/optimized/ and target/fast-compile/; --release remains an alias for the default. Profile-separated cache reuse and explicit two-profile regression tests accompany the change. Integration with call-site inlining preserves caller handling of C string conversion errors (issue #724) and shared cleanup for tail calls with managed locals (issue #725).

Checked-default validation

Branch-scoped validation for local PR #886. This records implemented adoption and bounded evidence, not a merge, universal ABI certification or performance superiority. The proposal records the design; implementation and validation records its integration history.

Compared contracts

The frozen baseline is 8e8fccd9: main + the common #712 Unit-verifier fix, not pristine or current main. The frozen candidate is 830763c9. Compiler, runtime, flags and native-tool provenance were authenticated separately for each role. Later controller repairs and benchmark-template adoption did not rebuild or replace these comparison compilers.

ModeCompilerArithmetic contract
TBaselineOriginal unadorned trapping arithmetic
CBaselineOriginal explicit checked recovery
ICandidateImplicit function-level checked arithmetic
UCandidateExplicit trapping unchecked:

T/U and C/I compare matching behavior. Trap-process and handled-error timings are not interchangeable. Operators produce plain successful values; functions and callbacks independently declare their public Result contracts.

Semantic and native qualification

The final common-fix native matrix passes 124 checks, 124 native builds, 284 exact runtime cases, six optimized call-retention gates and 24 cleanup cases. All 16 full 1000/4000-unit corpus configurations compile and execute their designated checksum entrypoints: 1004057 and 16016057. Four C/I calling-thread zero-allocation intervals have four genuinely allocating positive controls. These checks are not an absolute allocator census or an execution of every function in a corpus.

The uninterrupted post-repair host gate on isolated validation revision 8ca6fc5a passes 5669 tests, zero failures and eight normal ignores, with both actual zero-test doctest packages completed. All 27 required optional bodies execute, including real proxy, dataframe and linalg cases. Real HTTP fuzzing requires at least 20000 completed iterations; headless Neovim assertions execute.

That host revision contains only the separately reviewed Rust proxy-fixture patch from local PR #918 over the checked-default source. Production compiler, runtime, std, Cargo and native Plenty fixtures remain unchanged from 830763c9. This is not a direct gate of either untouched PR branch or all current main. Ten authenticated same-ID container removals and subsequent absences are proved; missing final wrapper classifications remain unknown. Older external engine builds establish compatibility, not fresh-engine qualification. Ten non-test example paths’ pre-clean absences remain unobserved; critical build freshness is separately verified.

The earlier uninterrupted gate remains failed: 5658 passed, one Caddy readiness failure and eight ignores. Two fixtures selected the same port. Later success does not relabel that historical result.

Repository benchmark template

The compiler benchmark generator adopts precise independent Result, Ok and callee ? contracts, with no per-operator wrapping. Original options, defaults, 1000/4000 scaling, declaration/operation order, warmups, timing boundaries, cache policy and all five existing tests are unchanged. All 1000 generated unit bodies match the frozen I corpus; only main replaces its old Failure with the precise I/O, allocation, overflow and zero-division union. Frozen inputs are unchanged.

Four new tests bring example coverage to nine passing tests. Both complete 1000/4000 sources type-check, freshly compile and execute exact original checksums. Failure cases preserve earlier field mutation and require independent callee/main error declarations. Fifteen arithmetic integration tests, formatting and strict workspace/all-target Clippy pass. A separate target reuses inactive dependencies only after cleaning both compiler/runtime packages and checking critical artifact absence. These are functional tests, not new performance data.

Final helper CI initially reports 333 tests with 60 errors and three normal skips: the historical-template hash guard correctly rejects the newly adopted live example. Issue #732 separates the inputs: an exact full historical Rust fixture retains SHA 14de3ba2, authenticated in one byte read before extraction. Current example changes cannot affect it; missing/tampered fixtures fail closed without a fallback. All 62 regenerated source files and 142 complete case expectations remain byte/value-identical to the frozen native inputs. The failed CI log and old helper remain preserved; focused 84 acceptance and 102 profile tests pass. No comparison compiler, frozen input, measured result or receipt is refreshed.

Controlled general collection

One serial, interleaved schedule retains 2466 timed invocations:

  • Runtime: 270 warmups and 1620 samples; five warmups and 30 samples per group.
  • Compiler: 96 warmups and 480 samples; three warmups and 15 samples per group.
  • Total: 366 warmups and 2100 post-warmup samples; 86 complete groups and 75 available paired-ratio distributions.

All slots pass their original oracles. The 350 timed exit-1 arithmetic traps are required outcomes, not waived failures. Recovered repeated errors remain separate outcomes. No sample, warmup, failure or outlier was discarded. Duplicate final command-log metarecords are not additional executions. All four general phases pass; the helper’s packet_complete=false and acceptance_complete=false remain unchanged. Collection success is not whole-proposal acceptance.

Selected paired wall ratios are median [p10, p90], not confidence intervals:

Runtime success caseU/TI/C
Tight loop0.986 [0.880, 1.076]0.982 [0.906, 1.052]
Calls0.993 [0.877, 1.141]1.003 [0.886, 1.144]
Text full output0.978 [0.903, 1.065]0.982 [0.926, 1.072]
Text success0.964 [0.948, 0.986]0.982 [0.964, 1.004]
Compiler action1000 units U/T; I/C4000 units U/T; I/C
Check1.002; 0.9851.012; 0.976
Debug object1.009; 0.9811.009; 0.987
Debug AOT1.002; 0.9861.000; 0.990
Release AOT1.015; 0.9901.008; 0.989

Complete distributions, including all error paths and available C/T and I/U pairs, remain in the local review evidence. Within-group (p90-p10)/median ranges from 2.43–55.13% for runtime and 1.12–14.77% for compiler groups. These whole-process observations do not establish a general speedup: startup, parsing, printing and teardown remain included. Compiler wall time includes the GNU-time wrapper; its maximum observed 327952 KiB RSS covers the compiler and descendants, not the whole machine or Rust compiler-build cost.

Code and allocation evidence

All 22 code reports pass, retaining 44 debug-object/optimized-linked-ELF structures and 110 nonempty compiler dumps. Corpus IR excerpts select work999/work3999; this is not a function-by-function audit of all generated functions. Linked ELF includes runtime/startup effects and is not an optimized application-object measurement.

Read-only optimized-IR/native analysis inspects 20 actual linked symbols: kernel, caller, text consumer and each corpus’s designated work function in all four modes. Counts below cover only each named symbol, excluding callees:

TargetOverflow guards T/U; C/IResult-tag tests T/U; C; IRelease sites T/U; C; I
Tight kernel1; 10; 0; 00; 0; 0
Caller2; 20; 1; 10; 0; 0
Text consumer1; 11; 1; 11; 2; 2
Each work999 / work39996; 21; 5; 62; 5; 4

The first three targets match within each behavior-preserving pair. Selected successful loop passes contain nine instructions in every tight kernel, and 21 in T/U versus 24 in C/I callers, excluding the retained leaf body. These are static path counts, not measured instruction totals or a cross-contract speed comparison. Arithmetic uses plain values and shared failure exits, not successful per-operator Result objects. Text’s second C/I release site is failure cleanup, not a second release on success.

Both corpus sizes independently show one additional C→I Result widening/tag test and one fewer release site: I sends normal list cleanup through the common exit. Stack reservation changes from 0x120 to 0x110. The six-versus-two local guard count does not mean lost protection: T/U inline squared-length arithmetic; C/I retain that independently fallible call. Earlier field updates and ordered cleanup remain before later failure handling.

Selected native returns use RAX alone for T/U kernel/caller values, and RAX/RCX for recoverable kernel/caller and every text/corpus result. Optimized CLIF agrees; no behavior-preserving pair changes its return carrier or introduces visible hidden return storage. This does not prove all functions or ABIs equivalent.

One official full-profile invocation passes all 16 separate compiler jobs: T/C/I/U × 1000/4000 × count/bytes, normal --emit-object. Every report retains 39 allocation and 39 timing rows, including all five required minimum phases and role-specific source locations.

ModeUnitsExact attributed requestsApproximate attributed bytes
T10003341485706.2 MB
T4000133738692.7 GB
C10004388551894.7 MB
C4000175289003.5 GB
I10004349233885.6 MB
I4000173822193.4 GB
U10003349154709.0 MB
U4000134144502.8 GB

Counts are exact attributed successful allocator requests, including realloc new-size requests above mimalloc. Bytes are formatted APPROXIMATE values using binary 1024 units, not recovered exact totals. Count and byte values come from separate processes: do not divide them to infer an observed average size. Exclusive attribution excludes nested measured guards on the same thread but includes unmeasured callees; worker attribution is not a complete process census. Calls mean phase invocations. Inclusive, potentially parallel profiling timings must not be summed as latency or compared with uninstrumented timings.

All eight approved C→I/T→U comparisons retain 312 common-phase entries: 118 exact count fractions, 118 approximate byte ratios and 76 unavailable zero-denominator ratios. Added/removed phase maps are retained; no missing phase was dropped. No retained-heap, generated-runtime allocation or instrumentation-equivalence claim follows.

The first two-job profiling attempt remains blocked before either producer ran because of issue #731’s system-tool hard-link guard. The reviewed correction admits only authenticated native-tool context, binds fresh link-count/ctime and consumes the exact authenticated record bytes. Source, proof, helper and output identities keep strict nonalias rules. A new independently reviewed driver, 58 synthetic regressions, a successful two-job smoke and the later full16 run are separate evidence; the original failure remains unchanged.

Observation limits

Collection was serial during a user-reserved window, without other managed builds, tests or writers. Fresh pre/post checks retained ordinary desktop activity without noise subtraction. Endpoint aggregate idle exceeded 98% for general collection and 97% for profiling; endpoints do not prove continuous inactivity. No cache flushing, affinity/governor changes or foreign-process signals were used. All raw reports, streams, artifacts, warmups and samples remain retained.

The bounded native/export audit preserves typed caller-owned error storage, Unit entry-stack validation, ordered cleanup/draining and earlier successful mutation. Executor-reduction opcode 9’s seven initialized operands are distinct from the separate collection dispatcher’s five-operand limit. Existing C adapters retain explicit status/output marshalling and reject unsupported error layouts; no general cross-language native Result ABI is invented.

This is x86-64 host evidence, not universal cross-ISA, sanitizer, all-FFI or all-Rust-arithmetic coverage. Independent archive-decimal and PNG-framing defects were recorded as issues #705 and #707, outside this arithmetic adoption. The archive decimal boundary fix addresses #705.

Completed library and application work

Completed-work index · Remaining work

The initial library slices below shipped. “Complete” refers to that bounded slice, not every extension of a module. The backlog owns all remaining candidates; linked references define current behavior and platform limits.

B38: Library breadth

Issue #49’s initial B39–B70 slices are delivered. Modules are implemented in Plenty unless native complexity/security warrants a wrapper (issue #505, local PR #578); large native dependencies use separately packaged engines (issue #504, local PR #579). The catalogue rationale records the scope decision, not a second task list.

DeliveryContract / evidence
XMLBounded UTF-8 events and validating writes without DTDs/external entities (issue #462).
Advisory file locksOwned shared/exclusive Linux locks, nonblocking acquisition and cancellable deadline waits; close-only lifetime preserves inherited descriptions (issue #774).
Filesystem watchBounded Linux inotify batches, overflow reporting and readiness integration (issue #484).
TemplatesBounded Jinja-style interpolation, filters, conditions, named inheritance and HTML escaping over explicit JSON (issue #499); reusable bounded text and quote-specific attribute escaping (issue #823).
LinalgFinite f64 matrices, solves and dense decompositions over allocation-safe faer kernels, with engine-parameterized build/packaging/CI (issue #490).
MIMERFC 9110 media parsing/building/formatting/matching and the extension table used by HTTP files (issue #485).
ImagesPlenty over std.compress: bounded owned 8-bit buffers, all PNG colour types/depths/Adam7 decoding, encoding, crop, conversion and nearest/bilinear resizing; dimensions/bytes checked before allocation (issue #476).
StatisticsFinite-f64 descriptions, inclusive quantiles and normal/uniform/exponential distributions with explicit fixed-draw sampling (issue #494).
SketchesDense HyperLogLog, reservoir quantiles, Misra–Gries bounds, static xor/scalable cuckoo filters, bounded memory and versioned checkpoints (issues #496–#498).
Key/value databaseNamed ordered byte maps, transactions and bounded scans in Plenty over SQLite; versioned file schema and engine decision (issue #495).
UUIDCanonical text/network bytes, OS-backed v4/v7 and allocation-free numeric sort keys (issue #461).
SystemBounded Linux procfs snapshots for CPU/memory/load/uptime/block devices/disk I/O/processes (issue #487).

Remaining breadth is B38 in the backlog.

Issue #705 corrects the archive PAX decimal parser to accept every nonnegative i64, including its final eight values. Boundary tests cover both optimization modes, overflow and malformed digits; the separate TAR content-length cap remains 2⁶² bytes.

B39–B70: Initial delivery slices

The local issues listed here recorded acceptance checks, ownership/allocation contracts and prerequisites. They are completion evidence, not active task assignments. Extensions retain their original B IDs in the standard-library follow-ups.

ID / local issueDelivered subset
B39 / #76Bytes: owned buffers, checked ranges, explicit copies, borrowed access, list-style indexing, loops, native fill and bulk scans (issue #115). Copying slices (issue #579), borrowed &[u8] subranges and mutable borrowed loops (issues #581/#594) shipped. Static bytes literals and ByteSource support in byte operations and std stream writes shipped (issue #156, local PRs #198/#203). Original lossless Unix transport remains list[u8] (issue #61).
B40 / #77Binary streams: statically dispatched reader/writer/seek/close, explicit partial progress, exact/bounded reads/writes/copy, fixed buffering and distinct flush/close failures, with file/process/memory adapters over Bytes. Sequential binary Cursor fields add checked reads/writes, payload caps and contextual truncation (issue #658).
B41 / #78Error context: typed operation/path/cause, explicit conversion, descriptions, stderr fallback reports, unit Results (issue #110), Formattable causes and readable entrypoint reports.
B42 / #79Time: checked durations, monotonic instants/deadlines, timestamps and sleep; typed std time APIs and cancellable waits (issue #635). Calendar/time-zone/RFC 3339 support is the official external plentylang/tz package (issue #463); Zoned.until/since calendar differences shipped in v0.3.0 (issue #566).
B43 / #80Environment/process: lossless variables/snapshots/cwd, argument-vector spawning without a shell, child env/cwd, piped streams, bounded deadlock-free collection, exit/signal status, termination and reaping. Shlex splits/quotes shell words without executing/expanding (issue #478).
B44 / #81CLI: validated schemas, typed/repeatable arguments, --, help/errors and lossless paths. Progress adds terminal-only throttled bars/spinners/counts/rates/ETA (issue #477); fileinput adds buffered fallible line iteration over native/raw command-line paths and stdin fallback (issue #553).
B45 / #82Formatting: static display/debug contracts, scalar/callback aggregate formatting, bounded partial-progress streaming and fallible strings. F-strings add field expressions, conversions, debug =, checked Python-style specs and nested fill/alignment/width/precision, returning FormatError; formatter output avoids intermediate strings (issue #157, local PRs #200/#202).
B46 / #83Logs: leveled typed events and generic caller-selected sinks (logfmt text, bounded memory and nonblocking channel/drain), clock policies and bounded lines.
B47 / #84Text builders/join/scalar wrapping, Python-style paragraph tools (issue #654), and Unicode 17 normalization, combining-class/Mark classification. UAX #29 segmentation/UAX #11 width ship in external plentylang/unicode-text (issue #465).
B48 / #85Regex: Plenty Pike VM with explicit pattern/nesting/repetition/instruction/memory/work/output/match limits; text/bytes captures, iteration, split and replacement.
B49 / #86Collections: natural/comparator heaps and generator drains, including tuple ordering/heap values (issue #111). B75 pipelines superseded the former std.iter adapters.
B50 / #87Random: owned version-stable seeded xoshiro256** generators, unbiased ranges, shuffle/index sampling/byte fill. Secrets: OS-only random fill/tokens/unbiased integers over getrandom, with no fallback.
B51 / #88JSON: strict RFC 8259 event decoder/validating encoder/value tree, depth/input/output/duplicate-key limits, located errors and std.io integration.
B52 / #89TOML: bounded strict TOML 1.0 parser/writer with located errors. Format-preserving edits add atomic in-memory changes and explicit path limits (issue #475).
B53 / #90CSV: streaming reusable-record reader/writer, explicit delimiter/quote/newline/width/size policies, strict RFC 4180 quoting and positioned errors.
B54 / #91/#489/#656Encoding: strict hex/base64/base64url/base32/base32hex, MIME wrapping, validated alphabets/padding and explicit-endian integer/bit-exact IEEE binary32/binary64 packing.
B55 / #92Filesystem: descriptor-rooted no-follow operations, anonymous publication, bounded copy/delete and byte globs; tempfile resources with explicit retention/cleanup. Recursive copying and selected metadata preservation add shared bounds, explicit partial-tree recovery and strict ordinary permissions/timestamps (issues #659/#660). Directories resolves HOME/XDG config/cache/data/state (issue #472), with Linux trusted-parent/publication limits documented. Explicit private creation modes and held-handle permission updates avoid create-then-chmod windows (issue #773). Link creation and canonicalization preserve native bytes and OS errors on workers, with explicit path limits and no confinement guarantee (issue #775).
B56 / #93Explicit thread/sync/channel/concurrent modules preserve builtin identities and ownership/effect guarantees. Atomics and mutexes shipped (local PR #362).
B57 / #94Test support: located assertions, Scratch, seeded RNG, fake clock/stream failures and short transfers; parallel package test processes, filters/list/capture/jobs/timeouts/seeds, per-test TMPDIR and allocation/file-I/O failure injection (local PR #142).
B58 / #95Every native primitive declares a worker effect, without std-wide exemption. Readiness supports bounded waits/cancellation and many-handle selection with direct token wakeups.
B59 / #96Sockets/DNS: owned TCP/UDP, bounded deadline/token resolver, descriptor-limit errors and measured accept bounds (local PRs #135/#140); immediate accept_now/read_now/write_now report WouldBlock for selector-driven TLS (issue #527). Initial platform: x86-64 Linux; Unix-domain sockets were declined.
B60 / #97URL: strict RFC 3986 absolute parsing/validated formatting, component-aware percent encoding, query pairs and bulk scans (issue #115).
B61 / #98TLS: verified TLS 1.2/1.3 client/server streams over std.net using vendored rustls/ring; PEM/system roots, deadlines/cancellation, certificate errors, client certs, ALPN/SNI, version/cipher choice, CRLs and selector-driven try operations (local PRs #145/#147). Recoverable rustls allocation, resumption/0-RTT and OCSP were declined for this slice.
B62 / #99HTTP/1.1: bounded head read/write and content-length/chunked/close-delimited framing, strict ambiguous-frame rejection and mutation corpus (local PR #139).
B63 / #100HTTP client: pooling, bounded deadlines/redirects, cross-origin credential stripping, proxies and idempotent replay; verified HTTPS, CONNECT, streamed uploads/Expect (local PRs #146/#154). Shared-client locking/atomics (issue #326, local PR #369) and pool accounting/cancellation fixes (issue #333, local PR #378) landed. ws/wss clients shipped (issue #467, local PR #593). Cookie jar/content decoding were declined; callers retain their own policies.
B64 / #101HTTP server: ordered typed routing, bounded admission/fixed workers, deadlines/logging, transport abstraction, graceful shutdown, parked idle keep-alives and accept-timed HTTPS handshakes (local PRs #143/#148). WebSocket upgrades/shutdown (issue #466, local PR #593) and detached close-delimited SSE shipped (issue #468, local PR #597).
B65 / #102Bounded forms/JSON/multipart temporary uploads, validated signed cookies/rotation, reverse-proxy/direct-TLS guidance and root-confined conditional static files (local PRs #144/#152). See body, cookie, proxy and files.
B66 / #103HTTP production qualification: framing mutations, curl/TLS interop, slow clients/load/shutdown, allocation/descriptor/heap failures, relocated tools and backend linkage (local PR #157). Host/absolute-form validation, parked heads/handshakes, signal drain, lingering close and TLS Finished acknowledgement fixes landed (issues #131–#133/#138/#142/#148). Nginx/Caddy reverse-proxy Docker checks, installed CLI/data and JSON-service workflows, HTTPS baselines and libFuzzer integration shipped (local PRs #171/#180/#183/#186). Production-ready label: 2026-10-18. Typed application routes shipped (issue #159, local PR #199). Proxy fixture port leases prevent cooperating same-process HTTP/TLS reuse across socket handoff and through cleanup (issue #719); deterministic non-Docker ownership tests cover concurrency and unwind. Separate parallel native qualification passes all ten worker/actor nginx/Caddy cases with authenticated removals and unchanged foreign containers (local PR #918), not a whole-workspace gate. External bind races remain; see reverse proxies.
B67 / #104Crypto: SHA-1 protocol use, SHA-256/512, HMAC and constant-time comparison via RustCrypto; BLAKE3 incremental/keyed/derivation (issues #473/#483). AEAD, Ed25519, ephemeral X25519 and HKDF use ring (issue #464). RSA/P-256, SPKI/PKCS#8/JWK import, strict PEM and Wycheproof checks shipped (issue #561); JWT builds on them (issue #488, local PR #697). RSA arenas make allocation failure explicit (issue #567); Ed25519 small-order rejection uses SmallOrderKey, while mixed-order keys remain accepted (issue #569).
B68 / #105Compression: bounded gzip/zlib/DEFLATE buffers/streams over zlib-rs, member/trailing-data/checksum policies and fixed per-stream memory. Brotli qualities 0–11 support bounded streams and fallible native allocation (issue #491).
B69 / #106Archives: TAR ustar/pax/GNU and ZIP/ZIP64 stored/DEFLATE streaming with root-confined staging extraction under explicit policy.
B70 / #107SQLite: vendored 3.54.0, typed statements, owned rollback-on-drop transactions, copied values and explicit length limits.

B48: Regex acceleration

Issue #742 adds bounded ASCII prefix filters across alternatives and classes, a lazy DFA for existence and fixed-length match offsets, and a one-pass engine for unambiguous ASCII transitions. The Pike VM preserves subgroup captures and handles unsupported patterns, Unicode fallback and exhausted DFA caches. Work limits include accelerated scans and fallbacks; optional scratch fits within the existing memory budget, and allocations remain fallible. Issue #752 adds pattern and search-budget compatibility checks before a cursor reuses cached transitions. Issue #753 removes per-byte text calls from the ASCII engines by borrowing UTF-8 storage once per search, iterates byte slices directly in the prefix scanner, and applies that filter at DFA restart states. The full regex-redux workload uses 44.36% fewer instructions than the merged #742 implementation. Issue #758 adds separate bit lanes for small alternative sets and bounded native substring batches for at most eight short ASCII prefixes. The unchanged full workload uses another 11.39% fewer instructions than main d951457d, with identical reference output. Scalar fallback lowering is isolated with CLIF variants and tracked in issue #760.

The unchanged regex-redux program measures cleaning, all nine variant counts, and five substitutions against the Rust regex reference. Reproducible instruction-count results and the C/Plenty prefix-loop comparison are recorded in experiments/regex/README.md. See the regex costs and coverage and benchmark workflow. B48 retains broader engine coverage, scalar scan lowering and Unicode feature expansion.

B71: Error composition and HTTP

The language/reporting foundation and precise-error migration are delivered: general sums, enum methods, structural protocols, Formattable, typed context, entrypoint reports and typed HTTP handlers. std.error.map_error and Result.map_error accept payload variants as function values (issue #556). Explicit retained-job errors/drop_job() preserve start/submission causes; Failure remains a last resort.

Typed I/O supplies stream helpers/legacy bridges. Typed HTTP body readers preserve concrete causes and partial progress, validate parsed framing and guard connection recovery. Typed parsing/writes/buffering serve the whole HTTP client, including CONNECT, uploads/redirects/pooling; stored-error retrieval was removed while the legacy Reader bridge and typed response composition remain.

Installed CLI/data and JSON-service workflows validate causes, reports, exit status, recovery and cleanup (local PR #472). See the composition evidence.

Issue #634 unifies report and display formatting on std.fmt.Formatter[W] and FormatError. Report methods use a method-local writer parameter, including anonymous-sum dispatch and conditional typed contexts. Reports can use arbitrary writers and floating-point format specs; stderr text/integer/boolean reports still allocate no memory. Canonical owned text methods on URLs, media types, socket addresses and Set-Cookie values are to_string(). See report formatting.

Remaining presentation/inference candidates stay B71 in the backlog.

B76: Columnar data

The optional dataframe foundation includes bulk numeric/string/bool construction/extraction, nullable schema inspection, conditional/cast/string/distinct/window expression plans, grouping, five join kinds, stable sorting/windows and CSV/Parquet streaming (issues #448–#451). The Polars engine is reached-only and separately packaged; version-matched release archives, compatibility/notices and deployment checks shipped (issue #456). Integer-key checked as-of joins shipped (issue #453); Date/Datetime units, naive/UTC metadata, nullable epochs and calendar formatting followed (issue #457). Numeric widths, owned binary columns and nested list/struct schema/access/ Parquet interchange shipped with issues #458–#460, preserving parent/child nulls. The original spike records its evidence.

The sales report application and examples/sales_report/ exercise catalogue validation, CSV/partitioned Parquet, nullable cleanup, UTC daily grouping and audited exports, with independent checks and an optional scaled workload. Classified rows are materialized once.

Primitive-field clist provides native-width packed columns, allocation-free empty owners, transactional reserve/append, scalar indexed fields, explicit row copies and borrowed scalar-valued pipelines. Ordered removal/clear retain capacity without allocating. Direct indexed access avoids per-field runtime dispatch (issue #432); bounded optimized integer count/filter SIMD is recorded under B23. It remains distinct from Polars-backed dataframes. Columnar evidence records whole-row tradeoffs. Remaining scope stays B76 in the backlog.

B78: Diffs

Myers/patience diffs over lists, Bytes and line/word/char tokens produce GNU-compatible unified output (issue #469, local PR #589). max_cost optionally bounds Myers work (issue #510, local PR #596). The scalar-list codegen improvement is recorded under B19, not an unfinished diff fix. Extensions stay B78 in the backlog.

B79: Markdown

Markdown is a Plenty port of commonmark.js, targeting CommonMark 0.31.2 with events/HTML and input/depth/output limits (issue #470, local PR #595). Extensions stay B79 in the backlog.

B80: Plotting

The partial plotting implementation delivers owned tables/record accessors, layer-specific data/inherited mappings, nine geoms (including supplied errors/ranges/ribbons), grouped counts/binning/bar positions, wrapped/grid facets, transformed continuous/discrete position scales with breaks/ labels, palettes/gradients, basic themes/legends, wrapped text layout preflight and streaming SVG. This is not a claim that the accepted core milestone is finished; its remaining scope stays B80 in the backlog.

B81: LRU cache

LRU implements fixed-capacity expected-O(1) operations in Plenty with no allocation after construction (issue #482, local PR #613). Copying getters followed use-site generic checking (issues #515/#516, local PR #619); optional reference/mutable getters replaced callback/trapping getters (local PR #623). No separate outstanding B81 item was recorded; future additions need their own use case/scope.

B82: Graphs

Graphs deliver stable directed/undirected handles, BFS/DFS, topological sort, SCC/components, Dijkstra, A*, Bellman–Ford and Kruskal (issue #486, local PR #614). Extensions stay B82 in the backlog.

B85: Actors

Actors deliver supervised single-thread execution, per-actor heaps on x86-64 and AArch64 GNU/Linux, bounded copied-message mailboxes and trap containment (issue #536, local PR #657). Typed trap exits, heap-safe spawn checks, allocation-free timers, timeout/gone request-reply and opt-in one-for-one fixed-window restart shipped (issue #537, local PR #658).

Actor networking adds actor-per-connection listeners, pause/refuse admission, bounded back-pressure buffers, one-timer idle/stall/read deadlines and nonblocking dial/lookup (issue #538, local PR #660). Actor TLS uses a nonblocking TLS session (issue #543, local PR #664). General Watchable handles, named readiness, spawn tags on Down and in-place connection-buffer compaction shipped (issues #540–#542/#545, local PRs #661/#662).

std.actor.http.server adds a second HTTP/1.1 server with isolated connection construction, bounded complete-request handlers, resumable TCP/TLS streaming, deadlines and draining. The shared std.http.decoder supplies byte-feed request framing. Stage observations, isolated spawning and immediate/destructor shutdown support its lifecycle; runnable deadline and live-stage cleanup fixes address issues #638 and #639. Fair child-exit delivery and safe runnable-slot reuse under connection churn address issues #640 and #642. Scalar-wrapper list references use their actual storage width (issue #643), preventing neighboring actor-address entries from being overwritten. The qualification record separates implementation delivery from production readiness and comparative measurements. The worker HTTP server remains independently supported. The qualification record includes 540,000 successful matched HTTP/TLS benchmark requests, a mixed-failure run with 400 contained traps and stable resources, and relocated native binaries. TLS sessions reconstruct a private configuration graph within their actor heap, so trapped sessions and repeated configuration rotation release retired generations without dropping interrupted rustls state (issue #641). This adds per-session configuration memory and setup work; the original TLS comparison predates that change. Completion acknowledgements and bounded retry timers reduce concurrent keep-alive mailbox retry overhead while preserving application mail, completion accounting and shutdown control (issue #644). The qualification record retains the original comparison and adds a controlled before/after HTTP measurement. A refreshed TLS comparison after both fixes records 270,000 successful requests, 9,000 verified handshakes, current idle RSS and 400 contained traps with stable after-batch resources.

A test-only budget and retained-stack investigation exercises the production heap router and trap boundary on two guarded stacks. Explicit checkpoints demonstrate peer progress, retained nested local borrows, trap recovery after resumption and cooperative cancellation with cleanup. It adds no production scheduling behavior; compiler insertion and the stage’s shared-context ownership remain unimplemented. See the probe evidence and limits.

Remaining stages stay B85 in the backlog.

Standard-library argument and error contracts

B92’s allocation/error-type work (local PR #804) gives std domain errors an Allocation(AllocError) cause, narrows allocation-only byte operations to AllocError, and reports invalid caller positions with named traps. Stream reads and read_exact validate buffer ranges before transport access, EOF, handshake work, or stored errors; TLS session transfers follow the same rule (issue #666). Encoding fields, UUID output, crypto finalization and CSV field positions trap on invalid caller offsets or indices; operations with no other failure return plain values (issue #667).

Unused byte-error adapters and unreachable range variants have been removed (issue #668). Real failures remain recoverable: truncation, malformed encoded text, allocation failure, transport failures and the capacities, limits and seek targets that still return named errors. BytesError.OutOfRange, kept then for actor networking connection limits, was removed by B95. See the invalid-argument convention and byte-stream contract.

B92’s remaining worker-certification subset is delivered in issue #661. Thin function and variant-constructor values are certified against all matching function addresses in the closed program. The 30 pure allocation adapters introduced by local PR #804 now use direct map_error calls at 227 current call sites, including nested domain-error wrapping. Helpers that attach partial-progress context retain that behavior. See worker callable certification for its signature-wide conservatism and remaining erased-callable limits. Static integer domains were completed by B95.

Dataframe byte-view inputs (issue #657)

std.dataframe.lit_binary accepts &[u8] from literals, owned buffers, byte lists and subviews. Its C strided adapter passes address, length and stride 1 directly, with no Plenty-side payload allocation. Binary-column constructors pass their packed Bytes through that adapter too, removing the additional list[u8] payload copy. Polars still owns a native copy; extraction still builds independent owned buffers. Mutable byte views remain unsupported by this adapter. See binary payloads and C borrowing.

Windows x86-64 dataframe packaging

The experimental Windows release builds the Polars dataframe engine natively and bundles plenty_dataframe_v1.dll with its licenses. The compiler generates an MSVC import library from the selected DLL, copies the DLL beside ordinary applications, and includes validated metadata and notices in --dist output. Release and portability workflows exercise Parquet queries and app relocation; Windows execution still requires native runner validation. Linear algebra and macOS engine packaging remain outside this delivery. See the engine installation reference.

macOS ARM64 dataframe packaging

The experimental Apple silicon release builds the Polars dataframe engine for Apple M1 and macOS 11, and bundles libplenty_dataframe_v1.dylib with its metadata and licenses. The dylib has a versioned @rpath identity; applications find their sibling copy through @loader_path. Ordinary output copies the dylib through a temporary sibling and atomic rename, preserving already-open engine files when rebuilding (issue #678); --dist adds validated notices and checks library identity and checksum. Selected-engine validation errors include the supported rebuild command. Packaging checks native dependencies, exports, deployment minimum and ad-hoc code signature. Native workflows exercise Parquet queries, moved installations and application-only relocation; native execution remains for CI to verify.

The shared native archive writer now gives libraries and notices mode 0644, resolving issue #677 for both Windows and macOS bundles. Broader B18.7 library qualification, linear algebra on these platforms and signing/notarization remain outside this delivery. See macOS engine installation.

Completed compiler performance and SIMD work

Completed-work index · Remaining work

These are delivery/investigation records, not current benchmark guarantees or an optimization queue. Workloads, methods and current measurements belong to profiling; remaining B19/B23/B74 scope belongs only to the backlog.

B19: Generated code and storage

  • The CPU audit (examples/cpu_codegen_audit/, local PR #431) led to issues #360–#392. --dump exposes operations/CLIF/assembly (issue #360). Unborrowed locals stay in registers, narrow helpers replace dispatcher calls in borrowed loops/byte/text operations, last owned reads move, held text avoids retain/ release, and SSE2 rounding is inline (issue #386).
  • Separate plenty_op_<N> symbols permit native section GC. The measured stripped hello executable fell from 790 KB to 356 KB (issue #501, local PR #577). Default stripping and lean runtime linkage are also recorded under B17.
  • Dense integer/tag matches use bounded jump tables at 256+ distinct values (issue #395; measurements). String matches with 16+ literals binary-search one sorted table (issue #565; measurements); other shapes retain chains.
  • Inline class field copies/address rewrites and destructor-free release for up to eight owned fields shipped (issue #544, local PR #663). One-field destructor-free classes use their field representation (issue #547, local PR #666); class fields are flat and strings release in place (local PR #668).
  • Inline copies/clears up to 192 bytes, in-place moved arguments and descriptor string-offset cleanup plans shipped (issue #550, local PR #669). Class components of tuples/multifield variants are flat; single-field Option/Result payloads keep the address layout (issue #551, local PR #670). See flat-layout evidence.
  • Release builds inline small constructors/__init__/new, avoid release of fields initialized first and omit provably zero releases (issues #548/#549, local PRs #665/#667).
  • Tail list.pop() avoids survivor relocation; other positions repair only the suffix (issue #52). Numeric/bool rows use natural 1/2/4/8-byte widths and lists omit the dictionary value slot (issue #531).
  • Borrowed integer/float/bool list rows use inline reads/writes with unchanged bounds reporting. The recorded 3k-line dissimilar diff fell from 265 to 100 ms (issue #509, local PR #598; measurements).
  • Collection access/removal, equality and string indexing work shipped (issues #18–#21). Layout and runtime constraints still follow the reference, not these snapshots.

B19: Compiler scale and caching

  • Warm single-file executable hits skip bundled-runtime extraction (issue #699). The measured no-op shebang median fell from 16.35 to 6.56 ms while preserving source/toolchain/native-input checks. See the startup measurements and method.

  • Cached collection element facts (issue #6), linear declaration/large-enum match checking (issues #53/#71), parallel native emission and the large-function single-pass allocation fallback (issues #55/#56) shipped. More than 2,048 branch arguments select the fallback with a code-size/speed tradeoff.

  • Operation checking/native IR construction/compilation run across cores with byte-identical output for any worker count (local PR #131); lookup, frame validation, tail-call preparation and adapter scans also parallelize (local PR #136). Declarations resolve once (issue #59, local PR #158).

  • Compiler peak memory was measured at about 1.8 KB/source line for checking and 2.8 KB for object emission after exact-length syntax lists/40-byte tokens, duplicate-body removal and earlier release of tokens/operations (issues #422/#54). Whole-program check work is linear in instructions and std imports no longer enable per-function whole-program work (issue #57).

  • Content-addressed compiled-function reuse shipped (issue #144, local PR #176). Buffered cache hashing/shared decoded stencils reduce repeated work (issue #316). Whole-object cache validation includes module text/import resolution (issue #317 stage 1, local PR #385).

  • Generic naming is linear; template/program instance limits and dependency depth bounds permit finite declaration chains (issues #428/#430/#435, local PR #516). Templates release when no pending lowering needs them, reducing generic/ordinary checking memory from 1.92× to 1.37× in the recorded 200×200 case (issue #434, local PR #521). Instance-time substitution, moved template bodies, template-ID cache keys, symbol-keyed requirements and cheaper parser tokens reduced the checking instruction ratio to 1.37× (local PR #534).

  • Cranelift 0.137’s reused regalloc2 context reduced debug compile instructions 4.2–4.7% in recorded workloads (issues #153/#421, local PR #528). Reusing cleared Cranelift Functions reduced ordinary construction allocation from 20.3 to 4.9 KB/function (measurements).

  • Batched codegen results preserve deterministic output with fewer messages (local PR #368). Mimalloc removed glibc arena-lock contention, so the measured 16-thread run was no slower than eight (issue #315, local PR #386). Linker preference chooses mold/lld while preserving overrides (issue #314, local PR #367).

Compiler architecture assessment

A source-based refactoring proposal records the compiler boundary assessment, file/symbol maps, alternatives and validation gates. Its six steps are delivered: structured diagnostics (issue #691), scoped recorder and teardown state (#692), the op and syntax-layer splits (#693, #694), grouped pipeline state (#695) and a frontend lookup profile that changed no representation (#696). The legacy stack syntax is removed (#698). No speedup is claimed; remaining scope stays in the backlog.

B19: Profiling and concluded investigations

--features profile exposes hotpath timing/calls/allocations per phase (issue #150, local PR #188); finer lowering/ownership/native rows and isolated compile_bench workloads supported investigations #301–#305 (local PR #343).

InvestigationRecorded outcome
#301Avoiding 128-bit slot split/join shifts halved AOT instructions and reduced register-allocation work 70% (local PR #351).
#303Borrowed loan-graph operations and liveness-buffer reuse cut ownership-check instructions 40% (local PR #352).
#302Shared token/nominal names first cut checking instructions 8.8% and allocations 25% (local PR #357); shared structural/callable children and FxHash tables brought the broader check-workload reduction to about 20% (local PRs #383/#389). Investigation closed.
#305The mixed-workload checking ratio was attributed to cache/TLB misses, not algorithmic work.
#304Warm-cache work was measured at roughly 45% frontend and 18% linking; linking experiments record levers.
#59Parallel parsing, speculative lowering and declaration parallelism were evaluated but not adopted: small gains and higher peak memory. Declaration reuse landed.
#321Mimalloc and skipped command-line teardown were retained; arenas/bump allocation were measured and not adopted (local PR #388).
#317Stored syntax trees/separately compiled module artifacts did not pay; the investigation identified imported-interface lowering as a prerequisite for a useful lowered-module cache (local PR #392). See module-cache investigation and module-edit baselines.
#436Substitution-environment lowering offered at most 2.6% and was set aside. Instance registration/header tokens explain the remaining generic gap. Closed.
#437Code placement affects SIMD measurements; retained placement research, no compiler fix. Closed as investigated/deferred.
#440Cranelift alias regions gave no gain and no sound general partition; reference targets can span frames/rows/fields/payloads. test_memory_order guards against unsound regions (local PR #529).
#441Cranelift 0.137 dead-store elimination was qualified against call/atomic cross-thread handoffs; release concurrency/coverage tests and >1 GiB frame diagnostics guard the contract (local PR #530).

These outcomes are not active claims. Any revisit follows B19 in the backlog, with new evidence.

B19: Traps and arithmetic lowering

One object-wide trap-site table and cold function check blocks reduced the recorded 200×200 object size 34% and single-core emission 9% (issue #420, local PR #505). Debug cmp; jcc overflow checks reduced that benchmark’s text 5.5% (issue #425, local PR #525). Cranelift 0.137 fuses unsigned add/all multiply checks and keeps pooled float-mask lanes in registers (issue #439, local PR #528).

PC-relative trap tables and executable ud2/SIGILL resume-to-report helpers reduced recorded text 14% debug/11% release and compile instructions 7–9% (issue #425, local PR #531). Libraries/fuzz/coverage retain helper calls. Issue #425 was closed as deferred to upstream changes, not as proof that every check/frame/spill cost disappeared. The runtime reference owns the contract; upstream revisit gates are in B19 of the backlog.

The Linux foreign-handler fixture repeats signal delivery until the tested read completes, instead of assuming a scheduling delay puts the reader in the syscall (issue #689). Restarting reads receive a pipe byte after repeated signals; non-restarting reads must report EINTR, with a bounded watchdog. Delayed-entry regressions retain the later checked-overflow report in debug and optimized builds. This changes test synchronization, not runtime handler behavior.

B23: SIMD

Automatic integer-column filter/count uses baseline four-lane i32/u32 SIMD with scalar tails in optimized builds. One/two total comparisons joined by and, inclusive/exclusive ranges, reversed operands and up to two immutable same-type scalar captures are supported.

The initial std.simd module provides four concrete numeric vector shapes, masks, checked/wrapping arithmetic, selection, safe list access and scalar fallback. Bounded lowering/local promotion avoid repeated materialization in supported kernels. Direct integer wrapping reductions avoid the spill reproduced in issue #444. These are delivered subsets, not automatic packed generic arrays or a backend switch. Remaining extensions stay B23/B76 in the backlog. This delivers the initial SIMD stage of the memory and parallelism assessment, which remains partly implemented for its public allocator design.

B74: Collection hashing

Named enums with supported key payloads now use structural hashing in sets and dictionary keys. Variant tags and all payload fields participate; nested named enums, tuples and classes, including custom cleanup, are supported. Anonymous sums with supported members also qualify; their active member tag and structural value define identity. Floats, boxes and recursive types remain excluded. These general key types use keyed runtime hashing. Supported finite enum and tuple domains with at most 256 values use the bitset specialization described here. See collections.

Collections use SipHash-2-4 with a per-process OS key (issue #329; issue #747 later moved to SipHash-1-3 with direct str, bytes and scalar key paths), and record traversal performs linear field-offset work per pass (issue #330). The built-in KeyType constraint now exposes the same key admission rule on functions, methods, classes and enums; see generics. Custom-cleanup keys use the same structural identity without making their cleanup obligations copyable.

Class keys hash their fields rather than their addresses. Move-only class keys, and enums containing them, transfer on insertion and owned extraction; lookups and views borrow them. Set algebra and key-view to_set() create independent copies, and affine keys_snapshot() requires an owned temporary, matching values_snapshot(). Keys containing custom cleanup support borrowing, transferring updates, in-place filtering and owned extraction. Non-mutating algebra, to_set() and explicit copying reject them. Duplicate insertion keeps the existing key owner and destroys the incoming owner. Regression tests cover nested cleanup, generic boundaries, duplicate ownership and allocation failures.

Small enum sets

The initial specialization used one membership word for payload-free named enums with up to 64 variants, without entry/order/hash buffers or entropy reads. The wider implementation extends this to 256. The collection owner still allocates once. Membership, length, insertion/removal, same-type set relationships and algebra use bit operations. Shared iteration uses immutable canonical enum values, while normal source loans remain enforced. Mixed operations with dictionary key views retain their existing semantics.

Allocation-failure tests cover the remaining owner allocation and allocation-free mutation; boundary tests cover 1, 63, 64 and 65 variants. These initial measurements cover payload-free enums at revision f4aba2de. Reproduction at that revision is cargo build --locked --bin plenty (without instrumentation features), then python3 scripts/bench_enum_sets.py target/debug/plenty --output /tmp/enum-sets.json: the same first 64 variants are exercised in a 64-variant bitset and a 65-variant hash set. It checks equal results and alternates execution order. Native programs use --release; measurements are observations, not performance assertions.

Measured on x86-64 AMD Ryzen 7 7840HS, with 200,000 rounds and median of five runs per case. Each lookup round probes 64 values; each mutation round adds and removes all 64. Union combines two disjoint 32-member sets; copy duplicates a 32-member set. These timings describe these workloads on this machine, not a general speed guarantee.

WorkloadBitset (seconds)Hash fallback (seconds)Speedup
Lookup0.0819690.3573194.36×
Add and discard0.3390451.5984634.71×
Union0.0126701.21075595.56×
Copy0.0136720.46108533.72×

Finite payload enum sets

The finite-payload stage (82bd5231) extended bitsets to named enums and tuples whose complete domains contain at most 64 values, recursively composed of booleans and named enums/tuples. The compiler checks sums and products against the limit before generating canonical values and variant offsets. Membership encodes the active payload with mixed-radix indexing; algebra still operates directly on the mask. Immutable, 16-byte-aligned metadata includes nested inline records, preserving stable shared references without runtime caches or allocations. The runtime type descriptor now carries an optional finite-domain descriptor.

Tests cover nested multi-field enum/tuple values, every member of a 64-value domain, the 65-value hash fallback, singleton/Option payloads, mixed dictionary views, copying and iteration after source destruction, and allocation failure. The benchmark at that revision also supports --payload to compare a six-boolean variant (64 values) with the same variant plus an unused extra case (65 values). Integer/string payloads and classes keep hash storage; the wider implementation extends the finite-domain threshold to 256.

On the same AMD Ryzen 7 7840HS, 200,000 rounds and medians of five runs with --payload produced the following observations. The workload counts match the payload-free benchmark, but each key now carries six boolean fields. These are measurements of these workloads, not general speed guarantees.

WorkloadFinite bitset (seconds)Hash fallback (seconds)Speedup
Lookup0.5843691.9614033.36×
Add and discard1.5942217.3732494.62×
Union0.0114054.927148432.02×
Copy0.0156941.980686126.21×

Wide enum bitsets

Finite enum and tuple domains select their mask width from the complete type domain: one 64-bit word for 1–64 values, two words for 65–128, and four words for 129–256. Operations dispatch to the selected width, including scalar membership, popcount and algebra for the smallest domains. Clearing or draining preserves the selection; sparse membership never narrows it. All alternatives fit within the existing insertion-order storage, so collection owners do not grow. The canonical payload-free tag table grows from 64 to 256 entries (one shared 4 KiB table); finite payload metadata remains proportional to domain size and inline layout. Wider masks do not change key admission or dictionary storage. No target-specific SIMD is needed.

Boundary tests cover 64/65, 128/129 and 256/257 values, including hash fallback. Randomized membership and mixed-end iteration tests exercise every word; native checks cover nested enum/tuple payloads, set algebra, dictionary views, copied values, owner failure and allocation-free mutation.

Requested heap storage measured with the runtime’s actual buffer capacities (excluding allocator overhead and immutable type metadata):

MembersBitset bytesPayload-free hash-set bytes
0176176
4176496
641764,272
1281768,368
25617616,560

Both owners are 176 bytes on this x86-64 build. The hash comparison uses a 257-variant enum and inserts the same tags without pre-reservation. Payload hash rows can require more storage; canonical payload data is shared per type, not part of these per-owner heap figures. For example, one eight-boolean variant emits 256 × 16-byte canonical slots plus 256 × 128-byte payload records: 36 KiB of shared value data, plus descriptors and alignment. This can outweigh the heap saving for a small number of sparse sets; the optimization trades static data for faster operations and smaller per-instance storage. Reproduce with cargo test -p plenty-runtime wide_enum_set_storage_measurements -- --nocapture.

The benchmark accepts --size 64|128|256, --payload, --members, and --workloads. Without a baseline compiler it adds unused variants to force a 257-value hash domain. --baseline-compiler instead compares identical programs against an older compiler; this also checks the existing 64-value specialization. Every result is checked, execution order alternates, and JSON records individual wall and child CPU times as well as medians.

The wide-domain comparison uses the pre-change 64-value compiler, native --release compilation without runtime instrumentation, and an AMD Ryzen 7 7840HS. Reported ratios use median user-plus-system CPU time over five runs. Other work was running on the host; CPU time excludes scheduling waits but cannot remove effects on clock speeds and caches. These are workload-specific observations, not speed guarantees.

For the full-domain cases, 50,000 rounds each probe all values, add and remove all values, union two disjoint half-domain sets, or copy a half-domain set. Payload cases use seven or eight boolean fields. Sparse cases use four values spread across the 256-value domain (two per input set) and 1,000,000 rounds. Reproduce a full-domain case with:

cargo build --locked --bin plenty
python3 scripts/bench_enum_sets.py target/debug/plenty \
  --baseline-compiler /path/to/pre-change/plenty \
  --size 256 --payload --rounds 50000 --runs 5 --output /tmp/wide-enums.json

Use --members 4 --rounds 1000000 for sparse cases, and omit --payload for payload-free enums.

Speedups over the older hash-backed implementation:

Domain / workload valuesLookupAdd/discardUnionCopy
128, payload-free3.75×4.23×136.24×115.01×
128, boolean payload4.15×4.46×696.99×410.92×
256, payload-free3.56×4.12×198.95×192.41×
256, boolean payload4.38×4.48×1342.00×849.87×
256, four payload-free values2.91×3.64×4.47×8.90×
256, four payload values3.64×3.79×26.00×22.92×

Raw measurements retain every wall/CPU sample, medians, settings, compiler hashes and the 64-value comparison. The gains at both wider sizes and in sparse cases support the 256-value cutoff.

The 64-value comparison uses seven alternating runs: 2,000,000 rounds for union and copy, 200,000 for lookup and mutation. Relative throughput changes against the old scalar implementation were lookup +0.3%, add/discard +3.5%, union +1.4%, copy +163.6%. These are measured medians, not guarantees of changes this small. Width is encoded once in the storage alternative, with scalar operations for small domains. Copying reuses the validated representation and allocates only the new owner, avoiding storage reclassification and unnecessary reservation or membership counting.

Proposals

Proposals record design rationale and rejected alternatives. The reference defines current behavior; the backlog tracks future work. If a proposal disagrees with the reference, follow the reference.

Status

Each main proposal opens with a status note linking to the governing reference, identifying differences in what shipped, and naming remaining backlog items. Companion pages identify their parent and whether their examples are hypothetical.

StatusMeaning
ImplementedThe design shipped. The page keeps the rationale.
Partly implementedSome of it shipped. The block says which parts.
Not implementedNothing shipped. The page is the design record for future work.
Investigation completeAn assessment reached a decision; it does not imply feature implementation.

Proposal text retains its original wording: it may describe shipped features as future work or show syntax that never shipped. Proposal code is not compiled by the test suite.

When a change implements or alters a design, update its status note. Move a proposal to completed/ when its agreed scope ships or its investigation concludes without an implementation commitment; retain its rationale and update links and book navigation. Alternatives and future extensions need not ship for the agreed scope to be complete. Keep partly implemented and unimplemented proposals here; retain superseded or abandoned designs separately in archived/ when needed.

The status scan used reference, implementation, and regression coverage at local main commit 74fa81d8 on 2026-10-20; it did not rerun every feature test. Later reviews reconciled HTTP acceptance, error composition, and anonymous sums at 4e40c4fc on 2026-10-23, and entry points, modules, executable lessons, and the next language phase at 161e1c63 on 2026-10-24.

Find a design by topic

TopicStart hereRelated decisions
Rust compiler architecture and performanceCompiler boundariesDiagnostics, frontend/session state, identity and facts, module replay
Runtime-independent networking protocolsCodec, session and driver architectureSession contract, runtime integration, WebSocket acceptance
Executable scripts and embedded metadataSingle-file scriptsModules and entry points, current packages
Mixed-edition packages and language evolutionPackage language editionsCurrent packages, script metadata
Protocol bodies, defaults, final, and private hooksProtocol method bodiesAgreed rules, short programs, existing wrappers and capabilities
Heterogeneous protocol objectsBoxed protocolsExplicit ownership and method contract, implementation design, research evidence
Error compositionComposition overviewAnonymous sums and the Failure policy, completed reporting, typed-stream evidence
Ownership and stateTypestateLinear obligations, resolution scopes, integration boundaries
Optional and fallible borrowsReturned enums that carry referencesReturned references, borrowed matching
Linked structures and graphsOwned structures and checked identitiesRecursive representation, arena prototype
CollectionsOrderingStable sorting algorithm, hashing and equality, slice syntax
Compile-time sizes and fixed inline arraysValue generic parametersFixed capacity and inline storage, portable std.simd API
Typed data formatsDerived serializationstd.json, std.toml, method annotations
Native platforms and librariesCross-platform assessmentFFI, export contracts, GCC backend
Allocation and parallel executionMemory, parallelism, and SIMDAllocation audit, shipped concurrency
Cooperative suspension and owned tasksAsync/await exploration, Go-style scoped tasksAsync ownership and cleanup, async spike evidence, stackful runtime choices, stackful spike evidence, linear execution responsibility
Iteration and adapter chainsFluent iterator pipelinesShipped generators, parallel execution
Portable vector operations and automatic SIMDPortable std.simdAPI contract, pipeline recognition, backend evidence
Columnar record storageColumnar listsOwnership and API, chunks and deletion, layout evidence, Plenty implementation spike
Analytical dataframesPolars embeddingNative interface and measured costs
Plotting and data visualisationstd.plotExamples beside ggplot2 and plotnine
Scientific numerics: optimization, integration, ODEs, special functionsNumerics coreAPI sketch, crate evaluation, std.linalg, std.stat
Exact decimal and financial arithmeticstd.decimalPolars embedding, ordering, dictionary keys, checked arithmetic
Physical units and dimensional analysisUnits of measureprecedents, spike evidence, domain types, value generics
Language consistency: copying, formatting, views, iteration, errors, callbacksSix rules for a consistent coreFluent iterator pipelines, user type hashing, error composition

Companion examples and evidence do not imply additional implementation commitments.

Completed

See Completed proposals for shipped designs and their reference pages. Further extensions remain tracked in the backlog.

Partly implemented

ProposalShippedRemaining
Value generic parametersValue parameters and BoundedB83 arrays/assertions; B96 std adoption
Recursive owned dataRecursive storage through Box[T]B11, B33
Typed ranges, generic calls, and expression-local inferenceTyped ranges, contextual literals, explicit type argumentsB12, B16, B32
C interop and dynamic librariesImports, linking, exports, and resident loadingB18, B27, B28
Generated library contracts and C headersExports, contractsB27, B28
Fallible memory, parallel execution, and SIMDFallible allocation, concurrency, initial SIMDB13 allocator control; B23 SIMD extensions; B24 automatic parallelization; B83 fixed arrays
Standard library scopeEmbedded library, including data formats, TLS, HTTP, SQLite, signals and testingRemaining B38–B70 scope
Keyword arguments, defaults, and static unpackingNamed calls and fresh defaults under the narrower designB36: splats and signature markers remain separate candidates
Units of measureDimensions and Quantity[T, D], erased to their storage type (issue #701)B97: std.units, sharing bodies that differ only in dimension

Not implemented

ProposalBacklog
Runtime-independent networking protocols — agreed codec/session/driver direction; complete shared WebSocket session and both drivers remain proposedB94, B85
Actor budgets and retained stacks — test-only heap/trap probe; compiler checkpoints and stage ownership integration remain unimplementedB85
Blocking work for actors — bounded native workers, independent job ownership and event-based completionB85
Async/await exploration — owned movable frames, fallible admission and synchronous cancellation; isolated spike onlyUnscheduled exploration; async implementation remains out of scope pending an adoption decision
Cosmopolitan and Actually Portable Executables — bounded compatibility experiment before any runtime/platform commitmentB18, B17
Package language editions — exploratory compatibility mechanism; no editions implementedB17
Using Rust crates in Plenty — three prototypes favor generated C adapters over a Cargo/rustc graph; production integration remainsB17, related B20
Compiling Plenty to Rust source — checked-IR scalar emitter, native companion round-trip and Rust build-cost measurements; ownership probes remain hand-lowered, no production backendB20, related B17
Linear obligations, including aggregate and error rules — opt-in non-abandonmentB72 (issue #149)
Synchronous resolution scopes — consuming scope policy; depends on linear obligationsB72 (issue #149)
Linearity, typestate, and execution responsibility — shared boundaries and deferred worker/async contractsB72 (issue #149)
GCC backend (plenty_codegen_gcc)B20
Portable std.simd and automatic pipeline SIMD — small portable API, bounded safe transformations, and backend spike evidenceB23; related B12, B19, B24, B75
Columnar lists — explicit field-column storage, a bounded object facade, and native layout/ownership spikesB76; related B11, B12, B23, B75
User-defined ordering — one three-way __cmp__ hookB37
Timsort for stable, adaptive sorting — fallible workspace, stability, and compatibility choicesB37 (issue #331)
User-defined dictionary keys and set members — derived structural hashing and a KeyType boundStructural class/enum/anonymous-sum keys, including custom cleanup, plus KeyType; copying operations require duplicable keys
Derived serialization for classes and enums — @serial derives hooks against a JSON-shaped std.serial data model shared by JSON and TOMLNone yet; candidate under B51/B38 (issue #471)
Returned enums that carry references — Option[&T] and Result[&T, E] returned under the first-reference-parameter rule, held in locals, never storedNone yet; extends B11 (issue #516)
Go-style scoped tasks, with runtime choices and native spike evidence — exploratory fixed-stack alternative; no lightweight scheduler adoptedB22 investigation; implementation uncommitted
Arbitrary-precision integers — Plenty limbs with an inline small value; malachite rejected on allocation failure and LGPLB38 (issue #481)
Thread-local reference counts — single-threaded phase first, then Lean/Koka-style thread-shared marks at certified boundaries; measured atomic count trafficNone yet; related issues #743–#746
Faster linking — measured link costs; caching the executable, running the linker directly and reusing the extracted runtime archive; object-format changes do not payB19; related B17
Checked arithmetic by default — function-level typed errors, explicit unchecked: with a validated no-failure precondition and retained local checked: recovery; core/repository adoption and bounded validation complete; local review and merge pendingB14
Exact decimal arithmetic — allocation-free 38-digit Decimal in Plenty, scale kept with numeric equality as in Python, exact operators under the integer trap and checked: rules, NonZero[Decimal] divisors, explicit roundingNone yet; related B76, B37, B74
Numerics core — std.numeric in Plenty over std.linalg; a scoped SciPy subset in three milestones; basin and numeris not adoptedB38 (replaces issues #492, #493)
Six rules for a consistent core — one copy rule, one formatting path, owner-to-view conversion, adapters on iterators only, error types that name only what can fail, every callable form as a callbackNone yet; proposes B88–B93; related B12, B49, B84 (issues #614–#636)

Completed investigations

AssessmentDecision / related backlog
Native macOS, Windows, and AArch64 supportNative ports need target-aware builds, ABI validation and OS/stdlib contracts before release packaging. B18, with B17/B12 dependencies.
Protocol method bodiesCore default/final dispatch and member-specific grants agreed; definition-time contract checking required. No automatic method injection. Not implemented or scheduled (B29).
Owned structures and checked node identitiesTested patterns and arena prototype; production identity/access contract remains B73, broader borrowing B11.
Typestate and owned transitionsExisting owned transitions and their limits; linear non-abandonment is a separate proposed extension (B72).

Cosmopolitan and Actually Portable Executables

Status: Not implemented. A bounded feasibility spike demonstrated two programs, not supported Cosmopolitan output. This proposal recommends an evidence-gated compatibility experiment before a platform commitment. Current behavior remains in runtime packaging and linking. Future work belongs to B18/B17 in the backlog.

Cosmopolitan offers a different deployment route from the native-platform assessment: generated programs could use Cosmopolitan Libc and the Actually Portable Executable (APE) container to run on multiple operating systems. It does not automatically make Plenty’s GNU/Linux runtime portable. Keep native GNU/Linux output unchanged and make any eventual Cosmo output explicitly opt-in.

The recommendation is not to implement a broad runtime port now. First establish a reproducible x86-64 runtime compatibility result, including failure paths, threads and execution of the identical artifact on a non-Linux OS. Then choose a restricted feature profile, a larger runtime port, or no support. The runtime design describes the unresolved build and ABI choices; evidence and qualification records what was actually measured and the go/no-go criteria. The proposed target facts and code-generation contract prevents host-Linux branch selection; a self-contained harness specification defines the first implementation deliverable without relying on scratch state.

Multi-OS is not multi-architecture

The spike used x86_64-unknown-cosmo-cc: its successful Plenty artifacts contain x86-64 code only. Multi-OS output on one ISA is useful independently of fat output. The ordinary cosmocc C workflow compiles both x86-64 and AArch64 and combines the results with apelink; it cannot manufacture AArch64 Plenty code from an x86-64 object. True fat Plenty output requires two qualified codegen paths, two compatible runtimes and dependencies, and packaging and testing of both slices.

Native Linux AArch64 work is being developed independently (local PR #706, not merged into the inspected main). Coordinate target descriptions, ABI checks, platform boundaries and tests with that work when available. Do not copy unmerged code or treat its Linux GNU runtime, signal context or release qualification as Cosmopolitan support. Native Windows/MSVC and Darwin ABIs are also different targets from the C ABI offered by Cosmo.

Support matrix and deployment contract

This matrix separates upstream scope from Plenty evidence. It is not a support announcement or a promise of every OS × ISA combination.

Environment / artifactEvidenceProposed admission
Native Linux x86-64 GNUImplemented baseline at inspected main 2851ef34Preserve existing support
Cosmo x86-64 on LinuxEmpty and print/allocation programs run; checked trap failsCompatibility experiment only
Same Cosmo x86-64 artifact on Windows, macOS or BSDNo Plenty execution testedOne real non-Linux OS required for experiment; each named OS/version needs separate qualification
Cosmo AArch64 on any OSNo Plenty evidenceSeparate ABI/runtime feasibility gate
Combined x86-64 + AArch64 APENo Plenty evidenceOnly after both slices qualify independently
BIOS / bare metalUpstream release mentions BIOS; no Plenty evidenceNon-goal for this proposal

The official toolchain documentation names Linux, macOS, Windows, FreeBSD, OpenBSD and NetBSD and describes the two ISA toolchains. These upstream claims do not establish Plenty’s OS versions, CPU baseline, capabilities or deployment guarantees. Choose and record those per qualified configuration rather than inheriting an upstream support list.

APE needs a working execution route. On the spike host a shell launched the successful programs, but direct Python subprocess execution returned ENOEXEC. The official documentation describes shell fallback, loader extraction into TMPDIR/HOME, optional system loader/binfmt configuration, and Apple Silicon loader installation/build requirements. Test writable/executable extraction locations, restricted environments and installed-loader operation explicitly. Do not require privileged registration by default or silently alter the host. Do not claim every deployment is a zero-prerequisite direct execve launch. Windows shell requirements for the toolchain wrappers are not a claim that every final Windows executable needs a POSIX shell.

The compiler should initially remain a native GNU/Linux tool producing an experimental artifact. Porting the compiler executable itself to APE, making all native compiler hosts cross-compile Cosmo, and fat output are separate questions, not prerequisites for the bounded experiment.

Alternatives and non-goals

  • Keep native ports only. Lowest additional maintenance; loses single-file multi-OS deployment. Prefer this if compatibility or qualification gates fail.
  • Linker override alone. Useful for investigation, rejected as a supported design: the spike already demonstrates a trap failure and ABI mismatch.
  • Restricted Cosmo runtime. Recommended candidate after the experiment; explicitly admit a small tested capability set instead of advertising the whole standard library. This still requires correct runtime fundamentals.
  • Broad runtime and dependency port. Potentially useful, but only justified after measuring Rust std compatibility and concrete module/dependency costs.
  • Dedicated Rust/Cosmo target or reduced runtime without Rust std. Could remove GNU assumptions, but carries substantial toolchain/runtime maintenance; compare against selective compatibility shims using evidence, not ideology.

This proposal does not replace Cranelift, change runtime language semantics, weaken fallible allocation/security contracts, implement native MSVC/Darwin ports, provide BIOS services, promise arbitrary dynamic C libraries, or bundle a new SDK into every compiler distribution. It supplies no production effort estimate.

Decision and risk policy

The major risks are latent GNU Rust std assumptions, signal/context corruption, TLS and synchronization bugs, incompatible native dependencies, and insufficient real-OS runners. A tiny program’s dead-section elimination can hide all of them. Loader policy and toolchain supply-chain maintenance add deployment risk even when program semantics pass. A restricted profile reduces exposed facilities, not the need to establish memory safety and thread lifecycle correctness.

Decisions about the final runtime build model, admitted capabilities, minimum OS/CPU versions, public option spelling and toolchain redistribution remain behind the qualification gates. Passing a gate permits a design decision; it does not silently authorize broader support.

Cosmo target, runtime and driver design

Part of the Cosmopolitan proposal. Recommendations here are not implemented behavior; build choices remain subject to evidence gates.

Separate target identity from the final linker

Current src/toolchain.rs::validate_target_pair accepts only the packaged x86_64-unknown-linux-gnu target. src/codegen.rs builds a host ISA and uses System V at exported runtime boundaries. build.rs builds and embeds Rust static archives for Cargo’s target; selecting --linker does not change those compile-time assumptions. See current runtime packaging.

An eventual target descriptor should distinguish compiler host, generated ISA, Cosmo C ABI, ELF object format, APE final container, runtime profile and toolchain identity. Do not describe the generated program as GNU/Linux merely because the compiler runs there. Preserve the internal Plenty Tail convention separately from Cosmo C calls. Verify entrypoints, helpers, callbacks, aggregate returns, i128/libcalls, stack alignment, atomics, relocations and direct/indirect Tail calls; familiar ELF and x86-64 are not a complete ABI proof.

For the experiment, retain object/runtime extraction and external link harnesses rather than prematurely committing to a public target spelling. For production, choose an explicit output-target option integrated with existing target selection; --linker alone must never imply support. Host execution of a generated APE is a separate operation from compilation, with a documented loader selection policy.

The proposed compile-time target and code-generation contract specifies host-independent target.os = "cosmo" selection and the early red-zone audit. Neither target facts nor object code can be corrected by final link flags.

Runtime build choices to resolve experimentally

Do not assume linking GNU-built Rust std into Cosmo is generally safe. Rust Linux cfgs, libc layouts, TLS, entropy, filesystem and thread implementations can survive final linking unchanged. Conversely, the successful allocation probe is evidence against declaring every GNU object unusable or requiring every dependency to be rebuilt without inspection.

Compare these bounded choices:

CandidateRequired evidence / cost
Retain selected GNU-built Rust components with Cosmo-header C shimsAudit reachable std/dependency paths and generated code; prove no incompatible OS layouts/syscalls/TLS assumptions escape the boundary
Build a dedicated Rust/Cosmo runtime configuration and compatible dependenciesEstablish target/std availability or maintained patches, build-script behavior, ABI, reproducible/offline builds and ownership of long-term maintenance
Reduce OS-facing Rust std use in a restricted runtimeMeasure how much replacement is actually necessary; preserve allocation, panic, reporting and thread contracts rather than invent a second language runtime

The first candidate is an experiment, not an endorsed distribution architecture. A shim fixes bindings it owns, not arbitrary std internals. If those internals cannot be bounded, stop or move to an explicitly approved runtime build project. Reconcile the chosen build with the native runtime build assessment, including separate output directories and consistent application/library modes. Do not recursively invoke Cargo against the enclosing build directory.

A narrow platform boundary

Prefer C compiled against pinned Cosmo headers where the libc ABI is complex or has runtime-valued constants. Expose small fixed-width/pointer interfaces to Rust; keep platform structs private to C, with build-time size/alignment and native round-trip checks. This is justified by ABI translation, not a reason to wrap ordinary Plenty library algorithms.

BoundaryRequired treatment
Signals and saved contextsReplace GNU sigaction, masks, flags, alternate-stack and register-context assumptions in traps.rs and stack_overflow.rs; install/chaining/restore and saved-PC/SP access belong behind the boundary
Trap reporting and stack guardsPreserve source-site diagnostics and exit semantics, inline page probes and main/worker alternate-stack lifecycle; distinguish owned traps/guard faults from foreign faults
TLS and threadsAudit Rust TLS lowering plus Cosmo fixups, initialization/destructors, pthread entry/attributes, stack bounds, join and start-failure cleanup; test per-thread allocation/trap state isolation
Synchronization and syscallsReplace or validate mutex.rs’s raw Linux syscall 202/futex assumptions; use qualified Cosmo primitives, testing wakeups, contention, cancellation and memory ordering
Filesystem and processesAudit filesystem.rs statx/readdir64 layouts, flags, paths, errors, rooted operations and process inheritance/cleanup; do not substitute unsafe path traversal for secure operations
Network, readiness and watchAudit pthread attributes in net.rs, sockets, DNS, clocks, nonblocking/error semantics and Linux inotify/readiness bindings; admit capabilities individually
Foreign librariesExisting dlopen/ELF residency/metadata assumptions in libraries.rs are not an APE portability contract; reject until independently specified and qualified

Signal-handler work must include async-signal-safety, previous-handler coexistence and unknown-fault forwarding. Prefer existing helper-call checked traps as an explicit experimental fallback if necessary, but still test the signal path; disabling stack guards is not an acceptable fallback. Libraries must continue to respect host handlers. Actors’ recovery and per-actor heaps need their own qualification rather than inheriting success from a main-thread trap.

Cosmo’s TLS fixups may adapt eligible GNU instruction patterns; the spike observed GS accesses, not a TLS failure. Neither that observation nor an isolated thread-local read proves Rust TLS destructor and worker lifecycle compatibility. Test scoped workers, executors, repeated spawn/join, foreign-thread entry if admitted, thread-local failure injection and trap caller-site reset between jobs.

Capabilities and dependencies

A candidate initial profile is executable-only core language, collections, allocation, reporting and basic standard output. Threads must be tested in the compatibility experiment even if public concurrency is withheld initially. Filesystem, processes, network/TLS, watches/readiness, actors, dynamic libraries, fuzz/coverage modes and optional engines start unqualified, not implicitly enabled. The final profile is a gate decision, not a promise that this list is sufficient.

Use target/profile capability checking to reject unavailable operations before linking, with the operation and missing capability named. Define whether the check applies to reachable code or all checked declarations; test generic and indirect calls so unused embedded std declarations do not disable every program. For capabilities genuinely determined at runtime, return an explicit existing unsupported error where the API permits it; otherwise reject compilation. Never provide success stubs, silently use a Linux implementation on another OS, or drop security guarantees. If the current error API cannot express absence, settle that API before admitting the facility.

Audit SQLite, ring/rustls, RustCrypto, compression and entropy paths separately, including C/assembly inputs and target cfg/build-script outputs. Pure algorithms may need only validation; native OS/assembly objects may need compatible builds or replacement. Retain specialist crypto libraries and their constant-time and entropy contracts. Optional dataframe/linalg engines require separately compatible builds and independent tests; shipping a Linux engine does not make it multi-OS. Dead stripping of uncalled modules is not a capability enforcement mechanism.

Package archives with a verified manifest: compiler/runtime contract identity, ISA and CPU baseline, C ABI/layout version, Cosmo release/hash, profile/features, Rust toolchain and dependency lock identity, application/library mode and native link inputs. Reject a GNU archive, wrong ISA/profile, incompatible ABI or stale cache entry before emission/linking. Extend existing target metadata and cache identity rather than create a second source of truth. Runtime extraction must include the manifest and exact link argument arrays.

A qualified driver should discover or accept a pinned Cosmo toolchain, validate its version/layout, construct profile-specific arguments and preserve paths with spaces. The spike’s missing -lutil justifies auditing native flags, not deleting all GNU flags or asserting unwinding works because -lgcc_s resolves. Preserve panic-abort, custom panic/OOM reporting, section retention and trap tables. Test stripping/debug sidecars with Cosmo tools instead of assuming host strip understands the final container.

Diagnostics should distinguish missing driver, incompatible archive/toolchain, unavailable capability, linker failure and launch-route failure. Verbose output should expose target/profile, toolchain identity and complete command arguments. Do not hide ENOEXEC by treating a program as arbitrary shell text; choose a documented loader/fallback route and preserve child arguments, status and errors. Initially reject library exports/dynamic loading and unqualified tooling modes explicitly, rather than emitting artifacts with untested contracts.

Pinning, provenance and licensing

The spike used Cosmopolitan 4.0.2 (bundled GCC 14.1.0). Use that as a reproduction baseline, not an assertion it is the best current release. Before implementation, review upstream changes/advisories and select an exact version, archive hash and source revision. Record compiler/runtime source, Rust version, Cargo.lock, patches, host tools, flags and artifact hashes. Fetch only declared inputs in a separate preparation step; exercise offline builds and reproducibility twice.

Start with an externally supplied pinned toolchain; bundling is a separate packaging decision. The official toolchain README’s Legal section distinguishes the GCC-based toolchain from permissively licensed runtime inputs and describes embedded notices. Audit actual shipped components and redistribution obligations, including loaders and any compiler sources/patches; do not label the whole SDK ISC or assume embedded notices replace Plenty’s release obligations. If code or data ships, update module headers where applicable, release third-party notices and supply documentation. This proposal copies no upstream implementation.

Cosmo compile-time target and code-generation contract

Proposed, not implemented. Part of the Cosmopolitan proposal. These contracts precede runtime qualification, not merely final packaging.

Build target is not the runtime operating system

Today src/frontend/target.rs derives facts from the packaged native triple, recognizes a closed set of target.os values, and discards unselected branches while parsing, before resolution and checking. See the implemented target conditionals. Using that native fact source for an APE would silently select Linux-specific code on a Linux compiler host, even when the artifact later runs elsewhere.

Recommend adding the closed value "cosmo" for target.os and deriving all facts from the explicit output-target descriptor, never the compiler host. Here os denotes the compile-time platform contract, not the executing kernel. For the proposed x86-64 profile the facts are os = "cosmo", arch = "x86_64", pointer_width = 64, and endian = "little". A later qualified AArch64 slice would have arch = "aarch64"; fat output evaluates conditionals independently for each slice. This does not introduce runtime dispatch or a runtime OS API. Runtime detection facilities are outside this proposal.

For Cosmo, existing comparisons with "linux", "macos", "windows" and "freebsd" remain valid but equality is false and inequality true. A chain selects its first true arm, or its else, just as native conditionals do. Thus an existing Linux/else program selects its else arm: that arm must be portable or explicitly reject Cosmo through capability checking. An else arm is not evidence of portability. Unknown spellings remain errors. Native targets retain their existing values and branch behavior; recognizing "cosmo" adds a value to the closed set, not an alias for any native OS. Older compilers will reject that new value when evaluated, so using it requires the new compiler. Unselected arms remain syntactically parsed but are not resolved/type-checked; this proposal does not change that language rule.

Required tests cover top-level selected/unselected imports (including absent modules), declarations and function-body branches, first-match/else and no-match chains, all native OS equality/inequality comparisons, boolean combinations, unknown-value diagnostics and native regressions. Compare identical Cosmo selection across compiler hosts wherever those hosts are supported; unit tests must independently vary host and output descriptors even during Linux-only experimentation. Cache tests must isolate native versus Cosmo and per-ISA/profile entries, including a source whose selected import changes. Include the output facts/conditional-semantics version in the shared target/cache identity so parsed or compiled results cannot be reused under a different platform contract.

Reject host-OS aliasing: it defeats multi-OS output. Reject treating target.os as a runtime expression: discarded imports cannot be restored at launch. Reject initially banning all OS conditions in favor of the explicit Cosmo value: a dedicated arm gives portable libraries a direct migration path while keeping native-only arms unselected. Capability rejection still guards unsupported calls in the selected code; OS comparison alone is not a capability check.

Early x86-64 red-zone and code-generation gate

The retained 4.0.2 x86_64-unknown-cosmo-cc wrapper (around lines 119–128) passes -mno-red-zone, except in its Linux-only optlinux mode, which selects -mred-zone. Multi-OS output must not assume optlinux compatibility. In contrast, build.rs invokes rustc with Cargo’s GNU target and no explicit no-redzone option. This is a compatibility risk requiring inspection, not evidence that red-zone use caused the observed trap failure. The signal-layout mismatch is measured; its causal role in that failure remains a hypothesis too.

Before any runtime-compatibility acceptance, audit Plenty-generated objects, Rust runtime code, bundled Rust std/compiler-builtins, vendored Rust crates and native dependency/assembly objects. Retain effective backend settings, rustc arguments/target specifications, C/assembly build commands, archive-member and reachable-symbol inventories, and disassembly covering leaf functions, spills, stack probes and signal/worker paths. Establish whether reachable functions use storage below the current stack pointer without reserving it, accounting for control flow and inlining/LTO; a textual search for negative offsets alone is not a proof. Unreachable members need a recorded reachability argument and profile enforcement, not an assumption that all dependencies disappear.

The recommended multi-OS contract is no red-zone reliance in reachable code. Verify the pinned Cranelift configuration and emitted code; configure/rebuild Plenty objects if necessary. Use -C no-redzone=yes for rebuilt Rust components and compatible C/assembly settings where required. A final Cosmo linker flag cannot retroactively recompile emitted Plenty objects, Rust std or archives. If an incompatible reachable member is found, rebuild its owning component (including std/compiler-builtins if implicated), replace it, or reject the capability/profile; then repeat the audit and behavioral tests. Do not assert that every Rust dependency must be rebuilt without that evidence. If provenance or analysis cannot establish compatibility, the gate stays closed. Record the verified code-generation policy and affected input hashes in archive/cache identity so changing the policy invalidates old objects.

Self-contained baseline harness specification

First implementation deliverable, not implemented here. All paths and invocations below are proposed. Part of the Cosmopolitan proposal. This specifies reproduction, not a passing portability test or another work queue.

Inputs and offline preparation

Check in scripts/experiments/cosmopolitan/ with a README, a Python 3 driver, a machine-readable input manifest and fixtures/. No invocation may depend on the original /tmp directory, prebuilt compiler or unpublished scratch files. The driver must use argument arrays, isolated output directories and explicit compiler/tool paths, including paths with spaces. It must not modify the host’s loader registration, main checkout or global configuration.

Pin the baseline source to 2851ef3481f70dec580eb76d2651b86524d16dfb and its tracked Cargo.lock; record the actual source tree hash and any patch diff. Pin Cosmopolitan 4.0.2 from the release archive, SHA-256 85b8c37a406d862e656ad4ec14be9f6ce474c1b436b9615e91a55208aced3f44 (bundled GCC 14.1.0). Retain the versioned upstream README, driver, signal/TLS headers and fixup source with URLs, release/source identities and hashes. See official sources.

The original installed rustc was 1.99.1 (58cf8f9c3 2026-10-13) and host cc was GCC 16.2.1 20260819; neither identifies the toolchain that built the retained Plenty binary. The harness implementation must resolve a complete Rust toolchain pin (compiler, Cargo, target std and component hashes) and native host-tool inputs in its manifest before running; it must not substitute a moving stable or pretend to reconstruct the prebuilt binary. If those exact host versions cannot be provisioned, record the deliberate replacement and treat output differences as discrepancies to investigate. The retained compiler hash on the evidence page is comparison metadata, not a required input.

Proposed preparation invocation (not an existing command):

python3 scripts/experiments/cosmopolitan/run.py prepare --manifest MANIFEST --inputs INPUTS
python3 scripts/experiments/cosmopolitan/run.py baseline --offline --inputs INPUTS --output OUTPUT

prepare is the only network-enabled phase: verify downloads, prepare the pinned source checkout and Rust/Cargo dependency cache/vendor inputs and host tools, and write a hash inventory. baseline --offline verifies that inventory, builds with cargo build --locked --offline --release --bin plenty in isolated build storage, and runs without network. Missing/corrupt inputs fail before building; no implicit downloads or fallback to a cached prebuilt compiler. Repeat in two clean output directories and compare input/object/archive hashes, recording any nondeterminism rather than requiring unexplained executable byte identity.

Complete original source fixture set

The following text is the required retained fixture content, not new runnable book examples. Preserve filenames and line breaks for source-site diagnostics. No upstream implementation code is copied by this proposal.

fixtures/hello.c:

#include <stdio.h>
int main(void) { puts("cosmo C control"); return 0; }

fixtures/empty.plenty:

def main() -> i32:
    0i32

fixtures/print-alloc.plenty:

def main() -> Result[(), AllocError | IoError]:
    xs = [1, 2, 3]?
    print("cosmo Plenty", xs)?
    Ok(())

fixtures/trap.plenty:

def divide(n: i64) -> i64:
    1 // n

def main() -> Result[(), IoError]:
    print(divide(0))?
    Ok(())

fixtures/abi-control.c:

#include <stdio.h>
#include <stddef.h>
#include <signal.h>
int main(void) { printf("sigaction size=%zu flags=%zu mask=%zu restorer=%zu\n", sizeof(struct sigaction), offsetof(struct sigaction,sa_flags), offsetof(struct sigaction,sa_mask), offsetof(struct sigaction,sa_restorer)); return 0; }

Command and artifact contract

Run the fixtures from an output-local copy so diagnostics use trap.plenty. Record complete commands, environment/tool versions, elapsed times, stdout, stderr and raw subprocess status separately for every case:

  1. Build hello.c with cosmocc/bin/cosmocc hello.c -o hello.com; shell launch must print cosmo C control and exit 0. This C control can be dual-ISA; the Plenty tests use only cosmocc/bin/x86_64-unknown-cosmo-cc.
  2. From the fresh compiler, extract --emit-runtime runtime. For each of empty, print-alloc and trap, use --compile NAME.plenty --no-cache -o NAME.native and --emit-object NAME.plenty --no-cache -o NAME.o. Native empty exits 0; print-alloc exits 0 with cosmo Plenty [1, 2, 3] plus newline; trap exits 1 with error: trap.plenty:2:5: division by zero on stderr.
  3. Test unchanged --compile NAME.plenty --no-cache --no-strip --verbose --linker ABSOLUTE_COSMO_CC -o NAME.as-is.com for empty and print-alloc. Baseline expects linker exit 1 mentioning missing -lutil.
  4. Read extracted runtime/link-args.txt as one argument per line. Link each NAME.o, runtime/libplenty_runtime.a, all extracted arguments except exact -lutil entries, then -o NAME.filtered.com, using the x86-64 driver. Record original and filtered arrays; do not delete any other flags. Baseline expects all three links to succeed, empty/print output as native, and trap raw status -4 (SIGILL) with no diagnostic. Disable core dumps and run via sh -c 'exec ./NAME.filtered.com' with an eight-second timeout, preserving child status rather than relying on a shell’s textual signal report.
  5. Compile/run abi-control.c with native cc and the x86-64 Cosmo driver. Baseline output is respectively sigaction size=152 flags=136 mask=8 restorer=144 and sigaction size=32 flags=8 mask=24 restorer=16, each with newline/status 0.
  6. Attempt direct Python subprocess execution of hello.com and the two successful Plenty APEs. Record errno separately from child status. The spike observed ENOEXEC (8); loader/binfmt-equipped hosts may differ and must be identified, not reconfigured to manufacture the original result. Also record successful shell launch routes.

Retain sources, manifest, compiler/runtime/object/executable hashes, extracted archives and link arguments, logs, disassembly/relocations and a JSON case report with expected/actual outcomes, discrepancy classification and launch route. Capture failures per case and continue independent cases; skip dependent cases with explicit reasons. Unexpected failures, timeouts, missing evidence or baseline discrepancies make the harness exit nonzero. Expected linker and trap failures are recorded baseline matches, not an early shell exit or portability success. A changed result from fresh source must be investigated, never silently blessed.

A separate future acceptance mode must require traps to diagnose at the source site and exit 1, plus the signal/stack/thread tests in the qualification gates. It must not inherit the baseline’s expected SIGILL as a pass. This proposal adds no harness, executes none of these cases, and claims no new experiment result.

Cosmo evidence and qualification gates

Part of the Cosmopolitan proposal. Spike observations are not implementation or support claims. Gates define required evidence, not a parallel priority list; scheduling remains B18/B17 in the backlog.

Retained experiment and provenance

The completed spike inspected main 2851ef3481f70dec580eb76d2651b86524d16dfb in /tmp/plenty-cosmopolitan-20260718. Its report is retained locally at /tmp/plenty-cosmopolitan-spike-report.md, with inputs, logs and binaries in /tmp/plenty-cosmopolitan-20260718-artifacts. These are local evidence locations, not required book assets or reproducible public downloads. The investigation worktree remains unchanged. The observations below preserve the essential results even when those scratch paths are unavailable.

The experiment used a retained prebuilt Plenty compiler and its embedded runtime, not a fresh build from the inspected source. It reported Plenty 0.7.1, x86_64-unknown-linux-gnu; retained binary SHA-256: 489623df196e84fd201417dae5a389574af184b4c7e1dc78f8a74c05e186d48f. The downloaded Cosmopolitan 4.0.2 archive SHA-256 was 85b8c37a406d862e656ad4ec14be9f6ce474c1b436b9615e91a55208aced3f44. Source inspection and binary execution therefore have different provenance.

Only Linux x86-64 was tested. No Windows, macOS, BSD, AArch64, BIOS, networking, worker-thread, native engine or unwind qualification was performed. No compiler/runtime source was changed by the spike.

ProbeRetained evidenceObserved result
C control with ordinary cosmoccc-control-build.log, c-control-run.logShell launch prints control message; not a Plenty runtime test
Native Plenty empty and print/list allocationempty.native-*.log, print-alloc.native-*.logBuild/run succeed
Existing Plenty linker overrideprint-alloc.cosmo-build.log, empty.cosmo-build.logLink fails: cannot find -lutil
Exact emitted object + extracted runtime, omit only -lutil*.filtered-command.txt, *.filtered-build.log, empty.filtered-run.log, print-alloc.filtered-run.logEmpty and print/list allocation link and run; output cosmo Plenty [1, 2, 3]
Division by zerotrap.native.run.log, trap.filtered.com.run.logNative reports source-site error and exits 1; Cosmo exits -4 (SIGILL) with no diagnostic
C sigaction layout measurementabi.native.log, abi.cosmo.log, abi-control.cGNU size/flags/mask/restorer offsets 152/136/8/144; Cosmo 32/8/24/16
Direct subprocess executiondirect-exec.logC control and both successful Plenty APEs return ENOEXEC; shell fallback succeeds

The ELF64 x86-64 objects linked with their measured PC-relative relocations. This establishes basic object/link compatibility for these inputs, not complete runtime compatibility. The signal-layout mismatch is measured; that it explains the trap failure is a strong hypothesis, not an isolated causal result from a patched runtime. Successful startup does not prove handlers were installed correctly. TLS fixup/source inspection and observed GS accesses leave TLS compatibility unresolved rather than demonstrating either universal safety or universal failure.

reproduce.sh retains consolidated commands, but was not rerun as a suite and ends with an expected failing as-is link. It is not a passing regression test. Do not rerun expensive probes merely to write this proposal; the first approved implementation experiment must replace the prebuilt provenance gap. Its first deliverable is the checked-in self-contained harness, including all source fixtures, pinned/offline inputs and per-case failure records.

Staged go/no-go criteria

Each stage requires review of the evidence before expanding scope. Stop with a recorded limitation if acceptance fails; do not expand an allowlist to bypass it.

GateBounded experiment / acceptance criteriaDecision enabled
Reproducible baselineImplement the harness contract first; fresh locked compiler/runtime, pinned offline inputs and retained sources/commands/artifacts; record expected linker/trap failures separately from passing acceptanceConfirm the spike applies to reproducible source, or investigate the discrepancy before proceeding
Target and code-generation compatibilityEstablish the target-fact and red-zone contract: host-independent branch/import selection, target/cache isolation, effective codegen settings and reachable-object audit across Plenty, Rust runtime/std and native dependencies; rebuild or reject implicated components before runtime acceptanceAdmit only code-generation inputs with a defensible multi-OS ABI contract; no final-link-only fix
x86-64 runtime compatibilityCosmo-header signal/context shim; checked traps with site/status, unknown-signal forwarding, stack overflow on main and worker threads, page probing, allocation failure/panic/OOM reporting, repeated thread start/join/failure, isolated TLS/destructors and contended synchronizationDecide whether a bounded GNU-component compatibility model is defensible; otherwise stop or approve a dedicated runtime-build investigation
Real multi-OS executionTransfer the exact same hashed x86-64 artifact to at least one real non-Linux OS; run core success/failure/thread tests through a documented loader route, without rebuilding it per OSDecide whether to adopt a restricted executable profile; Linux-only success is insufficient
Restricted preview integrationReviewed capability set, explicit negative tests, target/archive/cache validation, pinned driver and reproducible offline packaging; preserve native GNU testsAdmit only named profile/OS/version/CPU configurations as experimental, with limitations
Broader support qualificationModule/dependency and OS security semantics pass independently; extracted-release tests on every claimed OS/ISA/version baseline, deployment policy and notices completeSupport announcement only for the qualified matrix; no automatic admission of upstream’s entire list
Optional fat outputIndependent AArch64 Cosmo ABI/runtime qualification, matching features and contracts, apelink packaging, identical combined artifact exercised on real x86-64 and AArch64 systemsDecide whether fat output merits its maintenance cost; not a prerequisite for x86-64 multi-OS support

The runtime compatibility gate must not disable trap reporting or stack protection to obtain a green result. If signal-context rewriting cannot be made safe, compare helper-call checks explicitly while retaining proper stack fault handling; document the performance/semantic consequences before proceeding. No requirement here implies stack-overflow recovery: preserve the current executable diagnostic and abort behavior, and separately qualify any admitted actor trap containment.

Testing and CI design

Keep native GNU/Linux regression jobs unchanged. Add a separately named, opt-in experiment lane before release jobs: build the compiler fresh, record provenance, compile a behavioral corpus once, and publish hashes and artifacts for real-OS jobs. Non-Linux jobs consume those artifacts, not recompile convenient OS-specific replacements. Cross builds and emulators supplement ABI debugging; they cannot establish real signal, loader, filesystem-policy or scheduling behavior.

Exercise debug/release, cache on/off, paths with spaces, failure injection, large/recursive frames, integer and collection traps, foreign unknown faults, handler coexistence, thread startup failure and repeated cleanup. Test allocation balance and per-thread state after jobs. Check profile rejection via direct, indirect and generic calls, unavailable optional engines, wrong runtime manifests, stale caches, missing tools and direct-exec/loader errors. Add full compiler and runtime all-feature suites and executable documentation tests for admitted features; explicitly label exclusions instead of broadly skipping non-Linux failures. Library mode requires separate host-coexistence and ABI tests if admitted.

For filesystem/network expansion, include adversarial rooted paths, native names, resource leaks, cancellation/readiness, DNS, TLS entropy and handshake errors, process cleanup and unavailable facilities. Reuse the contracts in the native OS semantics assessment, not guessed Linux errno mappings. Check absence of accidental dynamic GNU/Linux dependencies and unqualified syscall paths in the final reachable runtime.

Release qualification additionally needs a clean extracted-package test with only documented tools, no Rust/Cargo or network for end users, correct status propagation, loader extraction failure/noexec cases, and OS download/launch policy including signing where required. Record which machines, versions, CPUs and routes actually ran. CI configuration alone does not qualify a platform; real runners and retained passing results are a go/no-go dependency.

Official sources and source-credit boundary

The spike checked official sources over HTTPS and retained copies. These links are versioned where possible; no additional external platform claim is needed for this proposal:

The spike’s ape-readme.md is a 404, not supporting evidence. No upstream code, data, toolchain or executable is added to the repository by this documentation proposal; implementation-stage redistribution requires the audit described in pinning and licensing.

Native macOS, Windows, and AArch64 support

Status: Research and design update (2026-10-31). Linux AArch64 is implemented in the ARM feature branch, with qualification limits recorded in the development guide. macOS and Windows remain proposed. This document does not announce their support. Priorities and implementation work packages live in B18 of the backlog, with B17 delivery and B12 path-contract dependencies.

Related design records cover FFI and dynamic libraries, generated export contracts, and the independent GCC backend candidate. This assessment concerns complete native toolchains, not only instruction selection.

Plenty can plausibly reach these platforms while keeping Cranelift, but this is a compiler/runtime/stdlib port, not just a release matrix expansion. Start with native builds: a compiler running on one OS/architecture generates programs for that same OS/architecture. Cross-compiling the compiler with Cargo is a separate build-system concern; compiling a Plenty program for a different target would additionally need a target ISA, runtime, SDK and linker. Defer that user feature until native ports are qualified.

Proposed scope

Native host and generated-program targetPresent evidenceProposed treatment
x86_64-unknown-linux-gnuExplicitly admitted, tested and releasedPreserve existing baseline throughout
aarch64-unknown-linux-gnuImplemented; extensive QEMU execution, native/release gates outstandingPreserve alongside x86-64 Linux during refactoring
aarch64-apple-darwinRejected by Plenty; native Cranelift/C ABI smoke tests pass in CIFirst macOS port after common target/build work
x86_64-apple-darwinNot admittedDeferred; no automatic promise from the ARM macOS port
x86_64-pc-windows-msvc / x86_64-pc-windows-gnuNeither admitted; native MSVC Cranelift/C smoke tests pass, GNU remains separateCompare SDK/CRT/dependency feasibility before choosing one initial Windows ABI
aarch64-pc-windows-msvcRejected today; ARM64 COFF relocation coverage is an additional backend riskSeparate feasibility gate before committing to a release

All proposed targets are 64-bit little-endian. ARM32, big-endian AArch64, Android, iOS, Linux musl, ARM64EC, and universal macOS binaries are outside this initial scope. AArch64 support is not a promise to support every ARM board, OS or ABI. WSL runs the Linux build; it is not native Windows support. Minimum OS/kernel versions and CPU baselines must be chosen and tested per target; do not inherit Rust’s support tiers as Plenty’s own guarantees.

Evidence baseline

This update inspects Plenty 869e1cb8087c126ec26a620149bd74d2290c4ca7 and Cranelift 0.137.0. The October 19 assessment described an older x86-only checkout; its assumptions about hard-coded native calling conventions and x86-only crypto assembly are superseded by the ARM implementation.

The current compiler has a two-target description, derives native C calling conventions through Cranelift, and builds matching ARM runtime dependencies. The target foundation centralizes frontend facts and binary identity in src/target.rs, and checks the compiler host against the packaged runtime. Metadata layout and object handling still implement ELF, and runtime services remain Linux-specific. This is a useful starting point, not a finished OS abstraction.

The immediate experimental scope is ARM64 macOS and x86-64 Windows. Preserve the Linux milestone in Git; perform independent work in child worktrees and merge validated increments into the portability branch. No local Mac or Windows machine is assumed: actual OS execution requires CI runners. Configuring a job or cross-linking an artifact is not execution evidence.

Keep portable language semantics and checked Plenty library policy above a small target-specific native boundary. Centralize target data: triple, pointer and scalar layout, C ABI, object format, linker flavor, artifact suffixes, runtime identity, supported facilities and minimum platform. Each source of this information must agree; an allowlist expansion is the last enablement step, after validation, not the first workaround.

Use separate runtime platform modules for Linux, Darwin and Windows operations. Prefer Rust std where its allocation, progress and error behavior meet Plenty’s contracts; use reviewed OS bindings where they do not. Avoid copying raw Linux constants into a generic module or translating every OS failure to a guessed Linux errno. Unsupported facilities need an explicit, documented outcome; security-sensitive operations must fail closed rather than silently weaken.

The detailed assessment is split into:

Study both OS ports before fixing shared interfaces, then establish a minimal macOS path and the selected Windows path. Port OS services against explicit contracts before claiming parity. Keep Cranelift as the production backend; Zig is a candidate C/dependency compiler and linker driver. User-facing Plenty cross-compilation remains a separate admission decision.

Interface boundaries and source mapping

Proposed organization, based on Plenty 869e1cb8, with the shared target foundation now extracted into src/target.rs. No new target is admitted by this design. Work packages and priorities belong to B18.

Separate three identities

Cargo’s build host runs build scripts. The compiler host runs Plenty and its linker driver. The generated-program target supplies the ISA, ABI, runtime and system libraries. Native releases currently equate the latter two; cross-building the Rust compiler does not change that relationship. Preserve separate identities even while enforcing equality at admission.

Use immutable target facts for triple, OS, architecture, object format, native calling convention, minimum system version and artifact naming. Derive these from one checked description; do not duplicate target-name comparisons in the frontend, cache, package tooling and release scripts. Cache identity must include runtime/ABI compatibility and features, not just the CPU name.

Focused interfaces

OwnerShared operationImplementation boundary
Compiler host/tool invocationDiscover tools; execute an argument vector and report errorsHost process/filesystem APIs; explicit target linker/SDK configuration
Target codegenBuild ISA; describe native calls; enumerate special probe sites; decode instructionsReuse Cranelift TargetIsa; small Plenty-specific architecture helpers
Object emission and inspectionRetain/read interface records; resolve source/trap records; export symbolsELF, Mach-O and COFF implementations owning their symbol/section state
Linking/distributionConstruct a link plan; locate dependencies; validate an installed engineDriver-specific argument arrays and format-specific dependency inspection
Runtime OS servicesOpen/close/wait/spawn; obtain entropy; classify errorsLinux, Darwin and Windows service implementations with typed resource ownership
Fault recoveryRead saved PC/SP; enter a helper; recover an actor or terminate safelyOS-and-architecture context/exception implementations, selected at build time
Plenty standard libraryPortable policy, ownership and meaningful resultsNative backend normalization; if const only at deliberate source boundaries

A single object combining these responsibilities would couple compiler state to runtime mechanics. Prefer modules with identical functions for build-time choices, small traits for genuinely interchangeable behavior, and data for facts. Keep stateful symbol/relocation work inside the emission component; passing the entire compiler through a platform trait would defeat the boundary. No global platform initialization, allocation or dynamic dispatch should be introduced into signal handlers merely to satisfy this organization.

Plenty protocols provide static constraints where a library genuinely accepts multiple providers. They need not be threaded through every application to use files or networking. Platform selection belongs behind public standard-library interfaces. Protocols do not supply missing OS behavior or dynamic dispatch.

Compiler changes with concrete owners

Current locationDependency to hideAcceptance evidence
src/target.rs, src/frontend/target.rs::factShared facts now cover the two Linux targets; OS admission and linker policy remain separate workFrontend facts agree with runtime manifest and emitted object for every admitted target
src/codegen/output.rs::finish, src/library_metadata.rsGNU ELF flags and ELF-only metadata readingFormat-aware objects; metadata survives executable/static/shared library dead stripping
src/codegen/exports.rs, sites.rs, traps.rs, coverage.rsSection names, relocation and linker retention assumptionsSource/trap/coverage lookup and exported contracts after relocation and installation
src/toolchain.rs, src/library.rs, src/toolchain/distribution.rsGNU options, ELF SONAME/RUNPATH and .so namesDriver argument tests plus real link/load/relocation tests with spaces in paths
src/codegen/stack_probes.rs, runtime signal_context.rsPinned ARM instruction sequence and Linux context fieldsRegistered probes only; native overflow and unrelated-fault tests; handler coexistence
build.rs, engine/release helpersDirect dependency builds and Linux assembly/tool conventionsMatching compiler/runtime/native objects; reproducible offline build and artifact audit

Cranelift 0.137 provides AppleAarch64 and WindowsFastcall, and Mach-O/COFF object emission. That is feasibility evidence, not a working Plenty port. Keep internal Tail calls distinct from native adapters. Test narrow signed/unsigned arguments, mixed float/integer register exhaustion, u128 and callbacks against both Rust and the chosen C toolchain. Windows GNU and MSVC are separate cases.

Audit unwind information explicitly: Plenty uses define_function_bytes, which bypasses the object backend’s automatic unwind processing. The pinned backend also has limitations for automatic Windows and Mach-O unwind emission. Retaining panic-abort does not settle OS exception/stack-walking requirements. Native stack probes, handler coexistence and actor recovery are separate admission gates.

Runtime and language contracts

Plenty surface / current dependencyReference implementation to inspectProposed boundary and required tests
files.rs, aggregates/filesystem.rs; Unix descriptors, no-follow, temporary publicationRust sys/fs/windows.rs; CPython Modules/posixmodule.c, Python/fileutils.cTyped resources and rooted operations; rename/link races, sharing, delete-open behavior, atomic publication and confinement
stdlib/std/path.plenty, environment and arguments; native Unix bytesRust sys/os_str, sys/process/windows.rs; CPython Lib/ntpath.py, _winapi.cLossless native names and explicit conversions; UNC/drive paths, unpaired wide units, invalid Unix UTF-8, Unicode and spaces
aggregates/readiness.rs, mutex.rs, cancellation; epoll/eventfd/futexRust sys/sync, sys/net; CPython Modules/selectmodule.cWait/wake contracts covering sockets, pipes and cancellation; lost wakeups, EOF, stale registrations, timeout and handle reuse
stdlib/std/fs/watch.plenty, aggregates/watch.rs; inotify records/masks/cookiesCPython selector backends help waiting, but do not supply equivalent watch semanticsNormalize events below portable policy; decide rename pairing, overflow and recursive behavior without inventing guarantees
aggregates/net.rs, stdlib/std/net.plenty, report.plenty; Unix handles and numeric errnoRust sys/net/connection/socket/windows.rs; CPython Modules/socketmodule.cDistinguish socket ownership and OS error domains; partial I/O, would-block, DNS lifetime, SIGPIPE and cancellation
aggregates/process.rs, files.rs; spawn/pipes/inheritanceRust sys/process/windows.rs; CPython Modules/_winapi.c, _posixsubprocess.cLiteral arguments, explicit inherited handles, bounded pipe draining, missing executable, timeout, termination and reap
threads.rs, actor_heap.rs, signals.rs, stack_overflow.rsRust sys/thread/windows.rs, sys/pal/{unix,windows}/stack_overflow*Fallible startup without hidden allocation; native fault/exception recovery, cleanup, shutdown notifications and concurrent first use
entropy.rs, crypto/TLS/compression dependenciesRust sys/random/{apple,windows}.rs; CPython Python/bootstrap_hash.cFallible secure entropy, no weak fallback; known-answer tests and CPU-dispatch checks

Reference paths are relative to the read-only checkouts and revisions recorded in delivery. They identify patterns and tests to study, not code licensed for uncredited copying. Prefer Rust facilities where their fallibility, allocation and progress semantics fit; borrowed implementation code/data needs its license and attribution handled explicitly.

The most consequential API decisions are Windows native path representation, readiness versus completion, filesystem event guarantees, OS error domains and shutdown signals. Record these before implementing backends. Security guarantees must survive the port; an unavailable operation must fail explicitly instead of silently following symlinks, losing filename data or weakening entropy.

Contracts requiring decisions before mechanical refactoring

stdlib/std/process.plenty encodes normal exits as 0–255 and POSIX signals as 256 plus the signal number. Windows native 32-bit exit codes require a distinct representation; preserve full status and distinguish termination from normal exit. Startup argument capture in text_io.rs also assumes C argc/argv bytes; compare Rust’s GetCommandLineW handling before choosing Windows behavior.

The generated mutex fast path assumes a u32 state word with states 0/1/2, while the runtime promises no allocation. Rust’s Darwin mutex implementation uses pthread primitives, and its Darwin thread parking uses public libdispatch semaphores rather than private address-wait APIs. A direct substitution must therefore prove layout, initialization and allocation contracts or change the Plenty representation/lowering deliberately. Rust’s Windows implementation uses address-based waiting; establish the supported Windows baseline before reuse.

control::Token stores a Unix OwnedFd even outside Linux-only operations. Separating function bodies alone will not make Windows compile: resource representations must also become platform-specific. Filesystem watching needs an explicit choice between retaining the Linux-native API and introducing a portable change/rescan contract; equivalent-looking notifications are not proof of equal rename, overflow or event-order guarantees.

Compiler, runtime build, and native ABI

Part of the cross-platform assessment. Proposed changes below are not implemented support.

Build the compiler and the runtime for the same machine

build.rs emits application and library runtime archives, native test hooks and a fuzz driver. The ARM work added target-aware C/archive tool discovery, matching Linux ARM ring assembly and dependency settings. Darwin/Windows selection still needs implementation: the current ring lists select ELF/Linux assembly, and direct Rust builds reproduce dependency build-script behavior.

Introduce one checked native-toolchain description consumed by runtime and native dependency builds. Distinguish Cargo HOST (build scripts run here), Cargo TARGET (compiler executable and embedded runtime run here), and any future generated-code target. Use native CI initially. For later cross-builds, require matching C tools, assembler, archiver, Rust target libraries and sysroot; reject inconsistent inputs before producing a package.

Prefer a separate, orchestrated runtime build using upstream Cargo build scripts and a locked dependency graph, then embed verified artifacts into the compiler. This avoids maintaining copies of ring/libc/getrandom platform logic. It must preserve the current self-contained compiler distribution, both runtime modes, offline builds, feature agreement and reproducibility. Do not invoke Cargo recursively against the same target directory from build.rs and risk lock contention. Compare this approach with a dedicated target-aware native build helper before changing the build architecture. If direct rustc builds remain, explicitly reproduce and test every dependency’s target-specific build-script outputs, features, native inputs and licenses.

Audit SQLite compilation and platform defines, compression CPU feature paths, crypto entropy/provider dependencies, native test hooks and libFuzzer separately. The runtime workspace’s Cargo build and the embedded build must behave alike. On Windows choose compatible CRT linkage and a documented SDK/toolchain; allocations crossing library boundaries must still be freed by their owner. Keep Rust panic-abort and Plenty’s no-unwinding contract. Do not introduce an implicit Rust installation requirement for release users.

Code generation and ABI correctness

Keep the internal Plenty Tail calling convention distinct from the target C ABI. Derive native signatures from the target and audit all boundaries: runtime helpers, entrypoints, threads, callbacks, imports, exports, dynamic library discovery, and compiler-generated adapters. Replacing one SystemV assignment does not cover the others. Small integer extension, booleans, float arguments, aggregate returns, hidden return pointers, stack arguments and callee-saved registers need C/Rust/Plenty round trips on each target.

Retain Plenty’s specified layouts where possible, but separately establish foreign layouts. Windows’ C long is not a Unix 64-bit long; OS structs, socket handles and pthread_t cannot be inferred from pointer size. Check i128 and floating-point lowering/libcalls, alignment of aggregates and atomics, thread entry storage, and library contract hashes against the selected ABI. CPU baselines must not accidentally depend on the release builder’s processor.

Validate direct and indirect tail calls with spilled arguments, deep recursion, large stack frames, closures, generators and worker threads. Keep stack probing and guard-page protection: disabling them to make a build pass is not a port. plenty-runtime/src/stack_overflow.rs requires separate architecture/OS handling, including Darwin fault context and Windows exception/stack guarantees. Library runtimes must continue respecting the embedding process’s handlers. A fallback may preserve safe termination without identical diagnostic text, but must be specified and tested rather than misclassifying arbitrary memory faults.

Objects, libraries, and linker drivers

ConcernLinux GNUDarwinWindows MSVC
Object / executableELF .o, conventional executable nameMach-O .o, conventional executable nameCOFF .obj, PE .exe
Native libraries.a, .so.a, .dylibstatic .lib, .dll plus import library
Driver and SDKGCC/Clang with libc development filesApple Clang and selected macOS SDKMSVC or qualified clang-cl/lld-link with Windows SDK
Retention/export policyExisting GNU sections/version scriptsMach-O section/segment limits, exported symbols and dead strippingCOFF sections, export tables/import libraries and retention rules

Represent link arguments as argument arrays, including paths with spaces; current runtime link strings are whitespace-split. Add driver-specific command construction and discovery rather than making MSVC pretend to be cc -o. Audit startup, system libraries, minimum OS flags, executable naming, object archive determinism and diagnostics for missing toolchains. LibFuzzer and sanitizers may initially be explicitly unavailable where not qualified.

src/library_metadata.rs requires ELF for the admitted Linux architectures and .plenty.interface.* sections. Define target-specific metadata sections and validate architecture, endianness, bounds and contract identity for Mach-O and COFF, including archive members. Mach-O naming constraints make simply reusing the full ELF section name unsuitable. Enable the required object reader features deliberately. Test object-only output, interface extraction, C headers, static exports and dynamic exports independently of ordinary executables.

Runtime loading currently uses dlopen, dlsym, dlclose and GNU RTLD_NODELETE. Darwin needs its own flags/lifetime implementation; Windows needs wide-path loader calls, safe DLL search rules, exports and handle ownership. Preserve the existing invariant that callable code stays resident after lookup leases close, along with allocator provenance and instance identity checks. If residency cannot be provided safely, reject the facility until it can; do not map the existing close operation to an unload that leaves dangling code.

Compiler tools are part of the port

Audit src/project.rs, src/cache.rs, src/lsp.rs and test helpers for Unix exit status, executable suffixes, chmod/symlinks, replacement/rename semantics, open-file deletion, shell invocations and cache directories. Cache keys already include target/ISA information; preserve it and verify runtime, ABI and feature changes invalidate artifact reuse. Test concurrent writes on Windows, where open-file sharing rules differ, and choose platform-native cache locations while preserving explicit overrides.

The LSP manually parses file:// URIs and emits paths through lossy strings. Add correct drive-letter/UNC handling, escaping, case behavior and non-ASCII round trips; keep OS-native paths until presentation. Test package roots with spaces, Unicode and mixed separators, Git dependencies, command arguments, timeouts and child cleanup. Compiler startup and --check are necessary but insufficient acceptance tests for a working developer installation.

Standard library and OS semantics

Part of the cross-platform assessment. These are proposed contracts to resolve before implementing native ports.

Paths, text, and filesystem guarantees

std.path.Path stores native Linux bytes, exposes as_bytes/from_bytes, and checks an absolute path by its initial slash. std.env and filesystem/process helpers also exchange native bytes. Windows uses UTF-16 OS strings, drive roots, UNC paths and device names; replacing separators or converting invalid text lossily would change filenames and could undermine confinement.

Choose a native-path representation with lossless round trips before Windows implementation. Recommended direction: an opaque OS-native Path and explicit text conversion; keep raw Unix-byte constructors/accessors platform-specific, and provide an explicit Windows wide-unit boundary if needed. Consider a documented lossless internal encoding, but never describe that encoding as UTF-8 text or portable filesystem bytes. This is a public API migration requiring compatibility decisions, tutorial examples and tests, not merely a private FFI change. Reject unsupported conversions explicitly. Process arguments and environment names/values need the same policy; source code remains UTF-8.

Preserve lexical operations separately from filesystem resolution. Test drive relative paths, UNC/device prefixes, reserved names, trailing dots/spaces, case-sensitive and case-insensitive volumes, Unicode normalization, long paths, symlinks and Windows reparse points. macOS paths are not automatically case-sensitive just because they are Unix paths. Archive member names are a portable format namespace; extraction into native paths must reject traversal and collisions under the destination filesystem’s rules.

aggregates/filesystem.rs uses descriptor-rooted no-follow operations and Linux O_TMPFILE plus /proc/self/fd publication. Darwin needs an alternative temporary-file/publication strategy. Windows needs owned directory/file handles, reparse-point policy, sharing/deletion rules and an honest atomic replacement contract. Security must be evaluated under concurrent rename/link replacement, not only clean-directory tests. Never replace rooted operations with unchecked string concatenation. If a promised no-follow/atomic operation has no adequate backend, return a documented unsupported result. HTTP static files, multipart uploads and archive extraction inherit these requirements.

Handles, errors, readiness, and networking

The current runtime uses Rust File/Unix descriptors for several resource kinds. Windows HANDLE, Winsock SOCKET and C-runtime descriptors are distinct; an integer cast and a common close are not a resource abstraction. Introduce typed owned native resources behind existing language ownership, with distinct close/read/write/wait behavior and cancellation-safe lifetimes. Audit runtime tags and compiler intrinsics that currently assume a file-shaped resource.

std.net and std.report classify numeric Linux errno values. Move portable classification to the native boundary and preserve original OS domain/code for diagnostics. Decide how existing public IoError/NetError variants expose that information without losing partial progress. Darwin errno, Win32 errors and Winsock errors are not interchangeable. Broken pipe, interrupted operation, would-block, timeout and cancellation must retain their specified distinctions. Review error formatting (strerror_r variants included), newline behavior, binary/text console I/O and SIGPIPE suppression.

FacilityCurrent assumptionProposed port boundary
std.ready, cancellationepoll/poll and Linux eventfdDarwin kqueue or qualified poll/wakeup implementation; Windows design for sockets and pipes/files plus wakeable cancellation
std.net, DNSLinux socket constants and hand-written structs; pthread resolver helperOS bindings, Winsock initialization/error mapping, bounded resolver lifetime and platform address conversion
std.signalLinux signal delivery on both admitted architecturesDarwin signal backend; explicitly define Windows console/service shutdown capabilities instead of pretending POSIX signals exist
std.processLinux implementation, Unix status/signals and pipesNative argument-vector spawn, inheritance control, Windows quoting/handle lists, termination/reaping and bounded output
std.secretsLinux getrandom; unsupported elsewhereFallible OS entropy primitive per platform, preserving no-weak-fallback guarantee
std.timeRust clocks plus deadline contractsVerify monotonicity, precision, overflow, suspend behavior and timed waits per backend

Windows IOCP is completion-based, while the public ready API is readiness-based. Choose the bridge deliberately: a readiness abstraction for supported resources, or a completion-backed adapter that owns buffers and preserves lifetimes. A socket-only select/poll replacement does not implement waiting on process pipes and cancellation tokens. Account for handle limits, stale registrations, EOF/hangup, cancellation races and deadline rounding. macOS also needs tests for event removal and descriptor reuse; sharing a Unix API shape is insufficient.

Threads, TLS, and higher layers

threads.rs directly declares pthread functions and fixes task storage to a machine-word thread identifier. Use target-specific thread creation/join and entry ABI while preserving pinned caller-owned storage, fallible startup and join-before-reclaim. Rust’s boxed thread startup is a useful semantic reference, not an automatic replacement for Plenty’s allocation contract. Review stack sizes, TLS initialization and cleanup, atomics/alignment and memory ordering. AArch64’s weaker ordering makes native concurrency stress especially important; channels and executors must not rely on x86 behavior.

Rustls protocol logic is broadly reusable, but the vendored ring build, entropy, socket progress, readiness and certificate paths must first work. Preserve verified TLS, caller-selected trust roots, mutual TLS, ALPN, SNI and revocation contracts. Native OS certificate stores would be a separate trust-policy feature, not an implicit change in a port. Requalify HTTP pooling, shutdown, deadlines and file serving after lower layers pass; parser tests alone do not prove an HTTP server port.

Collections, JSON/TOML/CSV, URL parsing, encoding and much of regex/compression should need fewer OS changes, but still require target execution, allocation failure and integer/Unicode compatibility checks. SQLite requires its correct OS VFS/build configuration, locking, durable-write and path tests. Math/float edge behavior, endian packing and CPU-dispatched crypto/compression need their own fixtures. Avoid blanket gating that turns a green test suite into an undocumented subset of the standard library.

A limited preview may explicitly omit unqualified modules. Before a general release, publish a capability matrix and ensure unavailable operations are rejected predictably without unresolved symbols, compiler panic, unsafe fallback or silent loss of guarantees. The current reference remains authoritative until each implemented port updates its contracts and learner-visible examples.

Qualification, delivery, and reference implementations

Part of the cross-platform assessment. These are dependency gates and design criteria; priorities remain in B18 of the backlog, not a second work queue.

Evidence required before calling a target supported

GatePrerequisites and acceptance evidence
Native backend experimentBuild an object, link and execute C/runtime calls, aggregate returns, indirect Tail calls, large frames and library metadata on the exact OS/ISA. Probe ARM64 COFF relocations before promising Windows ARM64.
Target/build boundaryCompiler and both embedded runtimes report matching target/ABI; target-aware native dependencies and drivers work offline; incompatible targets are rejected early. Both Linux architectures remain green.
Runtime coreAllocation failures, reporting, stack guards/overflow, thread start failure/join, channels, atomics, files and foreign ABI round trips pass natively. No unsafe success stubs.
OS and stdlibNative paths and resources, rooted filesystem guarantees, process cleanup, cancellation/readiness, DNS, TLS and HTTP pass the platform capability contract.
User toolingPackage build/run/test, cache races, LSP path round trips, static/dynamic libraries and supported fuzz tooling are exercised from an extracted release.
DistributionFresh machine with documented native-linker prerequisites, no Rust/Cargo and no network compiles and runs representative programs; dependency/OS baseline audits and signed-package policy pass.

The Linux AArch64 experiment can retain Linux semantics while exposing ABI, native assembly and memory-ordering assumptions. Darwin ports then exercise Mach-O, Apple ABI and Unix OS differences. Windows follows the decisions about paths, handles and readiness; Windows ARM64 depends on both the Windows port and backend feasibility. OS implementations can proceed independently once the shared contracts settle. Native release jobs must wait for their corresponding qualification gates; a compiler that prints --version is not enough.

CI and regression strategy

Add native test runners for each admitted tuple, with explicit toolchain/SDK and minimum-version policy. Emulators and cross-compilation can supplement coverage but do not replace real OS execution for thread scheduling, sockets, file races, signals, DLL loading or installer behavior. Runner availability and cost are implementation-time decisions. Plan for CI runners; the user confirmed no local Mac or Windows machines should be assumed. This assessment did not provision runners or publish any workflow.

Inventory Linux-specific fixtures: /proc/self/fd, /tmp, shell scripts, descriptor counts, errno numbers, ELF tooling, chmod, signals, executable suffixes and symlink privileges. Keep portable behavioral tests common and replace platform observations with equivalent checks. For example, verify resource return to an observed baseline, using descriptor counts on Linux and qualified handle accounting on Windows. Preserve adversarial leak/race/security tests; do not just skip every failure under a broad non-Linux condition.

Run compiler and runtime workspace suites with all features, tutorial examples, allocation injection and relevant release-helper tests. Test debug and release generated programs, cache on/off, library hosts and parallel execution. Include non-ASCII and non-text native names, paths with spaces, inheritance leaks, timeouts, repeated cancellation, DNS cleanup, TLS handshake errors and HTTP shutdown. Keep environment-dependent Unicode/version assertions semantic rather than tied to incidental toolchain output.

Release packaging and user prerequisites

Extend the existing tag-verified, locked, draft-then-publish workflow to build one archive per admitted tuple. Preserve tag/version checks, immutable published artifacts, checksums, licenses and third-party notices; the ARM work already parameterizes Linux compiler and engine artifacts and notices by target. No crates.io publishing is needed or proposed. Keep build, test and publication permissions separated.

Retain .tar.gz on Unix; use .zip on Windows, with plenty.exe and plentyc.exe. Update packaging/smoke helpers that assume one Linux suffix or archive glob. Every archive must contain target-specific installation guidance and the same required resources/notices. Test extraction and PATH setup as an ordinary user; .exe naming must propagate into packages and test runners.

For Linux AArch64 preserve and qualify the configured glibc 2.28 baseline and ARM dynamic-loader/dependency allowlist. Minimum-baseline release execution remains outstanding; configuration alone is not qualification. For macOS choose deployment targets, audit Mach-O architectures/install names and linked libraries, and test Apple Command Line Tools/SDK discovery. For Windows choose supported OS, SDK and CRT requirements; audit PE imports and run from a clean environment with only the documented native tools. The compiler still needs a native linker to build programs; bundling a compiler driver/SDK would be a distinct distribution/licensing decision.

The user-facing support promise must distinguish a full native port from a preview with explicit unavailable modules. A successful compiler build must not silently turn skipped actor/engine/watch tests into parity evidence.

Decide macOS Developer ID signing/notarization and Windows Authenticode policy before public downloads. Credentials, account access and signing runners are prerequisites for that release stage, not assumptions this proposal fulfills. Separate unsigned reproducible build checks from signed-artifact verification and document timestamp/signature differences. Test macOS quarantine/Gatekeeper and Windows download/launch behavior; signing is not a promise that every reputation prompt disappears. A preview must state unsigned status honestly.

Inspected reference implementations

The local checkouts were used read-only, without copying their implementations. Paths below are relative to the named checkout, making them useful even when the repos are not adjacent to a reader’s Plenty checkout. These snapshots are design evidence, not claims about the latest upstream support policy.

Rust, ../rust/, commit 080cc27146389dbb5c8772c476b1937be9c6110f:

Inspected pathTransferable lesson
library/std/src/sys/pal/mod.rsSelect platform implementations beneath a shared interface
library/std/src/sys/fs/windows.rsWide-path file APIs and owned handles, rather than Unix descriptor emulation
library/std/src/sys/process/windows.rsNative UTF-16 environment/command handling and argument quoting require explicit policy
library/std/src/sys/net/connection/socket/windows.rsWinsock resource ownership and socket errors differ from file handles
library/std/src/sys/thread/windows.rsNative entry ABI, startup failure cleanup and join ownership; do not copy its boxed allocation strategy blindly
library/std/src/sys/pal/unix/stack_overflow/{mod,handler_signal}.rs, library/std/src/sys/pal/windows/stack_overflow.rsStack protection/diagnostics need OS-specific setup and handler coexistence
library/std/src/sys/random/{apple,windows}.rsEntropy source selection depends on platform; adapt to Plenty’s fallible contract instead of copying panic behavior
compiler/rustc_target/src/spec/targets/{aarch64_apple_darwin,aarch64_pc_windows_msvc}.rsArchitecture, ABI, baseline and atomic capabilities belong in target descriptions; rustc support does not prove Cranelift coverage

CPython, ../cpython/, commit b3714a5fb7a3aadd72134ee0710b22519c1df09a:

Inspected pathTransferable lesson
Modules/posixmodule.c, Python/fileutils.cFile operations and lossless native names need platform boundaries; Python’s surrogate policies are precedents, not a requirement to alter Plenty strings
Modules/socketmodule.cSeparate socket closing/error behavior and platform capability detection
Modules/selectmodule.cpoll/epoll/kqueue and Windows socket selection are different capabilities; one selector call cannot cover every resource
Modules/_winapi.cWindows process/pipe/handle mechanics deserve a dedicated implementation
Python/bootstrap_hash.cOS entropy paths, size limits and failure handling; never weaken randomness for portability
Lib/ntpath.pyDrive, UNC and native path rules need a distinct model from POSIX slash operations

These references show patterns to adapt within Plenty’s ownership, bounded resource and explicit-allocation rules. They do not license assuming equal semantics across systems. No Rust or CPython changes are proposed.

Decisions still open

Before implementation, settle the supported OS/CPU baselines, Windows native path API migration, readiness/completion bridge, and reduced-preview capability policy. Then choose runtime build orchestration, target-specific tool discovery, and the scope of static/dynamic library support at first release. Confirm ARM64 COFF behavior with an executable experiment. Signing credentials and native CI capacity can be arranged in parallel, but are required before qualified releases.

The highest uncertainty is Windows resource/security semantics and backend coverage, followed by maintaining fallible allocation across portable helpers. Linux AArch64 now supplies the shared-baseline regression target; its native hardware and release gates remain open. The immediate port experiments are ARM macOS and x86-64 Windows, with MSVC versus GNU still a decision gate. No effort estimate is reliable until those focused experiments establish the actual backend and SDK constraints.

Native ABI and Zig toolchain experiments

Evidence from 2026-10-31. These are small toolchain probes, not Plenty port qualification. Decisions and implementation sequence belong to B18; interface boundaries describe the design.

Native Cranelift CI smoke tests

The standalone platform-probes repository now runs push-triggered native jobs on macOS ARM64, Windows x86-64 MSVC and Linux x86-64. At probe commit ded48e7d6570dba951a954d0de09e1e93749fc14, all three jobs passed. The fixture uses Rust 1.99.1 and Cranelift 0.137.0; each job checks its host triple, emits a native object, links a C harness using the platform compiler, and executes seven assertions. Objects, executables and logs are CI artifacts.

The assertions cover ten integer arguments, mixed integer/f64 arguments, signed/unsigned narrow arguments, generated-to-C calls for those signatures, and a native entry calling an internal Tail function. Native CI confirms these paths execute on Apple AArch64 and WindowsFastcall, rather than merely producing a Mach-O or PE file. This fixture does not compile Plenty itself.

The initial run exposed two harness configuration errors. Enabling Cranelift is_pic=true removed Darwin’s rejected absolute text relocations; Plenty already enables that setting. Replacing filesystem canonicalization with ordinary absolute output paths avoided MSVC driver mishandling of Windows verbatim path prefixes. Both corrections were verified by the successful run.

This is partial B18.1 evidence. Indirect/tail-return calls, large frames and stack probes, u128/aggregates, Rust ABI boundaries, unwind/exception handling, minimum OS versions and full runtime/dependency builds remain unqualified. Windows GNU is not covered by this native suite; MSVC is the working probe baseline, not yet a qualified Plenty release contract.

The Plenty repository’s portability.yml workflow additionally builds the actual workspace on these macOS and Windows runners and runs unit tests if compilation succeeds. On macOS it also runs the AArch64 stack-probe integration tests. It preserves build logs and reports failures while the ports are incomplete. Existing ci.yml jobs exercise Linux x86-64 and ARM64, including optional engines and Docker proxy tests. A successful standalone probe is not substituted for any of these compiler/runtime checks.

Two independent uses

cargo-zigbuild can help build the Rust compiler, its runtime and C dependencies. Separately, Plenty could invoke zig cc as the driver linking Cranelift-produced objects with a matching runtime. The latter needs Plenty’s own target-aware link plan and artifact selection; cargo-zigbuild does not add that user feature. Cranelift remains responsible for Plenty machine-code generation.

The important Zig mechanism is more than a linker executable: it carries target information, libc headers/sources or interface stubs, compiler support routines, and logic for producing target CRT/import artifacts. These inputs still have coverage and version limits. Full system SDKs and external libraries may remain necessary. Avoid promising arbitrary host-to-target builds based on a simple program, or confusing producing an executable with testing it on that OS.

Source and tool identities

InputInspected identity
Plenty869e1cb8087c126ec26a620149bd74d2290c4ca7
Local Zig source4dd65e63f23569782f1560d40217cc74aae8b6cb (development source)
Executed ZigInstalled /bin/zig, version 0.16.0; distinct from the source checkout
Executed cargo-zigbuild0.24.0; upstream tag resolves to 2ffd80658c96b5ff971085dd509e58e5b441f1c8
Executed Rust1.99.1

The Zig checkout’s src/Compilation.zig loads bundled Darwin libSystem.tbd and SDKSettings.json from lib/libc/darwin, and builds MinGW CRT/import libraries. Its src/main.zig has native MSVC library discovery. These are specific toolchain facilities, not evidence for portable Plenty runtime behavior.

The cargo-zigbuild documentation describes target-qualified compiler wrappers, glibc-version selection, SDKROOT, deployment targets and limitations. Its integration must reach all of Plenty’s direct C/archive commands, not only Cargo’s final Rust linker. Pin the tool version and inspect produced imports rather than assuming the requested libc baseline is sufficient proof.

Executed small probes

An isolated dependency-free Cargo package in /tmp/plenty-zig-port-probe compiles probe.c using CC_<target_with_underscores>, archives it using AR_<target_with_underscores>, and links it into a Rust executable. Its C function calls puts and returns its integer argument plus nine; Rust calls it with two and asserts eleven. The helper invokes each tool as an executable path, matching a relevant constraint of Plenty’s direct native build.

ExperimentObserved resultWhat it establishes
Rust+C aarch64-unknown-linux-gnu.2.28Build succeeded; ELF imported glibc symbols no newer than 2.28; QEMU Cortex-A53 execution exited zero and printed both messagesWorking wrapper/archiver handoff and small C ABI call; execution used glibc 2.41, so minimum-libc execution is unproved
Rust+C aarch64-apple-darwin, deployment 13.0, no SDKROOTBuild succeeded; ARM64 Mach-O reports minimum 13.0, SDK 26.4 and libSystem dependencyZig bundled stubs suffice for this tiny artifact; no macOS execution occurred
Darwin CoreFoundation headerCompilation failed without full SDK headersBundled libSystem stubs are not a complete Apple SDK
Windows MSVC C using stdio.hCompilation failed without SDK/CRT headersLocal environment does not supply the required MSVC development inputs
Standalone Windows GNU CProduced x86-64 PE32+ with UCRT API DLL importsSmall MinGW artifact construction only; no Windows/Wine execution, and no Rust GNU target installed for a Rust+C probe

Logs, sources and artifacts remain in that task-owned temporary directory. linux-run.json records the exact QEMU command and exit status. The successful builds do not compile Plenty’s embedded runtime, ring/SQLite/compression, optional engines, generated Tail calls or exception recovery. Those are explicit subsequent experiments, not inferred successes.

Reproduction inputs

The original fixture and exact environment are recorded in /tmp/plenty-zig-port-probe/REPORT.md. The executed build commands, from that fixture directory, were:

export CARGO_ZIGBUILD_CACHE_DIR="$PWD/wrappers"
export ZIG_GLOBAL_CACHE_DIR="$PWD/zig-cache"
export CARGO_TARGET_DIR="$PWD/target"
export CARGO_BUILD_JOBS=1
cargo zigbuild --offline --target aarch64-unknown-linux-gnu.2.28 -v
MACOSX_DEPLOYMENT_TARGET=13.0 cargo zigbuild --offline --target aarch64-apple-darwin -v
cargo zigbuild --offline --target x86_64-pc-windows-msvc -v
zig cc -target aarch64-macos -c framework.c -o framework.o
zig cc -target x86_64-windows-gnu windows.c -o windows-gnu.exe

The first two commands succeeded; the MSVC and framework commands failed as recorded above. framework.c includes CoreFoundation/CoreFoundation.h and references CFGetTypeID(kCFBooleanTrue); windows.c prints one message from main using stdio.h. All dependencies are system/toolchain inputs; no crates were downloaded for the fixture. The QEMU command was:

qemu-aarch64 -cpu cortex-a53 \
  -L /tmp/plenty-arm-toolchain/aarch64-buildroot-linux-gnu/sysroot \
  target/aarch64-unknown-linux-gnu/debug/plenty-zig-port-probe

Recorded SHA-256 artifact identities (build paths affect reproducibility):

ArtifactSHA-256
Linux Rust+C ELF9e248f61e6cedff324c9e0c7e25eb757432d8348bce96f5b8bd0a19d6ded360f
Darwin Rust+C Mach-O78784e5bc56a76a2f98552b486706120a6121816a72ecfe7e5eb27bc0a7804d1
Windows GNU C PE46b4e2498719abb500ba3f6a35b114e1b827882eb2bc04a665f1de2c3e32948c

Admission experiments

Run these in isolated probe packages before expanding Plenty’s target allowlist:

  • Use pinned Cranelift to emit native wrappers around direct/indirect Tail calls, spilled arguments and large frames for each candidate triple. Link and execute on CI; inspect actual relocations, stack probes and unwind records.
  • Build both actual embedded runtime variants and their C/assembly dependencies with the proposed toolchain. Compare Cargo and direct-build configurations, panic-abort aliases, archive member architectures, imports and notices.
  • Round-trip signed/unsigned narrow integers, u128, floats, stack arguments and callbacks through Rust/C/Plenty. Include separate Windows GNU and MSVC fixtures before selecting one distribution contract.
  • Exercise interface/source/trap/coverage metadata through static/shared links, dead stripping, relocated installation and library hosts. Verify resident code lifetime after loader leases close.
  • Build and execute dataframe/linalg using the supported packaging helper. Audit platform library identities/search paths and allocator ownership.
  • Repeat using pinned full Apple SDK/Windows SDK inputs and minimum supported systems. Audit imports and execute offline from extracted packages without Rust/Cargo. Cross-linking remains supplementary to native CI execution.

Decisions informed by the probes

Keep native macOS/Windows runners as the correctness baseline. Choose the Windows ABI after actual dependency and runtime probes; native MSVC is a useful first candidate, while MinGW is a separate CRT/link/distribution choice. A successful GNU C executable does not establish MSVC compatibility.

For macOS, distinguish a libSystem-only artifact from SDK/framework-dependent builds and record deployment target explicitly. For Linux, retain the existing glibc floor unless evidence warrants changing it; the user permits raising the floor to 2.30, but these probes do not require that change.

Do not bundle Zig, SDKs or libraries into Plenty releases without deciding installation size, redistribution/license obligations, offline behavior and version pinning. First prove whether invoking an installed driver meets the native-port goal. User-facing cross-compilation can later reuse the boundaries without forcing a toolchain-bundling decision into this port.

Standard library scope

Status: Partly implemented. The standard library reference owns the shipped contracts: collections, text/data formats, files/processes, streams, networking, TLS, HTTP clients/servers and helpers, crypto, SQLite, logging, signals, and testing. The catalogue below is the original scope, not a list of modules still missing. Issue #49 retains the original catalogue and rationale; it is closed by decomposition, not implementation. Remaining delivery is recorded only in backlog B38–B70. Module names and unimplemented signatures remain design candidates. This page adds no language syntax or runtime API.

The goal is useful CLI/data tools and network services without third-party application packages. Python’s library breadth and Go’s combined HTTP client/server scope motivated issue #49; compatibility with their APIs is not a requirement. A production HTTP server remains an explicit bundled-library goal, with separate acceptance criteria.

Boundaries

Keep the prelude small and import larger facilities under std. Ship source and approved native dependencies with the matching toolchain; ordinary builds must import the library offline after installation or relocation. Importing an unused module must not impose unrelated backend/linker requirements on a program. Issue #45 continues to own package distribution, manifests and project commands.

Ordinary algorithms, containers, parsers, routing and policy belong in checked Plenty source. Standard imports pass the same ownership and worker-effect checks as user modules. Runtime boundaries cover allocation, handles/syscalls, clocks, entropy, sockets/polling, native synchronization and approved bindings. Reuse maintained TLS/crypto, SQLite and suitable compression/math primitives; record backend versions, licenses, supported targets and update responsibility. Plenty or native sets when a native library is chosen. Library purity must not weaken correctness or security.

Current support is x86_64 Linux GNU. Lossless Linux Path and list[u8] binary transport have shipped; neither is a portable path abstraction or compact byte view. Further target support requires explicit platform contracts and tests.

Plenty or native

Write a std module in Plenty unless a native library is clearly better: cryptography, a large established engine, or performance the compiler cannot reach. A Rust crate named in an issue is a reference for API and test vectors unless the issue makes that case. Issue #505 records this decision.

Plenty code costs less in an executable:

  • Removal. Emission drops unreachable std functions one at a time before native code is generated (issue #400), and release builds substitute small leaf functions at the call site (issue #363). Native runtime code is removed only at link time, by --gc-sections, and only when nothing kept still calls it. A central runtime dispatcher keeps every operation it can reach (issue #501).
  • Allocation failure. Plenty code reports it as AllocError by construction. A wrapped crate must be checked, since Rust’s global allocator aborts on failure. rustls is the recorded exception (std.tls costs); new wrappers get none.

A native wrapper must:

  • report allocation failure as AllocError, through a fallible interface or a caller-supplied allocator, as zlib-rs does for std.compress. If the crate cannot, write the needed part in Plenty instead;
  • give each operation its own runtime entry symbol, as the std.crypto digests have, so the linker drops what a program does not call;
  • add no case to a shared runtime dispatcher;
  • add a row to NATIVE_MODULES in tests/test_executable_size.rs, which checks that hello links none of the wrapper’s code and a program using it does (size budgets);
  • when large, ship as a separate engine needed only by programs that import it, following the engine contract that the std.dataframe engine uses.

Module implementation choices

The policy applied to issues #461–#499. “Decide” means the issue settles the choice with a measurement or allocation check before implementation.

IssueModuleChoiceReason
#461std.uuidPlentyImplemented: byte formatting over std.secrets and std.time
#462std.xmlPlentyImplemented bounded UTF-8 event reader/writer; no DTDs or external entities. Pure Plenty keeps allocation failures explicit.
#463Time zonesPlentyPackage; packages carry only Plenty source
#464AEAD, signatures, key exchangeNativeCryptography; ring is already linked for std.tls
#465Unicode segmentationPlentyPackage; generated tables
#466, #467WebSocketPlentyFraming over std.http and std.tls
#468Server-Sent EventsPlentyText encoding over a response stream
#469std.diffPlentyMyers and patience are short algorithms over lists
#470std.markdownPlentyParser and renderer; pulldown-cmark allocates infallibly
#472User directoriesPlentyXDG rules over std.env
#473SHA-1NativeCryptography; RustCrypto sha1 allocates nothing, like sha2
#474Bytes.find_bytesNativeSIMD search the compiler cannot match; memchr allocates nothing
#475std.toml editsPlentystd.toml is Plenty; toml_edit allocates infallibly
#476std.imagePlentyImplemented: PNG decoding and encoding over std.compress, crop, conversion and resize, with limits checked before allocating; image allocates infallibly. JPEG, GIF and WebP deferred
#477Progress barPlentyTerminal drawing over std.io
#478std.shlexPlentySmall tokenizer
#480std.memmapPlentyA few system calls behind small runtime entries
#481std.bigintDecideMeasure Plenty limb arithmetic against native call cost; num-bigint allocates infallibly
#482LRU cachePlentydict and std.deque
#483BLAKE3NativeCryptography with SIMD; confirm hashing allocates nothing
#484inotifyPlentyA few system calls behind small runtime entries
#485std.mimePlentyParsing and a lookup table
#486std.graphPlentyUser data on nodes and edges; algorithms over lists
#487System informationPlentyText files under /proc and /sys
#488std.jwtPlentyComposes std.json, std.encoding and std.crypto
#489base32PlentySame as the existing codecs
#490Linear algebraNativeLarge engine with SIMD kernels; separate engine; check faer allocation first
#491BrotliNativeLarge codec; the brotli crate takes an allocator, so failure is reported
#492OptimizationPlentyClosed; replaced by the numerics core proposal: solvers in Plenty over std.linalg
#493numeris evaluationPlentyClosed; not adopted; std.linalg keeps faer
#494StatisticsPlentyFormulas over lists; sampling keeps std.random’s stable sequence
#495Key-value storeDecideLarge engine with its own file format; std.sqlite already covers transactions and ordered keys
#496SketchesPlentySmall algorithms over arrays and hashes
#497Xor filtersPlentyShort construction over a fixed key set
#498Cuckoo filtersPlentyA small hash table
#499std.templatePlentyAlready chosen; tera renders from serde values

Issue #471 designs derived serialization and makes no implementation choice. Issue #479 is closed.

Shared contracts

Establish binary buffers, partial-progress I/O, contextual errors and monotonic deadlines before consumers invent incompatible alternatives. Preserve typed Result errors, explicit fallible allocation and existing cooperative cancellation. Specify EOF, short progress, interruption, close/flush errors and cleanup after partial I/O, cancellation or allocation failure.

Structural protocols are statically dispatched. A proposed API must not assume associated types, existential storage, reflection or automatic codec derivation. Use implementable generics, explicit error/value enums and typed codecs. Consuming/borrowing, copied/streamed and in-place/allocating operations need distinct documented contracts. Binary data is not UTF-8 text; filesystem paths are not URLs; simulation randomness is not cryptographic entropy.

Every adopted module publishes its scope, errors, allocation/copy costs, ownership/thread safety, supported platforms and complexity where relevant. Add small reference pages, runnable tutorial examples and SUMMARY entries with the implementation. Use local fixtures, fixed seeds, fake streams/clocks, failure injection, external format fixtures and retained parser fuzz corpora. Test Plenty libraries through plenty test when issue #45 supplies its runner, alongside native primitive/lifecycle tests.

Capability checks and further breadth

The initial CLI/data check reads arguments/environment, loads TOML/JSON, walks files, processes CSV and reports contextual errors/structured logs. Existing math, text primitives, lossless paths, walking, sorting and deque support count toward it. It does not wait for a network execution model. The service check fetches verified HTTPS with cancellation/bounded streaming, persists records in SQLite and serves a routed JSON API meeting the HTTP acceptance gates.

Accurate summation/norms, float decomposition/adjacent values and special functions need numerical contracts; basic math has already shipped. Optional scope still includes timezone/calendar data, statistics, decimal/big-integer/ complex arithmetic, MIME types, HTML escaping/context-aware templates, WebSockets, diff tools, terminals and signals. Evaluate each against a concrete use case before adding an implementation issue/backlog entry. HTTP/2 is a follow-on protocol candidate; HTTP/3 needs a separate decision.

GUI toolkits, mail/FTP stacks, interpreter/debugger internals, arbitrary runtime loading and pickle-style serialization are outside the initial catalogue. YAML, an ORM and a complete web framework also require their own case. Closing the umbrella does not broaden these commitments or claim delivery.

Arbitrary-precision integers (std.bigint)

Status: Not implemented. This page compares ways to provide issue #481’s std.bigint and recommends one. It ships no module, compiler change or API. Scheduling belongs to B38. The current integer contract is numeric types; library operators are user-defined arithmetic operators.

Recommendation

Write std.bigint in Plenty, in three steps:

  1. Plenty module. A BigInt with an inline i64 for small values and u64 limbs for large ones. Schoolbook and Karatsuba multiplication, Knuth division and divide-and-conquer radix conversion, ported from num-bigint (MIT OR Apache-2.0) with credit. Every allocation is an ordinary fallible Plenty allocation.
  2. Compiler builtins for limb arithmetic. u64.widening_mul (Cranelift umulhi plus imul) and an add-with-carry form. The measured limb loop is 4.6× slower than Rust without them.
  3. Optional native kernel, only if measured to pay. A runtime entry for large multiplication (Toom-3 or an FFT) that writes into an output and scratch slice the Plenty caller has already allocated. It allocates nothing itself, so it needs no allocation-failure handling. Write it in permissively licensed Rust; do not use malachite.

Malachite does not fit. On allocation failure it aborts the process. It is LGPL-3.0-only, and the runtime archive is statically linked into user executables. It adds 1.2 MB to every executable that uses it and 35 s to a clean build. Its multiplication beats the alternatives only above about 10⁴ bits, and only through its AVX2 FFT on x86-64; the portable path was slower than every other crate, and ARM64 was not measured. It also parses decimal text 2–6× faster.

Requirements

  • Fallible allocation. Every allocation in std reports AllocError; a Rust allocation failure in the runtime aborts the process (Rust panics).
  • License. Plenty is MIT OR Apache-2.0. No current dependency is used under a copyleft license. The only license expressions that mention LGPL also offer MIT OR Apache-2.0.
  • Size and build cost. Code in the runtime archive costs nothing to programs that do not reach it, but grows the compiler and its build. The engine criteria move a module out of the archive above 2 MB per program or one minute of build time.
  • Platforms. Linux x86-64 and ARM64, macOS ARM64, Windows x86-64 MSVC (cross-platform assessment).
  • Project preference. Implement in Plenty unless native complexity or security justifies a wrapper (B38, AGENTS.md).

Allocation failure in candidate crates

Each crate’s source was read and run with a global allocator that refuses allocations over 4 KiB, then asked to square a 65,536-bit number.

CrateLicenseHow it allocatesOn failure
malachite-nz 0.14.0LGPL-3.0-onlyVec throughout (about 1,500 vec!/push/with_capacity sites, no try_reserve); the FFT allocates internallyhandle_alloc_error: abort
num-bigint 0.5.1MIT OR Apache-2.0Vec<u64>, no try_reserveabort
ibig 0.3.6MIT OR Apache-2.0alloc::alloc in memory.rs/buffer.rs, null handled by handle_alloc_errorabort
dashu-int 0.6.2MIT OR Apache-2.0alloc::alloc in memory.rs/buffer.rs; null calls panic_out_of_memory()panic, which is an abort in the panic=abort runtime
rug 1.30 / GMP 6.3LGPL-3.0+ / LGPL-3.0 or GPL-2.0mp_set_memory_functionsGMP’s manual: allocation functions “must terminate program execution. A longjmp or throwing a C++ exception will have undefined results.”

No crate exposes an allocator parameter or a fallible API. Malachite’s limb-level functions with caller-supplied scratch, such as limbs_mul_greater_to_out, are crate-private unless the test_build feature is enabled, and its FFT path allocates regardless.

Can the runtime’s arena catch the abort?

The runtime can already contain one infallible Rust library: ring’s RSA operations run inside actor_heap::within with a pre-reserved arena. An allocation failure there jumps back and reports AllocError (RSA allocation). Applying this to malachite would fail for these reasons:

  • Platforms. The arena router exists only on Linux x86-64 and ARM64. On other targets within runs the job directly and a failure aborts.
  • Value-dependent memory. ring uses a fixed amount of memory per operation, so its arena has a fixed, tested size. A bignum operation’s memory grows with its operands (the FFT path uses O(n log n)), so each call must estimate a bound and reserve it first.
  • Results cannot leave the arena. within forbids handing back allocations. Every result Natural would have to be copied into Plenty storage before the arena is freed: an extra O(n) copy and twice the peak memory.
  • dashu panics instead of calling the allocation-error handler, so the arena cannot catch its failures at all.

An engine (shared library) does not help either: Rust’s Vec aborts there too, and the linalg engine reports allocation failure only because it uses fallible faer APIs and a fork. Forking malachite (about 650,000 lines with malachite-base) to make it fallible is not maintainable.

License: LGPL-3.0 in a statically linked runtime

The runtime archive, including any Rust crate it wraps, is statically linked into each executable that reaches the module. Under LGPL-3.0 §4, everyone who distributes such an executable conveys a “Combined Work” and must:

  • give prominent notice and ship the GPL-3.0 and LGPL-3.0 texts;
  • use terms that do not restrict modification of the library or reverse engineering to debug such modifications;
  • either supply the application in a form the recipient can relink with a modified malachite (object files or source), or load malachite through a shared-library mechanism;
  • for “User Products”, supply installation information.

These obligations fall on every Plenty user who distributes a binary that uses std.bigint, not only on the Plenty project. Relinking is also hard in practice: the Rust code is monomorphized and inlined into the runtime archive, so the user would need the whole runtime archive rebuilt against the modified malachite. A shared engine (libplenty_bigint_v1.so) would meet the relinking condition with §4(d)(1), but it still leaves the allocation-failure problem, costs a dynamic call per operation and must be shipped with the application. GMP/rug raises the same issues. GMP also needs m4 and make to build and does not support Windows MSVC (MSYS2/MinGW only).

num-bigint, ibig and dashu are MIT OR Apache-2.0, so their algorithms can be ported into Plenty with credit in the module header and its ## Acknowledgements section.

Measurements

Measured 2026-10-31 on an AMD Ryzen 7 7840HS, rustc 1.99.1, release build with panic=abort and codegen-units=1, pinned to one core. Inputs are pseudo-random. The scratch crate lives under the worktree’s target/ directory and is not committed.

Throughput

Nanoseconds per operation. “Small” is 10 million &a + &b or &a * &b on values that fit in a word, producing a new value each time.

Operationnum-bigintmalachitedashuibig
small add8.27.37.21.1
small multiply7.65.86.05.1
multiply 256 bits30334829
multiply 4,096 bits2,2961,9471,7302,129
multiply 65,536 bits175,39859,958145,030154,566
multiply 1,048,576 bits11,081,5851,891,4937,316,1939,840,681
20000! by repeated multiply21.8 ms26.0 ms24.2 ms19.8 ms
format 20000! (77,338 digits)3.7 ms2.4 ms3.2 ms3.5 ms
parse 20000!4.2 ms0.70 ms1.5 ms1.7 ms

Malachite is 2.4–6× faster at 65,536 bits and above, and much faster at parsing. Up to 4,096 bits and on small values it is no faster than the others. Its large-size speed comes from an f64 FFT that uses AVX2 and FMA only when the std feature enables runtime CPU detection. With default-features = false (the portable path), 65,536-bit multiplication took 837,473 ns, slower than every other crate. The ARM64 path uses the wide crate and was not measured.

Plenty limb arithmetic

The same schoolbook multiplication was written in Plenty, compiled with --release. Plenty has no u128 or high-half multiply, so each 64×64-bit product uses four 32-bit multiplies. The Rust baseline uses u128.

SizeRust u128 schoolbookPlenty, product in a helper functionPlenty, product inlined by hand
256 bits23 ns403 ns357 ns
4,096 bits4,879 ns33,705 ns22,389 ns
65,536 bits929,096 ns7,814,643 ns4,878,989 ns

The inlined inner loop costs 5.5 ns per limb product against Rust’s 1.2 ns. The small-size cost is mostly fixed overhead: allocating and filling the result list and the call. Holding 32-bit limbs in u64 slots, so that each product needs one multiply, was slower (43,463 ns at 4,096 bits). The remaining cost is in the checked adds and the indexing, not the multiply. A widening-multiply builtin and a carry form would remove the emulation and most of the overflow checks.

A Plenty small-value fast path is as fast as the crates’ small paths: enum Big: Small(i64) | Large(list[u64]) with an __add__ hook that tries a checked: add takes 8.7 ns per (acc + one)?. Calling a native crate would add a call into the runtime and an owned native handle per result, on top of the crate’s own 6–8 ns.

Executable size and build time

The program parses one number, squares it and prints it. Sizes are of a stripped Rust executable; the baseline prints its argument.

CrateExecutableAddedClean build, 8 jobs
none339,416 B——
num-bigint411,912 B+72 KB2.4 s
ibig408,856 B+69 KB2.5 s
dashu-int472,400 B+133 KB4.5 s
malachite-nz (std)1,543,816 B+1.20 MB35 s

Malachite would remain under the 2 MB engine threshold. Its build time would use more than half of the one-minute budget, and every compiler build would pay it. A Plenty module adds only the functions a program reaches; it needs no Rust build.

Options compared

Plenty module (+ builtins, + optional kernel)Wrap malachiteWrap num-bigint / ibig / dashuGMP via rug
AllocError on failureYes, on every targetNo; arena on Linux only, with copy-out and per-call boundsNo; same arena limits; dashu cannot be containedNo (undefined if unwound)
LicenseMIT OR Apache-2.0 (ported, credited)LGPL-3.0-only obligations on every distributorPermissiveLGPL/GPL, same obligations
Size per programOnly reached functions+1.2 MB+70–130 KBGMP static library
Compiler buildNone+35 s+2–5 sC build with m4
PlatformsAll targetsAll, AVX2 FFT on x86-64 onlyAllNo Windows MSVC
Small values8.7 ns measured, no allocationNative call + handle + 6–8 nsSameSame
Large valuesSlower until builtins/kernel; 4.6× behind Rust schoolbook todayFastest above 10⁴ bitsComparable to each otherFastest

API direction

These signatures are a sketch for the implementation PR to settle.

from std.bigint import BigInt

class BigInt            # not a copy class; copy(x)? duplicates the limbs
    def __add__(self, other: &BigInt) -> Result[BigInt, AllocError]
    def __radd__(self, other: i64) -> Result[BigInt, AllocError]  # i64 + BigInt
    def __sub__, __mul__, __floordiv__, __mod__, __neg__, __lshift__, __rshift__,
        __and__, __or__, __xor__   # same shape
    def add_assign(self: &mut BigInt, other: &BigInt) -> Result[(), AllocError]
    def cmp(self, other: &BigInt) -> i64          # until __cmp__ (B37)
    def to_i64(self) -> Result[i64, RangeError]
    def to_f64(self) -> f64
    def display[W](self, out: &mut Formatter[W]) -> Result[(), FormatError]
    def to_str(self, radix: i64) -> Result[str, AllocError]
def from_i64(value: i64) -> BigInt                # no allocation
def parse(text: &str, radix: i64) -> Result[BigInt, ParseError | AllocError]
  • Ownership. Operator hooks take a shared self and a shared or copyable operand, and return an owned result. a + b borrows both operands and never consumes them. A result that fits in i64 is stored inline and allocates nothing, but the hook still returns Result, so callers write (a + b)?.
  • In-place operators. __iadd__ must return (), so it cannot report AllocError. Provide named fallible methods (add_assign) instead, which is the reference’s own advice (“use explicit assignment or a named fallible method”).
  • Division. Use floor semantics, as // and % have on i64. A zero divisor traps, as it does for i64. A checked variant can return ZeroDivisionError.
  • Equality and ordering. == is structural, so it is correct only if the representation is canonical. Every value that fits in i64 is Small, and limbs carry no high zeros. Classes have no ordering operators until the user-defined ordering proposal’s __cmp__ lands, so cmp is a method for now. BigInt cannot be a dictionary key until user-type hashing lands.
  • Limits. Text conversion of attacker-controlled input should either use subquadratic algorithms or take a digit limit, like other parsers’ Limits.

Open questions

  • BigInt + i64. Each operator has one left hook, so __add__ cannot accept both &BigInt and i64. i64 + BigInt works through __radd__. For the other order, callers write a + from_i64(n), which allocates nothing, unless hooks gain overloading or a mixed-operand sum type.
  • Names and signatures of the limb builtins (widening_mul -> (u64, u64), carrying_add(other, carry: bool) -> (u64, bool)), and whether Cranelift lowers them on ARM64 as cheaply as on x86-64.
  • The size threshold above which a native multiplication kernel is worth its code, measured after the builtins land.
  • Whether small helper functions in the limb loop need inlining. The measured call to mul_wide cost 1.5× in the 4,096-bit loop.

Acknowledgements

  • num-bigint (MIT OR Apache-2.0): the reference implementation suggested by issue #481; the recommended source for ported algorithms.
  • malachite (LGPL-3.0-only), ibig and dashu (MIT OR Apache-2.0), and GMP (LGPL-3.0 or GPL-2.0): evaluated and measured here; no code is used.

Checked arithmetic by default

Status: Core and repository adoption implemented; bounded validation complete. Function-level arithmetic boundaries, lexical unchecked:, independent closure contracts, std/callback APIs, fixtures, shipped examples, compiler benchmark templates and teaching/reference changes are implemented on this branch. Validation evidence records the isolated post-repair host gate, controlled timings/profiles, representative optimized code and native/export audit, with explicit source and observation limits. Neither local PR #886 nor its proxy-fixture dependency #918 is merged. This is not whole-proposal performance or merge acceptance. Remaining B14 conveniences stay in the backlog.

Recommendation

Make every ordinary function body a boundary for recoverable integer arithmetic failures. Operators still produce ordinary numbers on success; their failure returns Err from the function, whose declared Result error type must include it. Callers handle that ordinary Result with ?, match, or another existing operation. There is no ? after each arithmetic operator.

Keep checked: as an explicit, value-producing local recovery boundary, both in ordinary code and inside unchecked:. Give unchecked: the value of its suite, without adding a Result wrapper. Inside it, the programmer guarantees that affected arithmetic does not fail. This proposal validates that precondition with today’s arithmetic traps.

This improves recoverability by default and makes deliberate arithmetic traps searchable. It does not remove machine checks yet: terminating on overflow requires detecting it. “Unchecked” means unchecked by the typed arithmetic error mechanism, never silent wrapping.

Plenty has no external users or dependent systems to preserve. Adopt this as a coordinated breaking change across the compiler, standard library, tests and documentation. Existing syntax and APIs are inputs to the design, not compatibility constraints; no transition mode or edition mechanism is required.

Function contracts

def scale(value: i64, factor: i64) -> Result[i64, OverflowError[i64]]:
    product = value * factor
    Ok(product)

def invoice(quantity: i64, price: i64) -> Result[i64, OverflowError[i64]]:
    Ok(scale(quantity, price)?)

The multiplication has type i64; failure returns Err(OverflowError[i64](wrapped=...)) from scale. A signature of -> i64 is rejected for this body. The compiler reports the failing operation and the missing error type, with suggestions to declare a Result, handle a local checked: result, or deliberately opt out.

The implicit function boundary controls failure routing, not success-value conversion. Keep ordinary explicit Ok(...), Err(...), tail expressions, and return rules. Do not wrap an existing Result into another Result, auto-unwrap calls, infer public return types, or accept a bare i64 as the success return of Result[i64, E]. This is deliberately more precise than textually wrapping an entire function in today’s checked: expression.

Only failures that can escape this function boundary must appear in its error type. A locally handled checked: error need not appear; an error propagated with ? does. Wider declared error unions remain permitted. Functions with no escaping arithmetic failures can still return a plain type, including (). An Option-returning function cannot silently turn overflow into Nothing: handle the arithmetic locally, change the return contract, or opt out.

The initial scope is exactly today’s checked integer operation set:

OperationError contributed
Integer +, -, *, unary -, powOverflowError[T]
Signed // with a plain or NonZero[T] divisorOverflowError[T]
// and % with a plain integer divisorZeroDivisionError

Unsigned division has no overflow case. NonZero[T] removes the zero error; Positive[T] also removes signed division overflow. Compound assignment and forwarded integer @newtype operations follow the same rules and field error types as today. Type checking follows operators and operand types, not general range analysis: even n // 4 keeps today’s conservative error set. Optimizers may remove proven-unnecessary branches without changing that contract.

Float arithmetic, float div_checked, shifts, casts, negative integer pow exponents, indexing, allocation, I/O, and explicit unwrap() retain their existing contracts. Module constants still reject overflow at compile time. This is a proposal for recoverable integer arithmetic by default, not a promise that all runtime failures become recoverable.

The opt-out block

def scale_in_bounds(value: i64, factor: i64) -> i64:
    unchecked:
        value * factor

On success this returns an i64. As implemented here, overflow terminates the process in debug and optimized builds alike. It cannot reach an outer checked boundary as an Err; there is no unwinding or rollback.

Termination is the first implementation of the precondition, not its permanent meaning. A dependent follow-up will add a compiler mode that assumes the precondition and omits the checks, so a violation becomes undefined behavior, not wrapping. Trapping stays the default, and a separate unchecked trap: form will always check and trap. Code that uses traps to reject invalid input must move to that form or validate explicitly before the mode can apply.

unchecked is contextual, like checked: only unchecked: followed by a newline opens a suite; an existing typed binding unchecked: i64 = 0 remains valid. Support value positions already available to checked:: a binding, assignment, return value, function tail, or match-arm tail. Also support a standalone suite for mutations and control flow. Locals remain block scoped.

A value-producing suite yields its final expression’s type T, not Result[T, E]. If that expression already has type Result, it stays a Result and must be handled normally. A statement suite follows ordinary statement/discard rules; it cannot silently discard an owned value or a Result. An empty suite with pass, or an all-exiting suite, is allowed in statement position. No reference-valued block extension is proposed initially.

?, return, break, and continue retain their ordinary function/loop targets. In particular, unchecked: does not catch or trap a callee’s Err, disable allocation handling, or make an unchecked call variant:

def caller(value: i64) -> Result[i64, OverflowError[i64]]:
    answer = unchecked:
        scale(value, 2)?
    Ok(answer)

scale still returns its declared error and ? still propagates it. To get trapping arithmetic in a helper, the helper’s implementation must explicitly contain the opt-out. Inlining cannot change that behavior.

Keeping checked:

Retain its Result[T, E] expression semantics for arithmetic in the suite. Functions and closures defined there have their own declared error boundaries, as described in functions and callbacks. The block remains useful for recovering or mapping errors before they reach the function signature, even when the default is checked. Within unchecked:, it also restores recoverability for a smaller region:

def mixed(a: i64, b: i64) -> Result[i64, OverflowError[i64]]:
    result = unchecked:
        offset = a + 1
        sensitive = checked:
            offset * b
        sensitive
    result

The addition traps on overflow; multiplication produces the local Err, which is returned normally. The outer unchecked suite passes through that Result without trapping or wrapping it. A match on sensitive could instead recover locally and let mixed return a plain i64.

Lexical context of an integer operationFailure destination
Function body, without a nearer explicit modeFunction’s declared Result
Explicit checked:That block’s Result
Explicit unchecked:Precondition violation; currently process termination
checked: inside unchecked:Inner block’s Result
unchecked: inside checked:Precondition violation; currently process termination

Nested unchecked: is redundant but legal. Leaving any suite restores its outer mode. A failure in an explicit checked block releases that block’s locals; an implicit function failure releases function-owned values as ? does. Earlier mutations are retained, and a failing compound assignment performs no write. Neither boundary is transactional.

Keep today’s explicit checked: restrictions initially: unused results, reference results, blocks with no failing arithmetic, and blocks whose every path exits are rejected. Do not apply the last two restrictions to the implicit function boundary: ordinary infallible and all-returning functions remain valid.

Auditability and cost

Search for unchecked: to locate explicit arithmetic opt-outs, then review why each region cannot fail and whether its extent is minimal. Use a source-aware search to distinguish syntax from comments and text. Prefer comments explaining bounds or measured hot paths over generic “faster” claims. Real-world confidence in bounds is useful evidence, but today’s check traps if that confidence proves wrong, and an assumed-valid build would not.

This search is not a complete trap audit: indexing, unwrap(), negative powers, foreign code and existing runtime primitives remain separate trap sources. Std implementations must expose deliberate arithmetic opt-outs in source or documented native contracts; hiding unchanged trapping reductions behind safe-looking calls would weaken the proposal’s benefit.

Default code can reuse today’s straight-line arithmetic plus cold error exits. It need not materialize a Result for each operation. unchecked: can avoid error payload construction, recoverable cleanup paths and some return ABI costs; it cannot promise check-free arithmetic or a speedup. Retaining checked: does not itself imply runtime mode switches: the mode is lexical.

See implementation and validation for callback boundaries, repository-wide changes and the evidence needed before adoption.

Checked defaults: implementation and validation

Companion to checked arithmetic by default. Repository adoption and bounded validation complete. Collected results and limits cover the isolated host gate, all controlled timing/profile jobs, representative optimized code, native/export audit and compiler benchmark-template adoption. This is not universal performance/ABI or merge acceptance. Remaining B14 conveniences stay in the backlog.

Current implementation

Function bodies collect arithmetic failures against their declared public Result type, without changing successful return rules. unchecked: selects trapping operators and leaves suite values unwrapped. Nested functions no longer inherit a private callback effect. Generator arithmetic receives a targeted diagnostic; ordinary native tail returns remain supported.

tests/test_checked_defaults.rs exercises the new contracts, all eight integer widths, nesting, mutable and stored callbacks, worker error transport, fallible key sorting, cleanup, skipped compound writes, generators and tail recursion. The private callback error channel and its lifetime restrictions are removed; its tests are replaced by public-contract coverage. The retained local arithmetic suite passes.

The initial library migration covers std.math, std.bytes, std.time, std.encoding, std.io, std.io.typed, std.fmt, std.fs, std.env, std.secrets, core std.http, std.net, std.ready, std.text, std.random, std.test, std.url, std.process, std.http.decoder, std.http.typed and std.http.client, plus std.json, std.unicode, std.complex, std.stat, std.linalg, std.simd, std.log, std.crypto, std.tls, std.uuid, std.jwt, std.http.server, std.toml, std.regex, std.cli, CLI progress, std.dataframe, std.plot, HTTP body/cookie/proxy helpers, std.xml, std.csv, std.tempfile, std.system, std.fs.watch, SQLite/KV wrappers, std.compress, and actor core/TCP/TLS/HTTP-server adapters. Integer math magnitude/reductions return Result[T, OverflowError[T]]; separate *_float helpers retain IEEE float values. Serial pipeline sum() returns a public result for integer items. Sorting accepts Result[K, E] key callbacks and returns Result[(), E]. Bounds-proven byte/time operations use small documented opt-outs; fallible size and time calculations retain their existing error contracts. Encoding expansion sizes are checked before allocation, stream position/total limits return their declared errors, and filesystem walk reservation no longer counts pending frames twice (issue #702). Pure capacity helpers are tested at representation limits without allocating huge buffers. The HTTP decimal accumulator subtracts ASCII bias before checked multiply/add (issue #703), accepting the complete i64 range while mapping larger lengths to protocol errors. FakeStream.fail_after returns a public Result[(), OverflowError[i64]]; failures preserve its pending schedule. Counter exhaustion in the stream protocol maps to OutOfRange before mutation. URL expansion and process extents check representation limits before allocating or consuming another chunk. Process argument encoding reserves additional bytes without counting held bytes twice (issue #704). Incremental decoder counters are bounded by its per-feed, line, head and body limits. Typed chunk writers preflight the complete wire extent before delivery, preserving progress and transport causes. Client callbacks retain their own contracts; deadline saturation and bounded pool/retry/body arithmetic are explicit. No blanket unchecked: conversion was applied.

JSON retains its input/output budgets and maps one-based position exhaustion to JsonFault.InputTooLarge before advancing the failing cursor. Partial output compaction preserves delivered progress without allocating. Unicode normalization uses explicit scalar-ceiling, successful-append and positive-index bounds. The JSON/Unicode component passed 35 isolated-target runtime-check tests, including debug/optimized representation boundaries and the Unicode normalization corpus, and received an independent source review. A separately reviewed TOML/regex checkpoint preserves parser faults, edit atomicity, regex work/memory limits and replacement validation order. Public mutable regex matches expose length subtraction as Result[i64, OverflowError[i64]]; unavailable TOML coordinates report InputTooLarge at the valid offset with zero line/column, following the user’s explicit choices. Its 40 isolated-target tests pass, with six added boundary tests in both optimization modes. A separately reviewed XML/CSV checkpoint preserves parser faults and stream ownership. CSV position exhaustion returns the existing stream OutOfRange error; preflight keeps offsets exact, record MAX valid and explicit flushing of already-finished bytes available.

The reviewed numerical checkpoint preserves complex powers at i64.MIN, matrix shape/index invariants and statistical capacity causes. Ordinary integer SIMD arithmetic returns lane-typed overflow results; explicit wrapping and IEEE float APIs remain unchanged. The SIMD tutorial and generated declarations match. Observation-only logger, server-report and actor statistics saturate at their existing maximum, following the user’s explicit policy; semantic progress stays exact and fallible. Server output and unread extents preflight before delivery or buffer growth. TLS/crypto/UUID/JWT retain their existing public errors with source-visible buffer, DER, timestamp and time/leeway bounds.

Actor slots and timer/request records retire rather than wrap exhausted identity generations. Mailbox counters rebase all linked target-owned exit fences while preserving message-prefix ordering. An affine arithmetic error reaches its parent with both actor heaps reclaimed. Complete actor TLS, crypto, JWT, HTTPS and server/client tests pass after dependency integration. Combined targeted validation has 733 passes, two failures in unmigrated HTTP body/cookie, and one pre-existing ignored test. The two native-engine linalg bodies were still unrun at that checkpoint; the follow-up native-engine results are recorded below. At that checkpoint the registry still reported 29 failing module analyses; broad tutorial validation remains incomplete. These results are not all-features or performance acceptance.

A later reviewed checkpoint adds bounded CLI parsing/progress, checked dataframe extents, plot arithmetic and body/cookie byte budgets. Actor HTTP keeps exact response accounting and equivalent bounded lingering-threshold state; complete TCP/TLS/control/error tests pass. Initial optional-engine self-skips were followed by actual local-engine runs: all 17 dataframe and nine linalg tests pass, including relocation and allocation cleanup after fixture-only arithmetic fixes. This does not replace the native arithmetic audit. Reviewed proxy/helper fixtures restore descriptor, cancellation, RFC and bulk-scan checks; actor HTTP interop retains real curl and 600-mutation corpus behavior. Temporary/system/watch/SQLite wrappers and compression preserve existing errors and resource/progress contracts. Compression’s live gzip/Brotli interop runs; SQLite’s optional vector extension and Docker cases remain unrun. At that checkpoint the registry had 13 failing module analyses. The final reviewed std checkpoints now cover archive, diff/image, HTTP files/SSE/WebSocket, MIME/template/markdown and sketches. The registry/index passes both tests with no diagnosed modules; public mutable diff lengths return overflow Results. Focused HTTP files tests now execute all 11 filesystem/conditional/stream cases, not merely extracted size helpers. Oversized metadata returns InvalidContentLength before validators, following the user’s explicit policy (issue #706). Image and Markdown preserve format/domain errors, checked extents, exact parser and output progress, and cleanup. Their 23 and 10 tests pass respectively; representation-limit probes supplement, not replace, actual codec/parser tests.

The reviewed parallel integer sum now returns an ordinary overflow/pool Result, retaining serial-prefix semantics, the precise first wrapped payload and one callback invocation per item. Mapping drains before ordered addition and source cleanup; O(n) mapped storage and a serial combination pass are deliberate, unbenchmarked costs. Floats retain chunk grouping. Enumerate index overflow keeps its explicitly documented trapping contract. All 30 pipeline tests and both owned tutorial examples pass after integration.

Reviewed tutorial, reference and skill adoption uses independent public callback Results, explicit joins and precise propagation. Full executable documentation improves from 3 passing / 17 failing tests to 19 passing / 1 failing after the reviewed generic deque/heap/graph/LRU checkpoint (49 targeted passes). Its std public declarations, allocation causes and ownership contracts stay unchanged. Reviewed fallible fixed-tree and pipeline reducers now accept ordinary Result callbacks, including reconstructed pair items with exactly-once affine cleanup. Their public error unions retain pool failures and callback errors; deterministic tree-level/pair selection and chunk-fold policies remain distinct from integer sum’s serial prefixes. All 20 tutorial/skill tests, 24 parallel tests and 37 parallel-pipeline tests pass after integration. Registry analysis does not instantiate all generic bodies; executable coverage was needed in addition to its two passing tests. Reviewed native export/FFI fixtures restore 23 tests without widening C scalar/fieldless-error signatures or weakening native oracles.

Adoption remains incomplete. Repository fixtures/examples, full-suite/native arithmetic audits and performance measurements remain. A complete earlier all-features baseline compiled, including experimental emitter and profiling features, but reported 4,465 passes, 1,109 failures and eight ignores. The passes include 27 confirmed optional self-skips. Most failures reject old fixture arithmetic contracts; some hit the generic std bodies adapted since that snapshot. A newer complete gate at ff52bd7e compiled successfully and reported 4,813 passes, 813 failures and eight ignores, again including 27 optional self-skips. It has no newly failing outer name relative to that baseline and no remaining std-source diagnostic. Subsequently reviewed borrowing, tail/stack, control/trap and flow/shared batches pass all 568 parent-run tests, including 258 identical failed names from that newer snapshot. This is scoped evidence, not a measured current-head whole-suite result or acceptance. Public Result adoption also exposed the pre-existing contextual numeric-literal defect in issue #710. A reviewed non-evaluating success-payload hint fixes it without changing the original unsuffixed-call-result oracle. A rebuilt main baseline rejects that source; native normal/release regressions now pass, and all 109 callable fixtures and ten literal-form tests pass after parent integration. The user approved deterministic fallible-tree errors by level then left-to-right pair, accepting level-by-level scheduling while infallible reducers retain their optimization. Ordinary Result[(), E] initializers now produce exactly Result[Class, E], without adding AllocError unless declared. Reviewed generic-output deferral preserves whole-result parameters, aliases, defaults and constructor values. All 114 constructor/checked-default tests pass after integration, including partial-field, affine-error, default-failure and native inline/noninline cleanup. Issue #709’s partial-self capture guard is integrated; captures are checked when created, without crediting deferred closure-body writes. Infallible destructor/context-exit protocols and existing C-export error-layout restrictions remain unchanged. The tuple-adapter review correction is integrated with owned/enumerated/mapped pair, cardinality, managed-error and allocation-failure regressions. Earlier scalar-only passes did not establish that case; the correction received separate review before integration. Further reviewed concurrency and collection batches pass 183 and 428 parent-run cases while retaining actual worker, guard-page, allocation and ownership oracles. The user explicitly approved updating one unoptimized range-generator oracle to require the exact bounded helper and caller-provenance calls. It retains counted-source/no-iterator-allocation checks and native output, not a zero-call or performance-equivalence claim. The historical gate at af8aa15d had 5,375 reported passes, 259 failures and eight ignores, including 27 optional self-skips. After the reviewed final fixture and example adoption, the complete host gate at 2ef36dce passes: 5,650 passed, zero failed and eight existing ignores. Every one of those 259 failed names now passes; sixteen additional outer tests pass separately. All 27 known optional self-skips execute their bodies, including the selected dataframe/linalg engines, tutorial engine jobs and ten Docker cases. The real libFuzzer and Neovim bodies also execute. Two explicitly selected ignored trials pass afterward and are not folded into the whole-gate totals. External engines are compatible existing artifacts, not newly source-built engines; transient host-tool conditionals are not exhaustively traced. This is host test acceptance, not performance acceptance. Reviewed shipped-example and companion adoption subsequently passes all 20 structure tests and all three exhaustive-example tests, with the original stress, allocation and ownership oracles intact. An explicitly run sqlite-vec trial passes its real transactional example using deterministic test inference; this does not qualify a real embedding model. A local release-skill assembly passes 19 sync tests and checks 152 Markdown pages/837 local targets without missing links; no remote publication or agent trial is implied. The bounded native/generated-adapter audit’s inline sort-error ownership hazard (issue #711) is repaired with a caller-owned typed result slot. Relocation precedes comparator return, without heap allocation or restricting the public error type. Independent review and 21 parent-run checked-default tests cover range/nested managed errors, first and second-key failure, skipped callbacks, exact payload/drop counts and normal/release storage-layout checks. The pre-repair native probes happened to print correctly; the discriminating failure was structural, so no reproduced crash is claimed. The integrated whole gate is green at that immutable snapshot. Measurements remain pending. The independently reviewed #712 repair is integrated. The fresh post-repair gate was safety-aborted by an external Docker evidence-wrapper regex error before any container launch: 4,089 passed, two wrapper-caused failures and one ignore were observed; 1,575 registered tests were unobserved. All 25 checked-default tests and the five new verifier tests passed, but this partial run is not a complete gate. A second independently reviewed external harness attempt also stopped before container launch because the installed Docker emits lowercase absence diagnostics: 3,691 passes, no completed failure rows and one ignore were observed; 1,975 registered tests remained unobserved. Its corrected process monitor fully waited owned descendants, but that does not make the run green. Both failed attempts remain intact. Live diagnostic compatibility must be prevalidated before another complete attempt.

Untimed native execution of the initial acceptance harness completed 110 builds and 260 exact-oracle runtime cases, but ten source checks failed and sixteen planned cases remained blocked. Reviewed harness corrections fix the corpus error union and add honest mode selection. The reviewed shared Unit checked-suite repair (issue #712) now preserves the exact entry type stack for Unit success, without changing nonunit behavior or codegen. Five verifier and four native normal/release regressions cover strict stack checking, ordered mutations, destruction/propagation and floating division. The same reviewed production hunk is committed on a new baseline fork, labelled “main + #712 fix”; original pins and failed-probe records remain unchanged.

Reviewed call-heavy probes now name the live multiplication result before adding the bias, preserving operations and error payloads while retaining a genuine leaf call. All four optimized modes and both recovery variants pass actual native call-site extraction on the historical syntax pair. The controller fails closed when extraction cannot qualify a call; an additional I-side cleanup case proves no rollback after an earlier successful mutation. These historical results are now supplemented by independently reviewed final-pin untimed validation on the common-fix baseline and candidate: all 124 source/profile configurations pass checks and native builds, all 284 runtime cases pass their exact oracles, and all six optimized executable call-retention gates pass. Both full corpus sizes execute their designated checksum entrypoints in all four modes and both generated profiles. All 24 cleanup cases pass, including C’s formerly blocked Unit/no-rollback cases. Four C/I allocation-free intervals and four real allocating controls prove active calling-thread successful-allocation checks, not a process-wide allocation census. No timing or size acceptance is inferred. Both matching perf/check build pairs and the genuine allocation-profiling build pair also pass preparation. The new baseline also executes the previously rejected original C cleanup source in both native modes. Profiling will report exact allocation counts and clearly label formatted byte totals approximate; no profiling or timing samples have yet been collected. The guarded measurement controller and separate profiling adapter are now independently reviewed and integrated. The combined Python suite passes 311 tests with three existing skips; their execution paths remain mock-validated, not measured. Runtime timings will be whole-process costs; compiler costs cover checks, debug objects and AOT builds. Optimized linked-ELF sizes are distinct from optimized object bytes, and profiling timings are not uninstrumented latency.

Checkpointed validation retains exact target/test identities on the unchanged verified build. After reviewed external-wrapper corrections, all 5,667 registered pairs have observed outcomes across stages: 5,659 passes and eight normal ignores, including all 27 required optional bodies. Passed prefixes were not replayed to complete that checkpoint. Cross-stage coverage is not an uninterrupted gate.

The separate final uninterrupted command completed all targets and both actual zero-test doctest packages, but failed: 5,658 passed, one failed and eight normally ignored; Cargo returned 101, with full wait and no wrapper STOP. Two concurrent Caddy fixtures selected the same HTTP port, and readiness correctly rejected the other fixture’s identity. Issue #719 tracks this pre-existing fixture race in a separately authorized fixing task. No compiler/arithmetic regression is established by that failure. All ten observed containers had completed exact-ID removals; foreign container and image identities were unchanged. Missing final wrapper classifications remain unknown, not Docker success.

The user then approved applying only the reviewed Rust fixture patch to an isolated validation fork. Its entire proxy-test file equals the separately reviewed fix; all compiler, runtime, std, Cargo and native Plenty fixture files remain byte-identical to production revision 830763c9. On validation revision 8ca6fc5a, one uninterrupted gate passes 5,669 tests, zero failures and eight normal ignores, including all 27 required optional bodies and ten new fixture helpers. Both actual zero-test doctest packages complete. Independent evidence review accepts this isolated host-gate result, not either unchanged PR branch or a merge. The earlier failed run remains failed evidence.

All ten real proxy containers have authenticated same-ID removals and subsequent absence; foreign container and image identities are unchanged. Missing wrapper classifications stay unknown. Critical compiler/runtime/test build freshness is verified; ten non-test example paths’ pre-clean absence remains unobserved. Existing external engines remain older compatible builds, not fresh source builds. Process sampling is not exhaustive tracing or a controlled performance baseline.

Exploratory collection passed 16 runtime samples, 32 compiler samples and 12 code reports, with no warmups and one sample per selected case. These smokes qualified collection contracts, not speedups or acceptance-quality distributions. The first separate two-job profile attempt remains stopped before either producer ran: its identity guard rejected a packaged GCC hard link (issue #731).

The reviewed correction authenticates recorded native-tool identities, binds fresh link-count/ctime observations and preserves strict nonalias rules for inputs, proofs, frozen binaries and outputs. It consumes the exact authenticated record bytes, with deterministic record-swap regressions. All 102 profile and 182 combined controller mock tests pass. A new external driver then passes independent review and 58 synthetic methods, followed by a fresh successful two-job smoke. These do not relabel the original blocked attempt.

The full approved general schedule completes once: 2466 timed jobs, comprising 366 warmups and 2100 samples, plus qualification and code inspection. All 86 groups, 75 available paired distributions and 22 code reports retain their raw streams, resources and artifacts. No outlier or failure was dropped. All 350 required arithmetic traps correctly exit 1. The official full allocation-profile schedule separately completes all 16 jobs and retains 1248 phase rows; exact attributed request counts and approximate binary-unit byte displays are distinct from process allocation totals or retained heap. Independent evidence review finds no issues with these bounded collection components, not whole-proposal performance acceptance. The helper’s packet/acceptance-complete flags remain false.

Representative optimized-IR/native analysis accounts for arithmetic guards, Result construction/tests, cleanup and return carriers in 20 selected symbols. Trapping and recoverable pairs preserve their intended contracts. The C→I corpus has an additional Result widening/tag test, one fewer cleanup call site and a smaller local frame; this is not a measured speedup or universal ABI conclusion. The separate bounded native/export source audit finds no defect in changed typed storage, entry stacks, executor operands, partial mutation or explicit C adapters.

The repository compiler benchmark template is now mechanically adopted with independent precise errors and unchanged workload/measurement controls. Nine example tests, including full 1000/4000-unit type-check/native checksum cases and partial-mutation failures, pass alongside 15 arithmetic tests, formatting and strict workspace/all-target Clippy. These functional checks are not new timing samples. Frozen comparison inputs remain unchanged.

The validation record gives paired timing distributions, attributed allocation values, static code counts, source bridging and observation limits. Neither PR is merged; the proxy fixture patch remains only in the explicitly approved isolated validation fork. Independent pre-existing archive-decimal and PNG-framing limits are recorded as issues #705 and #707, not silently changed by arithmetic migration.

Reuse and limits

Source inspection at local main revision 96f6fc42 found useful existing machinery in src/frontend/checked.rs, src/codegen/checked.rs, and src/numeric.rs: a stack of checked scopes, per-operation error collection, cleanup paths, and cold exits producing a typed error. Scope records already distinguish function forwarding from arithmetic checking. The parser already recognizes contextual checked: suites in value positions.

Adding an explicit trapping mode and retaining nested checked: should be a contained extension of that machinery. Represent the nearest arithmetic mode explicitly; do not merely search past an unchecked scope for an outer checked target. Function failures use the declared public Result ABI. Remove the private wrapper and forwarding machinery for lexical checked callbacks under the uniform function rule below. Success returns, ?, early exits, cleanup and tail calls need auditing.

The whole change is not a small default toggle. Most arithmetic-bearing functions currently declare plain returns, and generic/protocol bodies, callbacks, generators, runtime helpers and std signatures depend on that. The core implementation reuses these paths; no timing experiment has been performed.

Functions and callbacks

Every named function and anonymous function has its own default error boundary, regardless of the mode surrounding its definition or call. Its declared return type must cover its escaping arithmetic errors. A checked: or unchecked: suite affects arithmetic in that function body, not a nested function’s body. Inlining must preserve this distinction.

Remove the special hidden error channel for closures defined inside explicit checked: blocks. That behavior was implemented for issue #614 under the old opt-in model; retaining it would give closures two different error mechanisms depending on their definition site. With checked defaults, an ordinary declared Result provides the same recoverability and a visible contract. Existing scope-bound callback behavior is not a compatibility requirement.

def make_incrementer() -> Callable[[i64], Result[i64, OverflowError[i64]]]:
    def(value: i64) -> Result[i64, OverflowError[i64]]:
        Ok(value + 1)

This closure needs no surrounding checked block or private error channel. Its arithmetic contract does not tie its lifetime to its definition scope; ordinary capture ownership and borrow rules still apply. A closure defined inside unchecked: likewise defaults to recoverable arithmetic. To opt its arithmetic out, put unchecked: inside the closure body, where it is visible.

Calls returning Result still require explicit handling, even within checked:. Using ? there exits the function as usual; it does not turn the call’s error into the enclosing block’s result. Local recovery from a callee uses match or ordinary Result operations. The cost of the uniform rule is that a checked block cannot transparently collect arithmetic errors from an arbitrary callback invocation; the callback and its consumer expose and handle them normally.

Update callback consumers to support the appropriate fallible return contracts. Review pipelines, sorting, generic helpers, worker jobs and boxed callables; change their APIs where that produces the clearest design. Do not preserve infallible callback signatures by blanket-inserting unchecked:. Worker error transport and resource ownership still need explicit contracts, but no lexical checked-block lifetime or private callback effect crosses those boundaries.

Special boundaries and library code

  • Generators: today’s Generator[T] cannot return a function-level Err from its suspended body. Reject unhandled recoverable arithmetic there with a targeted diagnostic; locally handle a checked: result or use a deliberate unchecked:. A fallible generator protocol is not implied by this proposal.
  • Destructors and protocol returns: revise protocols where fallible results fit their purpose. Where execution semantics require infallibility, such as cleanup with no error recipient, recover locally or explicitly opt out. Do not preserve a fixed return merely because it is the current API.
  • FFI and exports: choose explicit ABI contracts for fallible operations. Wrappers for foreign APIs must respect those APIs; Plenty-owned exports can change. Checked defaults do not invent a cross-language Result ABI. Native implementations need their own documented arithmetic audit because they are not Plenty source suites.
  • Generic functions and recursion: validate against declared returns and bounds, with specialization diagnostics where needed. Do not discover public signatures by a call-graph fixed point or change effects with inlining.
  • Std and builtins: apply the same default to Plenty implementations. Audit builtin integer reductions such as sum, which currently trap regardless of surrounding checked:. Prefer a public fallible contract where arithmetic can fail; retain a trap only as an explicit, documented choice. Fallible user operator hooks remain ordinary calls returning Result, not implicit effects.
  • Internal compiler/runtime arithmetic: compiler implementation language operations do not acquire Plenty source semantics. Generated operations that implement a public primitive must nevertheless honor that primitive’s contract.

These are implementation and execution-contract questions. Changing user-function lowering alone does not make all library arithmetic recoverable. Any public std signature changes need a std-conventions review and regenerated signature index as part of implementation.

Coordinated repository changes and teaching

Plenty has no external users or dependent systems. Breaking syntax, APIs and behavior is acceptable: select the clearest resulting design and update the repository together. There is no compatibility release gate, edition dependency, legacy mode, deprecation period or external migration plan.

Implementation still needs to update arithmetic-bearing functions, callback consumers, std implementations, examples and tests. This is ordinary work to make the repository obey the new contract, not a reason to preserve an inferior API. Diagnostics should explain missing arithmetic errors and the choices of propagation, local recovery or an explicit unchecked: precondition. Do not insert unchecked blocks throughout the repository merely to make the build pass.

Explicit checked expressions remain useful for local recovery. Recovery calculations can themselves fail and need the same scrutiny. Use a contextual unchecked: suite consistently with the block grammar; identifier compatibility is not a requirement if a different grammar proves clearer.

Teach simple arithmetic in a fallible entry point, with a short explanation of Result and Ok(()), then introduce local recovery and deliberate opt-outs. Update lessons and application examples to recommend the new behavior directly; no old-to-new migration lesson is needed. Keep tutorial/reference examples on current behavior until implementation lands; proposal examples use text fences.

Validation contract

Acceptance needs semantic tests for function propagation, precise unions, missing-error diagnostics, locally handled errors, all nesting directions, value/statement suites, and ordinary calls returning Result inside unchecked code. Exercise every integer width, division proofs, MIN // -1, MIN % -1, negative powers, constant operands, compound assignments and forwarded newtypes.

Ownership tests must cover resources and temporaries on failure, partial mutation, skipped writes, ?, explicit returns, loop exits and tail recursion. Callback tests must verify that named and anonymous functions keep their own declared error contracts when defined or called in either block mode. Cover stored and returned callbacks, generic borrowers, generators, worker boundaries and ordinary borrow-based escape restrictions; remove tests for the retired private checked channel. Inlining and optimization must preserve the lexical contract. Trapping tests need child processes and must verify termination in both debug and optimized builds.

Compare today’s unadorned trapping arithmetic, today’s explicit checked block, proposed implicit checked function, and proposed unchecked block on equivalent workloads. Inspect emitted IR/object code as well as timings: count guards, result construction, cleanup and return ABI differences. Measure tight loops, call-heavy kernels and realistic std consumers on success and failure paths. Verify the no-allocation arithmetic contract and report compiler/build-time and code-size costs. No speedup claim is justified before these measurements.

The original checked-arithmetic design records why local PR #678, which returned a Result from every operator, was abandoned. Preserve its lesson: use plain values between operators and a shared failure exit, rather than reviving per-operator wrapping and unwrapping.

Adoption also requires updated executable tutorial/reference examples, updated arithmetic and trap documentation, the std signature index, the full relevant compiler/runtime tests and the complete docs CI checks. The proposal itself changes none of those implemented contracts.

Unchecked arithmetic modes

Status: Partly implemented. --unchecked trap|assume and unchecked trap: are implemented; see the numeric reference, which is authoritative. A plenty.toml setting and the stdlib precondition audit remain under B14 in the backlog; until that audit, std always traps. Issue #764 also extends the mode to list, slice-view and Bytes index syntax. This builds on checked arithmetic by default.

Contract

Plain unchecked: transfers responsibility for arithmetic preconditions to the programmer. A compiler setting selects whether those preconditions are checked at runtime or assumed valid. Trapping is the initial default in every build profile; selecting optimization alone must not enable undefined behavior.

Source formGlobal trapping modeGlobal assumed-valid mode
Ordinary arithmeticDeclared recoverable arithmetic contractSame
checked:Local recoverable arithmetic contractSame
unchecked:Check arithmetic and trap on violationAssume arithmetic preconditions; omit their checks
unchecked trap:Check arithmetic and trap on violationCheck arithmetic and trap on violation

Both unchecked forms produce an ordinary success value, without adding arithmetic errors to the function’s return type. unchecked trap: makes termination on an arithmetic violation a permanent source-level guarantee. Plain unchecked: must not promise termination independently of configuration. The diagnostic mode catches violated obligations without changing signatures; passing that mode’s tests is evidence, not proof for all possible inputs.

In assumed-valid mode, overflow is not allowed and is not wrapping. Violating an arithmetic precondition is undefined behavior: the compiler may remove branches or transform loops on the assumption that violation never happens. Silent wrong results and memory corruption are possible; a crash is not guaranteed. An actual machine instruction wrapping does not define the source expression’s result. Use explicit wrapping operations for modular arithmetic; explicit wrapping, saturating and checked methods retain their own contracts in either mode.

Overflow obligations

The no-overflow obligation applies uniformly to signed and unsigned integers. Every intermediate result of ordinary integer arithmetic must be representable in that operation’s type, not merely the final result of the expression. For example, a * b // c violates the contract if a * b overflows even when the final mathematical quotient fits. Use a wider type, an algorithm that avoids the overflowing intermediate, or explicit wrapping when modular arithmetic is appropriate to the algorithm.

Arithmetic contractOverflow detection required?May assume overflow never occurs?
Explicit wrapping operationNoNo
Ordinary arithmetic in assumed-valid unchecked:NoYes
Ordinary arithmetic in trapping unchecked: or unchecked trap:Yes, unless proven unnecessaryOnly on paths where checking or proof establishes it

Wrapping already permits arithmetic without overflow checks. Undefined overflow adds optimization freedom: for example, the compiler may treat x + 1 > x as true when the addition carries the no-overflow obligation. This is not a valid way to detect overflow after the operation. The benefit beyond check omission depends on the backend and workload; the source contract does not guarantee a speed improvement over wrapping.

Arithmetic scope and boundaries

The operation contract must specify which preconditions a block transfers: representable integer addition, subtraction, multiplication, negation and powers; valid integer divisors and representable quotients; valid shift counts; and the existing validity conditions of numeric conversions and integer power exponents. Preserve Plenty’s existing successful arithmetic semantics, including floor division and modulo. Do not import a machine instruction’s different semantics or forbidden cases as language rules. In particular, a defined signed minimum remainder by minus one must remain zero if that is the existing contract.

IEEE floating-point arithmetic retains its existing behavior, including NaN and infinity. Float-to-integer conversion retains its existing valid-input rounding rules; its range/validity obligations must be specified separately from ordinary floating arithmetic. Compile-time invalid constants remain compile errors. No new recoverable shift or cast API is implied by this proposal.

Issue #764 extends the implemented mode to list, slice-view and Bytes index syntax: plain assumed suites require nonnegative, in-range indices and omit bounds checks and negative-index normalization. Trapping suites preserve normal negative indexing. Other indexing and collection methods remain checked.

Allocation failure, capacity/layout validation for memory allocation, ownership, borrowing, I/O failures and explicit unwrap() are outside this feature. Memory-management arithmetic must still establish its own safety invariants. The global mode is not permission to skip allocator validation or to assume an allocation succeeded.

Modes apply lexically to arithmetic, not dynamically across calls. The nearest explicit arithmetic suite controls an operation; leaving it restores the outer mode. A nested plain unchecked: uses the global setting, even inside unchecked trap:. A nested checked: restores local recovery. Named functions and closures keep their independent contracts; merely defining or invoking one inside an unchecked suite does not change its arithmetic mode. Inlining must preserve these boundaries. ?, returns, loop exits and ordinary Result values keep their existing meanings.

The setting must have explicit, documented CLI/package configuration and precedence. Its spelling is an implementation decision. It must participate in compiled-artifact and function-cache identities, including dependency builds, so changing the setting cannot reuse code with the opposite contract. Prebuilt libraries retain their compiled semantics; a caller setting cannot retroactively change them. A source unchecked trap: guarantee must survive all build modes, worker lowering, specialization and cross-module optimization.

Standard library rationale

Validate public inputs, then use a small plain unchecked region only where the validated inputs and maintained invariants establish its preconditions. If a block intentionally rejects invalid input by trapping, express that guarantee with unchecked trap: or explicit validation. Merely enclosing existing code in an unchecked suite does not establish a bound. Classify each audited operation by intent: established bounds permit plain unchecked:, intentional modular arithmetic requires explicit wrapping, and invalid-input rejection requires unchecked trap: or explicit validation.

Review arithmetic opt-outs already introduced by checked-default adoption before enabling assumed-valid mode for the stdlib. Preserve resource-failure mapping, earlier successful mutations, callback error propagation and ownership cleanup on defined paths. Use the repository’s consistency-check procedure for a repository-wide audit; proposal examples and historical fixtures must be identified separately from executable code.

Validation rationale

The modes need independent evidence. Invalid arithmetic belongs in trapping-mode tests, which must check the trap and source location without widening signatures. Never execute undefined arithmetic in assumed-valid mode to assert a particular crash, wrapped value or output. Its runtime tests use valid boundary inputs; static IR/native inspection establishes check omission and supported assumptions. Explicit trapping suites need invalid-input tests in both global modes. Cover signed and unsigned overflow obligations, including an overflowing intermediate with a representable final mathematical result, in trapping tests. Test explicit wrapping operations at overflowing inputs in both global modes and both unchecked block forms: those operations remain defined.

Check nested modes, callbacks, generators, compound writes, short-circuit paths, integer widths, divisors, shifts, conversions, powers and mode-sensitive cache reuse. Verify that allocation, indexing outside assumed suites and explicit checked/wrapping operations retain their contracts. Compiler backends must preserve valid-input semantics; omitting guards alone does not establish that all desired optimizer assumptions are represented.

Follow the emitted-code procedure: isolate small kernels, compare equivalent C contracts, attribute remaining instructions and measure optimized builds by instruction count. The programmer’s explicit precondition is the premise for removing a check in this feature; ordinary safe code still needs a compiler proof. Report backend limitations and measured results rather than promising that unchecked code is always faster. This contract is permission to optimize, not a speed guarantee.

Recursive owned data

Status: Partly implemented. Recursive classes, enums, aliases, and finite generic instances are implemented, with moves, matching, borrowing, and allocation-free drop. Recursive storage and boxes own the contract.

Shipped differently: this page first proposed that classes and enums stay heap records, so recursion needed no Box. Classes and enums are now inline values, and every storage cycle must pass through Box[T] or a collection. The source model below describes what shipped.

Remaining: deep copy, equality, and formatting of recursive values (B33); stored references (B11). Shared mutable graphs, weak edges, and a garbage collector are out of scope.

For practical linked-list and tree patterns, see the owned-structure assessment. Its checked-index arena experiment addresses graph identity and access; this proposal addresses finite storage and destruction. An index arena is not a new borrowing capability.

Source model and finite storage

Classes, tuples, and enums store their fields inline, so a type that contained itself directly would have infinite size. A recursive field goes through a heap boundary: Box[T], a list, a set, or a dictionary. A cycle without one is rejected, and the diagnostic names the boundaries to use.

class Node:
    value: i64
    next: Option[Box[Node]]

enum Tree:
    Leaf(i64)
    Branch(Box[Tree], Box[Tree])

Node(1, Nothing) and Tree.Leaf(1) allocate nothing and cannot fail. Some(node) is an inline ownership wrapper. Box(value)? is the one allocation per link, and its failure drops the value it consumed. Lists of children use their existing fallible allocation API.

Keep transparent alias cycles invalid: type Loop = Option[Loop] has no nominal ownership boundary and must not be expanded forever. Separately distinguish a finite nominal recursion from infinitely growing generic instantiations such as Node[T] containing Node[list[T]]. Neither a type spelling’s nesting limit nor the runtime value’s depth is a substitute for that distinction.

Ownership and destruction

Recursive classes and enums move under the existing rules. Aggregates containing recursive values are also affine, and automatic deep copying is rejected. Mutable collections or classes already make their containing values affine. Storing an owner inside its own descendant must fail the normal move/borrow checks. There are no stored references or weak edges in this design, so source ownership cycles are not introduced by recursive types.

Drop uses the existing intrusive destruction queue and must allocate nothing, even for a very deep chain. A class hook still runs before its fields, and explicit nested drops in a hook finish synchronously. Queue links reuse dead header storage only after atomic final-release ownership has been established. Recursive values do not become transferable across threads merely because their counts are atomic; the worker eligibility rules apply to every reachable field and destructor effect.

Compiler identity and bounded analysis

Use a compilation-owned table of nominal definitions. Type edges refer to stable identities; the table owns definitions, avoiding an Rc cycle that leaks compiler memory. Reserve identities before resolving fields, then finalize immutable definitions. Aliases resolve to those identities. Type equality and hashing compare nominal identity and concrete generic arguments, never recursively expanded fields.

Compute properties over the finite type graph, with explicit monotone rules for copyability and destructor presence; a temporary placeholder must not cache a wrong fact. The implementation uses visited identities and leaf elimination to find recursive storage and aggregates that reach it. More detailed component analysis may support future thread effects without changing source syntax. Keep finite inline-layout checks distinct from heap graph recursion. Emit native descriptor IDs before their edges, as the current metadata emitter already does. Every metadata/visibility/debug traversal needs visited identities or a bounded worklist. Declaration errors should name the actual dependency cycle.

Operations that need their own depth contract

Construction, movement, matching, field borrowing, and iterative drop are implemented. Deep copy, structural equality, hashing, and formatting recurse through runtime values, so on recursive data they would turn ordinary valid values into stack overflows. The compiler rejects them for any type whose stored values can recur.

B33 lifts that restriction. Prefer iterative fallible work storage for already-fallible copy and formatting. Equality and other allocation-free operations cannot silently add a fallible allocation; they need a bounded-memory traversal. Direct user-written iterative traversals remain possible. This restriction is preferable to a hidden small maximum on the depth of every recursive value.

The backlog tracks implementation and acceptance work; this page records the representation and behavioral constraints, not a second task queue.

Owned structures and checked node identities

Status: Investigation complete. Owned-chain and tree-rotation examples, an arena lesson, and a checked-source prototype exercise existing Plenty. No std.arena API or new borrowing capability has shipped. The recursive-storage, borrowing, and box references own current behavior. B73 in the backlog owns arena delivery; B11 owns broader borrowed traversal and collection-loan precision.

For representation and destruction rationale, see recursive owned data. This assessment focuses on how to write and manipulate those structures using today’s language.

Make the correct representation easy to use

The goal is that a safe, efficient implementation should also be the obvious and simplest implementation for its workload. Memory safety alone is not enough: failure can destroy data, stale indices can identify the wrong object, and a recursive traversal can exhaust the native stack even when automatic cleanup is bounded-stack.

The investigation distinguishes uniquely owned chains/trees from parent links, shared subgraphs, and intrusive collections. Direct box field access reduces syntax but does not change their ownership requirements. A generic arena is a better first library candidate than separate general-purpose list, tree, and graph APIs: it centralizes identity, deletion, and checked access without fixing application-specific edge semantics.

Evidence and limits

The examples and tests/test_data_structure_patterns.rs preserve these findings:

ExperimentResult
Owned linked stackFallible push, ownership-transferring pop, borrowed iteration, and mutable iteration work.
Tree rotationExisting boxes can be rewired without allocating, including transfer of the pivot’s middle subtree.
Deep chainOne million nodes traversed, mutated, and explicitly dropped with allocations disabled under a 256 KiB native stack on Linux. This is a regression check, not a throughput benchmark.
Failed insertionAllocating an empty node before linking preserves the existing chain. Detaching into a fallible box constructor does not.
Arena linksIndices express back/parent links and shared targets without ownership cycles. The TOML parser already uses this representation.
Borrowing limitsStored references, including a worklist of borrowed tree nodes, are unsupported. Different list indices do not establish disjoint mutable loans.

Bounded-stack cleanup is a useful semantic advantage, but recursive user calls and destructor hooks still consume native stack. The current box runtime carries a 32-byte header before its payload; simpler syntax is not evidence of lower per-node memory cost. No comparative timing benchmark was performed.

For ordinary stacks use a list; for queues use the existing std.deque. Its growth already preserves existing contents on failure. Keep owned chains and tree rotations as teaching examples until an application motivates a stronger packaged collection API.

Candidate arena contract

A candidate std.arena would own node values and issue typed handles. Handles identify nodes; they neither own nodes nor keep them alive. The arena validates identity before granting temporary access. Deletion invalidates the handle; slot reuse must not make an old handle identify a replacement.

  • Typed handles distinguish payload types. Arena identity must also distinguish separate instances with the same payload type, including after destruction.
  • Growing storage preserves logical IDs but does not guarantee stable addresses.
  • Reusing vacant slots changes a generation. Exhausted slots retire permanently rather than wrapping. Arena-identity exhaustion needs an equally explicit rule.
  • Failed insertion preserves all existing elements and handles. The initial candidate consumes the incoming value on failure, following Deque.append; an ownership-returning variant is a separate contract to evaluate.
  • Lookup and removal reject invalid/stale handles through a checked result. They do not trap just because a handle has expired.
  • Node removal does not silently rewrite inbound edges. Applications either remove those edges or handle a failed lookup. Unreachable nodes remain owned until removed or the arena is dropped.
  • Safe access to two nodes rejects equal or invalid handles before invoking the callback. A callback error restores storage ownership, but does not roll back mutations already performed by the callback.

Returning Option[&T] or a tuple of mutable references is not an available implementation strategy today. Scoped callbacks returning owned results fit the current storage rules. Their usability and move costs need evaluation before a public API is frozen. A separate language change could eventually offer direct disjoint loans; the library should not require one merely to demonstrate value.

Prototype

The prototype assessment describes the executable module in examples/arena/arena.plenty, its API, validation, and deliberate namespace limitation. It is ordinary Plenty and is not embedded under std.

This distinction matters: requiring callers to invent unique namespace numbers would move a correctness problem into application code. The prototype proves generation validation and scoped access, not a satisfactory identity issuer. A production constructor needs automatically unique identity within a documented domain. A non-reusing runtime token is one possible narrow primitive; an owned issuer with enforced domain separation is another. Concurrency, copying, serialization, and exhaustion determine which contract is sound and convenient.

std.atomic and std.sync.Mutex now provide synchronized mutation through scoped shared borrows. An atomic counter could serve one identity issuer, but separate counters can issue the same numbers: issuer uniqueness and lifetime remain part of the arena contract. Atomics do not by themselves provide shared ownership of an issuer or arena.

Issue #326 separately explores a counted shared owner for user data and a thread-shared HTTP pool. Its eventual access, cycle, and destruction rules may offer another representation for shared nodes or a shared identity issuer. These examples and the arena prototype do not depend on that design, and do not choose its cycle policy. Reassess the representation guidance when that feature ships; retain arenas for workloads that need checked logical identity and explicit removal independently of shared-owner lifetimes.

Rust and intrusive collections

Rust also supports safe owned singly linked lists and trees. It already provides field autodereferencing, so Plenty’s omitted dereferences are a modest ergonomic difference. Recoverable allocation is another difference: Plenty’s box constructor returns Result and destroys its input on failure. Compare failure contracts, not only the shortest successful-path expression.

An intrusive collection embeds links in application objects, potentially for multiple memberships. Pointer-based implementations can need stable addresses and lifetimes governed outside the collection. These requirements differ from an ordinary owning doubly linked list. An arena can emulate many link topologies while changing physical-address and lifetime semantics. It is not a replacement for all pointer-intrusive use cases.

Safe libraries can encapsulate trusted implementation machinery even when the source language has no unsafe keyword. Add such machinery only when a concrete workload needs stable-address/external-lifetime semantics. There is no evidence here about which kind of list programmers complain about most.

Relevant precedents, consulted for the original investigation: Rust linked-list guidance, checked disjoint slice access, slotmap’s versioned keys, and intrusive-collections.

The documentation teaches implemented behavior. This proposal preserves the representation decisions and alternatives; priorities and delivery work remain only in B73/B11 in the backlog.

Arena prototype assessment

This accompanies the arena design assessment. It is an experiment in existing language facilities, not a stable standard-library contract.

Run the demo from the repository root:

cargo run --locked --bin plenty -- examples/arena/main.plenty

examples/arena/arena.plenty owns list[Slot[T]]; each slot stores an optional value, a generation, and a free-list link. Key[T] carries the namespace, slot, and generation, with no stored T. The demo transfers between two integers, removes one, reuses its slot, and shows that the old handle fails lookup. examples/arena/parent_links.plenty stores typed parent handles inside nodes, duplicates a handle without copying its node, and detects a removed parent.

Prototype operationResult and behavior
Arena[T](namespace, generation_limit=i64.MAX)Fallible construction; namespace uniqueness is the caller’s responsibility.
insert(value)Result[Key[T], AllocError]; reuse a vacant slot or grow storage.
contains(key)Check namespace, bounds, generation, and occupancy.
remove(key)Option[T]; transfer the owner and invalidate its handle.
read(key, callback)Option[R]; callback receives &T and returns a storable result.
update(key, callback)Option[R]; callback receives &mut T.
update_two(a, b, callback)Option[R]; reject identical slots or invalid handles, then lend both values to a callback.
len()Number of live values, excluding vacant or retired slots.
key.duplicate()Another key for the same node, without allocation or payload copying.

The module uses a custom empty destructor to forbid explicit copying of arenas; otherwise copying storage would duplicate its namespace. Moving an arena preserves its keys. Key.duplicate reconstructs three integers; it avoids the fallible generic aggregate copy API and never duplicates a node payload. The reduced generation limit is an experimental test seam for exhaustion, not a proposed production tuning knob. Slots retire on exhaustion without wrapping.

Two-node updates without simultaneous indexed loans

update_two validates first, moves both values into local owners, lends those locals to the callback, then restores them before returning the callback result. The exclusive arena borrow prevents the callback from re-entering that arena. References cannot escape through the stored Option[R] result.

This does not allocate, copy payloads, or run their destructors during transfer. It works for affine payloads. It does physically move values, potentially several times for a large inline T; it does not promise stable addresses or zero memory traffic. A callback returning Err still restores both values, with its mutations intact. Fatal traps do not unwind; restoration after a process abort is not promised.

Borrowing distinct indices directly remains rejected. The prototype shows a library alternative, not a change to collection loans. It deliberately avoids temporarily allocating a second container or requiring copied payloads.

Failure and cost boundaries

Reusing a free slot and removing a value allocate nothing. Fresh slots use fallible list append; existing contents and keys survive failure, while the incoming value is destroyed. The free list lives in vacant slots, so removal does not need an allocation. Retired slots are retained as empty storage.

Lookup, removal, and reuse take constant time apart from payload operations; growth has list allocation/movement costs. Callback work and returned values retain their own costs and failures. An arena does not enforce tree/graph invariants, collect unreachable nodes, or repair incoming links.

What is proved, and what is missing

Native tests cover successful access, deletion/reuse, stale and foreign-namespace rejection, generation exhaustion, affine destruction, callback errors, allocation-free reuse/pair access, and preservation during failed growth. The demo and module are exercised by tests/test_data_structure_patterns.rs.

Namespaces are supplied integers. Two arenas given the same namespace can accept one another’s keys if the slot and generation coincide. Negative namespace values are not special. This is an explicit prototype limitation, not safe default instance identity. The production design must solve it before this can become std.arena. The module also does not supply serialization, iteration, compaction, shared concurrent access, or a node-address guarantee.

These findings support the generic arena direction while leaving its production identity and access API open for review. Delivery and acceptance criteria live in B73 of the backlog.

Typed ranges, generic calls, and expression-local inference

Status: Partly implemented. Recommended 2026-10-06. Typed ranges, contextual numeric literals, IntType, explicit type arguments, and anonymous def shipped; see ranges, numbers, generic inference, and function values.

Shipped differently: the compiler passes expected types down as hints; it has no inference variables or deferred defaulting. A generic parameter position gives a literal no context. An integer literal is accepted in a float context. Generic bodies are checked per instance.

Remaining: expected-result inference, partial explicit type arguments, and literals across generic arguments (B16); an iterator protocol (B12); mutable/consuming erased callbacks (B32). Explicit fallible shared-call erasure already supports heterogeneous shared-call storage. Inclusive ranges and indented anonymous bodies inside delimiters are unscheduled.

The completed callable surface review and stored-environment decision own the callback design history. Allocation-free ranges record representation, separate from the inference choices discussed here.

Recommendation

Yes: use range[u8](8) and support contextual inference from list[u8]. Square brackets should be the ordinary explicit type-argument syntax for generic functions, including qualified calls such as numbers.range[u8](8). No extra separator is needed before the brackets.

The proposal was that all three examples below produce list[u8]:

squares = [n * n for n in range[u8](8) if n % 2 == 0]?
squares: list[u8] = [n * n for n in range(8) if n % 2 == 0]?
squares = [n * n for n in range(8) if n % 2u8 == 0]?    # rejected

The first states the range’s type directly; the second states the desired collection type. Those are the clearest forms to teach, and both shipped. The third needs the constraint solving proposed below and is rejected: a filter operand does not select the range type.

Numeric suffixes such as 2u8, 42i32, 3.5f32, and 1e-3f64 already work. Keep them as exact type constraints. The new convenience is contextual typing of unsuffixed literals, not implicit numeric conversions.

Infer before defaulting or generating operations

At proposal time the compiler lowered a comprehension’s iterable before its result expression and assigned concrete numeric literal types immediately. It could not make the second and third examples work just by accepting a new call spelling. Introduce a typed-expression inference step before lowering to operations:

  1. Instantiate a generic signature with fresh inference variables, or substitute the explicit type arguments supplied by the caller.
  2. Collect constraints from declared types, explicit literal suffixes, argument and result types, operators, branches, and comprehension bindings/filters.
  3. Solve equalities and check bounds. Only then default unconstrained integer variables to i64 and floating-point variables to f64.
  4. Validate literal representability and lower concrete types to existing IR.

For the annotated comprehension, the result element must be u8. Multiplication has equal operand/result types, so both uses of n must be u8. That determines the range’s element type. Its stop literal 8, the remainder divisor 2, and the comparison operand 0 can then be checked as u8 literals.

For the suffix example, % requires equal operand types: 2u8 constrains n to u8. That propagates to the iterable and multiplication. Comparison produces bool but still requires its numeric operands to have the same type.

No constraint source silently overrides another. For example, an annotated list[u8] conflicts with range[u16](8) when the element expression is n * n. An explicit cast in the element expression can make that intentional instead.

Keep the inference region small: an initializer, an assignment RHS, or a return expression, including nested calls, comprehensions, and expression-valued branches. A function’s declared return type constrains its tail expression and each explicit return. Function interfaces still require parameter and return types. Do not infer signatures from callers or inspect callee bodies to infer call-site types.

Once an unannotated local initializer has been resolved and defaulted, its type is fixed. A later use does not retroactively specialize it:

values = range(8)                     # defaults to range[i64]
squares: list[u8] = [n * n for n in values]  # error

Use values = range[u8](8) at the earlier binding instead. Likewise, a declared return type constrains a comprehension used directly as the final expression; it does not reach back through an already defaulted local initializer. This boundary keeps diagnostics predictable and avoids whole-function inference.

An output annotation also cannot determine every input in an arbitrary chain:

values: list[u8] = [u8(n) for n in range(8)]

Here the cast explicitly separates input and output types. The range can still default to i64. Types flow through the known signatures of operations, not through guesses about their behavior.

Literal context is not conversion

Under the proposed rules, x: u8 = 8 and x: f32 = 3.5 become valid because the literal is initially unresolved. However:

limit: i64 = 8
values = range[u8](limit)  # error: an existing i64 is not a u8
values = range[u8](8i64)  # error: an explicit suffix fixes the type
values = range[u8](300)   # error: the literal cannot fit in u8

Retain explicit casts for numeric conversions. Initially keep integer-shaped unsuffixed literals in the integer family and decimal/exponent literals in the floating-point family. To spell a floating-point whole number, use 1.0 or an explicit suffix such as 1f32, which is already supported.

Contextual literals are a deliberate change to the current language contract. Some examples that currently teach small: f32 = 1.5 as a type error must become successful examples when this is implemented. Transparent aliases should supply the same context as their targets; they still need not create new literal suffixes. Integer arithmetic remains checked at its inferred width: u8 multiplication must not secretly widen just because the expression appears in a comprehension.

IntType is a compile-time constraint

The sketched declaration is workable:

def range[T: IntType](stop: T) -> range[T]:
    ...

Here the return spelling describes a concrete built-in iterable. The example is a signature sketch, not a requirement to implement this intrinsic in Plenty source immediately. A user-written yielding function would naturally return Generator[T] using the existing generator model once generic functions exist.

Define IntType initially as a compiler-known constraint admitting exactly i8, i16, i32, i64, u8, u16, u32, and u64, including aliases of those types. It excludes bool and floats. It is neither a runtime sum type nor a type alias for a union. Its contract grants the operations required for integer code: comparisons, checked arithmetic, and representable integer literals.

This does not require user-defined arithmetic protocols in the first release. Ordinary structural protocols remain the plan for user-defined method contracts. Keep these mechanisms distinct; a method name alone cannot establish built-in integer representation or arithmetic semantics.

Do not introduce a runtime-erased Iterator[T] merely to express range. The existing language distinguishes concrete iterable values and generators; an iterator protocol can later describe shared operations and borrowing rules. Returning a concrete range preserves cheap repeated iteration and indexing, which differ from consuming a generator.

Initially specialize reachable generic calls and reuse an instance for each declaration/type-argument combination. Check generic bodies against their bounds. Do not add overload search, implicit trait imports, compile-time arbitrary execution, or unconstrained specialization as part of typed ranges.

Range boundaries and descending unsigned iteration

The one-argument form is straightforward: both implicit zero and stop have type T, and yielded elements have type T. Inferring T from a typed argument should work too: range(8u8) implies range[u8](8).

Recommend that the three-argument form use T for start and stop, but a separate signed i64 for step. This allows:

range[u8](8, 0, -1)  # yields u8 values 8 through 1

The step is a displacement, not an element. Zero remains invalid. This deliberately limits the step magnitude to the supported i64 domain even for u64 elements; larger displacements can be a later extension. The concrete intrinsic may retain its existing one-, two-, and three-argument forms without committing the language to general overloads or default arguments yet.

Keep exclusive endpoints in T initially. Therefore range[u8](256) is a compile-time error, and an ascending exclusive u8 range cannot include 255. Do not silently widen the bound or wrap it to zero. A future inclusive-range API can cover the full domain; a separately typed bound would be an alternative design but would no longer have the simple stop: T signature above.

Iteration must check whether another element exists before an increment that would overflow the element type. For example, range[u8](254, 255, 2) yields only 254 and terminates normally. Use widened or checked unsigned-magnitude calculations for bounds, differences, length, and stepping, including u64 and the most negative i64 step. Do not lower these cases through signed i64 element arithmetic. If a length cannot fit the language’s length result type, report the limit rather than wrapping; specify that behavior alongside the API.

Reserve space for functions as values and closures

Generic application and indexing share brackets. Resolve f[T](x) as generic application when f names a generic callable; retain callbacks[i](x) for an indexed callable value once functions become values. The parser must preserve this distinction for name/type resolution, rather than assuming that every bracket followed by a call denotes type arguments. Type arguments resolve in the type namespace, with the same explicit import rules as annotations.

Multiline anonymous functions are compatible with this direction. A promising spelling, deliberately still provisional, is an anonymous def expression:

transform = def(value: i64) -> i64:
    adjusted = value * 2
    adjusted + 1

Keep its parameters and return type explicit, like a named function. Expected return types then constrain literals and generic calls in its body. Captures need no separately repeated type declarations: their types come from bindings.

An initial closure model should distinguish:

  • Read-only borrowing of captured owners.
  • Exclusive borrowing when the closure mutates a captured owner.
  • Explicit ownership capture for a closure that outlives the surrounding scope.
  • Captures consumed during a call, making the closure callable at most once.

The implemented capture syntax and limits are recorded in the closure reference and consuming closure reference. Do not infer an invisible deep copy, or decide capture ownership by whether a later compiler pass happens to let a closure escape. A borrowing closure cannot outlive its owners, and captured borrows must remain active while future calls may use them. Owned captures drop with the closure, subject to moves and consumption. The implementation extends loan liveness with environment dependencies, rather than only lowering code and data.

Use concrete environment layouts and static call contracts for generic callable parameters where possible. Avoid mandatory heap allocation, reference counting, or dynamic dispatch for every closure. A heterogeneous collection of callbacks would need an explicit representation decision later.

Anonymous suites nested inside call arguments also require a parser design: the current lexer suppresses indentation inside parentheses. Preserve the goal of multiline anonymous functions in expression positions without claiming the assignment example above settles nested suite termination or comma placement.

Suggested delivery

  1. Add contextual numeric literal inference and a typed-expression phase with deferred defaults. Cover initializers, calls, returns, branches, filters, and comprehension elements, including conflicting and unrepresentable constraints.
  2. Add explicit function type arguments and typed range representation/lowering, using the same generic-call model intended for user functions. Test unsigned descent, boundary termination, maximum-width values, and empty ranges.
  3. Expand to explicitly bounded generic functions and cached instantiation. A first IntType constraint can precede general structural protocols.
  4. Add callable values and closure capture checking, with a separately reviewed multiline expression grammar. The range work must not preclude these features.

This records the original design sequence. Typed ranges, contextual inference, generic functions, and structural protocols now have implemented paths; see the reference for their exact limits. Remaining inference and closure work is tracked only in the backlog. The learning guide continues to teach implemented behavior with runnable examples.

Package language editions

Status: Not implemented. Exploratory candidate under B17, not an implementation commitment. Plenty currently has one language mode; its manifest rejects edition. The package reference describes implemented behavior. All edition names, settings and examples below are hypothetical.

Assessment

Plenty could plausibly support packages written in different language editions in one compilation. The existing package-aware module loader is a useful foundation. A narrow mechanism for syntax and name-resolution changes looks like moderate compiler/tooling work, with little reason for extra runtime cost. It is not just a manifest field, and it is not a cheap way to preserve arbitrary historical language behavior.

Recommendation: keep this option open, but wait for a concrete incompatible change before implementing it. Prefer additive features and contextual keywords where they solve the problem. If an edition becomes worthwhile, require the first change to demonstrate mixed-edition interoperability using one type, ownership and runtime model. Do not promise a release cadence or permanent support policy before understanding that maintenance cost.

What editions buy

An application should be able to adopt a new spelling or resolution rule without waiting for every dependency to migrate. Conversely, an older application should be able to use a newer-edition library, provided its compiler supports both. The application selects its own edition, not the edition of the whole graph.

Rust uses opt-in editions for otherwise incompatible language changes. Crates from different editions interoperate and lower to a common internal representation. That constrains which changes can be edition-specific; it does not require a separate runtime per edition. See the Rust Edition Guide.

For Plenty, distinguish four independent settings:

SettingMeaning
Package versionVersion of the library/application and its API
plenty-versionMinimum compiler version needed to build the package
manifest-versionFormat of the package metadata
Proposed editionRules used to interpret this package’s source

A new compiler can support old and new editions. An old edition may still use new additive features, and an old compiler need not understand a new edition. Editions do not pin compiler bugs, standard-library releases, dependency versions or native ABI compatibility. They would not make a new compiler link arbitrary objects emitted by old compilers: the initial contract is compilation from source with one supported toolchain and runtime.

A package author’s experience

Use illustrative names e1 and e2, not proposed release dates:

# app/plenty.toml
manifest-version = 1

[package]
name = "app"
version = "0.1.0"
edition = "e2"

[dependencies]
legacy = { path = "../legacy", version = "^0.1.0" }

The legacy package independently declares edition = "e1". All its modules, library targets, executable targets and tests use that edition; importing it does not reinterpret its source as e2. Workspace members choose independently. Start without per-file or per-target edition overrides.

For a deliberately hypothetical change, suppose e2 reserves the word memo while e1 permits it as a local name:

# legacy/src/lib.plenty — e1
pub def answer() -> int:
    memo = 42
    return memo

# app/src/main.plenty — e2
import legacy

def main() -> Result[(), IoError]:
    print(legacy.answer())?
    Ok(())

The application’s call works; only writing a local named memo in the e2 application is rejected. Migrating legacy could rename its local variable, then change its manifest. No caller needs to change its edition.

Public names make even this example harder: an e1 library might export a function named memo. Before reserving it, Plenty would need a way for e2 callers to name that existing API, such as contextual member-name parsing or escaped identifiers. Neither spelling is selected here. Renaming a public API is not a harmless automatic migration. A new keyword that makes old exports inaccessible fails the interoperability requirement.

Boundaries that keep this manageable

Candidate changeEdition suitability
New reserved spelling, old syntax translated to existing operationsPromising, with public-name access and migration rules
Different import lookup or implicit namesPossible, if resolved in the defining package before combination
New opt-in feature that accepts all existing codeUsually no edition needed
Changed type identity, value layout, calling convention or allocator contractOutside the narrow design; needs a shared representation or a separate interoperability design
Weaker historical borrow/safety checks, different destruction or concurrency guaranteesDo not preserve through this mechanism; cross-package safety obligations must still agree
Removing a standard-library API or changing its result/error typeOrdinary API compatibility problem; an edition label alone does not solve it

Every package must expose types and callable contracts that the other editions understand. Protocol satisfaction, borrows, generic arguments and errors must continue to compose. A generic function’s body must retain its defining package’s interpretation when specialized for a caller from another edition; the caller must not change how that body resolves names. Prefer resolving and normalizing edition differences before shared checking and specialization.

Safety fixes apply to every edition even if previously accepted code stops compiling. More general compatibility exceptions need an explicit policy; “old edition” cannot mean “all past compiler behavior is frozen”.

Embedded std should use the edition selected by the compiler distribution, not the importing package’s edition. Its exported contracts remain usable from every supported edition. Generated test harnesses and interface source also need an explicit interpretation; never inherit an unrelated caller’s setting.

Selection and tooling

If adopted, missing edition metadata should mean a fixed baseline, never “whatever is newest today”. New-package commands should write the latest supported edition explicitly. Reject unknown editions with the package name and supported values; do not silently choose a nearby version. Validate the minimum compiler requirement independently.

Standalone compilation needs an equivalent explicit option, provisionally plentyc --edition e2 FILE, with the same fixed baseline when absent. A package graph supplies editions per package; reject a conflicting global override instead of applying it to dependencies. Compiler library APIs, the language server and documentation checks must use the same selection rules.

The proposed single-file script header could carry [package].edition. It should share the package schema and default rather than invent another source directive. An edition does not require a script runner.

Resolution continues to choose package sources/versions. Edition is metadata of the selected package, not a request to compile that dependency in multiple editions. A lockfile is not the authority for source interpretation. Generated package graphs must preserve the edition, and build reuse must be invalidated when it changes, including when no source bytes change.

Fit with the current implementation

This is a source inspection at local main 2d6998b4, not an implemented prototype or a measured delivery estimate:

AreaExisting foundation and required work
src/project/manifest.rsStrict keys and minimum compiler metadata exist; tests explicitly reject edition. Add validated metadata, defaults and generator support.
src/project.rsSourcePackage/SourceGraph already carry package identity, roots and dependency edges. Carry edition through resolution, graph serialization and synthetic test packages. Version the graph format so older readers cannot silently drop semantics; preserve the old format as baseline input if supported.
src/frontend/modules.rsLoader::file already receives a package index. parse/parse_buffered create a lexer/parser without an edition. Pass the owning edition through both buffered parsing paths and retain it wherever interpretation remains deferred.
Lexer, parser and checkerGate only the intended source rule and normalize it; audit generic specialization, interfaces and source-only entry points for caller-dependent interpretation.
src/project/fingerprint.rs and compiler cachesTarget fingerprints cover sources, selected graph fields and compiler identity, but currently have no edition. Include effective editions in relevant build/cache inputs and version affected formats.
CLI, IDE and test toolingAgree on package discovery, explicit standalone selection, generated code and diagnostics. Migration tooling is additional work, not supplied by storing the field.

Module loading currently deduplicates by ModuleSource. If one physical file can be assigned to different package contexts, implementation must either reject conflicting ownership/editions or include that context in module identity. Taking whichever edition happened to load first would be incorrect.

The inexpensive part is plumbing a value to the parser. The larger recurring cost is maintaining old rules and proving that their combinations still work. A syntax-only first edition should not need a second backend, runtime or dependency resolver. Rules that spread throughout checking or require adapters at package boundaries are a reason to reconsider the change, not evidence that editions in general are easy.

Evidence required before adoption

An implementation experiment should use a real proposed incompatible rule and prove old-to-new and new-to-old calls, including a diamond graph. It should cover public names that collide with new keywords, shared class/enum values, protocol satisfaction, borrows and cross-package generic specialization. Merely accepting two manifest values does not establish mixed-edition support.

Check omitted/unknown editions, independent workspace members, test harnesses, embedded std, interfaces, direct compiler/API use and IDE diagnostics. Warm builds must invalidate after changing only a root or dependency edition; graph round trips must retain it. Run existing baseline tests unchanged and measure compile-time/cache effects before claiming negligible cost.

Migration should first make source valid in both editions, then change the manifest and rerun checks/tests. Rust’s migration workflow uses compatibility diagnostics and verified edits; it also documents limits for generated code and configurations. Plenty could begin with diagnostics and a short manual migration guide for a small change. Automated public-API rewrites or claims of complete migration coverage need substantially more evidence.

Before committing, choose the first change, edition naming, baseline/default policy, supported-edition lifetime, and public-name escape policy. With no pressing incompatible change, retaining one language mode is the simpler choice. Scheduling and priority remain solely in the backlog.

Plenty toolchain manager

Status: Not implemented. Proposed under B17. The tool is named plenty-manager, written in Plenty, and intended for a new, separate repository named plenty-manager in the plentylang organization on GitHub. This proposal does not create that repository. Commands, configuration and storage below are proposed interfaces.

Motivation and existing behavior

Installing, upgrading and switching compilers should not require editing shell startup files for every release. A project should be able to commit the exact compiler version used by its developers and CI, while older projects keep their own versions.

Today, release archives are relocatable and can coexist in versioned directories. The first compiler on PATH runs. The compiler embeds ordinary standard-library sources and runtime code; Linux archives also carry the native engines in bin/. These properties make a manager possible without changing how compilation works.

The package manifest’s plenty-version is a minimum supported compiler version, not a selection instruction. Preserve that meaning. Dependency locking, compiler selection and proposed language editions solve different problems.

Rustup’s toolchain selection is the precedent: executable proxies select an installation before Cargo or the compiler runs. Adopt the small useful subset, not its complete component, channel or override system.

Ownership and implementation

Implement the manager and its executable proxies in Plenty in the separate plentylang/plenty-manager repository. Keep compiler release production and archive compatibility in the compiler repository. Agree on a versioned release metadata contract between them; neither repository should need to import the other’s implementation.

Distribute native manager executables, so installing Plenty does not first require Plenty, Rust, Cargo or a source checkout. Maintainers build the manager with an explicitly pinned, already published Plenty compiler. Retain documented manual archive installation as the bootstrap and recovery path. A manager release must not require the compiler release it is intended to install.

Use Plenty’s existing process, filesystem, HTTP/TLS, hashing and configuration facilities where suitable. Audit the required APIs and platform behavior before implementation; this proposal is not evidence that secure extraction, file locking or executable replacement already has every needed primitive. Keep manageable application logic in Plenty. Security-sensitive verification may use existing audited libraries through standard-library facilities rather than new cryptographic implementations. Record discovered compiler/library defects in the issues register, not as silent installer workarounds.

First user-facing interface

Illustrative versions below are placeholders, not published release claims.

plenty-manager install X.Y.Z
plenty-manager list
plenty-manager default X.Y.Z
plenty-manager show
plenty-manager run X.Y.Z -- plenty build
plenty-manager remove X.Y.Z
plenty-manager update

install fetches an exact version for the native host and leaves all existing versions intact. It does not change an existing default. On a fresh installation, setting the default is an explicit step. list shows installed versions and the default; show reports the effective version, selection source and executable path for the current directory, including a missing selected installation.

run explicitly selects an installed version for plenty or plentyc, with remaining arguments forwarded literally. It does not invoke a shell or search PATH for the selected compiler. Unknown tool names are rejected.

update explicitly discovers and installs the newest stable compiler supported on the host, then makes it the user default. It reports the resolved exact version, preserves previous installations, and never edits project pins. It is the only initial interface that resolves a moving release name. Interrupted or failed updates leave the old default working. Manager self-update is not part of this command or the initial scope.

remove rejects removal of the current default until another default is set. It cannot discover every project pin: warn about that limitation and report a clear missing-toolchain error when such a project is later used. Removal must not damage running tools; platforms that cannot remove an in-use installation must fail safely or defer cleanup.

Project pin and selection

Use a separate, committed file rather than overloading the library’s minimum compatibility requirement:

# plenty-toolchain.toml — proposed
[toolchain]
version = "X.Y.Z"

The initial schema accepts one exact published MAJOR.MINOR.PATCH version. Reject ranges, moving channels, paths, unknown keys and malformed files rather than guessing. Pre-release channels and local development toolchains are separate extensions, not implicit initial behavior.

A proxy selects, in order:

  1. An explicit plenty-manager run VERSION -- TOOL ... selection.
  2. PLENTY_TOOLCHAIN, containing an exact version.
  3. The nearest plenty-toolchain.toml, searching from the caller’s current directory upward to the filesystem root.
  4. The user’s configured default.

Invalid higher-priority selections are errors, not reasons to fall back. With no selection, explain how to install and select a default. With a selection that is not installed, show the exact plenty-manager install VERSION command. Neither proxy invocation nor a project file causes a network request.

Discovery deliberately follows the working directory, not a source filename or --manifest-path. For a project addressed from elsewhere, enter its directory or use an explicit selection. Workspace members normally share a root pin; a nearer member pin overrides it when invoked there. A single compilation still uses one compiler for the complete dependency graph. Dependency manifests and pins never switch that compiler during resolution.

A newer compiler remains free to reject a package for other compatibility reasons. The selected compiler continues to enforce every package’s minimum plenty-version; the manager does not solve a second dependency graph or silently select a newer compiler to satisfy it.

Proxy and installation layout

Add one per-user manager bin directory to PATH during explicit setup. It contains plenty-manager and proxies named plenty and plentyc (with native executable suffixes where required). Managed releases live outside that namespace, keyed by exact version and native host triple. The proxy invokes the absolute compiler path, never itself through PATH.

Choose platform-appropriate user data/configuration directories, with an explicit PLENTY_MANAGER_HOME override for CI and portable installations. Keep downloaded staging files distinct from complete installations. Do not write manager state inside project checkouts, require root, or modify shell startup files without consent. Keep archive-based installations usable outside the manager.

Forward arguments, working directory, environment, streams and process results without reinterpretation. On Unix, prefer process replacement for transparent signal handling; where a supervising process is necessary, preserve status, interruption and child lifetime. --version is handled by the selected compiler; plenty-manager show explains the dispatch. Preserve the explicit selection in child environments so nested tool invocations cannot accidentally switch back to a project pin or default.

Install the compiler, embedded runtime and standard library as one release. Optional native engines must match the release and host compatibility contract; do not reuse an arbitrary engine from another installation. Initial delivery may omit engine management, but must say so and retain documented manual engine installation. The manager does not install system linkers, SDKs, libc development files or unsupported cross-compilation targets.

Download trust, persistence and failure behavior

Download only from the configured official distribution endpoint over verified TLS. Release metadata must identify its schema version, exact compiler version, host triple, archive URL, size and cryptographic digest. Resolve moving updates to that immutable identity before installation. Reject unsupported hosts before downloading; available compiler archives do not prove that the Plenty-written manager is qualified on the same platform.

A checksum served beside an archive detects corruption but is not independent authentication. Before shipping automatic installation, settle authenticated release metadata or signed artifacts, trust-root distribution and rotation, and downgrade/replay policy. Do not describe HTTPS plus adjacent checksums as a complete supply-chain security design. Use reviewed verification primitives.

Bound metadata and download sizes. Treat archives as untrusted: reject path traversal, absolute paths, escaping links, duplicate/conflicting entries and unexpected file types; validate the expected layout before publication. Never execute downloaded hooks. Avoid shell construction from versions, filenames or URLs, and do not execute the installed compiler as a substitute for verification.

Use private staging directories, per-installation locking and atomic publication on the destination filesystem. Update defaults atomically only after successful installation. Concurrent installs must agree on one complete result; failed verification, disk exhaustion or interruption must never expose a partial compiler. Retain enough metadata to identify each installed artifact and reject conflicting content under an existing release identity. Define safe recovery for stale locks and staging files without deleting active work.

Dispatch, inspection, removal and execution work offline. Explicit installation and update fail clearly without network access; no command silently substitutes another compiler. A compiler pin improves repeatability but does not pin Git sources, native engines, linker/SDK versions or the system C library. See build reproducibility.

Delivery and acceptance

Keep priority and scheduling solely in B17. Before implementation, agree on the release-metadata trust contract, supported manager hosts, storage layout and optional-engine scope. Creating the separate GitHub repository and publishing binaries are later, explicit operations, not part of adopting this document.

The first delivery should prove exact-version installation, side-by-side use, proxies, project pins, explicit overrides, default changes, explicit updates, removal and recovery. Require integration tests for:

  • Two compiler releases used by separate projects without changing PATH, and manual archive installations still working.
  • Selection precedence, malformed/missing pins, workspace/member discovery, dependency-pin isolation and calls from outside a project directory.
  • Offline invocation with network access prohibited, including nested tools.
  • Argument fidelity, spaces/Unicode in paths, supported native argument bytes, exit status, signals and interruption on each admitted host.
  • Corrupt downloads, invalid trust metadata, hostile archives, unsupported hosts, disk exhaustion, concurrent installation/removal and interrupted update.
  • A downloaded manager installing its build compiler from a clean machine without Plenty or Rust already installed, plus manual recovery without it.
  • Optional engine/version isolation if engine management is included.

Document platform qualification, limitations and the compiler version used to build each manager release. Defer nightly/beta channels, target/component matrices, directory-override databases, automatic fetching on dispatch, self-update, arbitrary mirrors and automatic project-pin rewriting. A separate manager keeps these concerns out of the compiler while preserving a useful, small initial tool.

Faster linking

Status: Not implemented. This page records a measured study of what the compiler can do to make linking faster, and the levers it recommends. Linking and the compilation cache own the current contract. Compile and link latency is tracked under B19; no levers here are scheduled.

Summary

Linking costs a near-constant 18–19 ms per build, whatever the program’s size. That is about 86% of a warm --compile of hello world, 27% of an edit loop on examples/semantic_search, and 7% of the compile_bench 1000-unit workload.

Plenty’s own object adds only 0–2 ms of that, even at 10,000 sections. The remaining cost is in how the compiler drives the linker and supplies the runtime: the cc driver chain, writing the runtime archive on every link, reading that archive, and the linker’s own startup. Changing the object’s format does not pay; changing the link procedure does.

Measured (M) or estimated (E). Times are hyperfine medians pinned to cores 0–7 on x86-64 Fedora with gcc 16.2, mold 2.40.4, LLD 23.1.1, GNU ld 2.46.1 and wild 0.11.0. Differences under about 1 ms are noise.

#LeverSavingCost
1Cache the linked executable next to the cached object. On a whole-program cache hit, run or copy the stored executable.E: unchanged hello rerun 20.5 → about 2.5 ms (−88%); semantic_search 23.5 → about 5 ms (−80%). Only unchanged reruns benefit.Low. Extends the whole-object cache (issue #317 stage 1).
2Run the linker directly instead of through cc. Run cc -### once during the existing linker probe, store the resolved command, then run ld.mold or ld.lld with it.M: −3.5 ms per link. The cc → collect2 → linker chain adds 3–4.5 ms, including about 1.5 ms loading the LTO plugin. Clang as cc costs 12 ms more than gcc.Medium. The replayed command must match the driver’s. Fall back to cc when parsing fails or --linker/--link-arg is given. macOS and AArch64 need their own replay.
3Extract the 17 MB runtime archive once into the cache directory, keyed by compiler build, instead of writing it to a temporary directory on every link.M: −2 ms per link; peak memory −16 MB (hello 28 → 11.6 MB).Low. Keep temporary extraction when the cache is off.
4Strip debug sections and embedded LLVM bitcode from the embedded runtime archive at compiler build time.M: archive 17.4 → 9.9 MB; compiler binary −7.5 MB. No link-time change.Low. --no-strip output loses runtime debug information unless an unstripped archive is kept for it.
5Link run-only builds (plenty FILE, plenty test) against a shared runtime library. Not for --compile.M: mold link 12.5 → 8.5 ms (hello), 14.3 → 9.3 ms (semantic_search); each run starts 0.3 ms slower.Medium to high. A second runtime artifact, and run-only executables link differently from shipped ones. Only after 1–3.
6Track wild as a future linker.M: direct link 5.4 / 7.6 / 6.1 ms against mold’s 12.5 / 14.3 / 13.7 (hello / semantic_search / bench1000).Not ready: its --defsym cannot pull archive members, and it removes far less dead code (hello 393 KB against 116 KB), failing size budgets.

Levers 2 and 3 were prototyped together (M):

ProgramWarm --compileEdit loop
hello20.5 → 15.2 ms (−26%)–
actor_echo–55 → 47.7 ms (−13%)
semantic_search23.5 → 17.8 ms (−24%)69 → 62 ms (−10%)
bench100026.5 → 22.2 ms (−16%)263 → 257 ms (−2.5%)

After 2 and 3, mold still takes about 11 ms for hello: about 5 ms is the cost of linking an empty C program (crt files, libc, libm, libgcc_s), about 4 ms is process startup and its thread pool, and about 6 ms is the runtime archive (419 members, 24,632 sections). Only levers 5 and 6 reduce that further.

Measurements

End to end (M, ms). The edit loop disables the whole-program cache and keeps the function cache warm.

ProgramEmit objectWarm --compileEdit loopCache offLink share of edit loop
hello2.421.322.228.486%
actor_echo3.621.952.666.735%
semantic_search4.723.970.891.127%
http_server_bench4.622.782.4118.922%
bench10007.927.3261.7345.17%

Link only, by linker (M, ms). The first three go through cc -fuse-ld=X, as Plenty does today; the rest run the linker directly.

Programbfd (cc)mold (cc)lld (cc)bfdmoldlldwild
empty C main14.011.8–18.59.08.44.3
hello58.815.616.450.612.413.05.4
semantic_search113.317.920.1110.315.417.37.6
bench100066.416.217.559.414.215.56.1

On one core, for parallel plenty test runs (M): mold 16.9, lld 16.3, wild 8.9 and bfd 46.9 ms per hello link. Unpinned, mold uses 58 ms of CPU for 12 ms of wall time, which supports setting PLENTY_LINKER_THREADS=1 for parallel runs.

What the linker receives (M):

hello.osemantic_search.obench1000.o
Size12.7 KB898 KB3.9 MB
Sections752,04210,052
R_X86_64_PLT3246,55816,005
R_X86_64_PC3243,1139,012
R_X86_64_644798426
R_X86_64_GOTPCREL0120

The objects have no COMDAT groups, mergeable sections, debug information or .eh_frame. The runtime archive is 17.4 MB in 419 members: 5.8 MB of debug information and its relocations, 1.7 MB of embedded bitcode, 3.4 MB of code, 1.0 MB of read-only data and 0.3 MB of .eh_frame. Executables are not position-independent on this system and carry 139 dynamic relocations.

mold’s phases for hello (M, ms): 11 in total; reading inputs 5; symbol work before copying 5 (merging strings 2–3, dead-code removal 0, relocation scan 0–1); creating the output 1; copying 0.

Levers that do not pay

LeverResult (M unless marked)
Linker flags (--build-id=none, -z norelro, no --eh-frame-hdr)±0.5 ms
Static linking10 ms slower: it pulls in static glibc
Pre-linking the runtime into one object with ld -r, stripped or not0.5–1.5 ms slower: section count, not member count, is the cost
Removing duplicate -l stub librariesNo change
Linker thread counts4 ≈ 8 ≈ 16 threads for one link; the default is fine
PIE instead of non-PIE3,000 more relocations, 0.3 ms; default-PIE distributions are no slower
Removing unreachable code in the compilerAt most 1 ms of link time. Still worth doing for compile time: most of bench1000’s 262 ms edit loop compiles functions nothing calls
Several objects for parallel linkersE: none; mold and lld already parallelize within one object

Common object-format suggestions

These ideas help some toolchains; none applies to Plenty’s output.

  • Deduplicating types before writing the object. In C++ the linker merges repeated COMDAT groups and debug types from many translation units. Plenty compiles one unit and emits each generic instance once, so its objects have no COMDAT groups, debug types or duplicate symbols. Identical data sections total 10 KB in semantic_search. String merging time comes from the runtime’s members. E: under 0.1 ms.
  • Page-aligned sections. bfd, lld, mold and wild all copy each input section into the output and patch it; none maps input sections directly, so input alignment has no effect. Larger alignment would only pad the output.
  • Fewer, larger sections. Prototyped with one text section per object: link time changed by ±1 ms and objects were 15–17% smaller, but --gc-sections could no longer drop dead code, so bench1000’s executable grew from 341 KB to 2.39 MB. Keep one section per function. The runtime’s per-operation symbols (issue #501) are in the archive and unaffected.
  • Pre-computed relocations. Runtime calls are already direct (issue #372), executables are not position-independent here, and the relocation scan takes 0–1 ms for 25,000 relocations. Resolving calls at emit time would need one section, losing dead-code removal. E: under 1 ms.
  • Dependency hints for parallel linkers. No ELF linker reads such hints and ELF has no convention for them. lld’s --symbol-ordering-file and mold’s --section-order control output layout, not link speed.

Reproducibility and portability

B17 requires reproducible output.

  • Levers 1 and 2 leave output bytes unchanged if the replayed command is identical; an object-identity test should check it. The cache key must include the link command and linker identity, which the linker probe already records. Replaying cc -### captures distribution differences such as multiarch paths and default PIE instead of hard-coding them.
  • Levers 3 and 4 do not change stripped executables. Lever 4 removes runtime debug information from --no-strip output.
  • Levers 5 and 6 change output bytes. They suit run-only builds, or a separately pinned linker input.
  • macOS (ld64) and AArch64 need their own versions of lever 2, with cc as the fallback.

C interop and dynamic libraries

Status: Partly implemented. Researched 2026-10-05. C imports through trusted interface files, static and shared linking, C exports, and typed resident runtime loading are implemented. The interop reference and linking own the contract; the C libraries lessons teach it.

Shipped differently: the foreign library, symbol, bind, and ptr[const c_char] sketches below are not Plenty syntax. A .plentyi file declares opaque Name and extern def f(...) -> T = "symbol". Pointers are &T and &mut T, and text crosses as &str as utf8 or &str as c_string. There are no built-in c_int aliases. Linking is configured on the command line (--link-arg, --linker), not in source.

Remaining: unloading, callbacks, C records, and header-assisted bindings (B28); text and byte buffers, views, and broader error types (B27); more targets (B18).

The native-platform assessment expands the target/ABI requirements; a GCC backend would be a separate compiler backend choice, not automatic OS or FFI portability.

Plenty should support ordinary C libraries without making every application author become an FFI expert. A binding author supplies an explicit, trusted contract; application code uses typed functions, ownership, borrowing, Result, and automatic cleanup. The contract belongs in an importable interface module. Static archives, linked shared libraries, and libraries opened at runtime should use the same contract representation.

This does not require weakening ordinary Plenty ownership rules. It requires a well-defined place where the compiler trusts external claims. A mistaken contract or a buggy native library can still corrupt memory; annotations cannot prove C implementation behavior. Ease of use comes from reusable bindings and generated adapters, rather than pretending that the boundary is checked end to end.

What existing systems teach us

Rust supports C shared-library linking and producing cdylib artifacts. Loading a library at runtime is also possible, for example through libloading. The important limitation is that signatures and ownership promises must be correct; dynamic libraries are not fundamentally incompatible with Rust safety. We should avoid attributing ecosystem usage patterns to an inability to use C DLLs. Rust linkage reference, libloading Library API.

Cython’s .pxd files demonstrate the value of reusable declaration modules and compile-time namespacing. They are a useful aesthetic model for Plenty interface files, though Plenty needs additional ownership and lifetime contracts. Cython external C declarations.

Zig provides C type primitives, C-compatible records, exports, and header translation. Its documentation explicitly requires matching target and C compiler flags during translation: long, packing, enum representations, and other ABI details depend on them. Reusing a mature C frontend is attractive; copying the surface syntax alone would not provide this machinery. zig translate-c produces Zig code, not a ready-made Cranelift interface or ownership specification. Zig C interop.

ctypes offers a convenient model for loading libraries, typed function lookup, and conversions. It also documents that invalid non-null pointers can crash the process. Plenty should retain the convenience while resolving known signatures ahead of time and checking its side of each contract. Python ctypes.

CO3 assessment (2026-10-22)

CO3: The Language of FFI demonstrates native-facing declarations lowered to C companion types, conversion adapters, explicit ownership transfer, and finite generic dispatch. Its example switches a consumer between an ordinary dependency and C ABI bindings through build configuration. This supports Plenty’s existing separation of the useful language API from its native calling representation; it is not evidence that arbitrary C libraries share one ownership convention.

The article’s ambition to represent every source type is broader than Plenty’s near-term needs. It explicitly identifies missing captured-closure support, import/header verification, and C header generation. Treat these as design ideas, not demonstrated solutions. The CO3 0.8.2 API documents generated companions, conversion checks, and configurable conversion failure handling; this assessment does not audit their implementation or establish cross-version ABI compatibility. Plenty already generates headers and embedded contracts for its supported subset.

The useful extension is to make conversion policy as explicit as ownership:

Boundary operationProposed Plenty policy
BorrowKeep the owner alive; specify extent, access and non-retention. No implicit copy.
Copy into receiver storageOpt into a conversion that leaves the original owned value with its sender; expose allocation/validation failure and release partially converted values.
TransferMove the owner with its exact originating release operation; describe consumption on both success and failure.

CO3 defaults owned values such as vectors to borrowing at the boundary and cloning in the receiver. Plenty should borrow the distinction, not that implicit default: hidden allocation and copying would obscure fallibility and cost. B27 buffer adapters should distinguish a borrowed byte view from an independently owned copy without exporting Plenty collection layouts. Copying may solve an allocator mismatch, but cannot manufacture validity for a dangling source or justify reading storage while foreign code mutates it.

Header-assisted work in B28 should also offer conformance checking, not only binding generation: compare the declared native signature and record layout with the library’s authoritative header under the exact target, defines, include paths, packing and compiler options. Expose the lowered C declaration and conversions for binding review. A mismatch must identify the native argument/layout at fault; a match establishes ABI agreement, not ownership, retention or error semantics. Keep this an optional tool using a mature C frontend, outside ordinary builds.

For future callbacks, reuse ordinary import/export conversion and failure rules for arguments and results in both directions. A generated C entry thunk is distinct from an ordinary Plenty function pointer or a captured closure. Begin with the synchronous context-pointer model below; specify context ownership, cleanup and failure behavior before adding captured or retained callbacks. Do not adopt CO3’s default conversion-failure panic as a recoverable Plenty Result: representation errors need an explicit status policy where supported, while traps still terminate. Checks of tags, lengths or encoding cannot establish pointer liveness, provenance or the absence of conflicting foreign aliases.

Separate source visibility, binary linkage, and loading

These are three independent decisions:

ConcernProposed meaning
pubAnother Plenty module may name the declaration.
C exportEmit an explicitly named, C-compatible externally visible symbol.
Static dependencyLink an archive into the application or library.
Shared dependencyLink against a shared library; the platform loader loads it.
Runtime loadingOpen a selected library at runtime and resolve a known interface.

Making a function pub must not automatically freeze its native ABI. Ordinary Plenty functions can keep compiler-private conventions and symbol mangling. Export declarations generate C ABI entry adapters and, eventually, C headers. Likewise, source imports should not have incidental side effects that load DLLs. Runtime loading is an explicit operation returning Result.

The accepted generated export contract design adds versioned Plenty metadata alongside the ordinary C ABI. Adapters, metadata, and C headers with precise ownership, borrowing, cleanup, and error comments come from one checked export representation. The initial export subset implements numeric and Result APIs, owned and borrowed handles, and typed loading.

Interface files use the .plentyi suffix and are imported through the same absolute module namespace as Plenty source. The compiler distinguishes them by suffix; it does not decide how to interpret a file from its contents.

Two layers of declarations

Keep the actual C signature separate from the ergonomic Plenty signature. A C function returning an integer and writing through an output pointer should not be falsely declared as returning a native Plenty Result. The compiler generates the conversion between these layers.

Illustrative interface sketch, deliberately not executable:

# graphics.plentyi — proposed concepts, grammar undecided
foreign library graphics:
    abi = "C"
    opaque Image

    symbol image_load(path: ptr[const c_char], out: ptr[ptr[Image]]) -> c_int
    symbol image_destroy(image: ptr[Image]) -> ()

    # A declarative adapter, not the actual C function signature:
    pub bind load(path: str) -> Result[owned Image, ImageError]:
        call image_load(c_string(path), out image)
        success when return == 0
        on success own image with image_destroy
        on failure image is null

An initial implementation should support a smaller contract grammar and handwritten adapter bodies before attempting this full declarative convenience. ptr, const, owned, symbol, and bind here name concepts, not agreed Plenty keywords.

A pointer-to-immutable-C-data is not automatically a Plenty immutable reference: some other C alias may mutate it. The binding must establish that the stronger Plenty borrowing guarantee holds, or copy the data into a Plenty value. const alone neither proves this guarantee nor says how long memory remains valid.

What an interface contract must express

TopicInformation the compiler or adapter needs
Native ABITarget, calling convention, symbol name, scalar widths, argument/result lowering.
Buffer extentPointer plus length/capacity, units, alignment, and initialized extent.
NullabilityWhether null is permitted and whether it means absence, empty buffer, or failure.
AccessRead/write permissions, exclusivity where required, and whether native code retains an alias.
LifetimeCall-only access, borrowing from a named owner, or independent owned result.
TransferWho owns an input or output on both success and failure paths.
ReleaseExact destructor/deallocator, library instance, allocator context, size/alignment when needed.
FailureStatus mapping, immediate errno capture where specified, initialized outputs on failure.
CallbacksSignature, borrowed/retained context, registration lifetime, reentrancy, thread of invocation.
ThreadsThread affinity, concurrent-call permissions, movable/shareable handles.
Control flowNo unwinding or nonlocal jump through Plenty frames.

Useful default: pointer access lasts only for the call and native code does not retain the pointer. This must be an explicit binding-author promise, not an inference from an unannotated C header. Contracts absent from the declaration leave a raw operation requiring an explicit trusted/unsafe adapter.

Ownership transfer on failure matters. If a C call consumes a handle even when it returns an error, the source value remains moved on both paths. If a call accepts ownership only on success, an adapter must preserve or return the original owner on failure. A single owned marker without conditional semantics is insufficient.

Borrowed results initially require a named owner and simple lifetime relation, such as “valid while this image remains borrowed.” Reject relationships the checker cannot express rather than silently extending the lifetime. This is a reason to design FFI contracts alongside public borrowing, not after freezing it.

Representation and allocation

Normal Plenty class, enum, str, list, dict, set, Option, and Result representations remain private to the compiler/runtime. They should never become C-compatible merely because a function is exported. Current managed objects have ownership headers and runtime metadata; C code does not know those conventions.

Provide a separate opt-in foreign record representation later, with target C layout, fixed fields, alignment rules, and compile-time layout checks. Initially support scalar values and pointers to opaque objects, then plain records passed by pointer. Records passed by value, unions, bitfields, packed layouts, variadic calls, and platform-specific calling conventions need deliberate ABI work.

Foreign type aliases such as c_int, c_long, c_size_t, and c_char belong in an explicit FFI module. They describe target C types; they do not reintroduce ambiguous native Plenty int. C enum values and boolean representations also need declared conversion rules rather than unchecked reinterpretation.

The single native str stays UTF-8 with a length and embedded-NUL support. A C string adapter validates or rejects embedded NUL, creates a terminated buffer when required, and reports allocation failure. Returned bytes need explicit encoding, length, and ownership. No automatic assumption that arbitrary C bytes are valid UTF-8. A byte-buffer/view API should precede useful buffer-oriented FFI.

Never free a foreign allocation using Plenty’s global allocator by default. Owned foreign handles carry their exact release operation and any required allocator context. Exported APIs provide matching release functions for their own outputs. Crossing DLL boundaries on systems with different runtime heaps makes this particularly important. Custom allocators and their lifetimes should use this same provenance model, rather than a separate incompatible mechanism.

A fallible adapter must release partially acquired foreign resources if subsequent conversion fails. Conversely, an allocation error result cannot reliably rescue a native library that itself aborts on allocation failure. Bindings must document which failures can actually be returned. Native error channels cannot be guessed from names or inferred solely from C signatures.

Dynamic library lifetime and callbacks

The easiest first runtime-loading policy is to keep successfully loaded libraries resident until process exit. That avoids dangling function pointers, borrowed static data, callbacks, destructors, and native background threads after unload. Libraries can still be loaded lazily and report missing symbols as Result errors. Unload support can follow once all these lifetime relationships are representable.

If unloading is later supported, the library owner must outlive every function handle, owned foreign value, destructor thunk, callback registration, and borrowed piece of static data. Merely finishing the last direct function call is not enough. Even a registration destructor may need to wait for in-flight callbacks before releasing its context. Loader APIs themselves can run native initialization code; opening a library is not a passive metadata read. libloading loading and symbol contracts.

Start callbacks with synchronous, same-thread calls and an explicit context pointer. Reentrant callbacks must obey active borrows; they cannot acquire another mutable borrow of the object already lent to C. Retained or foreign-thread callbacks require a later registration/lifetime model and thread-safe runtime support. Do not mark handles transferable between threads just because they are represented by pointers.

Foreign exceptions, longjmp, and Rust unwinding through Plenty frames are outside the initial contract. C++ adapters catch exceptions on their side and return a status. Plenty currently terminates on runtime traps and its Rust runtime uses abort-on-panic; library exports initially share this limitation. A status-returning wrapper does not turn those aborts into recoverable errors. Rust FFI and unwinding.

Compiler and runtime implications

Repository evidence at proposal time:

  • src/codegen.rs emits native object files, enables position independence, uses CallConv::Tail for Plenty functions, and explicitly uses SystemV for runtime calls. Foreign calls need target-derived C conventions; hard-coded System V is not a portable DLL implementation.
  • build.rs embeds a host-target runtime archive. Shared-library output needs a reviewed runtime link mode, exported-symbol policy, and startup path without the executable’s main/plenty_main dependency.
  • plenty-runtime/README.md documents atomic counts, thread-local destruction, and compiler-private 64-bit slots. Native threads cannot safely share current Plenty objects merely because an FFI function spawns them.

Cranelift supplies native calling conventions and indirect calls. Its Tail convention is not a stable external ABI. Generate C entry/exit adapters around Plenty functions and preserve the internal convention. C aggregate classification and target layout remain frontend responsibilities; an IR function signature alone does not translate arbitrary C headers. Cranelift CallConv.

Do not require a JIT or libffi for runtime loading of statically known signatures. Resolve addresses once, then emit typed indirect calls in the AOT program. Arbitrary runtime-discovered signatures are outside scope. Keep header translation as an optional, cacheable binding-generation tool so ordinary Plenty builds remain fast and do not depend on a bundled C frontend.

Suggested delivery sequence

  1. Reserve distinct internal, C import, and C export ABI concepts now. Keep native object layouts private and module visibility separate from binary exports.
  2. Add target-correct scalar/pointer C imports, static/shared link configuration, explicit raw adapters, opaque owned handles, and call-scoped buffer contracts.
  3. Add C exports and static/shared-library outputs with generated headers, matching destroy functions, versioned symbol naming, and C integration tests.
  4. Add explicit runtime loading of known interfaces, initially without unloading.
  5. Add record layout, retained callbacks, header-assisted binding generation, and more expressive borrowed results as independent extensions.

These stages record the original progression; the status and reference links above identify what has shipped. Apply the CO3 lessons within the existing B27 buffer/view work and B28 tooling/callback work, without making universal type lowering or generic dispatch prerequisites. The export qualification strategy defines the evidence required for each additional boundary type.

Every stage needs native integration tests built against a small C fixture: successful/error transfers, partial initialization, matching destruction, embedded-NUL rejection, library lifetime, and reentrant callbacks. Validate each supported ABI with actual C callers/callees; metadata parsing tests alone are insufficient.

Interface naming and format, trusted-boundary syntax, and shared-library exports are settled. Further host ABIs (B18) and borrowed buffer types (B27) remain open. None requires adopting Python object layouts, a garbage collector, Rust trait lookup, or a public Plenty native ABI.

Using Rust crates in Plenty

Status: Not implemented; comparative prototype research (2026-10-23). B17 owns dependency tooling; backend alternatives relate to B20. Current packages, linking and runtime packaging are unchanged. The prototypes are experiments, not a new supported compiler mode.

Recommendation

The most tractable way to let Plenty projects use Rust crates is generated C-ABI adapters over a Cargo-built Rust dependency graph, with Rust-owned opaque objects and checked Plenty-facing bindings. Users select Rust APIs, not C APIs; upstream crate authors need neither rewrite their implementation nor provide C exports. Cargo and rustc still consume ordinary rlib build artifacts.

This broadens the original investigation beyond avoiding C ABI. Executed prototypes show that Rust ABI does not eliminate the ownership/marshalling layer for Plenty’s different value representations, and it adds compiler-version and lowering obligations. A Rust-source backend removes that boundary for supported types, but preserving the whole Plenty language/runtime is a much larger job. The recommendation concerns implementation cost and maintenance, not a claimed universal performance winner.

Keep native Rust ABI as an opt-in research route. The separate Rust-source backend proposal explores Rust-consumable libraries and native interoperability if this becomes a central goal. Evidence and measurements describe all three; the project/value contract sets the intended user-facing boundary.

What the three prototypes establish

All paths execute one source workload containing scalar calls, Unicode/NUL text, a generic identity operation, a Rust regex owner with query and destruction, and a local Rust Future polled from Pending to Ready. Source is checked by the existing Plenty frontend against generated binding signatures, then a small restricted parser lowers its actual operations. Both native routes emit a real Cranelift object. The third emits actual Rust source, not a substituted result.

RouteDemonstratedRemaining production work
Rust-ABI adaptersReal Cranelift calls to pinned scalar/pointer Rust-ABI functions, debug/release adapter builds; separate rustc-private ABI queries work.Connect matched compiler ABI information to lowering, pin compiler/helper identities, validate every admitted type and upgrade.
Generated C adaptersSame selected Rust APIs and ownership lifecycle, generated adapters and caller declarations, stable Rust compiler interfaces.Integrate generated modules/contracts with Plenty project tooling, error/allocation policy and runtime linking.
Plenty-to-Rust sourceChecked restricted Plenty calls and arithmetic become Rust calls, native Rust owners and borrows; no opaque-handle adapter boxes.Backend for checked IR, complete Plenty representations/semantics, std/runtime parity, diagnostics and compile-time qualification.

A separate pinned nightly helper queried FnAbi: i64 was direct, &str a pair, and a 24-byte repr(C) record passed indirectly by pointer under Rust ABI but on the stack under C ABI. repr(C) fixes layout, not the function calling convention. This makes compiler-assisted Rust ABI handling plausible, while showing why the successful scalar calls must not be generalized. The helper observes a different compiler build from the stable executable tests and is not connected to their Cranelift lowering.

rlib consumption and final linking

An rlib contains compiler-specific metadata and code, potentially LLVM bitcode. It is not an API schema or a bundle of all upstream Rust dependencies. Generic instances may be generated downstream; rustc must participate. Metadata from another compiler version is rejected. Libraries and metadata, monomorphization.

Rust’s supported mixed-language linking routes are a rustc-owned final link or one aggregate Rust staticlib linked by a foreign driver. Passing arbitrary rlibs directly to a foreign linker remains unsupported. Artifact format and calling convention are independent: C-ABI functions can live in rlibs. Both adapter prototypes use rustc to link their rlibs and the Cranelift object. Linkage.

For production, prefer a Cargo-managed final Rust target that includes the generated adapters, dependency graph, rebuilt Plenty runtime and application object. Alternatively produce one aggregate staticlib for today’s linker. Neither route means converting upstream crates to C libraries individually.

Plenty manifest + selected Rust APIs / small Rust companion module
                     |
                generated bindings
               /                  \
Plenty checked foreign owners     Rust C-ABI adapters
          |                       ordinary Rust calls
    Cranelift object              Cargo dependency rlibs
               \                  /
             one runtime graph, rustc final link

Generate exact-width scalar/pointer/length interfaces and explicit output/error contracts. Keep complex Rust values opaque; Rust constructs, borrows, mutates and drops them. Instantiate selected generic operations inside Rust. Plenty enforces owner moves, exclusive/shared loans and drop timing across calls. Binding metadata must specify these rules, not merely the machine signature. Automatic discovery can assist API selection, but Cargo metadata and symbol lists are insufficient to derive every lifetime/effect contract safely.

A companion .rs module is a practical escape hatch for trait-heavy APIs, callbacks and async executors. It can present a small reusable operation to Plenty while calling an unchanged crate. Publish/cache adapters independently of individual applications where their API and crate-version contract allows; do not make every user write unsafe pointer glue. Unsupported APIs should give a precise diagnostic and suggest a facade, not silently erase types/errors.

The prototypes’ async operation is a future with explicit polling, not async Plenty syntax or a reactor integration. C ABI does not prevent async use: an opaque operation can expose start/poll/cancel with owned state. Real executors, wakers, callbacks, retained buffers and cancellation completion need explicit lifecycle contracts before general async crate support.

Runtime and compiler integration

Today src/codegen.rs::link_object extracts an embedded staticlib and src/toolchain.rs::link invokes a cc driver. build.rs::build_runtime builds Rust std, vendored dependencies and an abort-on-panic runtime into that archive. Appending another independent Rust staticlib risks conflicting allocator, panic/std and native library implementations.

For Rust-enabled projects, ship the matching Plenty runtime source/build recipe and rebuild it in the same Rust graph. Reuse the runtime variant without C main and a Rust launcher, preserving Plenty application initialization and exit status; or produce one aggregate staticlib with the existing startup contract. Keep today’s embedded-runtime mode for pure Plenty projects. None of the three experiments integrates the real Plenty runtime into its final executable.

Project resolution and caching belong in src/project.rs; imports receive a resolved graph rather than fetching during type checking. Add foreign-owner types and loan/drop operations before backend lowering. Map generated Rust diagnostics back to binding requests. Stable C ABI removes dependence on compiler-private lowering, not the need for compatible Rust artifacts within each Cargo build.

The Rust-ABI alternative would obtain a monomorphic adapter’s FnAbi and layout from a matched helper and translate every accepted pass mode and attribute. Reject unsupported cases. Compiler APIs are unstable; rustup’s rustc-dev and llvm-tools enable investigation but do not make a portable ABI contract. ABI rules, FnAbi, external drivers.

Other boundaries and next evidence

Wasm components could supply isolation and a defined value-transfer interface, but need a Wasm runtime, supported crate targets, bindings and copy/lifetime policies. A subprocess/RPC boundary offers strong isolation and independent toolchains at the cost of serialization, process lifecycle and latency. These are useful for coarse operations, not substitutes for cheap in-process calls. Neither was prototyped here.

The next adoption gate is one real Plenty executable using a generated adapter, recoverable constructor errors, deterministic cleanup and a unified runtime graph. Then qualify a native build-script dependency, realistic async facade, offline rebuilds and version updates. Source-backend expansion and full native Rust ABI lowering remain separate decisions. The current evidence supports generated C adapters as the smallest route to useful crate reuse; it does not justify promising that every crate API will be directly importable.

Proposed Rust dependency contract

Companion to Rust crate dependencies. All syntax and behavior below are proposed, not implemented.

Project declaration

Distinguish Cargo crates from Plenty packages. One possible spelling is:

[rust-dependencies]
matcher = { package = "regex", version = "1", default-features = false, features = ["std"] }

[rust]
toolchain = "<project-selected supported Rust toolchain>"
bindings = "rust-bindings.toml"

This names a candidate, not a qualified crate. Registry versions, local paths and pinned Git sources should reuse Cargo semantics. Generate a deterministic private workspace with explicit manifest, target, profile and toolchain. Cargo owns feature resolution and transitive dependencies, including host proc-macros/build scripts. Use versioned metadata and build JSON for actual artifact paths, not guessed hash-suffixed filenames. Cargo external tools.

The recommended generated C-adapter route uses stable Rust interfaces. Pin the project toolchain for reproducibility; an exact compiler-private helper match is required only for the experimental Rust-ABI route. A Rust companion module can simplify unsupported APIs while the generator supplies the ABI glue.

Commit a Cargo-format companion lockfile, for example plenty.rust.lock, and record its digest and toolchain identity in the Plenty lock/build contract. --locked forbids changing either lock. --offline applies to both ecosystems and fails on missing sources/components. Imports never silently install a toolchain. Cargo builds execute dependency build scripts and proc-macros; this is not a data-only import.

Cache keys include compiler/runtime identity, adapter/schema versions, full rustc identity, target/CPU, Cargo lock and resolved features, profile/panic/LTO flags, relevant environment/configuration and source/build inputs. Honor Cargo invalidation; a lockfile is not a fingerprint of mutable path sources. Rebuild after toolchain changes. Arbitrary prebuilt rlibs without a verifiable compatible graph are unsupported.

Leave native flags, search paths and build-script ordering with Cargo through the final Rust target. Its links constraint prevents competing packages from claiming the same native library. Account for the runtime’s vendored native dependencies in that graph, rather than bypassing the checks. Build scripts.

API selection and value rules

A binding specification selects public Rust paths, concrete type arguments and Plenty-facing signatures. rustc checks generated Rust calls, including traits, visibility and enabled features. Cargo metadata and symbol lists alone cannot describe callable APIs. Safe Rust signatures do not establish the validity of raw-pointer adapters: the generated boundary must enforce its own preconditions.

A first useful surface could expose a constructor, borrowed query and destructor for an opaque Rust matcher/parser, imported as a generated Plenty module under the dependency alias. A small Rust facade can simplify unsupported APIs without translating the crate. Automatic discovery can later prefill binding selections.

Rust shapeInitial proposed treatment
Integers/floats/boolExact-width mapping, checked conversions and bool validity; target-check platform-sized integers.
&str, &[u8] inputsScoped validated pointer/length views when storage permits, otherwise explicit copies; no retention after return.
Owned records, String, Vec<T>Opaque move-only owners, constructed/dropped in Rust; explicit copy-out where needed.
Option<T>, Result<T,E>Destructure in Rust into specified tags and initialized outputs; preserve typed/opaque errors, never reinterpret enum layout.
Generic callsSelected concrete Rust instantiations; no automatic Rust trait implementation for arbitrary Plenty classes.
Shared/mutable methodsShared/exclusive loans on the owner; forbid aliases and use after consuming calls.
Borrowed returnsInitially reject or explicitly copy; general Rust lifetime relationships exceed current Plenty returned-borrow rules.
Closures, trait objects, async, retained callbacksNo automatic general import; allow a reviewed Rust facade with a defined environment/lifetime/executor contract. Explicit future polling works in the prototype, but no production executor contract exists. Macros run in Rust, not as Plenty syntax.
Unsafe APIs/raw pointersSeparately audited safe facade, not automatic safe exposure.

Boxing an opaque owner is an explicit fallible bridge allocation. Rust crates may allocate internally and abort on OOM: they do not automatically gain Plenty’s recoverable AllocError guarantee. Document copies and allocation effects. Deallocation stays with the originating runtime even if both use the system allocator. Drop exactly once on normal/error paths; invalidate consumed handles and specify output initialization separately for success and failure.

Rust values are not automatically Plenty-copyable or worker-safe. Start with thread-bound owners; later worker exposure needs compiler-checked Send/Sync and compatible Plenty rules. Pinned/self-referential objects need an explicit storage contract. Rust checks the adapter’s local borrows; Plenty must check lifetimes and aliasing across successive calls.

Failure and qualification

Use abort-on-panic initially, matching today’s runtime. Panic terminates the process rather than becoming Result; catch_unwind cannot catch an abort. No Rust unwind may escape through Plenty frames. A later catch-at-adapter policy needs compatible unwind builds and proof for every call and destructor.

Qualification needs compile-fail tests for unsupported APIs, aliasing and toolchain mismatches; drop counters across moves/errors; output-initialization checks; embedded-NUL/non-ASCII text; narrow integers and return lowering; panic in a subprocess; generics; a native dependency; and Rust owners alongside Plenty allocations. Check duplicate runtime/native symbols and cold/offline builds. The prototype evidence covers a subset: real Cranelift callers, generated adapters, a restricted Rust-source path, move rejection, cleanup, UTF-8/NUL input, generic calls, Pending/Ready polling and arithmetic traps. Production project/runtime integration and the broader matrix remain unimplemented.

Three Rust crate prototype results

Companion to Using Rust crates in Plenty. Reproducible code, commands and checked result snapshots are in the repository’s experiments/rust-crates/ directory; start with its README.md. These are bounded experiments against Plenty d7332b6c, not shipped features.

One workload, three paths

The original Plenty source executes scalar arithmetic, Unicode/NUL text length, generic identity, construction/query/drop of a Rust regex::Regex owner, and a local pinned Future polled twice. regex 1.13.1 is an actual external crate resolved by the experiment’s Cargo.lock. The future is a synthetic fixture that returns Pending once, then Ready; there is no network I/O or executor.

The existing Plenty parser/type checker checks the source against generated signature-only stubs. Those stubs model move-only classes and loans, never run, and do not automatically prove the selected Rust API matches the declaration. A second restricted parser converts the same source into an operation plan. It supports one run(i64) -> i64 function, simple/mutable local bindings from selected function calls, name/integer/text arguments, borrow markers and a final sum of names. It rejects branches, loops, arbitrary expressions and unknown APIs.

The Rust-ABI and C-ABI paths emit real Cranelift machine code from that plan, calling generated Rust adapters. The generator changes the ABI keyword; both otherwise use the same scalar/pointer interfaces and Rust-owned handles. Both link ordinary rlibs through rustc. Cranelift’s scalar lowering is experimentally matched to stable rustc 1.99.0 on x86_64 Linux, not derived from compiler metadata.

The source backend emits Rust statements and checked arithmetic from the plan, calling the shared Rust fixture directly. Rust owns native values and borrows; there are no handle-box adapters on that path. This is a real source-lowering experiment, but it does not consume Plenty’s internal checked IR, compile arbitrary Plenty, reproduce its string/allocation model, or execute its runtime. Passing the public checker before a second parser is a prototype compromise.

Correctness and failure observations

All three paths passed with adapter/generated-source optimization levels 0 and 3. For inputs -100, -1, 0, 1, 41 and 1000, the independently specified answer is 3 * input + 11; live Rust owner counters return to zero. The workload includes four Unicode scalars in five UTF-8 bytes (é, NUL, a, b) and two regex matches. The Future’s first/second poll yield 0/1; finish requires Ready and a second poll after Ready is forbidden. Unit tests separately assert these states and dropping an unfinished Future. This does not qualify foreign callback/cancellation races.

An inserted use-after-move is rejected by the real Plenty checker. Large inputs trap on integer-sum overflow in all executable paths: Cranelift exits by SIGILL, generated Rust by SIGABRT. The harness detects an incorrectly successful wrap followed by its fallback panic, rather than counting any abort as a pass. This preserves the limited no-wrap contract, not identical Plenty trap diagnostics.

Rust unit tests, Python grammar rejection/literal tests, formatting and Clippy passed. No compiler/runtime implementation was changed, and no full Plenty regression suite was run. Native dependency build scripts, real async I/O, fallible constructor errors, OOM recovery, aggregate ABI calls and cross-target compatibility remain untested.

Compiler-derived ABI observation

After installing rustc-dev and llvm-tools, the separate helper built and ran on nightly-2026-10-01 (21b707e3f, Rust 1.101.0-nightly). It invokes fn_abi_of_instance after analysis:

Function inputObserved pass mode
Rust ABI i64Direct, size/alignment 8
Rust ABI &strPair, size 16/alignment 8
Rust ABI 24-byte repr(C) recordIndirect, Pointer
C ABI the same recordIndirect, OnStack

Both record returns are indirect. This is concrete evidence that the query is accessible and layout alone does not determine calling convention. The helper’s results are not fed into the stable Cranelift caller: compiler identities differ, and complete pass-mode translation, attributes, monomorphic API discovery and compatibility checking remain work. Missing initial components were an environment issue, not evidence the approach was infeasible. FnAbi, PassMode.

Cost observations and interpretation

See the committed measurement snapshot at experiments/rust-crates/RESULTS.md for exact values and all samples. The initial isolated dependency/tool build took 85 seconds; subsequent fixture builds were warm. Adapter-plus-link builds were roughly a tenth of a second. Cranelift object emission took a few milliseconds. These are different stages, not cold-build comparisons of production backends.

Each timing has three runs, with 2,000 whole workloads or two million scalar iterations, black-boxing inputs inside the loop and outputs. Source-mode Rust may inline; native calls cross an object boundary. Scalar results measure the whole small loop/call, not a calibrated standalone ABI transition. The shared fixture and regex dependencies remain debug-built even when adapters are optimized. CPU contention/frequency, ordering and debug dependencies make these exploratory measurements, not a backend speed ranking. No benefit of Rust ABI over C ABI was established.

Both native adapter paths borrow existing UTF-8 bytes without copying payloads and add two opaque-owner Box allocations per workload. Source mode avoids those two boxes; all paths retain the future’s internal pin allocation and regex’s own allocations. These are source-level accounting observations, not allocator instrumentation or a general zero-copy guarantee.

Handwritten glue consists of a ten-function API-selection table and the shared Rust facade/fixture. Unsafe adapter bodies, native caller signatures and checked source stubs are generated; no handwritten C wrapper or C header is needed. The Rust-source route calls the same facade directly. Both still rely on a trusted hand-selected type/ownership contract rather than automatic reflection.

What changes the recommendation

All three routes can reach the same useful Rust operations. Rust ABI did not remove conversion or ownership glue, while the helper revealed additional ABI machinery that must be maintained. C adapters can expose ordinary crates and future handles using stable compiler interfaces. Rust-source lowering is concise for this grammar and avoids handle boxes, but a complete backend must preserve Plenty’s fallible allocation, strings, sums, borrowing, cleanup, std and diagnostics. Therefore generated C adapters are the most tractable next production experiment; the others remain viable research directions with larger scope.

Compiling Plenty to Rust source

Status: Restricted checked-IR emitter spike; no production backend (2026-10-24). B20 tracks optional backends; B17 tracks Rust dependency tooling. The current compiler still emits native code through Cranelift. No production command, target, Rust import syntax or compatibility guarantee is introduced here.

Recommendation

The checked-emitter experiment now generates a restricted scalar library from Plenty’s actual checked operations, including a Rust → Plenty → Rust companion round-trip. It measures generated Rust build costs separately from the earlier hand-lowered ownership probes. References, records, enums and dynamic owners remain outside that emitter.

The evidence supports continued bounded exploration of Rust-hosted Plenty libraries. Emitting a Rust crate could let Rust call Plenty functions, and Plenty call selected Rust APIs, with ordinary Rust owners, borrows, enums and callbacks. rustc would own type layout, calling conventions, generic codegen and final linking within that build. Users could write application logic in Plenty while retaining a Rust application’s ecosystem and tooling.

The opportunity is real; “seamless” needs a precise boundary. Producing valid Rust is easier than preserving Plenty’s semantics and exposing a natural Rust API. Replacing runtime values with Vec, String and derived Drop everywhere would change allocation, copying and cleanup behavior. Keeping all current runtime representations would preserve more behavior but expose a Plenty runtime API to Rust users. Prefer a small shared native-type subset, with explicit wrappers for richer Plenty values.

The Rust crate investigation already includes an actual restricted Plenty-to-Rust emitter alongside two native adapter routes. This proposal develops the source-backend direction as its own architectural choice. It does not overturn that investigation’s lower-cost recommendation of generated C adapters for adding Rust dependencies to today’s compiler.

Proposed compilation boundary

Plenty source ── parse / resolve / type and ownership checks
                                 |
                 typed source structure + places, loans,
                  cleanup decisions and source locations
                                 |
                       generated Rust crate
                       /                 \
             Plenty support crate     selected Rust APIs
                       \                 /
                         Cargo + rustc
                              |
                  Rust host library or executable

Keep Plenty’s own checker authoritative. Do not translate unchecked source and use Rust errors as Plenty’s type system: their admitted borrows and contracts differ. The independent Rust check is valuable defense and catches emitter mistakes, but a supported Plenty program rejected by emitted Rust is a backend problem, not a reason to silently redefine the language.

Start from the existing structured operations and their construction sites, not Cranelift IR or assembly. src/op.rs already retains typed slots, structured branches/loops, Loan, Access, Site, Try cleanup and DropLocal operations. It is not devoid of ownership information. However, some source constructs are specialized/desugared and runtime operations already assume Plenty layouts. No public checked-IR emission API exists today.

The next technical question is whether these facts suffice to reconstruct a small safe Rust body, or whether a thin typed source representation must be retained before representation-specific lowering. Prefer preserving the missing facts at that boundary to reverse-engineering lifetimes from machine slots. Do not commit to a new universal IR before the small slice answers that question.

User-facing direction

The first useful product would be a Rust-consumable library containing selected Plenty functions. A generated crate would have stable exported Rust names and an explicit API manifest; internal monomorphized names remain private. Rust can supply native data, call these functions and inspect native Results. Selected Rust imports would use checked signatures or a small Rust companion module whose ownership/effects are declared. A raw symbol list is insufficient.

Build integration can use an explicit Plenty generation step followed by Cargo, or a build script writing generated modules to OUT_DIR with complete input invalidation. Cargo already supports this kind of source generation. Separate generation from dependency resolution so checking does not unexpectedly fetch or execute build scripts. Pin the supported Rust toolchain/edition and lock dependencies; build one coherent runtime/dependency graph.

This is source interoperability inside a coordinated Rust build, not a stable Rust binary ABI or a promise to mix arbitrary rlibs. The Rust ABI has no stability guarantee. Native Cranelift Plenty objects cannot directly treat emitted Rust layouts as their existing layouts. Mixed-backend calls still need a defined boundary.

Why pursue it, and what it does not buy automatically

Potential benefitQualification
Rust calls Plenty as a normal cratePublic types and lifetime contracts must be designed, not just printed.
Plenty calls Rust without C marshallingOnly admitted types/APIs share that representation; ownership and effects still need checking.
LLVM optimization and Rust targets through rustcNo speedup or new Plenty target is established until runtime support, ABI boundaries and generated code are tested.
Cargo packaging, profiling and debuggingGenerated-source locations need mapping back to Plenty; dependency build cost remains.
Access to trait-heavy or async librariesA companion Rust facade can help now; arbitrary traits, macros and futures do not become Plenty features merely by changing backends.

The fast Cranelift path remains valuable. A Rust-source route is a stronger candidate for ecosystem integration or an optional optimizing build than a demonstrated replacement for interactive compilation. Two frontends plus Rust type checking and optimization add work; incremental Cargo builds may amortize some of it. Measure cold, unchanged and edited builds separately, including generation, dependency build, Rust compilation, link time and peak memory.

The semantic contract describes the hard boundaries. Probe evidence and adoption criteria explain what has run and what would justify a first implementation. Future work and priority remain in the backlog, not this proposal.

Rust-source semantic and interoperability contract

This is the proposed contract for the Rust-source backend, not implemented behavior. Preserve the language first; make unsupported slices explicit rather than accepting a program with different semantics.

Values and API shape

Plenty conceptCandidate Rust representation and constraint
Sized numeric scalars and boolCorresponding Rust scalar; emit explicit checked operations/casts where required.
Simple scalar-field classesNamed Rust structs, normally move-only even when fields are Copy. Do not derive traits that weaken Plenty’s ownership contract automatically.
Named enums, Option, ResultNative enums; explicit conversions for Plenty error widening and pattern tests. Stable public wrapper names for anonymous sums.
Shared/exclusive referencesRust references with generated lifetimes from checked place relationships. A returned first-parameter borrow gets that parameter’s lifetime.
Borrowed primitive list data&[T] / &mut [T] at selected Rust-facing boundaries; prove owner/lifetime stability and indexing behavior.
Owned collectionsInitially preserve a support-crate wrapper; optional Vec conversion must state ownership, allocation failure and whether it copies.
Plenty immutable stringsPreserve share/retain behavior in a wrapper and borrow UTF-8 as &str where valid. String is not a transparent replacement.
Arbitrary Rust ownersNative Rust values only for explicitly admitted APIs with known loan/drop/effect contracts. Rust generic/trait declarations require more than an ABI signature.

Do not transmute current runtime values to native Rust enums or collections. Plenty’s sum tags, owner handles, string headers and inline relocation rules are private representations. A source backend may choose different internal layouts, but std/runtime helpers must agree and every crossing must be checked. Initially isolate portable helpers behind a support crate rather than copying the whole runtime or linking two independent runtime/allocator graphs.

Ownership and destruction

Ordinary lexical locals and many ? exits fit Rust RAII. The probes validate one such sequence, not the full contract. Plenty specifies closure capture cleanup order and allocation-free bounded-stack recursive heap destruction; ordinary Rust captures and recursively nested owners do not automatically provide those guarantees. Rust’s destructor rules also make declaration order, temporaries and partial moves consequential. Generate explicit capture structs/scopes and cleanup helpers when necessary. Avoid blanket Option<T> slots, hidden clones, Rc<RefCell<T>> or heap boxes as a convenient way to appease the borrow checker.

Checked place relationships must survive lowering. A concrete example is Plenty’s simultaneous loans at distinct constant list indices: direct Rust index borrows fail with E0499. A split_at_mut translation works for the tested pair. General lowering must preserve argument evaluation, bounds failure sites, intervening effects and loan scopes; the two-index sketch is not a general algorithm. If unavoidable unsafe helpers are introduced, centralize and audit their place/aliasing preconditions. Do not emit pervasive raw pointers and call the result safe Rust interoperability.

Rust and Plenty impose different limits on stored/returned references. Continue to enforce Plenty’s current limits even when Rust could represent more. Conversely, a backend must either translate an admitted Plenty construct or diagnose its unsupported backend subset before emitting code.

Arithmetic, errors and allocation

Plain Rust + in release mode is not the implementation of Plenty checked addition. Emit checked_add plus the Plenty fatal trap path, and analogous handling for other arithmetic, division, shifts and narrowing casts. Preserve floating-point comparison and operation order; an optimization backend does not authorize reassociation. Trap helpers need Plenty source/caller locations and must not allocate on their failure path.

Native Result<T, E> is a good representation, but Plenty’s automatic error-sum widening needs generated mappings rather than accidental From implementations or boxing everything into dyn Error. Public error enums should implement the appropriate Rust traits and retain concrete causes. Rust panics are not recoverable Plenty error variants by default.

Fallible construction must reserve before evaluating list elements, propagate failure and destroy only initialized values. Rust’s Vec::try_reserve can implement one part, but Vec::new() does not perform Plenty’s empty-owner allocation. vec!, implicit cloning and unconstrained collection operations can introduce infallible allocation. Preserve CapacityOverflow versus OutOfMemory with explicit layout validation and an appropriate allocation wrapper; simply exposing TryReserveError is not equivalent. External Rust crates may allocate infallibly internally, regardless of generated code quality. Their contract must say so; do not claim arbitrary Rust APIs satisfy Plenty’s recoverable-allocation policy.

Panics, recursion and suspension

Plenty fatal traps do not unwind. The probe’s default Rust panic runs a destructor that Plenty’s overflow trap does not. Emit an aborting trap helper. For the initial foreign-call slice, require an explicitly agreed panic=abort build graph or reject integration; never silently change a Rust host’s panic policy. Supporting an unwinding host later needs a separate boundary contract, including callbacks and destructor panics. Wrapping a whole generated function in catch_unwind is insufficient: cleanup may already have run before the catch, and aborting panics are not caught.

Plenty also has required direct recursive tail transfers. An ordinary generated Rust call cannot be assumed to preserve that guarantee. Translate admitted self-tail recursion to a loop, use an explicit dispatch machine for supported mutual cycles, or reject the subset until implemented. Generic specialization must precede that analysis where it does today. Generators need explicit frame, suspension and cleanup lowering; a Rust iterator or async future is not automatically the same lifecycle. Concurrency, generators, boxed closures and runtime-specific intrinsics are outside a first library slice.

Generic and tooling boundaries

The present frontend specializes generics before native lowering. Emitting those instances as private Rust functions is simpler than recovering generic templates, but only exposes selected monomorphic APIs to Rust. Exporting a generic Plenty function as a Rust generic requires retaining the template and translating protocol bounds, not mechanically replacing bracket syntax. Rust trait coherence/orphan rules still apply to generated implementations; newtype facades may be necessary for foreign traits/types. Rust implementation rules.

Keep selected imports explicit first. Cargo metadata describes packages, not complete type/loan semantics. Automatic API discovery, procedural macros, associated types and async executors remain independent integration problems. The compiler should track a generated-byte-range to Plenty-span map, consume rustc JSON diagnostics, and distinguish user API mismatch from emitter failure. Path remapping alone cannot map generated Rust expressions back to Plenty. Runtime trap sites and debugger stepping are separate obligations. Neither source mapping nor debugger support has been implemented by this exploration.

Rust-source probe evidence and adoption criteria

This accompanies the proposal and semantic contract. Results are bounded evidence, not a completed backend or a performance comparison.

The subsequent checked-emitter spike adds actual checked-IR generation and compile/link measurements. The hand-lowered ownership results below remain separate evidence; they are not implemented by that scalar emitter.

What is already generated

The Rust dependency experiment has an actual restricted emitter in experiments/rust-crates/run.py. It checks Plenty against declared API stubs, reparses a small accepted subset and emits calls, borrows and checked arithmetic to Rust source. A real regex owner and synthetic future execute through that path. That establishes feasibility of source emission with native Rust owners; it does not consume checked compiler IR or implement the language. Its earlier results were not rerun for this proposal and should retain their original measurement qualifications.

New semantic probes

experiments/rust-source-backend/ contains dependency-free Rust candidate implementations, a separate Rust-host integration test, paired real Plenty programs and a runner. The Rust is hand-lowered, deliberately isolating semantic questions from implementing a second parser or a production emitter. Run python3 run.py --plenty /absolute/path/to/plenty from that directory. Inputs, expected observations and the recorded snapshot are in its README and RESULTS.md; machine-readable observations are written under ignored target/.

Tested on x86_64 Linux with Plenty 0.3.0 from main 00d32a13, rustc 1.99.0 (b940084d7, LLVM 23.1.1), edition 2024. The local Rust checkout inspected for Vec reservation and panic/abort implementation was 080cc27146389dbb5c8772c476b1937be9c6110f, distinct from the tested compiler revision. It was used as source evidence, not built or assumed identical to the installed toolchain.

QuestionObservationLimit
Can Rust consume native-looking APIs?Separate Rust test crate passes a native vector borrow, Cursor and consuming closure; returned element points into the original vector.API declarations are handwritten, not inferred. No general Rust crate importer.
Do scope cleanup and propagation align?Both ordinary/debug and release Plenty/Rust print 33, 2, 1, 3, 4: numeric result, reverse lexical cleanup on ?, explicit moved-owner drop, final marker.One concrete control-flow path; not closure/generator/recursive cleanup coverage.
Does checked overflow preserve fatal behavior?Explicit checked-add/abort fails in both modes with no destructor output, as Plenty does.Trap formatting, exact exit status, stack guard and source attribution are not reproduced.
Can ordinary Rust panic stand in for a trap?No: default panic prints destructor trace 99; separate panic=abort build does not.Does not qualify arbitrary dependencies, callbacks or unwinding Rust hosts.
Does native fallible reservation help?Impossible capacity fails before any element callback; normal construction succeeds.Capacity overflow only; no allocator fault injection or Plenty owner/error representation parity.
Can all admitted loans be printed literally?No: distinct constant list-index loans pass Plenty but naive Rust fails E0499; safe split translation passes.Only a two-element constant-index case.
Does Rust still check owner misuse?Moved-vector access is rejected with E0382.Redundant checking cannot replace Plenty checks or prove the emitter correct.

Two library tests and one Rust-host integration test pass in debug and release. The runner checks four differential executions, two rejected Rust sources and three panic observations. Cargo formatting and Clippy with warnings denied pass. No production compiler source changed, no external dependencies were added, and no cross-target or performance result is claimed.

Decision gate for a first backend slice

The checked-emitter experiment now satisfies the scalar part of the original gate: real checked-frontend emission, native companion calls, source mapping and build measurements. The remaining ownership-bearing gate is to admit simple move-only scalar records, named enums/Results, structured branches/loops, local borrows and a returned first-parameter borrow. Expose monomorphic exports and a small explicitly declared Rust companion API. Exercise a genuine Rust host that calls an exported Plenty function which in turn calls that companion. Restrict recursion and resource features until their contracts have a translation; unsupported cases must fail explicitly.

The adoption decision should require all of the following evidence:

  • Differential tests against Cranelift for admitted semantics, including overflow, early exits, drop order, moves, aliases and fault-injected allocation if owned dynamic storage is admitted.
  • A public Rust API usable without unsafe calls or per-call marshalling glue, with documented ownership and panic behavior and no hidden copy workaround.
  • Source-located Plenty diagnostics for unsupported inputs, plus at least one generated rustc error mapped back to the triggering Plenty source.
  • Cold/warm/edit build and representative runtime measurements, separately attributing source generation, Rust checking/optimization, dependency builds and linking. Compare the same accepted workloads, not tiny warm Cargo no-ops.
  • A documented list of retained frontend facts, support-runtime dependencies, unsafe helper invariants if any, and costs of keeping both backends correct.

If useful native APIs require broadly changing Plenty’s semantics or pervasive unchecked lowering, stop expansion and prefer the generated-adapter route. If safe monomorphic library exports work with a small support layer, that alone can be worthwhile without committing to a whole-language compiler replacement. This gate describes the technical decision; scheduling remains in B20.

Checked emitter and Rust build costs

This follows the Rust-source proposal. The experiment in experiments/rust-source-backend/emitter/ generates Rust from the real checked frontend. It is deliberately smaller than the ownership-rich hand-lowered probes: its purpose is to test a real emission boundary and the cost of compiling the resulting source, not to declare a production backend.

Implemented experiment

Build the developer driver with cargo build --locked --release --features rust-source-spike --example rust_source_spike --bin plenty. The hidden emission API is absent from normal builds. It loads a normal Plenty module, runs frontend and independent operation/ownership checking on the complete input, then selects explicit export roots and their reachable functions. There is no second parser, alternative Plenty type system, public backend flag, or shared-IR redesign.

AreaAdmitted slice
API/storageNative signed/unsigned integers through 64 bits, bool, unit results
ComputationChecked add/sub/mul, floor division/modulo, wrapping add/sub/mul, integer conversions, comparisons
Control flowStructured branches, scalar matches, short circuiting, while, break/continue, returns and direct calls
GenericsConcrete scalar specializations already produced by Plenty; no generated Rust generic functions
RecursionSelf-tail calls become an explicit loop with evaluated argument temporaries; other recursion is rejected
Rust cooperationExplicit checked stub → Rust companion replacement, called from a native Rust host through an exported Plenty function
DiagnosticsSource-located unsupported operations; sidecar line map demonstrated against JSON rustc type errors

Every accepted value is a Copy scalar. This avoids inventing a storage, allocation or lifetime model. References, records, enums/Results, dynamic owners, floats, generators and concurrency are rejected, not silently converted into Rust types. Input imports still use Plenty’s normal resolver; only reachable admitted functions are emitted. All input bodies are checked before pruning.

The checked Op representation is sufficient for this slice: typed slots, structured control flow, specializations and source sites survive lowering. Snapshot temporaries preserve evaluation order; scalar branch values become Rust tuple joins. Already-validated ownership-only markers have no emitted action for this Copy-only storage. That does not establish that the same approach suffices for references or destruction.

Generated Rust has no dependencies beyond the installed standard library, no unsafe code, heap marshalling, trait-object dispatch, macros or derives. Checked failures call an aborting helper, and the generated module requires panic=abort. It does not reproduce Plenty’s trap wording/status, stack guards, runtime caller traces or debugger source information. A companion is an explicit integration contract, not automatic discovery of arbitrary Rust APIs.

Measurement method

run.py checks the semantic slice and bench.py measures it. Every benchmark variant compiles a real Rust library, compiles/links a Rust host, and runs it; its exit-status checksum must match the same Plenty workload compiled through Cranelift. An edited incremental library is checked against a freshly compiled native program too. The source generator creates 1, 100 or 1,000 scalar functions, each with twelve bounded multiply/add/modulo steps, plus an aggregate loop.

The Rust library boundary prevents the host’s fixed input from being folded through the whole library. Library compilation and the tiny host’s compilation/linking are separate measurements; the latter is not a pure linker timer. --emit=metadata is another full invocation to bound checking costs, not an exact phase subtraction. No linker-only or peak-memory claim is made.

Each artifact-cold sample starts a new compiler process without incremental reuse. OS pages, the installed Rust sysroot and Plenty runtime artifacts are warm. The native baseline sets PLENTY_CACHE=off; it is not Plenty’s warm compiled-function-cache path. Explicit Rust incremental results distinguish first build, unchanged source and a real first-function body edit.

All Rust variants use edition 2024, aborting panics, no debug information, embed-bitcode=no, the installed default linker, and no cross-crate LTO, nightly options, third-party compiler cache or external crates. The rustc codegen documentation describes codegen-unit parallelism, incremental reuse and omitting unneeded embedded bitcode. The table below is measured evidence on this machine, not a promise that these choices dominate for every crate. Rustc’s default local ThinLTO across optimized codegen units remains enabled.

Observed build tradeoffs

Recorded 2026-10-24 on a Ryzen 7 7840HS, Linux x86_64, rustc 1.99.0 (b940084d7, LLVM 23.1.1), Plenty base be665bb0 plus this experiment. Three samples per cell; compiler processes were restricted to CPUs 0–7 (four physical cores). The small/flat development runs were quiet. During the large optimized/grouped runs, another worker performed validation on separate cores 4–7 (CPUs 8–15). Later validation exposed substantial tmpfs occupancy (roughly 32 GB) and a full 8 GB swap allocation; completed build artifacts were moved off tmpfs before the final measurements. Per-sample memory pressure and frequencies were not recorded for these exploratory snapshots, so shared thermals and memory contention can confound their ratios. The shape effects exceed sample spread, but that alone does not rule out changing machine load. Raw samples are flat-measurements.json and grouped-measurements.json in the experiment directory.

FunctionsNative dev compile/linkRust inline-check library, opt 0Rust helper library, opt 0Rust host compile/link after helper library
141 ms33 ms24 ms49 ms
10055 ms509 ms154 ms46 ms
1,000229 ms5,300 ms1,452 ms48 ms

The separate Plenty check/emission process adds approximately 2/13/125 ms for these helper cases. These are independently measured stages, not totals from a single timer. Generating one monomorphic helper per arithmetic operation/type reduces repeated Rust checking work and emitted source size (1,000 functions: 6.4 MB inline versus 2.8 MB with helpers). One versus sixteen codegen units did not materially help the flat development case. A separate Rust metadata-only invocation took about 1.26 s versus 1.45 s for the helper library, suggesting that much of this cost precedes native code generation. This is not an exact phase partition.

Flat optimized output exposes a much larger problem. The experiment also emits groups of 32 sorted function symbols into Rust modules, and optionally applies #[inline(never)] to source functions while leaving arithmetic helpers eligible for normal optimization. This is not a Plenty module cache. Body edits preserve these groups; inserting symbols can renumber them.

Helper emission, opt 3100 functions: library build1,000 functions: library build1,000 functions: execution
Flat, ordinary inlining1,081 ms81,920 ms4.78 ms
Groups of 32391 ms4,074 ms4.89 ms
Flat, source functions not inlined758 ms10,255 ms6.16 ms
Groups of 32, source functions not inlined337 ms2,901 ms4.60 ms

Execution includes process startup and 100 aggregate iterations. Native release took 5.27 ms in the flat snapshot; these tiny absolute times do not establish a general Rust runtime advantage. Development helper execution took 21.89 ms versus native development’s 10.45 ms. Full binary sizes, including linked runtime material, are in the snapshots; they are not generated .text sizes.

Grouping and inlining control offer a credible way to avoid pathological optimized build costs without sacrificing this workload’s runtime. The exact LLVM attribution was not profiled; these emission comparisons suggest a useful direction, but the machine conditions prevent attributing the full ratio to one change or a specific optimizer pass. Grouping made the development build slower here, so there is no single winning configuration.

For the 1,000-function development library, explicit incremental reuse reduced the flat first/unchanged/body-edit rustc times to 2,148/795/1,092 ms. Grouped output measured 1,655/656/697 ms. The edit timings are single observations, with a checksum checked against a fresh native build. Add emission and host compile/link for an application edit; these are not full rebuild times or Cargo no-op timings. Rust still reads and checks the generated crate.

Final end-to-end comparison

finalist-measurements.json adds a single timer around checking/emission, Rust library compilation and host compilation/linking, with three independent samples. After the user corrected the CPU frequency limitation, the two finalist configurations and matching native baselines were rerun in a coordinated quiet window on 2026-10-24. Sampled frequencies were 3.25–4.09 GHz. Workloads, three repetitions, 100 aggregate iterations, CPU affinity and compiler flags are unchanged. The raw snapshot records the command, source commit and environment. The initial exploratory emission-shape measurements were not rerun.

FunctionsNative devRust helper dev, end to endNative releaseRust grouped/non-inlined release, end to end
142 ms83 ms42 ms94 ms
10061 ms246 ms65 ms343 ms

The previous roughly 544 MHz results are preserved separately in finalist-throttled-measurements.json and superseded by this table; the two hardware regimes are not pooled. The fresh samples support practical subsecond builds for these small Rust libraries, while the native path remains faster. These two small synthetic workloads do not set a general build-time budget.

Recommendation: continue the Rust-hosted library experiment with compact monomorphic helpers and an explicit development/optimized emission policy. Small libraries can have practical build times, and stable function grouping can prevent very expensive optimization. This evidence does not make rustc a replacement for Cranelift’s fast build path. Native ownership-bearing API emission remains the next semantic gate; more backend breadth should not be inferred from the scalar benchmark.

Validation and limits

The differential suite covers all admitted integer widths, narrowing/widening, wrapping arithmetic, negative floor/modulo and MIN % -1, branch values, short-circuit skipped traps, loop exits, scalar pattern matching, specialized generics, unit calls and a 100,001-step self-tail recursion with swapped arguments. An independent Python oracle additionally checks every one of the 65,280 nonzero-divisor i8 pairs for floor/modulo, in native dev/release and generated Rust opt 0/3 with both helper shapes. The sole overflowing quotient is skipped there and checked separately as a fatal case; its modulo is still checked. Overflow, zero division/modulo, division overflow and checked casts fail in both native and generated debug/release builds. Exact signal numbers are not compared.

Rust → generated Plenty → Rust companion execution is checked with native integer signatures. A wrong Rust companion type produces a real rustc error mapped back to its Plenty declaration. Invalid Plenty in an unexported body still fails frontend checking; references, unsupported numeric types and unsupported recursion are rejected. A default unwinding Rust build is rejected.

These measurements cannot establish owner cleanup, native Rust borrowing, fallible allocation, collection interoperability or arbitrary crate support. They qualify a restricted compiler experiment; the broader proposal’s adoption criteria remain open. Scheduling stays in B20.

Generated library contracts and C headers

Status: Partly implemented. Accepted 2026-10-07. Numeric and Result exports, owned and borrowed handles, generated headers, embedded interfaces, extraction, compatibility fingerprints, and typed resident loading are implemented. Exports, owned exports, and contracts own the shipped contract. C interop records the wider boundary design.

Shipped differently: there is no standalone manifest. The contract is a generated .plentyi interface, embedded in the binary and written beside NAME.h and NAME.link-args.txt. The image API below is an illustrative sketch: generated headers differ in naming and in how Result exports report status. Exported error types include numeric values, AllocError, ParseError, RangeError, Failure, and nongeneric payload-free user enums; payload-bearing enums and error sums remain unsupported.

Remaining: text and byte buffers, returned views, and broader error types (B27); callbacks, unloading, and loaders generated from extracted contracts (B28).

One contract, several consumers

A Plenty-produced static or shared library should expose an ordinary C ABI and carry a richer Plenty interface. Generate its C entry adapters, C header, machine-readable .plentyi contract, and header contract comments from the same checked export representation. Consumers using Plenty get ownership checking and automatic cleanup; other languages get explicit instructions for satisfying the same contract. They do not need a Plenty compiler to call the C exports.

The generated contract describes both the actual C signature and its Plenty interpretation. For example, a status code and output handle may represent a Result containing an owned object; its release symbol supplies automatic destruction. This does not expose Plenty’s private Result, class, or collection layouts as C layouts. Borrow checking uses the interface without reanalyzing the library implementation.

Keep opaque handles and explicit C representations at this boundary. An owner must be released through its originating library and allocator context. Ordinary pub declarations do not become C exports or acquire a stable binary layout. Generic exports need concrete specializations; metadata alone cannot instantiate new machine code in an AOT consumer.

Comments are part of the exported interface

Generate precise, human-readable comments beside each declaration. Use stable templates driven by the contract, rather than inferred prose or a separately maintained description of ownership. Author-written purpose, examples, and domain-specific notes can supplement these generated facts, with their source distinguished from the generated requirements and guarantees.

SubjectWhat the header must explain when applicable
InputsNullability, alignment, initialized extent, valid ranges, and required object/library identity.
BorrowingRead/write access, alias restrictions, duration, whether pointers are retained, and whether callbacks can access the same storage.
OwnershipWhether an argument is borrowed or consumed; who owns every value on each success and failure path.
OutputsWhich outputs are initialized for each status, whether they must be initialized before the call, and whether previous contents are released.
DestructionExact release function and context, whether null is accepted, which aliases become invalid, and why ordinary free is inappropriate.
Returned viewsThe owner keeping a view alive and the mutations, calls, or destruction that invalidate it.
Text and buffersEncoding, byte/element units, lengths and capacities, terminators, embedded NUL rules, and null behavior for empty buffers.
FailureStatus meanings, allocation failure behavior, partial effects, and whether input/output state is preserved.
ExecutionThread affinity, concurrent-call and reentrancy rules, callback lifetime, and prohibitions on unwinding or nonlocal jumps across the boundary.
Library lifetimeHow long code, release functions, borrowed data, and callbacks must remain available.

Describe obligations and guarantees separately. For example, an exclusive borrow requires the caller to prevent conflicting access during the call; the callee guarantees not to retain that borrow afterward. A C const qualifier alone does not communicate these requirements. Avoid vague statements such as “caller owns the result” when error paths or borrowed subobjects change what that means.

Illustrative generated header fragment, not an implemented image API:

#include <stdint.h>

typedef struct ImagesImage ImagesImage;

#define IMAGES_OK UINT32_C(0)
#define IMAGES_INVALID_SIZE UINT32_C(1)
#define IMAGES_ALLOCATION_FAILED UINT32_C(2)

/**
 * Requires:
 * - image is a live, non-null handle from this library instance.
 * - Call on the thread that created image.
 * - No other access to image or its storage occurs during this call,
 *   including access through aliases or reentrant callbacks.
 * - No outstanding borrowed view into image's storage exists.
 * Guarantees:
 * - Borrows image exclusively for this call; retains no new alias.
 * - Ownership stays with the caller on every return path.
 * - IMAGES_OK: image has the requested dimensions.
 * - IMAGES_INVALID_SIZE: width or height is outside 1..65535 inclusive.
 * - IMAGES_ALLOCATION_FAILED: dimensions are valid but storage allocation fails.
 * - Both failure statuses leave image unchanged.
 * - These are the only returned status codes; runtime traps terminate
 *   the process and are not converted to status codes.
 */
uint32_t images_resize(ImagesImage *image, uint32_t width, uint32_t height);

/**
 * Requires:
 * - image is null, or a live owned handle from this library instance.
 * - For a live handle, call on its creating thread with no outstanding
 *   borrows and no concurrent or reentrant access to image or its storage.
 * - The originating library remains loaded throughout this call.
 * Guarantees:
 * - Null is a no-op; otherwise consumes and destroys image.
 * - All aliases and views into the destroyed object become invalid.
 * - Use this function exactly once per owned handle; never use free().
 */
void images_destroy(ImagesImage *image);

The unchanged-on-error promise above is a property of this illustrative API, not a default for all exports. Emit only guarantees established by the checked implementation or explicitly declared trusted dependencies. Do not invent rollback, thread safety, non-retention, or error recovery from a C signature or a function name. Unsupported contracts must produce export diagnostics rather than silently generate a safe Plenty view. Explain trusted foreign assumptions where they are part of the public contract.

Plenty checks its callers against these contracts. Foreign callers must uphold the documented obligations themselves. Metadata and comments cannot stop invalid C pointer use; runtime detection, where provided, is a separate API guarantee.

Packaging and compatibility

Emit the versioned .plentyi interface beside the library and embed the same contract in a dedicated binary section/resource or static-archive member; no second manifest format is needed. Include target/ABI information, stable exported type identities, contract format version, and an interface fingerprint. Preserve discovery metadata through supported stripping, archive extraction, and linker garbage collection. Read metadata at compile time without loading or executing library code, and validate it as untrusted file input.

Runtime loading uses a library-specific versioned discovery symbol or function table and checks the expected contract before exposing callable handles. Unique symbol namespaces also permit multiple libraries in one static link. The fingerprint detects interface mismatches; it is not authentication or proof of a foreign implementation’s behavior. AOT clients compile against known interfaces; metadata does not provide arbitrary runtime specialization.

Keep loaded libraries resident initially. Any future unloading must account for owners, release functions, borrowed data, callbacks, and in-flight calls. Library output also needs a runtime packaging mode without application startup symbols; metadata does not solve allocator or runtime-state composition by itself.

Validation requirements

Compare generated header contracts with their .plentyi facts, including ownership on failure and output initialization. Compile and run C callers against generated headers for static and shared libraries, and test the corresponding Plenty consumer’s borrow and cleanup behavior. Cover unsupported contracts, incompatible metadata, allocation failures, matching destruction, and the supported strip/link settings. These checks belong with export and loader implementation, not solely with documentation snapshots.

Qualifying new boundary types

The CO3 assessment reinforces generating native representations and conversion code together. For Plenty, extend this to a reviewable contract for each supported conversion, rather than promising that every language type automatically has a public ABI. Before admitting a new B27 buffer, view or error representation, specify:

  • Its native declaration, target layout and valid values independently of the private Plenty layout; include pointer/length relations and tag/payload rules.
  • Whether each direction borrows, copies or transfers, with allocator and library provenance, invalidation, and cleanup on every conversion failure.
  • Which preconditions remain trusted and which the adapter checks; define the error result or termination behavior before converting invalid input into a language value. Validate readable storage only after its foreign validity obligations are satisfied; generated checks cannot safely probe arbitrary pointers.
  • The semantic facts included in compatibility checking. A borrow-to-transfer change can break callers despite an identical C prototype. New lowering rules must update the contract/fingerprint scheme as necessary, with explicit treatment of old consumers; a stable-looking C layout alone is insufficient.

Qualify the same representative API through a C caller, a Plenty consumer of the generated interface, and typed runtime loading where supported. Exercise static and shared linking, valid success and error paths, allocation failure, and exact destruction. For malformed-value tests, use valid backing storage with invalid tags or encoding, and check size overflow before dereferencing buffers. A plausible length exceeding the real allocation remains a foreign contract violation, not something the adapter can generally detect. Use compiler rejection tests for forbidden lifetime escapes. Test callbacks in both directions when that capability is added. Compare results and ownership effects, not only emitted header text or metadata. Existing C fixtures are the starting point, not a reason to defer B27 until all B28 facilities exist.

CO3’s finite specialization and tagged-dispatch examples are useful alternatives for a later concrete generic-export need. Plenty should initially require named, concrete wrappers. If specialization is added, record a finite exported set, stable symbol names and full lowered contracts. Shared machine representation does not imply interchangeable ownership or destructor semantics; tagged dispatch also needs validated tags and an explicit unknown-tag policy. Neither generic export syntax nor a universal companion-type framework is required for buffers, views, header verification, or the first callbacks.

Fallible memory, parallel execution, and SIMD

Status: Partly implemented; retained for B13 allocator design. Originally researched after b690afe on 2026-10-05 against Cranelift 0.131.1; that backend evidence is historical, not the current dependency. The design sketches below are not current API documentation.

  • Recoverable allocation: implemented. Allocating operations expose allocation errors through Result, and ? propagates; see allocation and traps. Public allocator selection, budgets, and bounded containers remain B13.
  • Parallel execution: implemented. See concurrency. Automatic parallelization is B24.
  • SIMD: initial subsets implemented. Explicit std.simd supplies four numeric vector shapes, masks, scalar fallback and bounded native lowering. Integer-column filter/count has automatic SIMD. Broader shapes, packed views and transformations remain B23.

This page remains the B13 design record. The newer portable std.simd proposal refines B23 with an API contract, automatic pipeline SIMD and measured backend evidence. The completed allocation audit and native concurrency contract record the delivered subsets. Checked-index arenas solve node identity; they are distinct from the allocator-lifetime proposal here.

Shipped differently: AllocError has two variants, OutOfMemory and CapacityOverflow. Class, tuple, enum, generator, and range storage is inline and needs no allocation; component expressions and fallible class initializers retain their own errors. .unwrap() exists; .expect(...) does not. Only object headers record an allocator; container buffers use the global one. Threads start through a scoped with manager, and a pool is ThreadPoolExecutor(max_workers, queue_capacity) with map, map_result, and reduce_tree.

Current delivery and remaining design

The original recoverable-allocation foundation and explicit native concurrency have shipped. B23’s delivery record describes the initial explicit and automatic SIMD subsets. Generic collections are not packed numeric arrays, and ordinary loops remain serial.

Public allocator switching, per-container allocator handles, allocator lifetimes, and buffer provenance remain unimplemented. Objects and container buffers use the global allocator; objects no longer carry an allocator prefix (issue #814), so allocator selection must record provenance without enlarging global objects. Fixed inline arrays and constant value parameters are separately accepted but unimplemented under B83; bounded containers build on that design. These remaining contracts keep this proposal partly implemented. Scope and priorities belong in B13/B23/B24/B83.

The allocation, concurrency and SIMD sections preserve the original rationale and alternatives. Where they describe a shipped area, the linked references and completed audits take precedence over their historical sketches.

Original recommendation

Make recoverable allocation failure a language contract, with Result and ? as the normal way to handle it. This requires more than making collection growth fallible: error values, metadata, cleanup, and runtime helpers must also work without an allocation succeeding. Establish this foundation before advertising OOM recovery, custom allocators, or embedded suitability.

Keep the public model simple: one str, ordinary collections using a configured default allocator, and explicit constructors accepting an allocator when needed. Start parallelism with owned tasks, bounded channels, and scoped borrowing; add explicit parallel collection operations after those are sound. Start SIMD with a small set of explicit fixed-width vectors. Neither automatic parallelization nor a general auto-vectorizer should be necessary for the basic language.

Historical implementation audit

This snapshot predates fallible allocation, inline sums/ranges/generators, static type metadata, and atomic ownership counts. It explains the original proposal’s motivation, not current behavior. See the implementation status and native concurrency audit for the current contracts.

AreaOriginal behavior and implication
Raw object allocationplenty-runtime/src/memory.rs::allocate calls alloc_zeroed, then handle_alloc_error on null; layout overflow exits through fail.
Collection growthaggregates.rs::insert uses try_reserve for entries, but turns failure into process termination. Hash-table rebuilding uses an infallible vec! allocation.
Type metadataDescriptors are parsed into String, Vec, and Rc<Type> at runtime. Even constructing a small collection can allocate metadata.
Classes and enum valuesRecords have owned heap allocations. Standard Result and Option use enum machinery; constructing an error is not an allocation-free escape hatch.
Text and I/ODynamic text, input buffering, formatting, and collection rendering allocate. String literals are immortal compiler data.
Indexing and rangesString indexing/iteration creates a new managed one-scalar string; constructing a range allocates metadata and a managed range object.
Structural equalityaggregates.rs::equal memoizes aggregate pairs in a Rust HashSet, avoiding exponential work on shared graphs whose floating-point contents prevent identity shortcuts. Equality and membership can therefore allocate internally.
GeneratorsCreating the frame allocates; suspended values own resources. Returning Option from iteration also uses enum machinery.
DestructionAn intrusive thread-local queue avoids allocating a separate worklist while traversing owned object graphs. User __del__ code can still allocate.
SharingObject reference counts are non-atomic u64; metadata uses non-atomic Rc. Moving a collection does not prove its reachable immutable strings are unshared.

Rust does have fallible APIs: Vec::try_reserve preserves existing contents on failure. The familiar infallible APIs choose a different policy, and the standard allocation-error handler normally aborts with std. Plenty can expose a different policy while using Rust underneath, but must select or implement fallible operations consistently. Rust Vec, allocation error handling.

Recoverable allocation contract

This is the original design rationale. The completed allocation audit records the delivered per-path behavior, including inline classes, sums, ranges and generator frames. The third error category, fallible generator-frame construction and heap-backed class construction discussed here were not adopted; .expect(...) remains unimplemented. None is an additional implementation commitment.

Allocation-free error transport comes first

Introduce a small, non-owning AllocError value distinguishing exhaustion, capacity overflow, and an unsupported allocation layout. Its data should be fixed-size integers/tags, not an allocated message or backtrace. It can be formatted later if resources permit; an emergency reporting path writes a static message directly without growing a buffer.

Represent ordinary finite Result[T, E] and Option[T] values without a new heap allocation for their wrapper: use an inline tag/payload layout, register tuples, or a caller-provided result slot according to a private Plenty ABI. Moving an already-created payload into Ok or Err must not allocate. This is a concrete reason to generalize the current single-word managed-value representation. Recursive values still need explicit indirection; neither inline errors nor stack placement imply arbitrary recursive types have a finite layout.

Implementing ? before this representation change is useful for normal errors. It must not be described as OOM-safe while its generated error path still creates heap-backed enum objects. Compiler-emitted immutable type metadata is the next useful change: replacing runtime descriptor parsing removes hidden allocation and a future cross-thread Rc hazard at the same time.

Which operations return Result?

Recommend fallible public signatures for operations whose contract permits allocation, even on calls that happen to fit existing capacity. Their type must not change with optimization level, input size, or whether the compiler managed to place a particular object on the stack.

For example, a future items.append(value)? propagates AllocError, and a future copy(value)? can fail when duplicating owned data. Explicit .expect(...)-style termination may be offered for applications choosing that policy; it is not a silent default selected by release builds. A terminal failure is distinct from a recoverable allocation error and does not promise user cleanup.

There is a syntax decision to make before changing existing programs: allocating literals, comprehensions, string concatenation, heap-backed class construction, and generator creation have no explicit fallible method today. A consistent candidate is for these expressions to produce a Result, with explicit ? at the use site, including ([f(x) for x in source])?. This makes allocation visible but changes many familiar expressions and needs learner examples before adoption. An alternative is a narrowly specified allocation effect with an explicit handler boundary; it adds more language machinery and should not be quietly introduced as implicit ? insertion. The Result with explicit ? form was chosen.

The required behavior is clearer than the syntax: a program must be able to handle every language-runtime allocation failure without an unadvertised abort. Audit these paths together, not just list.append:

Allocation siteRequired recovery behavior
Lists, dicts, setsReserve entries and hash storage before committing a mutation; report checked-size overflow separately from exhaustion.
ComprehensionsDestroy the partial result and pending owned temporaries on failure. Already-completed I/O or other external side effects are not rolled back.
StringsConcatenation, formatting, conversion, slicing that copies, and input buffering expose failure; literals and merely retaining existing immutable storage need no allocation.
String indexing and rangesRemove their internal allocation through a suitable representation, or expose failure even though these operations currently look like simple value expressions.
Equality and membershipReplace allocating memoization with an allocation-free strategy, or make its failure explicit. A bool result must not conceal an abort when temporary comparison storage is exhausted.
Classes and enum payloadsClean up only successfully initialized fields if construction fails. No call to a whole-object custom destructor on a partially initialized object.
copyDestroy a partially built deep copy; retain the original unchanged. Resource objects with custom cleanup keep their existing copy restrictions.
GeneratorsFrame creation is fallible. Allocation errors in the body need an explicit yielded Result or a designed fallible-iteration contract, not an indistinguishable end-of-stream.
Metadata and dispatchPrefer emitted static tables. Any remaining lazy caches or adapter allocation must return errors or be preallocated explicitly.
Printing and diagnosticsStream through a nonallocating writer or propagate allocation/I/O errors. Formatting an error must not recursively allocate another error.
Scheduling and channelsPool creation, task records, queue storage, thread stacks, and message construction have distinct fallible creation/submission contracts.
FFIAn external allocator/library can still abort; an imported declaration must not claim recoverable failure unless its implementation provides it.

Mutation, moved inputs, and cleanup on failure

Guarantee the strong property for allocator-aware collection mutation: if reserve, rehash, or allocation fails, the existing collection’s logical contents remain unchanged. Capacity may have increased during preparatory work. A replace operation must not destroy the old value until all required allocation succeeds. Allocator failure must leave the original allocation valid on failed resize.

Recommend a simple initial ownership policy: an ordinary owning append consumes its argument on the call, including failure, and destroys it if it was not stored. The caller cannot conditionally reuse the moved binding after observing Err. Document this prominently. A later recovering variant can return the uninserted item together with AllocError, but that composite error must itself have an allocation-free representation. Another explicit workflow is reserve first, then move the value into already available capacity. Do not hide input cloning in the error path.

Destruction itself must not allocate. The existing intrusive queue is a useful starting point, not a complete proof: custom destructors, allocator callbacks, formatting used by cleanup, and cleanup of partially initialized values all need auditing. A destructor has no caller to receive Result; keep __del__ and scope exit hooks infallible and make fallible close/flush an explicit operation before drop. OOM-safe cleanup must handle any fallible work locally and continue releasing owned resources. A later compiler-checked noalloc effect could verify critical cleanup and request-rejection paths; initially keep this a documented runtime contract with tests, not an unimplemented guarantee. See the companion context-manager proposal.

The runtime needs a failure-injection allocator: fail allocation number N for every N in a test, then verify unchanged source values, balanced resources, partially initialized cleanup, and no secondary allocation on the failure path. Include failures during hash growth, nested copies, generators, output, and user cleanup. Success-only accounting and sanitizer tests cannot establish these failure guarantees.

What recovery can and cannot guarantee

An allocator reports only failures it can observe. On an overcommitting OS, successful reservation can be followed by process termination when pages are touched; a Result cannot intercept the kernel killing the process. Bounded allocators and request budgets make admission control more useful; reserved and committed-memory policy remains platform-specific. Linux explicitly documents multiple overcommit policies. Linux overcommit accounting.

Compiler OOM, process startup, stack overflow, and third-party C libraries are separate limits. Do not advertise survival of arbitrary system-wide exhaustion. For services, the practical guarantee is narrower and useful: reject work when a controlled allocator cannot satisfy a request, clean up, and continue using previously reserved service resources. An application needs a preallocated path for that rejection response too.

Custom allocators without multiplying everyday types

The user’s proposed separation is viable. Prefer one ordinary constructor with an optional allocator argument over unrelated list_custom/dict_custom names. Exact syntax depends on generics, but conceptually list[T](allocator=pool) uses the same operations as list[T](); literals and comprehensions use the configured default. An explicit collection builder can later direct a comprehension into a particular allocator without introducing ambient thread-local allocation scope.

Use a small runtime allocator handle containing a context pointer and operations for allocate, grow/resize, and deallocate. The contract includes alignment, checked sizes, thread eligibility, failure behavior, and ownership of the handle. This is compatible with statically known adapters for user classes; it does not require every ordinary collection type to expose an allocator generic parameter. The tradeoff is a stored handle and indirect allocation calls. Specialized allocator-generic collections can remain a later optimization.

Rust’s per-object Allocator contract is useful prior art, including matching deallocation and allocator lifetime, but the inspected stable documentation still marks it an unstable API. Plenty can implement its own stable runtime interface without exposing Rust’s allocator ABI. Rust’s GlobalAlloc also permits an implementation that aborts instead of returning null: a Plenty allocator advertised as recoverable must impose the stronger return-failure rule. Rust Allocator, Rust GlobalAlloc.

The difficult part is lifetime and provenance, not constructor spelling:

  • Each object and internal buffer remembers the allocator that created it, and frees through that allocator even if the default later changes. Growth stays with that allocator. Start with default selection fixed at startup to avoid unnecessary ambient state and initialization races.
  • The allocator’s context must outlive every live allocation and destructor using it. Start with process-lifetime or explicitly retained allocator handles. Handle creation is fallible and its own allocation must not form an ownership cycle.
  • A scoped arena is a borrow-checking feature: forbid reset/destruction while any arena-backed value or reference can survive. Deferring resettable arenas is better than using a runtime pointer that silently dangles after a scope ends.
  • An arena does not make resource cleanup optional. Destructors of files and other resources still run; bulk byte reclamation cannot replace them.
  • Inserting an existing string or nested list into a pool-backed list does not migrate its storage. The outer buffer uses the pool; each child retains its own allocator. A future explicit copy_into(pool) must define which reachable storage it duplicates and may fail partway through.
  • copy should default to the source allocator; changing allocators is explicit. No pointer allocated in a DLL may be passed to another DLL’s unrelated free function. FFI ownership contracts need the matching release operation and its library lifetime; Plenty’s allocator handle remains a private representation.
  • Thread-confined allocators make their values thread-confined. Merely making a reference count atomic cannot make an allocator or its destructor thread-safe.

Supporting arbitrary allocator callbacks is itself a trusted low-level extension; the compiler cannot prove that foreign callbacks honor alignment and lifetime. Built-in budget/pool allocators should be the first supported implementations.

Fixed capacity and inline storage

The ideas in heapless and smallvec solve different problems. heapless containers store fixed-capacity backing storage inline and report capacity exhaustion; SmallVec stores a small prefix inline and spills to the heap when needed. Inline means inside the containing value, not necessarily on the stack. heapless, SmallVec.

The original sketches were Array[T, N], BoundedList[T, N], and SmallList[T, N], with constant generic capacities. The accepted value-generics design now specifies array[T, N], but is not implemented; the bounded and small-list names here remain illustrative. A bounded container returns a capacity error without allocating; a small list still needs fallible spill/growth. Element construction can allocate even when the container does not. A borrowed slice/view should let APIs accept their contents without copying into the ordinary heap-backed list. Defer this family until basic constant parameters exist; inline aggregates have shipped. It should not complicate the first custom-allocator interface.

Threads, channels, and parallel collections

This section preserves the original concurrency design. Scoped threads, bounded channels, executors, ordered mapping and fixed-tree reduction are implemented; the concurrency reference owns current APIs, worker eligibility, ordering and shutdown contracts. The suggestions below are not additional pending concurrency work.

Runtime prerequisites

The native concurrency audit records the current runtime and boundary contract. Type metadata is static and immutable, and object counts now use atomic ownership handoff, including immutable strings: a list move may leave a string alias in the originating thread. Atomic ownership alone does not make a mutable payload safe to share or grant foreign thread permissions.

Retain static immutable type metadata and use atomic reference counts for storage that can be shared between threads, including the single public str type. Benchmark the cost before choosing whether all headers should use the same atomic implementation or whether internal unshared objects can keep a cheaper path. Either choice preserves one user-visible str type. Revisit the destructor queue algorithm when changing the count field: zero-to-destruction handoff, memory ordering, and recycling the dead count as a link need a coherent atomic design.

Define compiler-known structural properties for transferable ownership and shareable immutable borrows. These are safety facts, not protocols users can satisfy merely by writing a method with the right name. Ordinary methods remain explicitly resolved; importing a module must not enable extra thread eligibility. Check all captured values recursively, including allocator handles, generator frames, destructor requirements, and FFI handles with thread affinity.

Initial user model

  1. Owned tasks move captures and results. Thread creation returns Result, and join is explicit; no silent detached thread holding resources beyond main.
  2. Scoped tasks may borrow until the scope joins every task on every exit path, including ?. Shared borrows need thread-shareable values; mutable borrows require disjoint storage proven by the checker. Scoped threads are established prior art in Rust’s thread::scope.
  3. Bounded channels own preallocated queue storage. Sending transfers ownership; failed or disconnected send returns the undelivered value. Creation and payload construction are fallible; preallocating the queue does not eliminate allocations inside payloads. Specify closure, blocking, and shutdown behavior before adding unbounded channels.
  4. A configurable thread pool supports explicit parallel map/collect/reduce. Keep the executor API replaceable: the language need not promise Rayon as a dependency, and any chosen executor must meet the allocation contract.

Rayon demonstrates explicit parallel iterators and configurable pools, including num_threads; its parallel fallible traversal may observe multiple errors and does not specify which one is returned. These are useful comparisons, not a reason to inherit unspecified language behavior accidentally. Rayon pool configuration, Rayon fallible traversal.

For an initial Plenty parallel collection operation, preserve input order in the successful result. If multiple items fail, returning the failure at the lowest input index is easier to reproduce; it requires completing earlier in-flight work before reporting it. Cleanup drops all partial outputs and joins started tasks. It cannot undo I/O already performed by workers. A simpler first-completed error mode is an alternative, but must be explicitly documented if selected. Dictionary/set construction must still resolve duplicates using source order. The order of destructor side effects across threads is not the serial iteration order and must not be promised accidentally.

Cancellation is cooperative and happens at defined boundaries, not by killing threads during mutation or drop. Restrict break/continue in higher-level parallel loop forms until their semantics are specified. Floating-point reductions need an explicit order policy: reassociation changes results; a parallel sum cannot silently claim ordinary left-to-right arithmetic semantics.

Automatic parallelization is deferred. Borrow independence alone does not prove equivalence: allocation failures, I/O, externally visible destructor order, floating-point reductions, and first-error ordering are observable. Start with explicit opt-in operations, measure workloads, then consider an effect analysis for a small deterministic subset. A runtime thread-count setting is reasonable for these operations, but should not silently change ordinary loop semantics.

SIMD that fits Cranelift

This is the original backend assessment, not an outstanding initial SIMD plan. The explicit implementation record describes the shipped API and representation limits; the portable SIMD proposal separates delivered subsets from broader transformations. Current values use copyable inline enums, with bounded local promotion to vector registers, rather than a public packed-vector layout or vector C ABI. The initial automatic integer-column filter/count slice is separate from general auto-vectorization.

Cranelift represents lane-typed vectors and provides vector arithmetic, but an IR type’s existence does not promise every instruction or width works on every backend. The upstream IR guide explicitly discusses target-dependent support. Cranelift IR, instruction builder.

The installed cranelift-codegen 0.131.1 source was checked directly: src/ir/types.rs tests cover I32X4 and F64X2; src/isa/x64/lower.isle includes fadd rules for F32X4/F64X2; src/isa/aarch64/lower.isle lowers vector fadd. This is evidence for a small 128-bit starting set, not a blanket claim about 256/512-bit vectors, scalable vectors, arbitrary lane counts, or all operations. The docs.rs pages for that exact crate version were unavailable to the research tool, so the source inspection, rather than latest-version docs, establishes the version-specific finding.

Propose explicitly imported vector types/functions first: 128-bit shapes such as four f32 lanes or two f64 lanes, with splat, checked load/store, arithmetic, comparisons, masks, and lane extraction. Names such as simd.f32x4 are sketches. Represent vectors inline in SSA/registers or stack slots; do not heap-box each vector as general managed objects were boxed at the time of this assessment. Efficient vectorized kernels also need contiguous typed buffers/views: the current generic collection entry storage is not a guarantee of a contiguous C array of lane values.

Specify exact integer overflow behavior, mask representation, lane order, alignment, bounds, and floating-point behavior. Scalar fallback must preserve those semantics, not merely produce approximately similar answers. Unaligned loads are a useful safe default where supported; never read past a slice end to fill a vector, even if discarded lanes appear unused. Keep scalar tails explicit at first. SIMD reduction order and NaNs must follow the documented operation; relaxed reassociation and fused arithmetic require opt-in semantics.

Compile a documented baseline target with scalar fallback for unsupported operations. Offer explicit native-target compilation separately, recording the required CPU features; shipping native-target binaries to an older CPU can fail. Runtime feature dispatch and multiple compiled variants are later library/compiler features with code-size and compile-time costs. SIMD vectors should not cross the initial public C ABI by value: use pointers and lengths until each target’s ABI and alignment rules are deliberately supported.

Before adopting a vector operation, test native and scalar implementations on boundaries, NaNs/infinities, overflow, short/unaligned buffers, and each supported target. Inspect generated code and measure throughput and compilation time. Explicit vector lowering is compatible with Plenty’s fast-compilation goal; global loop dependence analysis and speculative vectorization are not required.

Original implementation sequence

This sequence records the original dependencies, not the current work queue. Delivery status is reconciled below it; remaining scope lives in the backlog.

  1. Add ordinary ? and clarify cleanup on propagated errors. Explicitly document that current heap-backed Result is not yet an OOM-recovery mechanism.
  2. Design allocation-free finite sum/aggregate layouts and emitted static type metadata; measure compile time and ABI complexity before wider runtime changes.
  3. Introduce a fallible allocation ABI and failure-injection tests, then migrate every runtime allocation path. Settle allocating-expression syntax with small tutorial examples before changing all public APIs.
  4. Add a configured default allocator and explicit per-container allocator handles, starting with a bounded budget allocator. Keep scoped resettable arenas deferred until lifetimes can be enforced.
  5. Add contiguous buffers/views and bounded inline containers as generic/value support permits. These also establish a practical foundation for SIMD.
  6. Audit thread safety and add transfer/share checking, atomic shared ownership, owned/scoped tasks, and bounded channels. Then explicit pool-based parallel collection operations with documented ordering and failure behavior.
  7. Add the narrow tested SIMD surface. Automatic parallelization, auto-vectorization, allocator specialization, and runtime CPU multiversioning remain optional later work rather than prerequisites for useful programs.

Steps 1-3 and 6 are complete; their descriptions of the then-current runtime are historical. Step 4 remains B13. Step 5 is mixed: inline aggregates, list/byte slice views and packed columnar storage have shipped, while fixed inline arrays and value parameters remain B83 and bounded containers remain B13. Step 7’s initial explicit vectors and bounded automatic integer-column SIMD have shipped; B23 tracks extensions, not an unimplemented SIMD foundation.

Allocating-expression syntax, the private sum-value lowering contract, and task error ordering are settled. Existing collection operations also define ownership on failure; see the collection reference. Any future recovering-insertion API would be a separate design, not an unresolved contract for today’s operations. Allocator handle lifetimes and buffer routing remain open B13 design questions. Published SIMD operations already have scalar contracts; broader operations require separate decisions in the portable SIMD design.

The private lowering contract is not a frozen ABI or a public C representation; it must leave room for future FFI, vector values, target-specific calling conventions, and further representation improvements.

Thread-local reference counts

Status: Not implemented; stage 1 measured and rejected. Design note for issue #746. Stage 1 was built (issue #751, local PR #944) and removed every locked count instruction from k-nucleotide without a measurable cycle gain, so the PR was closed and stage 2 is not recommended; see measured result. The memory and destruction and concurrent runtime pages own the current contract. Issue #744 (no count pair around borrowed arguments) and issue #745 (sole-owner release) have landed.

Summary

Every runtime count update is a lock-prefixed read-modify-write, although most objects never leave their thread. This note proposed the Lean 4 and Koka design, a per-object thread-shared mark set at certified thread boundaries, starting with a process-wide single-threaded phase that needs no marking.

Stage 1 was built and removed every locked count instruction from k-nucleotide, with no measurable change in cycles. On the measured processor the locked instructions are not the cost, so neither stage is recommended. The design below is kept as the record of what was considered.

The note also found that the retain half of issue #745’s fast path is unsound without a thread-locality guarantee; see the sole-owner fast path.

Measured result

Stage 1 was implemented as described below (issue #751, local PR #944): a live-thread count raised only in plenty_thread_start and lowered only after plenty_thread_join returns, with plain count updates while it is zero and a runtime-checks trap for count updates from unregistered threads. Measured against main at d7a70ede (issues #744 and #745 landed) on an AMD Ryzen 7 7840HS (Zen 4), with callgrind and perf stat -e cycles:u pinned to one core:

WorkloadLocked count instructionsInstructionsCycles
k-nucleotide (fasta 50,000)1,399,543 to 0+0.2%Within noise (under 2%)
reverse-complementAbout 0 before and afterUnchangedWithin noise
Parallel k-nucleotide (7 pool jobs)Unchanged: workers are live+0.4%Within noise
Retain/release loop, 50M iterationsAll removedAbout 3 more per operationUnchanged

In the loop benchmark, plenty_retain and plenty_release fell from 46% to 7% of cycle samples while total cycles stayed the same.

The premise of this note was wrong for this processor. Issue #743 attributed k-nucleotide’s 26.5% of cycles in plenty_release to the lock decq, because 96% of that function’s samples fell on the instruction after it. An uncontended locked operation on a cache line the thread already owns is cheap on Zen 4; the samples land after the first access to the object’s header, which is the likely real cost, attributed to the next instruction. This has not been confirmed with cache-miss counters, and processors with costlier atomics (some AArch64 cores, older x86) were not measured.

Consequences:

  • Stage 1 adds about three instructions per count operation for no measured gain, so it was not merged.
  • Stage 2 removes the same locked instructions on parallel programs and adds marking walks; on this evidence it is not worth building.
  • The cost to attack is touching object headers: fewer count operations (issue #755, a last-use move after a borrow), fewer separately allocated objects, and confirming header misses with cache counters before further work on count atomicity.

Why counts exist and why they are atomic

Plenty has single ownership at source level, but runtime objects begin with {AtomicU64 refs, destroy} (plenty-runtime/src/memory.rs). A count above one arises from:

  • copying an immutable heap str, or an inline value containing one, which retains instead of copying;
  • Shared[T] handles, whose share() retains one box;
  • native helpers that return retained projections and builder aliases, and the retain/release pairs still emitted around some borrowed arguments and receivers (issues #744, #741).

Strings of up to seven bytes are inline and own nothing; their count calls return before touching memory. Literals use the immortal count u64::MAX.

Counts are atomic because counted owners can cross certified boundaries, and because a worker can retain through a borrow: a scoped worker that copies a str out of a borrowed list[str] increments a count the parent may be incrementing at the same time. A release site cannot tell whether that ever happened to its object.

Release uses fetch_sub(Release) and an acquire fence on the final count; retain uses a relaxed compare-and-swap loop that rejects overflow. After the final release the count word becomes the destruction queue’s link (plenty_release, continue_after), on the destroying thread only.

Where an object can reach another thread

BoundaryWhat crossesConcurrent count access?
Scoped spawn with &T arguments or spawn(&job) capturesBorrowed parent graphYes: worker and parent both retain
Scoped spawn with moved, copied or &mut T argumentsOwned or exclusively borrowed graphYes: a str inside can share an object with a parent local
Scoped join resultWorker resultNo: the worker’s frame is gone and join orders it
pool.submit and consuming spawnJob environmentYes, as for moved arguments
Executor map’s shared environmentOne environment borrowed by every jobYes
Future resultJob resultNo: the adapter finishes, including environment cleanup, before Phase::Ready is published under the job lock (executors.rs)
Channel sendMessageYes
.par pipelines, ordered map, fixed-tree reduceBorrowed list and callback capturesYes
Shared[T] constructionContents, and the box itselfYes: every holder retains the box and copies from the contents
Store under a Mutex lockThe stored valueYes, when the mutex is reachable from another thread
std.actorMessages are copied into the receiver’s heap; all actors run on one threadNo
FFIForeignPtr cannot enter workers; exports require one caller threadNo
GlobalsNone exist; constants are immortalNo
Runtime-internal threadsThe detached DNS lookup thread in aggregates/net.rs holds only Rust dataNo

Every Plenty thread starts in threads::plenty_thread_start (scoped tasks and executor workers alike). threading::check in src/threading.rs already walks each crossing type for eligibility, with src/effects.rs classifying the native operations a worker can reach; both are compile-time type walks. Marking adds a runtime walk over values of the same types.

Prior art

SystemMechanismFit for Plenty
Lean 4m_rc > 0 single-threaded, < 0 multi-threaded, 0 persistent. lean_dec_ref inlines the rc > 1 case; lean_dec_ref_cold does the atomic fetch_add on negative counts. lean_mark_mt walks the graph with a worklist and negates counts; alloc_task calls it on a task’s closure, and task results are marked when resolved. Marks are never removed. Lean copies a shared object instead of updating it in place, so a marked object never points to an unmarked one.Good. Boundaries are explicit and compiler-known. Plenty mutates in place, so it needs a store barrier where Lean copies.
Koka (Perceus)Follows Lean: thread-shared objects have negative counts, so one inlined rc <= 1 test takes the slow path for both “shared” and “last reference”. tshare marks recursively; starting a thread marks its argument; storing into a thread-shared mutable reference is the other sharing operation.Same as Lean; Koka’s mutable-reference rule corresponds to Plenty’s Mutex stores.
Biased RC (Choi, Shull, Torrellas, PACT 2018); CPython 3.13 free-threaded (PEP 703)Each object has an owner thread with a plain local count; other threads use an atomic shared count. A negative shared count queues the object for the owner to merge. Swift prototype: RC operations over twice as fast, client programs 22.5% faster on average. CPython uses separate ob_tid, ob_ref_local and ob_ref_shared fields, immortal objects and deferred counting.Poor. Needs an owner id per object and a thread-id comparison per operation; the 16-byte header has no room, and owner-thread merge queues conflict with the destruction queue’s reuse of the count word and with pool threads that finish. Its advantage, no walk at a boundary, matters less when the compiler knows every boundary.
SwiftAtomic counts on every class reference. OSSA copy_value/destroy_value and copy propagation delete redundant pairs. -assume-single-threaded makes all reference-counting instructions nonatomic, unchecked. Ungar, Grove and Franke (DLS 2017) report ARC taking up to 80% of time and propose dynamic atomicity: a store barrier plus generated tracers that mark objects reachable from shared storage.Plenty’s current model. Pair elimination is issue #744. The phase flag below is a checked, dynamic -assume-single-threaded. Dynamic atomicity is the per-object design with a store barrier.
RustRc is non-atomic and not Send; Arc is atomic. Owned values have no count.Would split str into local and sendable forms, a user-visible distinction the language avoids. Shared[T] already plays Arc’s role.
Nim ARC/ORCCounts are never atomic; isolate checks at compile time that a subgraph has no outside references, and Isolated[T] moves it between threads. --mm:atomicArc is the thread-safe variant.Moved Plenty values are not isolated, because str copies share storage. Copying strings at each boundary would add hidden allocation, and Shared[T] contents are read by several threads anyway.

Recommendation

Stage 1: single-threaded phase

Keep a process-wide count of live Plenty threads other than the main one. plenty_thread_start increments it before pthread_create; the parent decrements it after the corresponding join or pool shutdown. While it is zero, retain and release use plain (relaxed load and store) updates:

release(o):
    if inert(o): return
    r = o.refs.load(relaxed)
    if r == IMMORTAL: return
    if live_threads.load(relaxed) == 0:
        if r == 1: queue o for destruction      # no fence: no other thread
        else: o.refs.store(r - 1, relaxed)
        return
    atomic path as today

Soundness: only the main thread runs while the count is zero, and it alone changes the count from zero. Its plain updates before a pthread_create happen before everything the new thread does; every update while workers are live is atomic; pthread_join orders every worker update before the plain updates that follow. A pool’s idle workers keep the count above zero until shutdown joins them. The destruction queue, continue_after, immortal counts and actor heaps are unchanged.

Cost: one load of a read-mostly global and a predictable branch per count operation, next to the existing inert and immortal tests. No marking, no header change, no codegen change. Programs that start threads lose nothing while workers are live.

Stage 2: per-object thread-shared mark

Only if a parallel workload shows atomic count cost while workers run.

  • Encoding. Bit 62 of refs marks a thread-shared object; u64::MAX stays immortal. One unsigned compare, refs < SHARED_BIT, selects the plain path for both retain and release; shared and immortal objects take the slow path, where immortal is tested first. Counts stay below 2^62 - 1.
  • Marking. At each “yes” boundary in the table above, the compiler emits a call to a marking walk for the crossing value’s type. A type that cannot reach a heap str, a Shared box or an aliased helper result needs no walk: list[i64] or Bytes costs nothing, list[str] visits each heap string. The walk uses the destruction queue’s worklist pattern, not recursion, and plain stores: the objects are thread-local until the boundary publishes them.
  • Store barrier. Plenty mutates in place, so a marked container can later receive an unmarked value. Stage 2 marks values stored under a Mutex lock, and walks do not stop at already-marked objects. Stopping early would need a barrier on every store into a marked container, as in Ungar et al.
  • No unmarking. A marked object stays marked, as in Lean and Koka: after a join another thread may still own a copy received through a channel.
  • Destruction. The final release of a marked object keeps today’s release/acquire handoff. Its children are marked, so the destroying thread updates them atomically. The queue link overwrites the mark only in a dead object, on its destroying thread.

Costs: the walk at each crossing, proportional to the reachable heap strings, boxes and aliases (a scoped spawn over a large &list[str] walks all of it, once per spawn); a compare per count operation, which replaces the lock instruction on the common path; the Mutex store barrier.

The sole-owner fast path

Issue #745 proposes that a release reading refs == 1 destroy without an atomic operation, and that a retain reading 1 store 2. The release half is sound: the releasing owner holds the only count, and no borrow of that owner can be live on another thread while it is released, so no other thread can retain concurrently.

The retain half is not. Retains happen through borrows, not only from owners:

parent owns s: str          # refs == 1
spawn(worker, &s)           # worker copies s: reads 1, stores 2
copy s on the parent        # concurrently reads 1, stores 2

Both threads store 2 for three owners, and the object is freed while one copy is live. The same happens with two pool jobs copying an element of a Shared[list[str]]. A plain retain needs a thread-locality guarantee, which stage 1 provides for the single-threaded phase.

Expected gain

Executed lock-prefixed count instructions, counted with callgrind --dump-instr=yes --compress-pos=no at the addresses of lock decq in plenty_release and lock cmpxchg in plenty_retain, on main at fe11d0a2 (release compiler, --no-strip):

Program (input)Release callsAtomic releasesFinal releasesRetain callsAtomic retains
k-nucleotide (fasta 50,000)4,320,7061,815,225222,0252,569,7721,315,181
reverse-complement (fasta 50,000)1,536,7041,536,70431,536,7011,536,701
fasta (100,000)1,033,4011,033,40171,033,3921,033,392
regex-redux (fasta 50,000)412,613281,838246232,686231,533

The other benchmarks perform under 1,000 count operations each. The calls not counted as atomic are inline strings and literals.

  • Almost every atomic operation is a non-final pair at refs >= 2. In k-nucleotide the call sites in count_frames are the pair around get_mut(&key) (issue #744) and pairs around borrowed Bytes receivers; in reverse-complement all of them are the Bytes.get/Bytes.set receiver pairs of issue #741. Issue #745 handles the final releases: 12% of k-nucleotide’s atomic releases, almost none elsewhere.
  • Once those pairs are gone, stage 1 removes whatever atomic operations remain in these single-threaded programs. The size of that remainder is unknown until issues #744, #741 and #745 land; from the call sites it is expected to be small, so the additional gain on k-nucleotide and reverse-complement is likely a few percent at most.
  • The upper bound, if pair removal covered none of them: in k-nucleotide at 250,000, plenty_release and plenty_retain take 30.4% of cycles (issue #743); in reverse-complement about 60% (issue #741).
  • Stage 2 gains nothing on these programs over stage 1. Its gain on parallel programs is unmeasured; BRC’s 22.5% average for Swift client programs is an indication, not an estimate for Plenty.

What to measure

  1. Before and after each stage, the executed lock count-instruction totals above, the final-release count, and callgrind instruction totals for k-nucleotide and reverse-complement; then bench.py run timings.
  2. A concurrency-heavy workload with workers live during the counting: a parallel k-nucleotide counting each frame on a pool job, as the Rust #1 reference does; a channel pipeline of str messages; and readers of a Shared[dict[str, i64]]. Stage 1 should leave these unchanged; they decide whether stage 2 is worth building.
  3. For stage 2, the marking cost: spawns in a loop over a large borrowed list[str], and channel sends of list[str] messages.
  4. Correctness: runtime tests in which several threads retain through a borrow, including the scenario above, and a stage-1 test that the counter returns to zero only after join and pool shutdown.

Decisions

  1. Stage 1 was accepted as the first step, then measured and rejected; see measured result.
  2. The live-thread count was to return to zero after the last join, decremented by the joining thread after join returns; detached or unjoined threads keep it raised.
  3. Issue #745 dropped the retain fast path and landed the release half only.
  4. The parallel k-nucleotide variant in benchmarks/game (local PR #940) is the threaded workload.
  5. Any future thread that runs Plenty code, such as the proposed actor blocking-work threads, must start through plenty_thread_start. This only matters if count modes return.

Sources

Portable SIMD without everyday SIMD bookkeeping

Status: Initial slices implemented; broader design remains proposed. The explicit module now supplies concrete portable lanes, masks, scalar fallback and bounded native lowering. See its implementation evidence for API differences and representation limits. This proposal for B23 was researched against Plenty 7654ecca on 2026-10-23. The current pipeline contract implements serial fusion and bounded scalar callback substitution. The first implemented automatic SIMD slice is integer-column filter/count; the broader API extensions and recognition patterns below remain proposals. This refines the SIMD portion of the earlier memory and parallelism proposal.

Recommendation

Give ordinary numeric code SIMD benefits without making its author select an instruction set, vector width, mask layout, alignment, or tail strategy. Introduce two cooperating facilities:

  1. A small portable std.simd module for algorithms that genuinely need explicit lanes, comparisons and selection. Follow Rust’s portable semantics and value model, not its entire trait, pointer and intrinsic surface.
  2. A bounded compiler transformation for recognizable fluent pipelines. Prove that a transformation preserves the existing scalar program; otherwise keep the scalar program. No .simd() marker or purity assertion should be required for the safe common cases.

An ordinary pipeline should be the recommended starting point. Explicit SIMD is an advanced tool, not a prerequisite for efficient arithmetic. Compile-time diagnostics can explain missed opportunities without making correctness depend on which optimizer or CPU is selected.

The API contract places bounds, alignment and representation complexity in the library/compiler. The automatic transformation design states exactly what the compiler must prove. The spike evidence includes reproducible native tests and backend comparisons.

Portability is a semantic promise

Public names describe lanes and operations, never SSE, AVX or NEON. A fixed logical vector can lower to one instruction, multiple narrower instructions, or scalar operations. The chosen lowering must preserve the same lane results, bounds behavior and arithmetic failures. “Portable” does not mean every target has equal speed or an implemented Plenty runtime today.

Start with tested 128-bit logical shapes and a scalar implementation of every published operation. Keep masks opaque and vector layout out of the public ABI. Avoid arbitrary lane-count generics, scalable-vector types, native register exposure and CPU multiversioning in the first API. Those are separate design choices, not hidden prerequisites.

Contiguous numeric storage matters more than vector syntax. Existing generic lists are not promised to be packed arrays of lane values. Do not silently copy them into scratch arrays to claim automatic SIMD. Counted ranges can benefit without new storage; packed numeric buffers/views need a deliberate B12/B19 contract before direct vector loads become an everyday list optimization.

Relationship to automatic parallelization

Both transformations need source bounds, visible callback bodies, effect summaries, and a cost model. Share that analysis and explanation machinery with B24 and the fluent pipeline design, but keep the decisions separate. SIMD runs on the current thread and does not introduce scheduling, thread transfer, cancellation, synchronization or thread-pool overhead.

“Serial” continues to mean serial observable semantics: the implementation may evaluate independent, total scalar operations in vector lanes. This clarifies the earlier proposal’s “always serial” wording without enabling implicit parallel workers. Existing explicit pool operations retain their own ordering contract; vectorizing inside a worker is an independent later opportunity.

Cranelift or LLVM?

Keep the API and candidate analysis backend independent. The spike shows that Cranelift can generate useful 128-bit SIMD for x86-64 and AArch64 when supplied vector IR. It also shows real limits: direct 256-bit i32x8 was rejected on both tested targets, scalar CLIF loops stayed scalar, and LLVM produced materially better code for a simple constant-multiply kernel. See the evidence page for measurements, tool versions and the limits of that comparison.

Cranelift therefore does not block the initial facility, but fast compilation alone is not a reason to accept persistently poor hardware utilization. Do not build a growing general optimizer in Plenty just to avoid evaluating LLVM. Use these adoption gates before a production implementation commits to a backend:

EvidenceDecision it should trigger
Bounded recognition plus 128-bit lowering reaches useful speedups at low compilation costShip a narrow Cranelift path with scalar fallback
A small, semantics-preserving local rewrite closes a measured gapAdd it only with differential tests and a bounded complexity budget
Important workloads need wider/scalable vectors, gathers, sophisticated dependence analysis or repeated backend-specific rewritesPrototype the same typed kernel representation through LLVM; reconsider backend timing
LLVM remains substantially faster across representative application kernels after equivalent arithmetic and alias contractsConsider an optimizing build mode or an earlier primary-backend switch
LLVM benefits are isolated, compile latency or packaging costs dominateRetain Cranelift and explicit specialized kernels; do not maintain two full backends by default

For the decision, measure cold and incremental end-to-end compilation, optimizer time, executable size, runtime and memory on multiple input sizes and targets. Include bandwidth-bound, reduction, branchy/filter, and small-input workloads. Compare equivalent safety checks, target features, alias assumptions and floating semantics. This spike is a reason to investigate LLVM, not a migration benchmark. The earlier GCC backend investigation is related context, not a dependency of either SIMD approach.

Implementation boundaries and acceptance

B23 in the backlog owns prioritization. A sensible dependency order is a private lane-value representation with scalar oracle, the small public module, then narrowly recognized automatic candidates. Range reductions can be investigated alongside the module; neither depends on automatic threading. Packed-buffer algorithms depend on the storage contract, not on finishing all allocator work.

Before adoption, require native/scalar differential tests, target capability tests, tail and alignment checks, preserved diagnostics, and code-generation evidence. Run the AArch64 cases on hardware before claiming execution support. Teach ordinary pipelines first when the transformation ships, then show explicit vectors only where they clarify a real algorithm. Until then, keep these sketches out of the reference, signature index and executable tutorial.

A small portable std.simd surface

This records the proposed explicit API for portable SIMD. The initial module implements a subset using existing copyable enums, module factories and named operations; its implementation assessment explains differences from this sketch. Packed views, static type factories, operator syntax and fully opaque logical masks below remain proposed.

Values users can understand

Start with concrete types such as simd.f32x4, simd.f64x2, simd.i32x4 and simd.u32x4. Each is an inline fixed-size value with ordinary copy semantics, not a heap object or a promise of one hardware register. Qualify additional integer widths only when their operations and scalar fallback are tested. Keep a closed initial set rather than making users learn const generics or backend capability traits. Named aliases can survive a later generic design.

Rust’s Simd<T, N> is a useful model: lane operations, splats, comparisons, selection and portable load/store behavior. Its full API is not the starting scope. In particular, do not expose raw pointer vectors, unsafe loads, arbitrary gather/scatter, dynamic swizzles or architecture-specific intrinsics by default.

SurfaceProposed contract
splat(value)Repeat one scalar into all lanes; allocation free
Fixed-arity new(...), lane(index)Construct without allocating a list; lane index is checked
Floating +, -, *, /Scalar lane semantics, without implicit fusion/reassociation
Integer +, -, *Checked like ordinary Plenty; first implementation may scalarize checks
Integer wrapping_add/sub/mulExplicit modular arithmetic, matching existing scalar methods
eq, lt, le, etc.Produce an opaque lane mask, not one boolean
mask.select(yes, no)Select already-computed lanes; this is not lazy conditional execution
mask.any(), mask.all(), mask.count()Boolean/count summaries; no exposed integer encoding
load(view, offset), store(view, offset)Check a complete vector fits; accept scalar-aligned memory; no overread
wrapping_sum() on integer vectorsExplicit modular reduction, independent of grouping
sum_ordered(initial) on float vectorsScalar left fold in increasing lane order; no speedup promised

Each comparison mask is compatible only with vectors of the corresponding lane count/shape; there are no implicit integer-mask casts. Avoid operator-overloading ambiguity around equality returning a mask: explicit comparison methods suffice. Mask bit representation, alignment and calling convention are private.

Keep Plenty’s arithmetic contract

Rust SIMD integer operators wrap. Plenty’s ordinary integer operators trap on overflow, so copying Rust’s operator behavior would make changing a scalar into a vector silently change the algorithm. Keep checked operators, expose wrapping by its existing name, and allow checked vectors to use scalar fallback initially. For an explicit checked operation, report the lowest failing lane and operation site. Do not publish partially written stores after a failed bounds check.

No initial integer division, shifts with new count rules, saturating casts, approximate reciprocal, fused multiply-add, or min/max with underspecified NaN behavior. Add each only with a complete scalar contract and target tests. Explicit fused or relaxed operations can be considered later; their names must make the semantic difference visible.

Floating lanes follow Plenty’s scalar rules for signed zero, infinities, NaNs and subnormals on a supported target. Do not promise bit-identical NaN payloads across platforms beyond the scalar contract. Rust documents subnormal flushing on some older architectures; Plenty must either provide an appropriate fallback or document the target’s scalar contract before enabling SIMD there. Do not silently weaken scalar semantics to accommodate an instruction.

Bounds, tails and ownership belong below the algorithm

Safe load/store accept the eventual checked contiguous numeric view, retaining its owner and borrow lifetime. No caller-supplied alignment proof is needed. Loads do not allocate; stores require an exclusive mutable view and validate all destination bounds before writing. The initial C ABI uses pointers/lengths and scalar elements, never a vector passed by value.

For manual kernels, provide a library block-view helper once packed views exist: conceptually simd.blocks4(view) splits a borrow into full four-lane blocks and a scalar tail. Each block has a safe load()/store() pair. The helper checks the split once, does not allocate or create overlapping mutable loans, and requires the tail to be handled explicitly. This is an ergonomic target, not an implemented borrowing API. Do not require every user to write alignment prefixes or masks for ordinary remainders.

Prefer full blocks plus scalar tail over padded masked evaluation initially. Padding memory safely is insufficient if inactive lanes divide by zero or trap on a cast. A future masked-load API needs an explicit inactive value and memory contract; selection alone does not suppress evaluation of either operand.

For everyday algorithms, automatic pipeline lowering performs the split and tail handling invisibly. Users reach for explicit vectors only when lane relationships are part of the algorithm. If common examples require repeated mask and tail bookkeeping, improve the safe helper or the automatic pattern before broadening the intrinsic catalog.

Target policy and explanations

Use a documented deployment baseline by default. A native-target build may require newer CPU features, but that is a build contract, not a different source API. Runtime dispatch, feature probing and multiple compiled variants remain separate extensions with code-size and startup costs.

Lack of a SIMD instruction must never make a valid portable program fail to compile: split or scalarize. A vector IR type’s existence is not a support check. Query a tested operation/type capability table before lowering and exercise its fallback path in tests. A backend compilation error should be a compiler defect, not the user’s mechanism for selecting vector widths.

Initial explicit SIMD implementation

This implements the first std.simd module, following the API proposal with deliberate limits. It is separate from the compiler’s packed-column automatic SIMD.

API and representation decision

Rust’s local source checkout at 080cc271463 supplied the model, particularly library/portable-simd/crates/core_simd/src/vector.rs, masks.rs, and ops.rs: logical lane values, splats, simd_* comparisons and mask selection. Plenty preserves checked integer arithmetic under add/sub/mul; its wrapping_* methods explicitly request Rust-style modular lane arithmetic. No Rust source was copied into the implementation.

The first types are concrete i32x4, u32x4, f32x4, f64x2, mask32x4, and mask64x2. They use existing copyable inline enums, so ordinary ownership, borrowing, aggregates and function boundaries work without new syntax or a new runtime ABI. Module splat/load factories and the Lanes constructor replace Rust’s static methods. Logical mask booleans are public; native mask encoding is not. Safe list loads/stores assemble ordinary list elements and require a full vector, rather than promising packed numeric list storage.

Every published operation has an ordinary Plenty scalar body. The optimized x86-64 path substitutes only trusted embedded methods with exact signatures, using a closed table of 128-bit operations supported by the baseline backend. Other hosts retain the scalar bodies. This adds no supported target or runtime ISA dispatch. Checked integers and ordered float folds remain scalar.

Integer wrapping_sum lowers directly to lane extraction and modular adds. This avoids both canonical argument materialization and an ABI call at loop exits; the second accumulator in separate reductions no longer has to remain live across the first reduction’s call (issue #444).

Keeping local values in registers

A first prototype emitted vector instructions but materialized every result as four tagged scalar slots. Its integer recurrence took about 172 instructions per iteration versus 13 for four scalar accumulators, and roughly 28 times the elapsed time. That was an architectural failure, not an acceptable SIMD result.

The implementation therefore promotes exact numeric-vector locals to private vector SSA variables. Parameters, generators, masks, and any local that is mutably borrowed or moved retain canonical storage. Shared borrows materialize valid snapshots; the ownership checker prevents source mutation during the loan. Fallback calls and container operations still receive normal enum values.

Cranelift conservatively retains stores to unused stack slots. The lowerer records its private snapshot instructions, roots snapshots whose SSA results reach any ordinary instruction, propagates dependencies between snapshots, and removes only unobserved groups. Calls, returns and block arguments are all uses. The vector-producing arithmetic is outside these groups. Thus supported local chains and loops retain vector values without repeatedly scattering lanes. This is bounded to the explicit module, not a general escape optimizer.

Externally stored four-lane values still have 64-byte tagged payloads plus the ordinary handle. The public API promises logical lanes and no allocation, not a packed 16-byte representation, one register, or a foreign vector ABI.

Evidence and limits

The reproducible experiments in experiments/simd/explicit/README.md in the repository exercise checked results, dependency chains, and throughput on the same host. They retain unfavorable cases rather than selecting only wins. On the initial Ryzen 7 7840HS runs, eliminating materialization reduced the integer recurrence to approximately 14 instructions per iteration. Its SSE2 multiply sequence was still slower than four scalar accumulators. A single four-lane float recurrence was close to scalar speed; four independent vectors of divide/add work showed a useful throughput gain. Retained final timings and counter samples live with the experiment.

The integer multiplication assessment isolates baseline instruction synthesis, dependency chains, newer-ISA behavior and an independent exit-shape spill limitation with the same integer workload.

The module is useful for explicit lane algorithms, but it does not make generic lists packed or remove all ABI costs. Measure the actual algorithm. Packed views, wider vectors and more capable target lowering remain separate decisions; the implementation does not implicitly relax floating-point order to improve a benchmark.

Differential tests cover integer endpoints and all comparisons, modular and checked arithmetic, distinct float lanes, NaNs, signed zero, subnormals, mask selection, full-fit bounds, no partial stores, nested values, copy independence, mutable projections/captures, shared references, loops/joins and scalar fallback. Code-generation checks require baseline vector arithmetic while checked integer operations remain scalar.

Why the integer multiply probe is slower

The initial module keeps the recurrence in vector registers, but baseline x86-64 lacks a single instruction for all four low 32-bit products. The original probe compares four independent scalar accumulators with one vector accumulator. Both compute exactly the same four modular multiply/add recurrences.

What actually executes

Optimized CLIF contains imul.i32x4 followed by iadd. Cranelift 0.137.0’s isa/x64/lower.isle (lines 1168–1183, source commit ebe5e8df1f4e12c5fc2f5954926881d8d61367d8) selects this baseline sequence:

pshufd      value, odd-lane-order
pshufd      factor, odd-lane-order
pmuludq     original-value, original-factor
pshufd      even-products, low-halves
pmuludq     odd-values, odd-factors
pshufd      odd-products, low-halves
punpckldq   even-low-products, odd-low-products
paddd       increment, result

The scalar loop has four independent imul plus add/lea pairs. The vector loop has shuffle, multiply, shuffle, interleave and add dependencies before its next iteration. Similar instruction totals therefore do not imply similar critical paths. The hot loop has no tagged-lane gathering or scattering. Its one invariant stack load is bookkeeping; canonical vector construction and final extraction are outside the loop.

An experimental compiler enabled only has_sse41; the production compiler still uses its existing baseline. The same CLIF then becomes pmulld plus paddd. Intel documents PMULLD as the SSE4 operation retaining each product’s low 32 bits; Rust’s matching intrinsic requires SSE4.1. See the Intel intrinsic reference, page 136 and Rust intrinsic implementation.

Same workload, controlled comparisons

Final measurements on the Ryzen 7 7840HS use one million iterations per timing sample and ten million for counters. The numbers below are median user-space cycles per iteration from three counter runs, including small process overhead; they are observations for these loops, not instruction-latency specifications.

KernelScalarBaseline SIMDExperimental SSE4.1 SIMD
Four lanes, constant factor4.087.124.06
Four lanes, runtime factor4.077.114.05
Eight lanes, two independent vectors8.098.174.27

The runtime-factor variant removes scalar immediate-multiply specialization; its result is essentially unchanged. Two independent vectors hide more of the dependency cost: baseline throughput reaches scalar parity, and SSE4.1 is about 1.9 times faster. A single SSE4.1 vector only reaches parity with four scalar chains. Fewer instructions alone are not a speedup guarantee.

Instructions per iteration for the runtime-factor case are approximately 13/14/8 for four lanes and 21/22/10 for eight lanes (scalar/baseline/SSE4.1). All variants check identical modular results. Reproduction sources, raw samples, counter output and load context live in the repository’s experiments/simd/explicit/ directory; start with its README.md.

The original exit-shape spill and its fix

The first eight-lane variant finished with v0.wrapping_sum().wrapping_add(v1.wrapping_sum()). Its generated loop spills one vector with a movdqu load/store pair, costing about 16.2 baseline cycles or 13.0 SSE4.1 cycles. Changing only that final expression to v0.wrapping_add(v1).wrapping_sum() removes the spill and yields the table above. This rearrangement is valid for modular addition; it is not a license to reassociate checked or floating arithmetic.

The spill-free loop uses only six XMM registers. Issue #444 traced the important distinction: the second vector remained live across the first reduction’s scalar ABI call and caller-location hooks. On the tested System V target, XMM registers are caller-clobbered, and the allocator placed that vector’s spill across the loop. The combined reduction had no second live vector at its single reduction call.

The fix lowers the existing integer wrapping_sum directly to lane extraction and modular additions. It removes the reduction calls and their argument materialization, so the original separate-sums spelling also keeps both vectors in registers. It does not change floating reductions, arithmetic order, ABI or ISA policy, or claim to fix general register-allocation heuristics. The integer_spill.plenty probe retains the original source shape; before/after measurements are in spill-measurements.json in the experiment directory. With the baseline ISA unchanged, the separate-sums kernel improves from about 16.2 to 8.1 cycles per iteration (3.369 to 1.676 ms per million iterations), reaching scalar parity. The single-vector multiply dependency cost remains.

The recommendation is to accept the portable baseline with these measured limits. The original slowdown is explained by its instruction set and dependent chain, rather than lingering tagged storage in that hot loop. Newer target selection can improve throughput, but requires a separate explicit compilation or dispatch policy. It should not be silently enabled to make this benchmark win.

Automatic SIMD for recognizable pipelines

Part of the portable SIMD proposal. This is a bounded proposed transformation, not implemented auto-vectorization. Fluent pipelines already preserve source and callback structure; the compiler should use that information before it disappears into low-level control flow.

Start with code people already write

These are current Plenty spellings; the proposed optimization changes only their implementation:

total = iter(range(0u32, 1000u32)).map(
    def(x: u32) -> u32: x.wrapping_mul(3u32).wrapping_add(7u32)
).fold(0u32, def(acc: u32, x: u32) -> u32: acc.wrapping_add(x))

even = iter(range(0u32, 1000u32)).filter(
    def(x: &u32) -> bool: x & 1u32 == 0u32
).count()

The first has a pointwise modular map and associative modular reduction. The second needs a comparison mask and a count, not a compacted intermediate list. Neither needs heap allocation, a thread pool, or a new annotation. The checked i64 count is safe only when the source bound proves it fits. Guarded induction must avoid overflowing while constructing the final vector of range values.

PatternAdmission rule and initial disposition
Sized range → total scalar maps → wrapping integer foldRecognize exact modular reduction; vector accumulators plus ordered scalar remainder
Sized range or packed read-only view → bitwise/comparison filter → countMask count; bound the result to i64; no materialized filtered collection
Packed numeric input → total pointwise map → distinct preallocated destinationVector loads/stores once views and exclusive destination proofs exist
Equal-domain zip of packed sources → pointwise mapLater bounded extension; preserve left-first consumption and prove both lengths before evaluation
skip/take on a sized pure source before mapsNormalize bounds only when it preserves evaluation and count rules
Floating left fold, ordinary checked integer foldPreserve order; exclude reassociation without a proof or a separately specified operation
filter(...).map(...), filter(...).collect(), take_while, any, findDefer: evaluation suppression, compaction or early stopping needs additional reasoning
Generators, indirect calls, I/O, mutable captures, allocation, ownership cleanup, atomicsScalar fallback in the first implementation

A recognized floating map can preserve each element’s multiply-then-add order; a floating reduction cannot freely change its accumulation tree. Even ordinary integer addition is not freely associative under overflow trapping: a sequential prefix may overflow although a regrouped sum fits. Do not infer safety from a final result bound alone.

Proof before profitability

For each candidate establish all of these facts:

  1. Source: finite length, element type, traversal order and storage stride are known. Prove each full-block access lies within the source; use scalar tails. No speculative reads into a later page or adjacent object.
  2. Body: a bounded visible expression graph uses only allowed total scalar operations and immutable scalar captures. Reject unknown calls rather than trusting a user purity assertion. Purity alone does not establish totality.
  3. Effects: no observable callback order, writes through aliases, allocation, cleanup or failure can move across a block boundary. Preserving only the successful result is insufficient.
  4. Terminal: a specific reduction law or destination layout is known. A function named add is not evidence of associativity. Match typed operations after substitution; retain scalar fallback for arbitrary fold callbacks.
  5. Target and cost: every widened operation has legal lowering or a useful fallback. Trip count, load stride, scalar remainder, instruction expansion and code size justify the transformed loop.

Known-length collect is not automatically admitted. It allocates, may reserve or grow, and has ownership/failure behavior. First implement preallocated numeric destinations or nonallocating reductions. Later preserve the existing reservation point and capacity policy before changing its inner loop; do not introduce an infallible hidden temporary array.

Checked element maps could later use a vector failure mask and replay the first failing block scalarly, but only when replay has no effects and reproduces the first source operation and item that fail. This is substantially harder than wrapping arithmetic and is not an initial fast path. Never run speculative trapping instructions and then attempt to recover after selection.

Placement in the compiler

Current src/frontend/pipelines.rs builds an Iteration body while adapting a pipeline; it does not retain an optimizer-ready list of typed stages. Extend the plan with bounded source/adapter/terminal metadata instead of reconstructing source expressions from arbitrary CLIF loops. Keep a scalar lowering as the semantic oracle. Preserve ownership, loan and cleanup validation before any optimization decision.

Small callback substitution already exists in src/codegen/leaf_calls.rs, small_calls.rs and scalar_captures.rs. Reuse its bounded body visibility, but do not confuse “eligible for substitution” with “nontrapping and safe to speculate.” A separate whitelist/effect summary must certify widening. Analysis may happen after specialization, with plan identity and source sites retained through substitution. Cache keys must include the transformed plan, target features and optimization policy.

Lower an admitted plan into a backend-neutral kernel description: logical lane operations, source bounds, output/reduction policy, scalar tail and original sites. Cranelift can receive explicit vector IR; LLVM can receive equivalent scalar loops with truthful alias and arithmetic facts, or explicit vectors. No noalias, nsw or nuw annotation may be guessed to coax better code.

Keep the analysis bounded by chain/body size; do not add whole-program alias analysis or arbitrary loop-dependence solving as a prerequisite. Share summaries with automatic parallelization, but neither optimization grants the other’s safety proof.

Make behavior easy to understand

Optimization misses are not errors. Offer an opt-in report with the source span and one actionable reason, for example:

SIMD: map/filter-count uses four lanes; scalar tail for remaining items
SIMD skipped: this callback can trap on integer overflow
SIMD skipped: ordered floating fold must preserve accumulation order
SIMD skipped: source layout does not provide contiguous numeric lanes
SIMD skipped: seven items are below this target's profitable threshold

The report and a compiler option to disable automatic SIMD are debugging tools, not new language semantics. Do not require users to read IR or choose an ISA to understand a missed optimization. Explicit std.simd remains available where an algorithm needs lane control, with the same portable contract and scalar fallback.

Acceptance includes empty and short inputs, every remainder, range boundaries, unaligned views, overflow and failure ordering, NaNs and signed zero, callbacks with effects, zip length mismatch, scalar fallback, and disabled-optimization equivalence. Correct results alone are not evidence of SIMD: inspect generated instructions and benchmark enough sizes to reveal overhead and memory limits.

SIMD spike evidence and backend decision

Evidence for the portable SIMD proposal, collected on 2026-10-23. These experiments do not implement a Plenty vectorizer or the proposed module. They separate current-language semantics, explicit backend lowering, and another optimizer’s recognition of equivalent scalar loops.

This page retains the original 0.131.1 experiment. The implemented clist slice has separate Cranelift 0.137 validation and current measurements; use those for the production backend’s present limits.

Reproduce

The checked-in experiment (experiments/simd/README.md in the repository) contains a locked isolated Rust package, C comparison/verification harness, a Plenty pipeline program, and a runner that preserves objects, assembly, compiler remarks, exit status and command timings. No changes to the production compiler or runtime are required.

Measured environment: AMD Ryzen 7 7840HS, Linux x86-64; Rust 1.99.0 (b940084d7, LLVM 23.1.1); Clang 22.1.8 (Fedora 22.1.8-4.fc44); Cranelift codegen/frontend/module/object 0.131.1. The isolated lockfile resolves some Cranelift helper crates to 0.131.3; it is separate from Plenty’s root lockfile. Plenty was freshly built from the task worktree based on 7654ecca. Default Clang configuration is disabled. Both comparisons use baseline targets, not -march=native; no fast-math or floating contraction.

Current Plenty chains

pipelines.plenty runs these actual fluent chains:

Input and terminalResultWhat it establishes
range(0u32, 1000u32) → wrapping multiply/add map → wrapping fold1505500Current syntax and scalar modular oracle for a narrow recognition candidate
Same range → even predicate → count()500A filter need not materialize or compact data when the terminal is a count
Borrowed [1e20f32, 1.0f32, -1e20f32, 1.0f32] → floating left fold1.0Reassociation changes observable results; a regrouped computation gives 2.0

The worktree compiler produced 1505500 500 1.0, checked by the runner. This validates scalar semantics and realistic input patterns, not recognition or a speedup inside Plenty. The C/CLIF kernels below use packed arrays, whereas these Plenty range pipelines need no array and a general Plenty list does not promise that representation. Bridging the typed plan to vector IR remains work.

Native execution and generated code

The Rust probe builds scalar and four-lane CLIF loops for x * 3 + 7 on u32 and f32, at opt_level=speed. Both use the same loop construction; the four-lane version is supplied explicitly. The C wrapper runs full blocks then the scalar remainder. The C comparison gives LLVM ordinary loops over restrict input/output pointers. These pointers are in fact disjoint in the harness.

Native x86-64 checks passed for:

  • Integer lengths 0–129, all four-lane remainders, an input offset by one scalar from malloc alignment, modular overflow, and an output sentinel after the end.
  • Floating lengths 0–9 containing signed zeros, infinities, NaN and minimum subnormals. Finite results agree bitwise; NaNs are compared by classification.
  • LLVM filter-count results against a scalar count and the ordered floating fold.

These are focused probes, not exhaustive memory-safety tests: a sentinel detects overwrites, not every possible overread, and sanitizers cannot automatically instrument the generated CLIF object. Guard-page tests and fuller floating coverage are adoption requirements. The generated kernel requires a lane-multiple length; only its wrapper accepts arbitrary lengths.

Probex86-64AArch64
Scalar CLIF arithmetic mapExecuted; scalar integer/mulss/addssObject compiled only
Four-lane CLIF mapExecuted; packed integer operations and mulps/addpsObject compiled only
Eight-lane CLIF mapRejected: Unexpected SSA-value type: i32x8Same rejection
LLVM scalar-source mapsVectorization width 4; packed arithmetic verifiedAssembly shows four-lane arithmetic
LLVM scalar-source filter-countVectorized; executed and checkedVectorization remark and emitted assembly only
LLVM strict floating foldScalar ordered adds; executedRemark says vectorized, but assembly widens loads and retains ordered scalar adds

The eight-lane probe stops at its first integer kernel, so it does not independently test f32x8 or all target feature settings. AArch64 was not executed. No claim of a production Plenty AArch64 runtime follows from successful object emission. The floating-fold example shows why an optimizer remark alone is insufficient: vector loads do not imply a reassociated vector reduction.

Small runtime comparison

Integer map, nanoseconds per item, best of five trials. Each trial processes roughly 16 million items through repeated calls; output contributes to a volatile sink. Allocations and compilation are outside the timed region.

Items per callCranelift scalarCranelift four lanes + tailLLVM from scalar source
70.5330.5440.448
1,0240.4200.1710.062
262,1440.4190.1590.070

For the larger sizes, explicit SIMD improves the CLIF loop by roughly 2.5–2.6×; LLVM is another roughly 2.3–2.8× faster in this kernel. For seven items, vector setup and the tail erase the benefit. Earlier runs varied by several percent, with more variation for tiny calls. The CPU was not pinned and frequency was not controlled; best-of-five suppresses interference but is not a confidence interval. This is a hot repeated microbenchmark, not a cold-cache bandwidth test or representative application performance claim.

Assembly helps explain the gap: baseline CLIF expands four-lane multiplication into pmuludq plus shuffles, and reloads constants in the loop. LLVM strength reduces multiplication by three into adds and interleaves two vectors. This compares the output of each optimizer on the straightforward equivalent kernel, not matched instruction sequences. A local strength reduction or stronger target baseline might narrow the gap; neither was measured here. It would be premature to infer a general backend ranking from one arithmetic map.

The runner also records process elapsed times: the warm Cranelift object-emission invocations were around 1–3 ms for two tiny functions, while Clang invocations were tens of milliseconds for four C functions. These include process startup and different frontend/object work. They are not a controlled compilation latency comparison. A backend decision needs matched IR and end-to-end Plenty incremental/cold compilation measurements.

Source evidence and consequences

Rust checkout ../rust/ was inspected at 080cc27146389dbb5c8772c476b1937be9c6110f, under library/portable-simd/crates/core_simd/src/:

  • core_simd_docs.md describes architecture-independent semantics, scalar lowering and older-target subnormal exceptions.
  • vector.rs defines Simd<T, N>, documents wrapping integer operators, and implements checked/fallback and masked loads/stores.
  • mod.rs separates numerical, comparison, pointer and mask facilities. Plenty can adopt the useful value model without exposing the entire surface.

Installed Cranelift 0.131.1 source in src/isa/x64/lower.isle and src/isa/aarch64/lower.isle contains vector arithmetic lowering, including F32X4 and vector multiplication rules. Source inspection motivated the probe; actual object emission is stronger evidence for these exact tested operations. It is not evidence for every instruction on every vector type.

Plenty’s src/frontend/pipelines.rs, src/codegen/leaf_calls.rs, src/codegen/small_calls.rs and src/codegen/scalar_captures.rs establish the existing fusion and bounded substitution paths. src/codegen/collections.rs still routes scalar list access through runtime helpers, so callback substitution alone cannot provide contiguous bulk loads. The current number contract keeps checked arithmetic distinct from wrapping methods.

The evidence supports a small portable layer and narrowly recognized pipelines. It also gives a concrete reason to evaluate LLVM early if broader hardware utilization becomes a requirement. It does not justify growing an unbounded Plenty optimizer, promising auto-SIMD for arbitrary chains, or switching backends before a representative comparison.

Columnar lists with an object-shaped interface

Status: Initial primitive-field implementation. See the current reference for the supported interface. It uses packed native columns and borrowed, scalar-valued local pipelines. The original generated Plenty spike remains historical evidence. Extensions in this proposal are not implied by the initial implementation; B76 tracks remaining candidates.

Recommend an explicit clist[T] storage choice for repeated scans of a few fields of many records. Keep ordinary class declarations and ordinary list[T] unchanged. Start with flat, fixed-width scalar fields and a small indexed-field/column interface; earn a richer object facade through separate ownership and code-generation evidence. Do not require an ECS, dataframe query language, automatic parallelism, or a new backend to try this idea.

class Particle:
    x: f64
    y: f64
    mass: f64
    kind: i64

mut particles = clist[Particle]()
particles.append(Particle(1.0, 2.0, 3.0, 0))?
particles[0].x = 4.0

This is a storage decision at the collection declaration, not a change to Particle everywhere. The empty owner requires no allocation; the first append is fallible. Append errors retain the submitted row; see the ownership and API rationale.

The original layout evidence and Plenty implementation spike support the narrow-field use case and expose tradeoffs. The Plenty experiment establishes bounded scalar scan benefits with ordinary list-backed columns, not packed primitive storage, a final chunk size, or safe general row references. The native implementation follow-up records the packed representation and its comparison with that baseline.

What changes in memory

For four particles, ordinary array-of-structures (AoS) storage groups each record’s fields. Structure-of-arrays (SoA) groups each field’s values:

AoS:  [x0 y0 m0 k0] [x1 y1 m1 k1] [x2 y2 m2 k2] [x3 y3 m3 k3]
SoA:  x: [x0 x1 x2 x3]     y: [y0 y1 y2 y3]
      m: [m0 m1 m2 m3]     k: [k0 k1 k2 k3]

Scanning mass then brings useful values into cache without the other fields. This concerns cache lines, prefetching and memory bandwidth; it need not involve OS paging. Contiguous columns also make ordinary scalar loops simpler and give vectorization a better input representation. Neither SIMD nor multithreading is implied by choosing the layout.

Whole-row access is not uniformly slower. Sequential full-row work can stream several columns efficiently, and compiler vectorization can change the result. Random whole-row access often needs several unrelated cache lines. Appending touches more buffers but can remain amortized constant time; deletion costs depend on ordering and identity promises. Measure the actual workload rather than naming a universally superior layout.

Plenty’s current runtime already keeps inline class payloads in type-sized collection rows (plenty-runtime/src/entries.rs); it is not simply a list of heap-object pointers. Its u128 storage words and bookkeeping also differ from tightly packed native structs. A clist implementation must decide how packed scalar columns cross that representation boundary. Reusing eight ordinary lists is a useful semantic prototype, not evidence of packed columns.

Learn from adjacent systems

PrecedentUseful partBoundary for Plenty
Arrow columnar formatStruct arrays have equal-length child arrays; primitive columns support sequential scansArrow is an in-memory interchange layout, not an ownership checker. Adopting SoA alone does not give Arrow-compatible strings, validity bitmaps, alignment or zero-copy interchange.
Parquet row groupsDivide rows into groups, each containing a chunk per columnParquet describes a file format. Its disk-oriented group sizes and compression/page machinery are not defaults for an in-memory container.
Flecs archetypesGroup entities with the same components for efficient iterationComponent columns need not split each component’s fields. A fixed-schema clist needs neither dynamic archetype migration nor an entity scheduler.
soa-rsSeparate generated row-reference and column-slice types from owned rowsIts references contain references to individual fields. Plenty currently forbids stored aggregate references; this cannot be copied as an ordinary library class today.
DuckDB zonemapsConservative min/max bounds can skip groups, especially when values are clusteredArbitrary Plenty callbacks can have effects and traps. Skipping their execution needs a recognized safe predicate or an explicit query operation.

Small initial scope

The initial candidate is a compiler-recognized collection for a fully concrete class whose fields are integers, floating-point numbers or booleans, with no custom destructor. Normal construction may happen before insertion; clist does not rerun a constructor per field. Custom constructors do not automatically justify exposing otherwise private fields. Exclude nested aggregates, text, collections, owned closures and recursive fields until their movement and destruction contracts are proved.

Provide length, fallible reserve/append, scalar field reads and writes, explicit whole-row materialization/removal, and a typed immutable column projection feeding existing fluent operations. Do not add a second family of map_field, filter_field, and reduce_field methods. The detailed contract makes the incomplete facade visible.

Start with dense flat columns. Preserve insertion order; use explicit ordered removal and defer cheap tombstones. Keep physical storage private so fixed-size chunks can be evaluated later without changing the element type. The chunking and deletion assessment records how to extend it without turning clist into a database prematurely.

Fit with pipelines and SIMD

Column projections should be borrowed sources for the existing fluent pipeline model, with zip for multi-column work. The source carries type, length, stride and ownership information; callbacks keep ordinary evaluation order, traps and allocation behavior. A projection must not allocate a temporary list. The first implementation may use scalar loads and still benefit from fewer fetched bytes.

The SIMD proposal can later recognize safe kernels over these sources. Integer overflow must retain Plenty’s checked semantics unless the program explicitly uses wrapping operations; floating reductions stay ordered. Contiguous storage does not authorize reassociation, speculative callback execution, or automatic threads. The original layout experiment uses LLVM-generated native kernels; the implementation follow-up uses Plenty and Cranelift. Neither is a controlled backend comparison. The existing SIMD evidence remains the backend comparison. Avoid building a dataframe optimizer inside Plenty simply to compensate for a backend limitation.

Adoption gate

There is no demonstrated fundamental layout blocker. There is a substantial semantic boundary between cheap field access and a transparent list[T] replacement. Before accepting production implementation, require:

  • A small generated layout path whose scans touch only selected columns, with bounded compile-time and code-size cost per class specialization.
  • Scalar differential tests against ordinary lists, including empty/tail inputs, checked arithmetic, field mutation and ordered removal.
  • Allocation failure at every required reserve, unchanged logical contents, exactly-once destruction, and useful unsupported-element diagnostics.
  • Compile-fail borrowing tests for mutation/reallocation with live projections, private-field access, whole-row borrowing and callback escape.
  • End-to-end Plenty benchmarks for narrow and full-row work, growth and deletion, including small inputs and at least one non-x86 execution target before making portable performance claims.

The proposal is a reason to try that bounded implementation, not a commitment to the later row facade, stats, vacuum, or interchange APIs.

Columnar-list ownership and API boundary

This records the original proposed contract for columnar lists. The implemented subset uses scalar-valued column pipelines and ordered removal, and does not expose packed field references. The alternatives below remain design rationale.

An honest object facade

items[i].field can lower to a field-column address at row i, evaluating the index once and preserving normal bounds diagnostics. It need not create an object or a proxy allocation. A full &items[i], however, cannot be an ordinary &T: there is no contiguous T at that location. This is an ABI and lifetime fact, not something a clever overload can hide.

For the initial version, support direct scalar field reads/writes and explicit copy_row(i) -> Option[T], which reconstructs an owned class from scalar fields without allocation. This is a deliberate duplication operation even though classes themselves remain affine. An out-of-range index returns Nothing. The copied class can be passed to ordinary methods; changes to it do not write back. Do not implicitly gather/scatter around a method call: methods may return borrows, invoke callbacks or exit early, and hidden writeback changes observable semantics. Plain items[i], ordinary whole-row borrows, and direct row-method calls should give a diagnostic pointing to field access or copy_row.

A later local row binding could allow for row in &items: row.mass, using an ephemeral compiler-known projected place, not a stored T or &T. That would need its own call, reborrow, escape and generator-suspension rules. Existing local pipelines show that restricted compiler-known values are possible, but do not implement this row representation. Treat general row iteration and method adaptation as separate work rather than presenting them as free syntax.

Candidate minimal operations

OperationMeaning
clist[T]()Empty owner; no column allocation until needed
len(items)Live row count, identical for every column
items.reserve(additional)Fallible capacity for that many additional rows; no logical change on failure
items.append(value)Move one complete T; return Result[(), AppendError[T]], retaining the intact submitted row on failure
items[i].fieldChecked scalar access, with ordinary field visibility; negative indices follow list conventions
items[i].field = valueReplace one field; keep the index and RHS evaluation order consistent with existing assignment
items.copy_row(i)Allocation-free Option[T] for the initial scalar-only class subset
items.remove(i)Option[T]; materialize and transfer the removed row, shift all columns together, preserve order
items.column(T.field)Borrow a typed immutable sequence of that field’s values, without allocating

T.field in the last row would be a new compile-time field selector, not a runtime string lookup or arbitrary class-variable expression. It must carry the owning class and field type and respect visibility at the use site. The compiler already knows class fields, but it does not expose selectors or generic reflection. This proposal requires frontend support rather than a magic generic library implementation. Concrete generic class instances may be eligible after checking all instantiated field types; no runtime schema changes.

For example, this proposed pipeline uses scalar field references, not a fabricated class reference:

def add(total: f64, mass: &f64) -> f64:
    total + *mass

total = iter(particles.column(Particle.mass)).fold(0.0, add)

Extend existing Iter source lowering for a read-only column view. Keep it local, borrowing the owner until consumption; do not require a general stored reference feature first. Two immutable column views can feed zip; they have the same length and row order. No public raw slice pointer or alignment promise is necessary. If chunks arrive later, a column view remains a logical sequence, not a promise of one allocation spanning all rows.

Loans and mutation

Initially, any live projection or field reference protects the entire clist against structural changes: append, reserve, removal, clear, movement and drop. Do not advertise stable addresses merely because a chunk currently has room. Scalar field references may use genuine &Field/&mut Field because that individual field has contiguous storage. Begin with conservative owner-wide conflicts; field- or chunk-disjoint mutable views require a separate proof. Even two integer indices do not create simultaneous mutable access by themselves.

Column views cannot escape into stored classes, enums, collections or suspended generators. Shared access from scoped workers must obey existing worker eligibility rules; the initial proposal does not add parallel mutation or automatic partitioning. Row positions are not entity identities: ordered removal shifts later indices, just as in a dense list.

Allocation and ownership transaction

An append reserves every affected buffer before moving any field out of the input row. Only after all reserves succeed may it scatter fields and publish the new length. The commit phase must be infallible and call no user code. If a later reserve fails, earlier capacity increases may remain; logical contents, length and input ownership must not change. Overflow when computing byte capacities follows the allocation-error contract. Never use independent fallible pushes after publishing the first column.

The failed append’s typed error owns the complete T, permitting retry or explicit destruction. A convenience that discards it can be evaluated later; do not silently lose an affine owner. The Rust spike checks this transaction boundary with injected reserve failures; it is not a proof of Plenty’s future allocator implementation.

The scalar-only restriction makes movement and row reconstruction simple. Supporting affine fields later requires compiler-owned field extraction that does not introduce unrestricted user partial moves. It must transfer/drop each field exactly once. Custom class destructors need a whole-row receiver and defined ordering relative to fields; initially reject such classes instead of silently skipping or duplicating their destructor.

Storing str handles in a column would make handles, not string bytes, contiguous. Packing variable-width payloads needs offsets, invalidation and mutation contracts of its own. Nested lists, nullable values, enums and recursive data likewise do not automatically become Arrow arrays. Conversion to/from ordinary lists or Arrow would be explicit and potentially allocating; no FFI layout, zero-copy conversion or serialization compatibility is promised.

Chunks, deletion, and statistics

These are extensions considered for columnar lists. The initial candidate stays dense and flat. Keeping these choices separate lets early measurements answer whether the basic storage interface is worthwhile.

Fixed-size chunks

A chunk contains the same bounded number of rows in every field column. A 1,024-row chunk of eight 64-bit fields holds 64 KiB of payload; a 4,096-row chunk holds 256 KiB. Those are experiment sizes, not chosen public defaults. A scan walks the selected column of each chunk and then advances to the next chunk.

Chunking can bound relocation, permit incremental allocation and release whole empty chunks. It also adds a directory, allocation count, tails, address calculations and transitions between buffers. A single aligned chunk allocation with generated column offsets is an alternative to one allocation per field; the spike uses the latter and does not settle the former. Tune byte footprint as well as row count, since class width varies.

Parquet’s row-group model is a useful analogy for grouping rows across columns, but its pages are encoded file data. In-memory chunk size should follow workload/cache measurements, not Parquet’s disk-oriented recommendations. The measured flat SoA scans beat both tested chunk sizes; chunking needs a growth, mutation or pruning benefit to earn its cost.

Deletion is an identity decision first

PolicyBenefitCost or observable change
Ordered dense removalFamiliar list positions and iterationShift later values in every field
Swap with last rowConstant-sized transferChanges order and the moved row’s position
TombstonesAvoid immediate relocationHoles complicate indexing, scans and storage reclamation
Generational handlesIdentity can survive relocationMapping storage, stale-handle checks and generation exhaustion

Do not give clist[i] both dense-list and physical-slot meanings. A tombstoned design needs either an explicit sparse-slot/handle API or rank/select metadata to map logical live indices to storage. The latter has costs even before vacuum. Do not reuse a tombstone for a new appended row if iteration promises insertion order, unless an ordering structure preserves that promise. A bitmap branch can also inhibit vectorization; processing dense live runs or vector masks is a later optimization.

If tombstones are adopted, deletion should normally destroy owned payloads at deletion time, clear liveness and decrement the live count. Vacuum must move only live owners, never read destroyed slots and never run their destructors again. Keeping resources alive until vacuum would be a different, visible ownership contract, not a storage detail.

An explicit vacuum() is preferable to an unexplained latency spike. Specify whether it compacts within chunks, across chunks, or also releases spare capacity. In-place stable compaction need not allocate, but it may not return buffer capacity to the allocator. Repacking into fewer allocations can fail; reserve destinations first and retain a valid container on failure. Outstanding loans prohibit either operation. A slot-based API must say that vacuum invalidates positions, or use handles whose mapping is updated. A default threshold cannot substitute for that identity contract.

The ownership experiment exhausts all 256 deletion masks of eight rows, including repeated deletion and repeated vacuum, with counted affine owners. It validates immediate destruction and stable compaction, not a production bitmap, chunk directory, relocation handle scheme, or deletion-speed claim.

Statistics need a validity contract

DuckDB’s zonemap guidance explains why per-group bounds help clustered data much more than shuffled data. Parquet page indexes similarly separate bounds metadata from payload reads to skip irrelevant pages. For clist, start with optional conservative integer bounds for explicit field comparisons, only after chunking itself is justified.

Deleting rows may leave old min/max values as a wider conservative bound: that loses pruning opportunities but does not hide a matching live row. Appending or mutating a value outside the bound must widen it or mark the statistics invalid. A mutable field loan must mark the affected stats dirty before exposing the address; recompute on demand or scan without pruning. NaNs, signed zero, nullable values, string ordering and unknown comparisons need explicit rules; “invalid/unknown means scan” is the safe fallback.

Exact sums and counts have stronger maintenance requirements. Cached sums cannot silently change checked-overflow traps or floating-point reduction order. Nor may a filter skip an arbitrary callback that prints, mutates a capture, allocates or traps. Recognize a bounded effect-free field comparison or expose an explicit typed query API; otherwise run the ordinary pipeline. This is the same semantic restraint required by automatic SIMD.

The spike’s min-bound test finds one candidate group out of eight for sorted integers and all eight for a permutation of the same values, with identical results. It does not time metadata construction or claim a general query speedup. Compression, secondary indexes, joins, transactional updates and automatic query planning remain outside this container proposal.

Columnar-list spike evidence

This accompanies the columnar-list proposal. Sources and reproduction commands live in experiments/columnar-list/README.md in the repository. These are bounded feasibility experiments, not a production container, Plenty performance results, or a backend comparison.

This page records the original 2026-10-23 experiment. The subsequent generated Plenty implementation spike tests real runtime failure paths, borrowing, and Cranelift-generated scans separately.

Layout experiment

The dependency-free Rust program compares an array of 64-byte records (eight u64 fields), eight flat columns, and groups of 1,024 or 4,096 rows with one allocation per field. Every layout contains identical generated values. The kernels sum one field, filter on one field and sum another, mix all eight fields in sequential row order, and perform the same whole-row work in an odd-stride permutation. That permutation visits every row once; it is a repeatable dispersed-access workload, not a model of all random workloads.

All arithmetic deliberately wraps. This is equivalent across the experiment, but does not model Plenty’s default overflow traps. Checked integer and floating-point kernels need separate end-to-end validation before adoption. Timed paths allocate nothing; construction is outside the scan measurements. Append and vacuum are correctness experiments, not performance measurements.

On 2026-10-23: AMD Ryzen 7 7840HS, Linux 7.2.11, rustc 1.99.0 (b940084d7), LLVM 23.1.1, native x86-64 target, CPU affinity 4. Per-core L1d is 32 KiB, L2 is 1 MiB, and shared L3 is 16 MiB. Each layout has 32 KiB, 2 MiB or 64 MiB of row payload at the three tested lengths; the single selected column has 4 KiB, 256 KiB or 8 MiB. All four layouts coexist.

Release builds use -C target-cpu=native; a second build also disables loop and SLP vectorization. Seven samples rotate layout order; each sample traverses 4,194,304 rows in total. References and outputs cross black_box, every result is checked, and individual scan functions are not inlined into the timed repetition. These are repeated warm scans with no cache flush. Frequency is not locked and the machine is shared: load average was approximately 8.1–9.7 on 16 logical CPUs. Treat close rankings and exact ratios as provisional.

Large input: median nanoseconds per row

1,048,576 rows; lower is better. “Scalar” means both LLVM vectorizers disabled, not a simulation of Cranelift. Native may use all supported host instructions.

Build / workloadAoSFlat SoAChunk 1,024Chunk 4,096
Scalar: sum one field2.4630.2510.4460.334
Scalar: filter two fields1.3630.3180.8170.513
Scalar: all fields, sequential1.9043.5223.0062.626
Scalar: all fields, permuted24.14647.17958.05752.378
Native: sum one field4.2170.0780.2030.154
Native: filter two fields6.8950.2920.7360.458
Native: all fields, sequential10.2852.2542.7902.385
Native: all fields, permuted22.90449.42656.29857.228

Sample ranges matter: scalar AoS sequential whole-row work ranged 1.707–2.325 ns/row and flat SoA 3.453–4.638. Native permuted work ranged 21.686–27.734 for AoS and 45.205–64.939 for flat SoA. This is enough to identify tradeoffs, not to choose final tuning parameters.

Size sensitivity: one-field scan

RowsScalar AoSScalar flat SoANative AoSNative flat SoA
5120.4030.2400.5600.040
32,7680.8100.2560.7350.064
1,048,5762.4630.2514.2170.078

The large single-column working set fits shared L3 while the complete AoS does not. Cache residency and reduced traffic are part of the benefit, but these warm-scan numbers must not be advertised as sustained DRAM bandwidth or cold-scan speedups. No hardware cache-miss counters were collected.

Code generation changes the story

Inspection of emitted assembly found vpgatherqq in the native AoS narrow, filter and sequential whole-row kernels. Flat SoA uses contiguous vector loads/arithmetic (vpaddq, and vprolq for whole-row mixing), without those gathers. With both vectorizers disabled, these kernel bodies contain no vector instructions. Chunk scan functions call the corresponding SoA kernels.

Thus the striking native ratios combine layout, cache residency and a poor AoS gather strategy on this compiler/CPU. Scalar results still favor columns for narrow work, but the sequential whole-row ranking reverses. Do not claim that all full-object access is slower in SoA or that LLVM always improves a loop. The experiment also charges flat SoA’s generated indexed row access its bounds checks and chunked random access its directory/division work. Future Plenty lowering may optimize those differently.

Flat columns won the measured narrow scans. Fixed chunks did not provide a scan-speed reason to adopt them here; growth limits, metadata pruning and deletion behavior need separate measurements. This supports starting flat, without foreclosing chunked storage later.

Ownership and metadata experiments

Four Rust tests check:

  • Equal row values and scan results at lengths 0, 1, 2, 7, 1,023, 1,024, 1,025 and 4,097, including chunk tails.
  • Failure before either required column reserve: no partial live row, both owned fields returned intact, then destroyed exactly once. Real try_reserve errors share this path. Injection is a deterministic gate, not process-wide allocator failure instrumentation.
  • Every deletion mask of eight rows: owners destroyed at deletion, repeat deletion harmless, stable in-place vacuum preserves row alignment/order, repeat vacuum harmless and no duplicate destruction. Safe Option slots model holes; production bitmap/uninitialized storage is not implemented.
  • Conservative min-bound pruning over identical sorted/permuted values: a < 64 query visits 1/8 groups when sorted and 8/8 when dispersed, returning the same sum. Statistics are rebuilt for the test, not maintained under mutation. This is a correctness demonstration, not a timing result.

These tests reduce implementation risk but leave real runtime allocation failure, destructor ordering, chunk relocation, and borrowing diagnostics for the production gate. There is no benchmark of noncopyable/string fields, concurrent updates, growth, tombstone scan density or memory reclamation.

Current Plenty feasibility

Using a compiler freshly built from 45798945, columns.plenty creates two ordinary lists, reserves both before appending, and zips borrowed columns into a fold. It prints 90. This is checked language code with ordinary collection storage and no generated schema support. It establishes that a concrete SoA wrapper need not wait for clist, not that its elements are packed native words.

reject-row-reference.plenty fails with references cannot be stored in class fields. reject-clist.plenty fails with `clist` is not a generic function. The rejection fixtures document the baseline only, not new defects. The implementation inspected in plenty-runtime/src/entries.rs stores collection rows in u128 words; the borrowed-list row helpers in plenty-runtime/src/aggregates.rs expose contiguous row addresses. A columnar source needs new field/projection lowering and must not masquerade as that ABI.

All four native tests pass, including in release mode; formatting and Clippy pass for the isolated crate. The Plenty checks are reproducible through check_plenty.py. Existing language/reference behavior is unchanged.

Generated Plenty column-container spike

The primitive-only restriction makes a useful column container feasible with Plenty’s existing ownership rules. A generated facade passes actual allocation failure and borrow checks, and a one-field scan is substantially faster than ordinary row storage. It also exposes expensive indexed access and whole-row reconstruction. These are reasons to implement direct column projection carefully, not to ship the generated wrapper unchanged.

This 2026-10-24 follow-up to the proposal uses Plenty 00d32a13. Sources, commands and preserved timing samples are in experiments/columnar-list/plenty/ in the repository. The initial spike changed no compiler or runtime; the indexed-access follow-up below records the later compiler fixes. clist[T] remains proposed.

What was implemented

An experiment-only generator reads one public, nongeneric, field-only class. It accepts bool, signed/unsigned 8/16/32/64-bit integers, f32 and f64; it rejects methods, destructors, private fields, nested/owned fields and empty schemas. For Particle it emits a checked Plenty ClistParticle class with one private list per field, length, reservation, append, scalar getters/setters, whole-row copying, and borrowed column methods.

mut items = ClistParticle()?
items.append(Particle(1, 2, 3, 4, 5, 6, 7, 8))?
xs = items.column_x()
total = iter(xs).fold(0, add)

This is specialization into existing language constructs, not a new parser feature or an ordinary generic library implementation. Each column uses current 16-byte scalar list slots; the eight-field row list uses 128-byte row payloads. Neither is packed native-word storage. Empty construction allocates eight list handles; this differs from the proposed allocation-free empty owner.

Append reserves every column before publishing any value. A failure returns an error containing the intact submitted row. Earlier capacity growth may remain, but lengths and existing values are unchanged. Subsequent scalar appends cannot allocate after reservation; the generator makes that invariant explicit with unwrap(). Row reconstruction creates an independent value without allocation. It never manufactures &Particle pointing across unrelated allocations.

Ownership and failure evidence

The freshly built compiler with runtime-checks passes 35 generated fixtures (31 executing programs and four compile-fail cases):

  • Every empty-constructor allocation and every append reserve can fail, both before the first row and while growing eight existing rows. Failed append returns all eight input fields, and retry succeeds. Runtime allocation accounting finishes balanced on every exit.
  • Reserved append, row reconstruction and a borrowed two-column zip/fold perform no successful allocation within checked regions.
  • Differential tests against list[Particle] cover lengths 0, 1, 2, 7, 8, 9, 33 and 1,025, field mutation, every field’s value and negative indexing. Invalid row copies return Nothing, including extreme signed indices.
  • All eleven accepted primitive types round-trip, including signed extrema, unsigned maxima and floating negative zero.
  • Live column/pipeline loans reject owner mutation; moved owners cannot be reused; callers cannot mutate private backing lists independently.

Two generator unit tests reject unsupported schemas. The three original Plenty feasibility probes still pass. No custom destructor, removal, stable row handle, stored view, cross-thread operation or affine-field semantics are implemented. Column loans conservatively borrow the entire owner.

Issue #427 recorded a frontend limitation found here. Its follow-up now accepts iter(items.column_x()) directly and retains the returned loan through consumption. The fixture suite includes this positive case and rejects mutation while such a pipeline holds the owner.

End-to-end scalar measurements

AMD Ryzen 7 7840HS, x86-64 Linux, CPU 4, Plenty --release, current Cranelift 0.131.1. Eight checked i64 fields per row; values repeat every 1,024 rows. Both layouts remain allocated. Seven samples follow one warmup, each sample repeats a scan 20 times, alternating layout order. Every checksum is checked independently in Python. No cache flush, frequency lock, counters or non-x86 execution was used. These are warm scans on a shared machine.

Median nanoseconds per row; lower is better:

Kernel8,192 rows: list8,192: columns262,144 rows: list262,144: columns
One-field sum1.1550.6614.0750.711
Two-field filter/fold2.0302.6052.8842.630
All fields, direct indexed columns2.48334.0963.21036.340
All fields, copy_row sequential2.53766.1903.53566.882
All fields, copy_row permuted6.34167.31449.377169.975

The clear result is the one-field sum: about 1.7× faster at the small size and 5.7× at the large size. Large rows occupy 32 MiB of payload, while one column occupies 4 MiB; shared L3 is 16 MiB. Cache residency is part of the benefit. The large one-column samples ranged 0.687–1.389 ns/row versus 3.556–4.651 for the row list. Two-column results are mixed and close enough at the large size that they do not establish a robust improvement.

The all-field comparison includes real facade costs, not just memory layout. Assembly inspection at the original spike revision showed:

  • The one-column fold hoists length, base and stride, then directly loads and adds one scalar per iteration. There is no callback call in its hot loop.
  • Two-column zip hoists both sources, but calls the scalar fold callback for each pair. Checked arithmetic and original evaluation order remain intact.
  • Direct indexing of eight borrowed columns called plenty_list_scalar_get eight times per row. Issue #429 records this cost.
  • copy_row adds a method call, reconstruction/Option handling and cleanup machinery. Avoiding reconstruction roughly halved the sequential cost there, but indexed access dominated the direct-column version.

The reserved construction pass took 145 ns/row for the row list and 454 ns/row for columns at the large size. This is one observation per layout, including allocation and append, not a repeated growth benchmark. Eight reserve/check/ append paths per row are a visible cost of this generated implementation.

Indexed-access follow-up (#429)

Ordinary for loops now cache length, row pointer and stride for immutable shared scalar-list bindings already in scope. Indexed reads use direct loads with separate bounds checks for every column, including negative indices. No runtime container layout is exposed, equal column lengths are not assumed, and the original helper handles the cold out-of-bounds diagnostic. Mutable, retargetable and owned sources keep their existing path.

Repeating the same benchmark against baseline 1c1b8ca1, without overlapping task builds, gave these medians (ns/row):

KernelRowsBeforeAfter
Eight indexed columns8,19233.2485.653
Eight indexed columns262,14437.3108.896
One-column fold262,1440.6480.647
Whole-row reconstruction262,14464.46563.062

The targeted large scan improved about 4.2×, but still trails ordinary row iteration (3.113 ns/row in the after run). Reconstruction is unchanged by this optimization. The eight-column function grew from 804 to 1,856 machine-code bytes; the two-column issue reproduction grew from 371 to 568. Cold compile/link medians were 91.47 versus 91.09 ms for the full benchmark and 47.26 versus 47.23 ms for the small reproduction: no meaningful compilation-time change in these seven alternating samples, using debug-built compilers and release output. The size increase is a real tradeoff, not evidence for applying this everywhere.

experiments/columnar-list/plenty/indexed-measurements.json retains raw samples, compiler hashes and methodology. Reproduce runtime results with the existing bench.py command in that directory’s README against each compiler. Use --release --dump asm --dump-filter soa_stream to inspect code size and the call-free hot loop; time --release --compile with PLENTY_CACHE=off and PLENTY_LINKER_THREADS=1 for cold compilation.

Consequence for the design

Proceed with the primitive-only restriction and explicit borrowed column operations. The spike validates their ownership model and demonstrates a useful scalar scan benefit without SIMD or a backend switch. Ordinary list storage remains preferable for the measured whole-row work.

A production implementation needs a deliberate representation and projection path: packed primitive buffers if compact storage is promised, an empty owner without eight heap handles, and direct checked loads instead of repeated helper calls for recognized field scans. These findings concern frontend/runtime representation as well as backend optimization; they do not establish a need to replace Cranelift with LLVM. The remaining adoption scope belongs in B76, including removal, richer classes and portability checks.

Initial native column implementation

The current interface implements the bounded primitive-field version of clist[T]. The earlier generated-container experiment supplied the ownership and performance baseline; its eight ordinary lists are not the native layout.

This page preserves the initial implementation’s measurements. The subsequent indexed-access work removes generic dispatch from field access and row reconstruction; consult that report for the current comparison.

Decisions at the representation boundary

One allocation holds a header and a contiguous buffer for each field. Native widths are 1, 2, 4 or 8 bytes, including one-byte booleans. Rounding row capacity to a multiple of eight aligns every column start. This avoids separate column allocation failures: reserve prepares a complete replacement before publishing it, and append advances the single row count only after all fields are written. Failed append returns the unchanged input object in AppendError[T].

Ordinary Plenty references point to tagged value slots, so they cannot safely point into these packed buffers. A column therefore yields scalar values through a local Iter[FieldType] while borrowing the entire owner. Its pointer and length are loaded before the loop; iteration emits native scalar loads. Existing fluent operations consume these values without new column-specific map/filter/reduce methods. A live plan prevents owner mutation or movement.

copy_row materializes an ordinary scalar-field object without allocation. There is no general &T row view, no packed field reference, and no mutable column plan. The initial indexed field access was checked through runtime helpers. Strings, nested classes, enums, custom destructors and fallible initializers are rejected. These restrictions make ownership and failure cleanup explicit without committing to a future reference representation.

Reproducing the comparison

Build without allocation instrumentation, then run the extended original probe:

cargo build --locked --release --bin plenty
python3 -B experiments/columnar-list/plenty/bench.py target/release/plenty \
  --native-clist --skip-generated --out /tmp/plenty-native-columns \
  --rows 8192 262144 --repeats 20 --samples 7 --cpu 8

The runner checks every checksum, discards the first sample, alternates layout order, and retains both owners during measurement. This final comparison uses native clist and ordinary list[Particle]; it omits the earlier generated eight-list spike. The earlier three-owner run is superseded, so its absolute timings are not mixed into this two-owner comparison. Each row has eight i64 fields. The kernels scan one field, filter using two fields, read every field by index, materialize sequential complete rows, and materialize rows in a fixed permutation. The historical random kernel uses (i * 65537) % rows: this is a predictable permutation, and degenerates to sequential order at 8192 rows. It is not an arbitrary random-access workload. Construction is measured separately.

These are scalar x86-64 measurements. No automatic SIMD or parallel execution is introduced, and no other architecture’s performance is inferred. Contiguous loads give later vectorization and partitioning work a useful representation; they do not remove checked arithmetic, effects, aliasing or scheduling questions.

Results and limits

experiments/columnar-list/plenty/native-measurements.json retains all samples, compiler hash, frequency samples and host information. These measurements use reviewed commit d6e9a2cf on an AMD Ryzen 7 7840HS, pinned to CPU 8 with the performance governor, after competing builds and timings finished. Frequency samples taken every 250 ms during execution ranged from 4.089 to 4.093 GHz; before-run samples ranged from 3.271 to 3.511 GHz. This replaces the initial throttled final numbers. It remains a workload-specific measurement on one host.

Median nanoseconds per row, over seven measured samples:

RowsOperationOrdinary row listNative clist
8,192One-field fold1.4630.560
8,192Two-field filtered fold2.3552.753
8,192All fields by index3.021116.088
8,192Sequential whole rows3.04796.809
8,192Permuted whole rows7.52096.842
262,144One-field fold4.4700.495
262,144Two-field filtered fold4.0613.787
262,144All fields by index3.694117.517
262,144Sequential whole rows3.97196.098
262,144Permuted whole rows49.437132.525

The ordinary all-fields and sequential whole-row controls borrow rows directly; the column kernels respectively index fields and materialize rows. They perform the same calculation using the available interfaces, so this compares practical access paths rather than isolating memory layout alone.

The large one-field scan was about 9.0× faster than the row list. The two-field median was only about 7% faster, with overlapping samples: native 3.660–3.845 ns/row and ordinary 3.648–5.448. This is not convincing evidence of a two-field speedup; the small case was about 17% slower. The earlier 4.3× two-field claim is withdrawn. Native columns use 64 bytes of payload per eight-i64 row, compared with 128 tagged-slot bytes in the row list; one large native column occupies 2 MiB. These layout and code-generation benefits depend on the calculation.

Whole-row work is a substantial weakness of this first implementation: sequential row copying was about 24× slower, and indexed eight-field work about 32× slower than the row-list controls. The native indexed kernel makes eight generic dispatcher calls per row. Reconstruction walks the schema, recomputes column prefixes and snapshots tagged row slots. These are current lowering costs, not inherent costs of columnar storage. Issue #432 records this performance concern; the follow-up addresses that dispatch overhead.

Reserved construction took about 174 / 145 ns per row for the large ordinary / native collections. These are single construction samples, not a stable construction benchmark. They do not justify a general append-speed claim.

Generated code and correctness

nm -S on the large-case executable gives these kernel sizes in bytes, excluding shared runtime helpers and measurement wrappers:

KernelOrdinary row listNative clist
One-field fold161294
Two-field filtered fold235620
All fields by index3621,269
Sequential whole rows362831
Permuted whole rows560936

The native one-field assembly has two setup helper calls, followed by a direct eight-byte indexed load/add and the existing overflow check. There is no helper call per element and no SIMD instruction in that loop. Its larger setup code is another reason not to assume a win for tiny collections.

Twenty-one focused cases cover scalar widths and floating-point bits, generic schemas, module privacy, owner loans, nested enum storage, source evaluation order, overflow/allocation failure recovery, and allocation-free reserved work. The full workspace suite with all features, Clippy with all targets/features, tutorial examples, Markdown links and the book build pass. The whole-row performance limitation motivated the subsequent indexed-access work.

Direct indexed column access

Issue #432 identified generic runtime dispatch as the dominant cost of indexed clist fields and copy_row. The initial implementation already emitted direct loads for column pipelines, but indexed eight-field work made eight dispatcher calls per row. Row copying walked the schema, recalculated column prefixes, and copied the resulting tagged row again.

Implementation and constraints

The compiler now calculates column prefixes from the checked primitive schema. Reads and writes use native-width memory operations. A row copy checks its index once, loads the fields, and writes directly into fresh ordinary row storage. The existing column-pipeline setup is unchanged. The compiler and runtime share layout offsets, checked against the runtime’s repr(C) header.

Null owners are checked before reading the header. Negative indices are normalized without signed overflow, and an unsigned comparison rejects both still-negative and too-large indices. Indexed access retains its source-located trap through a small cold helper; copy_row returns Nothing for missing rows. Field types determine the actual load/store width and row-slot encoding, including signed narrow integers, unsigned maxima and floating-point bit patterns.

There is no new frontend analysis or pointer cache across source operations. Each operation reads the current owner, so reserve, append and replacement between operations cannot leave stale metadata. Existing loans, evaluation order and checked arithmetic are unchanged. General loop hoisting, automatic SIMD and parallel execution are outside this change.

Instruction-count evidence

experiments/columnar-list/plenty/count_access.py runs the same indexed and sequential-copy kernels as bench.py, one at a time, for 200 passes over 8,192 rows. Linux perf stat counts user-mode instructions and cycles in three alternating before/after samples, pinned to CPU 8. Every checksum is checked. Compilation finishes before counting. This is useful under CPU contention; cycles and elapsed time still require a separate quiet timing comparison.

Counts include startup, construction, output and teardown. A zero-pass control reports their cost separately; the table does not subtract it or claim isolated kernel counts.

experiments/columnar-list/plenty/indexed-instructions.json retains the samples, compiler/source/binary hashes and counter scope. The baseline is local commit 697aedb5626d74b7f97f1a51bcd3334823df5744.

WorkloadBefore instructionsAfter instructionsReduction
Construction-only control15,870,29615,870,567Essentially unchanged
Eight indexed fields3,043,642,962255,083,61111.9× fewer
Sequential row copies2,391,559,687240,339,98210.0× fewer

The counts varied by fewer than 400 instructions across the three samples for each case. They support the removed-work explanation, rather than substituting for elapsed-time verification.

Reproduce with two compilers built without allocation instrumentation:

python3 -B experiments/columnar-list/plenty/count_access.py \
  /path/to/before/plenty /path/to/after/plenty --out /tmp/clist-counts \
  --rows 8192 --repeats 200 --samples 3 --cpu 8

The reference and tutorial continue to recommend column pipelines for selected field scans. Whole-row reconstruction still gathers separate fields and moves ordinary tagged row storage; removing dispatch does not make this free or prove that clist is universally faster than an ordinary row list.

Final timing verification

experiments/columnar-list/plenty/native-indexed-measurements.json retains final samples, compiler/source/executable hashes, kernel sizes, frequency readings and rejected attempts. These use the same AMD Ryzen 7 7840HS, pinned to CPU 8. Both compilers were built without allocation instrumentation. The runner waited for competing builds/tests to finish, checked host processes every 250 ms during execution, and rejected one large after-run when a build began. Earlier interfered timings are not mixed into these results. This excludes detected build interference, not all operating-system noise. Median sampled frequency was about 4.092 GHz; the artifact retains occasional lower readings as well.

The existing bench.py program checks every checksum, warms up once, alternates the ordinary/native layout order, and measures seven samples. Large cases use 20 repetitions; tiny cases use 100,000. All executables were prepared before the final timing pass. Median nanoseconds per row:

RowsIndexed fields beforeIndexed fields afterRow copy beforeRow copy after
1119.7787.49697.59614.830
8105.7036.47780.81312.457
6499.2236.51980.06012.780
8,192115.9516.41994.51612.331
262,144117.9836.52696.02912.558

At 262,144 rows, indexing improved about 18.1× and sequential copying about 7.6×. The after-run ordinary-row controls were 3.985 and 4.031 ns/row: indexed clist work is still about 1.6× slower and row copying about 3.1× slower. The copy kernel still moves the unwrapped ordinary row into local storage with a 128-byte memmove. Removing schema dispatch does not remove that cost.

The large indexed function shrank from 1,269 to 991 bytes; the copy function grew from 831 to 961 bytes because its field loads are now explicit. These sizes exclude shared helpers. Neither valid indexed access nor reconstruction calls the generic collection dispatcher. Cold range-validation code elsewhere in the benchmark remains separate from the per-row access path.

The one-column control is a limitation: its large median changed from 0.497 to 0.743 ns/row, despite identical emitted kernel assembly and unchanged 294-byte function size. The 8,192-row case similarly changed from 0.494 to 0.736 ns/row. Code placement/alignment is a candidate explanation, not an isolated causal result. An exploratory change to inline column setup was removed; the final patch retains the original pipeline setup and loop. This observation is why instruction counts alone do not justify a universal speedup claim. The target indexed/copy improvements remain clear in both counters and elapsed time. Issue #437 records the scan-throughput follow-up separately.

Bounded placement check (issue #437)

The follow-up found reproducible address sensitivity, so the observation cannot be dismissed as timing noise alone. probe_scan_layout.py compiles the one-field workload once and relinks that same object with GNU ld at eight text-section addresses. Linker alignment produces four distinct function placements. Each binary performs 20,000 scans of 8,192 rows; checksums are checked, order is shuffled with seed 437, and one warmup precedes three measured samples per placement. Two independent runs produced the same object hash and 294-byte scan function.

Function address modulo 64Cycles/row across the two runsFirst-run time, ns/row
92.045–2.0660.500–0.502
252.048–2.0650.505–0.507
413.048–3.0700.747
574.055–4.0840.996–0.997

Retired instruction counts remain essentially identical. Hardware counters cover the whole process; internal elapsed timing covers the repeated folds. The second run has noisier elapsed times, while the cycle groups remain distinct. Modulo 64 is an address label here, not a claim about the CPU’s exact fetch-block size. Instruction-fetch/decode alignment is a plausible mechanism, but the probe moves the whole text section and does not isolate a specific CPU front-end effect or prove the precise cause in the original two binaries.

The probe reused an available compiler executable instead of rebuilding Rust; its hash, the object hash and all samples are retained in experiments/columnar-list/plenty/scan-layout-measurements.json. Reproduce on x86-64 Linux with GNU ld, nm, and access to user-mode perf counters:

python3 -B experiments/columnar-list/plenty/probe_scan_layout.py \
  /path/to/plenty --out /tmp/clist-scan-placement

This completes the bounded investigation. No compiler change is proposed. A universal padding/alignment policy would need broader benchmarks and code-size tradeoffs; issue #437 remains a low-priority performance concern. The placement experiment is evidence, not a hard-coded alignment recommendation.

The subsequent function-alignment evaluation tests 16-, 32- and 64-byte boundaries against short scans, a two-column fold and an unrelated scalar loop. It rejects a blanket alignment change: some placements improve while others regress.

Reproduce the timing programs on each compiler, without competing builds:

python3 -B experiments/columnar-list/plenty/bench.py /path/to/plenty \
  --native-clist --skip-generated --out /tmp/clist-access-large \
  --rows 8192 262144 --repeats 20 --samples 7 --cpu 8
python3 -B experiments/columnar-list/plenty/bench.py /path/to/plenty \
  --native-clist --skip-generated --out /tmp/clist-access-small \
  --rows 1 8 64 --repeats 100000 --samples 7 --cpu 8

The full all-features workspace suite, Clippy, formatting, documentation links and book build pass. Focused cases cover development/optimized modes, mixed primitive widths, growth, copies returned across calls, independent copies, extreme indices, source-located traps, and absence of generic reconstruction dispatch. Existing tests retain evaluation-order, enum-projection, allocation failure and owner-loan coverage.

Function-alignment follow-up

Issue #437’s placement probe found a repeatable scan-throughput difference with identical function bytes. The next experiment tested a small potential remedy: raise the alignment of every generated function to 16, 32 or 64 bytes.

Do not adopt a blanket function-alignment change from these results. It improves some placements and worsens others. In particular, 64-byte alignment makes the one-column scan consistently slower than its best unaligned placement. This follow-up changes the experiment and documentation, not compiler behavior. Issue #437 remains open at low priority.

The subsequent online research connects the observed cycle groups to documented fetch-block constraints and a closely matching upstream incident. It finds no verified small remedy and recommends deferring further optimization rather than applying blanket alignment.

Available backend control

The pinned Cranelift 0.131.1 exposes log2_min_function_alignment. Its x86-64 backend specifies a minimum alignment of one byte and a preferred alignment of 32 bytes, but ObjectModule uses the compiled buffer’s required alignment. Our baseline object therefore has byte-aligned functions (some sections need eight-byte alignment for constants). Preferred alignment is not a guarantee that the object linker places every function on a 32-byte boundary.

This version has no public loop-header alignment control. Its bb_padding_log2_minus_one option inserts dummy zero bytes for code-generator stress testing; it is not a production loop-alignment setting. These observations come from the pinned crate’s shared/settings.rs, isa/x64/inst/mod.rs, machinst/buffer.rs, and the object backend, rather than an assumption about what another version or Wasmtime does.

Controlled comparison

The compiler was freshly built from 68108734 without allocation instrumentation. Each workload was compiled once. GNU objcopy then raised all function-section alignments, and GNU ld linked each variant at eight requested text addresses. The baseline retains the original object, including constant-pool alignment. The probe verifies unchanged function-section contents and actual function addresses. It never inserts instructions or changes arithmetic semantics.

Measurements use the AMD Ryzen 7 7840HS, pinned to CPU 8, after the compiler build finished. Host-visible process checks found no competing builds around the measurement runs. Each sweep shuffles variant order with seed 437, excludes one warmup, and takes three samples per variant; every checksum passed. This is a small single-machine rejection test, not a representative architecture survey.

Ranges below are the minimum and maximum per-placement median cycles/row. Counters cover startup, construction, repeated calls and teardown; internal timers cover repeated calls. In the short scan, call/setup costs are substantial.

WorkloadOriginal alignment16 bytes32 bytes64 bytes
One-column fold, 8,192 rows2.04–4.072.04–3.062.04–3.053.04–3.11
Two-column filtered fold, 8,192 rows11.17–11.7210.87–12.7110.76–12.2410.79–10.91
Checked scalar range sum, 8,192 iterations3.01–3.022.01–3.022.01–3.023.01–3.02
One-column fold, 8 rows6.60–8.686.73–7.706.48–7.917.85–7.90

The one-column scan’s first sweep took 0.409–0.819 ns/row with original alignment, 0.409–0.618 with 32-byte alignment, and 0.609–0.624 with 64-byte alignment. A second complete sweep reproduced the same groups: 2.04–4.06 cycles/row originally, 2.04–3.06 at 32 bytes, and 3.04–3.06 at 64 bytes. Use these fresh within-run comparisons; earlier reports were measured at different CPU speeds. Across the first sweep’s alignments, retired instructions varied by fewer than 200 out of about 1.33 billion. Identical instruction counts are not identical throughput.

The scan function remains 294 bytes. At matching requested text addresses, 16-byte alignment adds 80 bytes to the linked text section, 32 adds 192–208, and 64 adds 416–464. Executable file size happens to remain unchanged because of existing segment padding. Across the other controls, text growth remains under 500 bytes; throughput tradeoffs, not this small size cost, reject the blanket change. Larger applications could accumulate more padding.

Interpretation and reproduction

Function alignment fixes the function’s starting boundary, not the hot loop’s boundary within it. The one-column loop has a condition at function offset 0xc6 and a body at 0xfd; aligning the function to 64 bytes does not align either block. Moving both blocks can improve or worsen instruction delivery. The exact CPU mechanism is still not isolated, and the probe moves other code as well. These results do not justify a benchmark-specific offset or NOP patch.

A further fix would need loop-aware backend work and broader validation; changing the global minimum alignment alone does not resolve this issue. That exceeds this bounded follow-up. No new user-facing flag is proposed.

The probe and all samples are in experiments/columnar-list/plenty/: probe_scan_layout.py and alignment-policy-measurements.json. Requires x86-64 Linux, GNU binutils, a C linker driver, and access to user-mode perf counters. After building the compiler, run the following sequentially without competing builds. Repeat the first command with a new output directory to check stability.

python3 -B experiments/columnar-list/plenty/probe_scan_layout.py target/release/plenty \
  --out /tmp/alignment-one --alignments 1 16 32 64
python3 -B experiments/columnar-list/plenty/probe_scan_layout.py target/release/plenty \
  --out /tmp/alignment-two --alignments 1 16 32 64 --kernel two --repeats 5000
python3 -B experiments/columnar-list/plenty/probe_scan_layout.py target/release/plenty \
  --out /tmp/alignment-scalar --alignments 1 16 32 64 --kernel scalar
python3 -B experiments/columnar-list/plenty/probe_scan_layout.py target/release/plenty \
  --out /tmp/alignment-tiny --alignments 1 16 32 64 --rows 8 --repeats 1000000

Why identical scan code can run at different speeds

Online follow-up to issue #437, researched on 2026-10-24. The research found a closely matching upstream incident and a plausible explanation of the measured cycle groups, but no verified small fix or dependency upgrade. My recommendation is to close this investigation as understood enough to defer, retaining its reproducer and evidence. That is not a claim that performance was fixed or that the exact CPU mechanism has been proved.

This report adds no benchmark runs or compiler changes. It interprets the existing measurements and inspects the retained scan’s disassembly.

What AMD documents

AMD’s Zen4 guide describes next-address generation in aligned 64-byte blocks; a branch target near the end leaves a short block and reduces fetch bandwidth. For very hot loops it discusses aligning the loop’s start or end, and keeping branch placement within predictor limits. These concern the loop itself, not merely the enclosing function’s address. See sections 2.8.1.1 and 2.8.3 of AMD publication 57647, revision 1.01, April 2023 (mirrored PDF).

The same guide distinguishes that next-address path from traditional instruction fetch/decode: section 2.9 specifies 32-byte fetches aligned to 16-byte boundaries and contained within a 64-byte block. Section 2.9.1 describes a decoded-op cache that bypasses ordinary decoding and has its own entry and transition limits. Consequently, warm instruction caches and equal retired-instruction counts do not establish equal instruction-delivery throughput. The document is AMD’s primary material, hosted by a mirror; AMD’s current document endpoint returned an authorization error during this research.

A simple model fits the retained scan

For ordinary successful iterations, the 294-byte function executes two separated instruction regions:

  • The condition and taken branch occupy offsets 0xc6..0xce inclusive.
  • The body, untaken overflow branch and jump back occupy 0xfd..0x117 inclusive.

The function’s exit epilogue lies between them. Consider a deliberately simple model: count the 64-byte blocks touched by each region separately, then add the counts. Do not count only unique cache lines across the whole loop: a taken branch starts the next region. For function address modulo 64 equal to d, a region [a, b] touches floor((d+b)/64) - floor((d+a)/64) + 1 blocks.

Function address modulo 64Condition fragmentsBody fragmentsSumMeasured cycles/row group
9, 16, 25, 32112about 2
0, 41, 48123about 3
57224about 4

This arithmetic matches the rounded median cycle count of all 64 variants in the retained one and repeat sweeps in experiments/columnar-list/plenty/alignment-policy-measurements.json. It also explains why 64-byte function alignment selected the three-cycle group: it fixes d = 0, leaving the body to straddle a boundary. This is new analysis of existing data, not a new experiment or an AMD-specified cycle formula.

Inference, not attribution: the correspondence makes instruction delivery a strong candidate. It does not separate fetch/prediction sequencing, op-cache packing, decode/fusion behavior, or other placement effects. The experiment moved other code too, and collected no counters isolating these mechanisms. Do not generalize this two-region formula to arbitrary loops or label it an instruction cache miss penalty.

A close Cranelift precedent

Wasmtime upstream issue 8573 reported roughly twice the execution time for identical hot-function bytes after a different function changed size. It used an Intel i5-9500T, so it is an analogy, not a diagnosis of our AMD CPU. Maintainers specifically identified a loop split around its exit epilogue, suggested better block layout, and discussed partial fetch blocks. The eventual mitigation was to prefer 32-byte function alignment; the current x86-64 backend source still links that incident beside preferred: 32 and minimum: 1.

This precedent is unusually close to our assembly shape. It also explains why simply adopting that historical mitigation is insufficient here: the local evaluation already tested 32-byte alignment and found both favorable and unfavorable placements. We have not identified a new fix hidden behind that source comment.

As checked on the research date, the published Cranelift shared settings identified themselves as 0.138.0-dev. They expose minimum function alignment and dummy basic-block padding for debugging, but no loop-alignment setting. The latter padding remains unsuitable as a production fix. This API check and the upstream searches found no verified upgrade remedy; they do not prove that newer code generation could never improve this case. No newer backend was built or timed for this research.

Decision

There is useful explanatory evidence, so the outcome is not “nothing was found.” There is still no bounded, validated compiler change to make. Closing issue #437 as investigated/deferred is reasonable under the requested effort limit. Preserve the address-sensitive benchmark as evidence; do not describe closure as a performance fix, dismiss the observation as noise, or adopt global padding.

If this becomes important again, the concrete question is whether keeping the hot loop together, or applying loop-aware alignment in the backend, improves multiple workloads and architectures. The upstream incident supports that direction, but does not supply a drop-in patch. It does not justify switching backends solely for this microbenchmark.

For future causal diagnosis, AMD’s uProf pipeline-utilization documentation separates front-end bandwidth and latency bottlenecks, branch misprediction, and SMT contention. Those measurements would distinguish hypotheses better than repeating cycles and instruction counts alone. They were not collected here, and no additional local measurement campaign is proposed by this report.

Automatic SIMD filter/count

The first automatic SIMD implementation recognizes one i32 or u32 column, one filter comparing its borrowed item with a same-type constant, and count(). It runs in optimized (--release) native builds. The reference defines the supported pattern; the tutorial shows ordinary source code. There is no public SIMD API or new syntax.

The subsequent runtime-bound extension accepts immutable scalar captures and two total comparisons joined by and. The reference above describes current behavior; the measurements below retain the original constant-only slice.

Why this boundary

The initial recognition inspected the checked loop and a bounded scalar leaf predicate. It accepts exactly one comparison, either in a named function or a closure without captures. Calls, arithmetic, extra adapters and other shapes keep their existing scalar path. The original scalar loop is retained for the tail. No callback effects, traps, floating-point operations or allocation failures are reordered.

The source has a stable borrowed pointer and length. Fresh zero counters and the known increment establish that the count fits i64. Four-lane loads only run below the length rounded down to a multiple of four; they have no alignment promise. Comparison masks widen to two 64-bit accumulators before summing, so large columns cannot overflow 32-bit lane counters. The final scalar item local is preserved as well as index and count.

Cranelift 0.137.0 emits 128-bit SSE2 instructions for this pattern. The current machine (AMD Ryzen 7 7840HS) has AVX2 and several AVX-512 features, but this slice does not need them. Using the existing target baseline keeps binaries portable among supported x86-64 machines and avoids runtime dispatch overhead and policy. This is not a claim that Cranelift can exploit all this CPU’s wider vector features; fresh backend upgrade probes establish the current width boundary and distinguish useful new backend features.

Validation and reproduction

Integration tests compare all six signed/unsigned comparisons over lengths 0–129 with scalar expectations, including extreme bit patterns, empty owners, column offsets and every four-lane remainder. Separate cases retain effects, captures, checked arithmetic and additional adapters on the scalar path. Dump tests check actual vector IR and packed comparison instructions.

The experiment in experiments/simd/clist/ compiles the actual Plenty pipelines, then links a C fixture against their generated functions. It constructs read-only internal-ABI owners solely to place columns immediately before a protected page. Lengths 0–129 and a null empty owner pass. Odd lengths need four poison bytes before the guard to keep the header aligned; scalar-oracle comparisons detect counting that padding. This fixture is not a supported foreign clist interface.

Run from the repository root:

cargo build --locked --release --bin plenty
python3 experiments/simd/clist/run.py target/release/plenty --out /tmp/clist-simd-check --verify-only
python3 experiments/simd/clist/run.py target/release/plenty --out /tmp/clist-simd-bench

The retained benchmark measurements used a captured threshold to retain the scalar pipeline and a constant threshold to enable SIMD, both comparing against five. It builds one object and relinks it at four text addresses, 16 bytes apart, then shuffles scalar/SIMD executions. Each variant has one warmup and three measured samples. Internal timings exclude input construction; perf instructions/cycles include startup and construction. Tiny calls therefore include substantial fixed scan setup. This is a hot repeated microbenchmark, not an application or memory- bandwidth claim. Code placement matters, as issue #437’s research demonstrated; do not select just the best scalar/SIMD address pair. The current runner uses an explicit scalar column loop as its reference, since captured thresholds now qualify for SIMD; rerunning it measures that updated comparison.

Final measurements

Measured on 2026-10-24 on the Ryzen 7 7840HS, pinned to logical CPU 8, with a fresh Cranelift 0.137.0 release compiler incorporating main c6e3f8b9, including the executable trap-table changes. Host process monitoring sampled 16 times at half-second intervals throughout the final run and observed no concurrent cargo/rustc work. The retained experiments/simd/clist/measurements.json records all 72 measured executions, checksums, actual function addresses, instruction/cycle counts, compiler/object hashes and the method. Earlier exploratory runs are not mixed in.

Each cell below spans the four placement medians, not individual samples:

Rows per callScalar ns/rowSIMD ns/rowScalar instructions/rowSIMD instructions/row
72.157–2.4801.745–2.03442.72438.010
8,1920.668–1.0010.116–0.22413.0373.037
262,1440.682–1.0070.115–0.22413.0583.058

At matched link placements, the two larger sizes improve 3.94–5.93× in elapsed time with roughly 4.3 times fewer instructions. SIMD uses about 0.54–1.03 cycles/row, while the scalar control varies between about 3.0 and 4.6. The small case improves only 1.19–1.24×; setup costs dominate and the captured scalar threshold has extra fixed setup. This compares two paths in the same compiler, not identical source compiled before and after the change.

The previous 0.131.1 run is retained as experiments/simd/clist/measurements-cranelift-0.131.json. The new vector loop retains its comparison constant in a register and executes about 0.25 fewer instructions per row on the larger inputs. Placement sensitivity is stronger in this run, so the upgrade does not imply a uniform elapsed-time improvement.

The SIMD function occupies 592 bytes including its constant pool; the scalar control occupies 513 bytes. Assembly contains unaligned movdqu, pcmpgtd, mask shuffles and paddq/psubq, followed by the scalar tail. The comparison does not call a predicate per row. The gains justify this small recognition slice; they do not establish a general backend ranking, wider-vector benefit, or speedup for other predicates and workloads.

SIMD after the Cranelift 0.137 upgrade

The automatic column filter/count slice was rebuilt and revalidated against Cranelift 0.137.0, after merging local main through c6e3f8b9. The original PR used 0.131.1. Production lowering needed only the new MemFlagsData type and the explicit signed-immediate builder methods (band_imm_s, iadd_imm_s, ishl_imm_s, icmp_imm_s). The loop’s legality checks, four-lane width, baseline target and scalar tail are unchanged.

Fresh evidence

Post-integration validation passed 145 compiler tests, 3,787 integration tests (two ignored), 183 runtime tests (four ignored) and five example tests. The full workspace/all-features run exposed issue #442 in the newly merged trap-table test: heap-to-stack offsets can exceed its signed 32-bit range. Keeping its synthetic code/sites/table in one allocation fixes the test without changing production behavior; the entire runtime suite then passed in both default and all-features configurations. This includes signed/unsigned comparison and fallback tests, generated-code checks and protected-page scans at lengths 0–129. Clippy passed for all workspace targets with warnings denied. Both compiler profiles were rebuilt from an empty task target directory; compiler debug information was disabled to keep build storage bounded.

The final four-placement measurements use the new release compiler. Its SIMD loop keeps the comparison constant in a register instead of reloading it each four-row iteration. The measured large scans execute about 0.25 fewer instructions per row. At the initial backend upgrade, scalar control assembly was unchanged while SIMD register allocation and block layout changed. Subsequent main changes replace executable trap helpers with PC-relative trap tables, shrinking the SIMD/scalar functions from 608/525 to 592/513 bytes. Measurements were repeated after integration; retired instructions per scanned row remain the same. Direct reference-store changes introduce no new alias-region assumptions. We do not attribute all elapsed-time differences to one optimizer pass.

Vector-width probe

examples/clist_simd_backend_probe.rs compiles a load/compare/store function with the verifier enabled and the workspace’s pinned dependencies. The experiment instructions (experiments/simd/clist/README.md in the repository) reproduce it. On the Ryzen 7 7840HS:

IR typeBaseline x86-64Detected native features
i32x4 (128 bits)Compiles, SSE encodingsCompiles, VEX encodings using XMM registers
i32x8 (256 bits)RejectedRejected
i32x16 (512 bits)RejectedRejected

Both rejected widths report Unexpected SSA-value type. This is a compile-only probe, independent of Plenty’s pattern recognition. The pinned upstream x86 register-class selection accepts vector SSA values up to 128 bits. Native feature detection changes available instructions but does not remove that restriction. The versioned upstream file was checked against the installed crate source.

Useful upstream changes

Cranelift 0.137 corresponds to Wasmtime v50. Release notes across v45–v50 describe optimizer and target improvements. In particular, v48 adds dead-store elimination in alias analysis and VNNI dot-product lowering; v50 improves alias analysis and introduces initial APX support. APX is not a wider SIMD-vector facility. Winch SIMD entries concern a different compiler backend.

The pinned x86 source has new AVX-VNNI and AVX-512-VNNI feature flags and dot-product patterns for packed signed/unsigned byte multiplication and accumulation. These could help a future quantized or packed-byte column calculation, provided its semantics and target selection are established. They do not replace this integer comparison/count kernel. No VNNI lowering, runtime dispatch or public SIMD API is added here; broader priorities remain in the backlog.

The upgrade improves the backend we already use without requiring a broader ISA policy for this slice. It does not demonstrate general auto-vectorization or unlock ordinary 256/512-bit vector SSA operations on x86.

SIMD counts with runtime bounds

The next bounded step after constant comparisons recognizes immutable runtime scalar captures and two total comparisons joined by and. This makes queries such as “count scores between these two input limits” use the same four-lane SSE2 path without exposing vectors, alignment, tails or hardware selection in the source. The tutorial shows the supported idiom; the reference defines its limits.

Legality boundary

Predicate recognition now has separate bounded metadata rather than expanding general leaf-function inlining to control flow. It accepts one comparison or the exact checked-IR short-circuit form for two comparisons joined by and. Both compare the item with a same-type constant or an immutable scalar capture; either operand order is accepted. Captures are limited to two by-value i32 or u32 values matching the column. The actual closure environment is read once before the vector loop, preserving its captured snapshot rather than rereading the original source binding.

Two total comparisons can be evaluated eagerly as vector masks because neither can call, allocate, mutate, trap or read externally mutable state. Combining their masks therefore preserves observable short-circuit behavior. Borrowed captures, mutable closures, arithmetic, calls, or, extra comparisons and extra pipeline adapters retain the scalar path. The original scalar loop handles the tail, with the same bounds and 64-bit count guarantees as before.

Validation

Tests compare normal and optimized scans with scalar expectations for all six comparisons, inclusive/exclusive range endpoints, reversed/equal bounds, lengths 0–33, and explicit signed/unsigned minimum, maximum and sign-bit boundaries. They check snapshot preservation after the source binding changes and across successive calls. Short-circuit side effects and skipped overflowing arithmetic remain scalar. IR/assembly checks confirm vector emission for runtime, named, constant/captured mixed and reversed comparisons, and scalar fallback for borrowed captures, arithmetic and disjunction.

The protected-page fixture now covers constant, runtime-threshold and range scans over lengths 0–129 and a null empty owner. Each scenario checks actual compiled Plenty functions against a scalar oracle.

Performance method

experiments/simd/clist/run.py --scenario threshold and --scenario range compare optimized pipelines with explicit scalar column loops. Both receive their bounds as runtime function arguments from the C fixture. The scalar reference has no per-row callback call; measurements therefore do not inflate the SIMD benefit with callback overhead. The original constant-only reports remain historical evidence, since their captured scalar reference now qualifies for SIMD.

Each scenario uses one compiled object, four verified code placements, randomized execution order, one excluded warmup and three samples per variant. Internal elapsed time covers repeated scans; instruction/cycle counts also include fixture startup. Small scans expose setup/tail costs, and large hot scans are not a claim about cold-data bandwidth or application-wide speedup.

Final measurements

Measured on 2026-10-24 on the Ryzen 7 7840HS, pinned to CPU 8, with the final Cranelift 0.137.0 release compiler. Two monitoring windows each sampled host processes 24 times at half-second intervals during the respective runs and saw no concurrent cargo or rustc. Raw reports are retained as experiments/simd/clist/measurements-runtime-threshold.json and measurements-runtime-range.json, including compiler/object hashes, checksums, function addresses and all 144 measured executions across the two scenarios.

Each range spans the four placement medians:

QueryRowsScalar ns/rowSIMD ns/rowScalar instructions/rowSIMD instructions/row
Runtime threshold71.613–1.9802.152–2.21236.86740.010
Runtime threshold8,1920.625–0.9310.107–0.21411.0343.039
Runtime threshold262,1440.623–0.9430.108–0.21511.0583.058
Runtime range71.957–2.3012.978–3.05541.58153.438
Runtime range8,1920.856–1.3180.162–0.21815.0344.542
Runtime range262,1440.845–1.2700.160–0.21215.0634.563

At matched placements, the large threshold scans are 2.91–8.71× faster and range scans 4.41–8.01× faster than the explicit scalar loops. Instruction reductions are stable across placements; elapsed time remains placement-sensitive, so the whole range matters. Seven-row pipeline scans are 12–34% slower for thresholds and 30–56% slower for ranges than those loops: closure/vector setup and scalar tails dominate. This does not compare against the old scalar callback pipeline, and is not evidence that every small query should vectorize. No new profitability threshold is inferred from these three sizes alone.

Generated function sizes are 640 bytes for the threshold pipeline versus 309 for its scalar loop, and 752 versus 352 for the range. The extension keeps the existing SSE2 baseline and does not add target dispatch or parallel execution.

Dataframes backed by Polars

Status: A bounded foundation is implemented as std.dataframe. It uses a separately built optional Polars engine, bulk numeric transfers and explicit expression plans. clist remains a distinct lightweight mutable collection. The text below records the original spike and its rationale; the reference describes the supported API. See the original measurements and B76 in the backlog for future priorities.

Why use an existing engine?

The original columnar-list vision overlaps with a typed dataframe: ordinary records describe the data, columns provide efficient storage, and bulk operations expose parallelism. Polars already supplies a much wider analytical engine, including expressions, joins, grouping, file readers and query planning. Its lazy optimizer can push filters and column selection into scans and simplify expressions. Embedding it lets Plenty focus on a clear, safe language interface instead of implementing another analytical engine. Polars lazy optimizations.

The tested boundary

The spike pins Polars 0.55.2, disables default features and enables lazy and parquet. Their transitive features are substantial; this is not a tiny dependency. A separately compiled Rust shared library owns DataFrame, LazyFrame, expressions and typed column builders. An ordinary Plenty trusted interface wraps opaque handles in move-only classes with destructors. Existing foreign-call support is sufficient: there are no compiler or runtime changes.

Plenty creates an expression tree; Polars optimizes and evaluates it. No Plenty callback runs once per row. For example, the experimental module supports:

groups = [0, 1, 0, 1]?
values = [10.0, 60.0, 80.0, 90.0]?
data = df.from_lists(&groups, &values)?
predicate = df.col("value")?.gt(df.lit(50.0)?)?
weighted = df.col("value")?.mul(df.lit(2.0)?)?.alias("weighted")?
total = df.col("weighted")?.sum()?.alias("total")?
result = data.lazy()?.filter(&predicate)?.with_column(&weighted)?
             .group_by_sum(df.col("group")?, &total)?.collect()?

This illustrates the API, not new multiline syntax. The runnable, single-line chain is in experiments/polars-embedding/example.plenty. It produces totals 160 and 300, then repeats the operation through a written and scanned Parquet file. Polars runs inside the same process, with no Python or subprocess query execution. Cranelift compiles the Plenty orchestration; Rust/LLVM compiles the engine separately. Ordinary Plenty compilation does not invoke Rust or Cargo.

Types and semantics need their own design

The spike has typed i64 and f64 input builders and checked typed extraction. It does not implement DataFrame[T]: column names and query result schemas are checked by Polars at runtime. Only a two-column constructor and a small expression subset are exposed. A production typed facade must distinguish statically known schemas from schemas discovered in files.

Null extraction becomes Option[T], independently from query failure in Result. Floating-point NaN remains a value; null is not converted to NaN or zero. Polars defines the query’s arithmetic, aggregation, comparison and ordering semantics. These are not silently promised to match an ordinary Plenty loop: parallel floating reductions can change rounding, groups have unspecified order, nulls require explicit policies, and integer overflow needs a deliberate contract. The executable check observes that Polars keeps NaN in a value > 50.0 filter, whereas Plenty’s ordinary floating comparison returns false. Translating an ordinary Plenty predicate into this expression without an explicit semantic boundary would therefore change the program’s result. The spike only builds floating multiplication and aggregates; it does not claim checked Plenty integer arithmetic inside a query. Polars data types.

Handles retain Polars-owned data after the Plenty source frame dies. Inputs are borrowed by calls, query operations clone native handles, and errors leave inputs owned by the caller. Native destruction uses the native allocator. Recoverable failures currently have coarse error categories. Ordinary Rust panics are caught at the boundary, but allocation failure may abort the process: Result does not imply Plenty’s recoverable AllocError contract. Rich error messages and a documented allocation policy remain important production choices.

Copies and deployment

Current foreign adapters support scalars and borrowed text, but no numeric slice borrowing. The spike copies lists element by element into native typed builders, then materializes Arrow columns. Small results are copied back one scalar at a time. This is intentionally measured. A bulk numeric-buffer adapter would avoid per-element calls and intermediate Option vectors even without zero-copy.

True zero-copy needs compatible layout, an owner that outlives the array, correct alignment and null representation, and safe mutation restrictions. Ordinary Plenty lists use tagged slots; they are not Arrow buffers. Even clist columns cannot simply be handed over without an ownership/release protocol. Parquet scans avoid the initial Plenty-list conversion altogether.

The engine is optional and prebuilt, with a fixed feature set and its own deployment artifact. The measured build uses x86-64-v3; this is a chosen spike hardware floor, not runtime ISA selection or Plenty’s normal baseline. A product must choose supported hardware/targets and package compatible binaries. Statically adding another independently built Rust runtime archive is not assumed safe; the shared-library boundary keeps allocation and Rust ownership on their own side.

Keep current clist behavior available while evaluating this direction. The spike tests analytical functionality, not row-mutation compatibility, full-schema typing, API completeness, or a migration. Its results can support prioritizing Polars integration without claiming that every small mutable columnar collection should pay for a query engine.

Polars embedding evidence

The proposal describes the boundary and semantic limits. Reproduction scripts and raw observations are in experiments/polars-embedding/. The experiment is separate from the compiler’s Cargo workspace and dependency graph. Normal Plenty builds do not compile or link Polars.

What runs

  • A Plenty-native facade constructs two typed columns from Plenty lists.
  • Explicit expressions filter values, derive a column, group by a key and sum.
  • The same query runs on a native lazy Parquet scan, returning values to Plenty.
  • Negative cases cover missing columns, mismatched input lengths, bounds and wrong-type extraction. Null and NaN remain distinct, negative zero survives extraction, empty schemas work, and a query outlives its source builders/frame.
  • Private constructors prevent source programs from manufacturing invalid native owners. Rust tests exercise null handles, unchanged outputs after errors and caught ordinary panics.

Measurement method

run.py compares the same deterministic data and equivalent filter/count or filter/multiply/group/sum work in Plenty with Polars, direct Rust with Polars, and a hand-written Plenty clist implementation. Input generation, import and repeated query execution are separate measurements. The grouped clist code specializes to eight known integer groups; it is not a general hash grouping implementation. All results are checked against an independent checksum.

Polars queries are built once and collected repeatedly, with five warmups. Query times include collection and copying the scalar or eight aggregated values out. They do not include input conversion or startup. Trials rotate implementation order. One-thread and four-thread settings use distinct physical cores; clist remains serial. Tiny and million-row inputs expose dispatch overhead as well as scan throughput. The startup experiment measures process launch with warm file cache, not cold disk load or the first query’s thread-pool initialization.

The built shared engine is copied beside Plenty binaries and found through $ORIGIN. Compilation timings measure optimized whole-program compile/link with caches disabled and a repeated cache hit with the engine already built. They are not a claim of separate Plenty module compilation.

Observations

Measured on 2026-10-25, AMD Ryzen 7 7840HS, Rust 1.99/LLVM 23.1.1. The engine uses explicit x86-64-v3 flags, stable Rust and no Polars nightly feature. Plenty programs retain their normal x86-64 baseline. CPU affinity is 2 for one thread and 2,4,6,8 for four threads. Seven rotated trials are recorded, with 500 query repetitions at 1,024 rows and 20 repetitions at one million rows.

RowsQueryPolars threadsPlenty + PolarsDirect Rust + PolarsPlenty clist
1,024Filter/count141.3 µs39.8 µs0.96 µs
1,024Filter/group/sum172.9 µs70.1 µs4.11 µs
1,000,000Filter/count10.728 ms0.756 ms0.935 ms
1,000,000Filter/group/sum14.574 ms7.982 ms3.941 ms
1,000,000Filter/count40.710 ms0.709 ms0.961 ms
1,000,000Filter/group/sum42.721 ms2.723 ms3.922 ms

The large four-thread grouped query is about 1.44× faster than this specialized clist loop, while small queries pay substantial engine overhead. The count comparison uses f64, not clist’s SIMD-specialized integer filter/count path. The wrapper and direct Rust paths are close in most cases. One-thread large grouping is an unresolved exception: the separate direct Rust executable is consistently slower. Removing an unnecessary rename from its plan did not remove the difference. These are distinct native binaries and construction paths; this bounded spike does not attribute that difference to wrapper overhead or claim that calling Polars from Plenty speeds up Polars itself.

For one million rows, constructing the two Plenty input lists costs about 95–96 ms, then copying into Polars costs about 36 ms. Direct Rust constructs its inputs in about 5.8 ms and imports them in about 6.3 ms. Building clist from the same Plenty lists costs about 41 ms. A single sub-millisecond query does not amortize these conversions. Keep data inside the engine across operations, read files directly where appropriate, and evaluate bulk transfer before promising efficient mixed Plenty/Polars workloads.

The instruction-count cross-check covers the whole process, including input generation/import, 100 grouped queries and Polars’ five warmups. It records about 9.62 billion user instructions for Plenty + Polars and 12.18 billion for clist, yet the corresponding cycles are about 2.78 and 2.46 billion. These are not isolated kernel counts or a substitute for the measured query times.

Build and distribution costs

The clean optimized engine build takes 279.9 seconds at three Cargo jobs, with dependencies already downloaded. GNU time reports 1,709,584 KiB maximum process RSS; this is not total parallel-build memory. The lockfile contains 332 packages including the adapter. A later direct-baseline-only rebuild takes 1.12 seconds. The clean-build observation predates removal of an unused helper and that baseline adjustment; it measures the same dependency and engine scope.

The shared library is 90.4 MiB, stripped of debug information. A Plenty benchmark executable is about 3.2 MiB and uses that shared artifact. The optimized Plenty/Polars benchmark compiles and links in about 53 ms without caches, or 41 ms with a warm cache; clist is about 49/41 ms. These process-level times include compiler startup and runtime extraction. They do not include building the engine. This is a good separation of engine build cost from everyday Plenty compilation, but a material distribution dependency.

Warm-file-cache launch with a single expression construction is about 2.79 ms, versus 1.15 ms for the plain executable. That includes subprocess and taskset overhead; it does not measure a cold launch or first-query latency.

Assessment

Embedding is tractable with the existing language boundary. Explicit expressions reach Polars’ optimizer, broad analytical operations and Parquet work without a compiler change, and the engine can be shipped precompiled. The results support prioritizing this direction over building those analytical facilities ourselves.

They do not establish a drop-in replacement for clist. Scalar/list transfer, typed versus discovered schemas, null/NaN/overflow semantics, diagnostics and native allocation failure need deliberate contracts. Tiny collection operations remain much cheaper without a query engine. Current clist users need not migrate as a consequence of this experiment.

Exploring async/await without exposed pinning

Status: Not implemented; exploratory proposal with a standalone native spike. Async/await is not accepted or scheduled for Plenty. The user requested this investigation while the language can still change freely, despite its original deliberate exclusion of async/await. Keeping it out remains a valid outcome. The backlog records that boundary; concurrency and generators describe what works today. Nothing here changes the tutorial or production compiler.

The question and provisional answer

Could Python-shaped async syntax coexist with Plenty’s explicit, fallible allocation and deterministic ownership, without exposing Pin or requiring asynchronous destructors? A restricted design looks possible: owned, concrete movable frames; no surviving borrow across suspension; explicit task admission; and synchronous cancellation only for operations whose pending work can be safely abandoned. The native spike demonstrates those mechanics for a tiny language. It does not demonstrate that the restrictions are ergonomic enough for HTTP/TLS or that a production scheduler is affordable.

This could improve simplicity for Plenty’s narrower contract, not improve on Python or Rust in general. In particular, avoiding arbitrary borrows across await gives up a major capability of Rust’s design. Refusing implicit async cleanup gives up automatic completion of some protocol obligations. Moving those costs into adapters or a runtime does not make them disappear.

What to borrow from Python and Rust

Python separates a coroutine from a scheduled task: calling an async function creates a coroutine, while awaiting or scheduling drives it. async def and prefix await fit Plenty’s surface syntax. Python also specifies separate asynchronous context-manager operations; these are more than alternate spelling for ordinary deterministic drop. PEP 492 describes that syntax and protocol separation.

Python task groups provide an attractive execution-ownership model: their exit waits for child tasks, and failures initiate sibling cancellation. Cancellation is delivered through CancelledError, can be suppressed, and needs careful handling in cleanup. Task creation can also opt into eager execution, so Python is not a mandate for one immutable scheduling rule. Plenty should specify lazy construction and task admission itself. Python task documentation is the comparison source, consulted 2026-10-22.

Rust’s await machinery polls a child and suspends its parent on Pending; its completion output can itself be a Result. That separation of suspension from failure suits Plenty. Rust’s await reference and Future contract also make readiness notification and the poll interface explicit.

Rust permits address-sensitive generated state, including a reference to another field in the same future. Pinning protects the resulting address invariant; it is not arbitrary syntax that can simply be deleted. The pinning documentation explains that motivation. Rust’s AsyncDrop remains an experimental nightly API in the documentation consulted. This proposal avoids relying on it and makes no claim that Rust’s cleanup design is settled or inherently impossible to simplify.

Candidate surface and evaluation order

The following is proposed Plenty syntax, not accepted source:

async def fetch(request: Request) -> Result[Response, TransportError | AllocError]:
    response = (await exchange(request))?
    Ok(response)

fetch(request) evaluates arguments immediately and moves/copies them into a concrete Coroutine[Result[Response, ...]]. It does not execute the body or schedule anything. Frame construction is inline and adds no allocation; argument expressions retain their own failures. await consumes a coroutine and produces its declared result. It may finish immediately or suspend; it does not imply another task, a thread, a failure, or a fairness yield. Parentheses deliberately leave await/? precedence undecided. ? propagates only the result error and destroys active owned locals through the ordinary cleanup path.

Like today’s Generator[T], Coroutine[T] would retain a concrete producer identity rather than secretly box different producers into one type. Returning coroutines from factories, generic specialization, frame-size limits, recursive async calls and containers need explicit contracts. Initially reject recursive inline layouts and erased/dynamic coroutine storage. A coroutine value is affine: it cannot be copied or awaited twice. Dropping an unstarted one destroys captures without executing the body; this is a deliberate difference from Python’s unawaited-coroutine warning behavior.

async would be legal only on functions in the initial scope. await would be legal only inside them, plus a library root runner that consumes a coroutine. No implicit async main, async iteration, async closures, arbitrary awaitable protocol, async generators or async with follows from adopting this core. These restrictions keep a possible first implementation independently assessable.

Scheduling is a separate choice

Direct await is sequential composition. To overlap work, an explicit bounded scope admits multiple roots. A candidate API is scope.start(coroutine) -> Result[Task[T], StartError[Coroutine[T]]]: capacity or allocation failure returns the entire unstarted owner. The scope owns execution; dropping a public handle must not orphan tasks. Results need their own owned, allocation-audited storage and a policy when unclaimed. Prefer one result consumer over implicit reference-counted shared results.

The minimal affine scope can synchronously cancel its remaining abort-safe children at exit. A stronger scope promising asynchronous graceful completion needs explicit awaited closure or retained responsibility elsewhere. It cannot silently borrow an enclosing stack and let that stack return. The distinction is developed in ownership and cleanup.

Existing Future[T] represents an executor job’s result and has different drop/cancellation rules. Do not reinterpret it as a coroutine or make .result() secretly await. Blocking code called from async remains blocking. A future adapter must submit work fallibly, wake the async consumer safely, preserve result/error ownership, and define cancellation when the worker continues. No such adapter exists here.

Alternatives and the decision still needed

DesignBenefitCost or unresolved boundary
Keep current threads, bounded executors and readiness APIsNo second calling convention or async ecosystem; existing fallible admissionMany mostly idle connections may need manual state machines or many native stacks; measure a real workload
Owned movable async frames (spike)Inline composition, no pin API or frame allocation per callNo stored loan across suspension; ownership-return I/O can be cumbersome
Runtime-owned stable framesPotentially allow broader borrows without user-facing pin typesFallible per-frame/arena reservation, lifetime proof, no relocation, fragmentation; address stability is internalized, not removed
Stackful tasks / goroutine-style syntaxSynchronous-looking call chains can suspend deeplyFixed stacks consume reserved memory and can exhaust; growing/moving stacks must preserve pointers, FFI and allocation failures; blocking calls and task admission still need policies
General Rust-like borrowed futuresBroad zero-copy composition and explicit low-level controlAddress-sensitive frame projection and poll contracts recreate the complexity we want to avoid

The independent stackful-task exploration investigates bounded fallible stack creation, guard pages and overflow behavior, cross-thread movement, foreign calls and cancellation. Stackful tasks are an alternative to syntax-directed stackless transformation, not a consequence of it. Current OS workers already provide one stackful baseline. Neither syntax alone solves protocol cleanup.

Before considering adoption, compare at least one bounded connection workload and one file/worker workflow using existing APIs and each plausible prototype. Measure allocations, reserved/live memory, throughput, latency, cancellation latency, compile time and API friction. The spike establishes neither a performance win nor a requirement for async. The main open decision is whether owned-operation ergonomics justify a second execution model; a full borrowing redesign should not be smuggled in as a small syntax convenience.

Suspension, allocation and cleanup boundaries

Companion to the unadopted async/await exploration. These are candidate constraints, not implemented Plenty contracts. The spike exercises only the small subset explicitly identified there.

Movable state without pinning

An inline frame can move between polls only if moving it cannot invalidate a surviving reference. Scalars, owned heap handles and owned child frames fit that rule when children obey it recursively. A short borrow during one poll is fine if it ends before returning Pending. A pointer into an independently owned heap buffer can stay valid when its handle moves, but owner/lifetime and concurrent-access rules still need proof; a pointer into the frame cannot.

Start from the restrictions already used for generators: no borrowed parameters/captures, and no live loan across suspension. Extend the check through awaited children and hidden operation state, not just syntactically obvious local variables. A child awaiting a borrow into its parent’s frame makes the combined frame address-sensitive even if the parent source declares no named reference. Compiler-created temporaries must be checked too.

async def invalid() -> Result[(), IoError]:
    mut bytes = ...
    count = await read_into(&mut bytes)   # rejected under the movable design
    Ok(())

The straightforward alternative transfers owned state:

async def receive(stream: Stream, buffer: Bytes) -> ReadOutcome:
    await read_owned(stream, buffer)

ReadOutcome = ReadOk(Stream, Bytes, Count)
            | ReadErr(Stream, Bytes, Cause)

These names are illustrative. An error may need to distinguish a reusable stream from one whose framing is uncertain. Losing the stream on cancellation may be safe locally but undesirable; an ownership-return cancellation API can preserve it. Returning buffer/stream owners repeatedly is the chief ergonomic cost, especially with nested protocol layers and multiple buffers. Ordinary borrowed readers cannot automatically become async readers. The spike does not validate this API or Plenty’s generic ownership checker.

Reject frame-address escapes through FFI, callbacks, readiness registrations, self-references and user-defined awaitables until they have checked contracts. Use stable task identities or independently owned registration records for wake notifications; never retain an inline frame pointer in the reactor. Deregister before freeing the associated storage, including stale wakeups and concurrent completion races. If an OS operation retains a buffer until completion, dropping the frame cannot free that buffer merely because cancellation was requested.

A stable-storage alternative can hide address stability behind a consuming owner API. That removes Pin from user spelling, but still requires non-moving storage and lifetime/drop-order proofs inside the compiler/runtime. It also changes fallible construction and task admission. Neither the spike nor existing generator implementation proves that alternative sound.

An allocation ledger, including failures

BoundaryCandidate contractEvidence / remaining work
Inline coroutine creationNo new heap allocation; argument failures remain visibleScalar captures demonstrated; owned captures and general layouts unimplemented
Nested awaitChild frame stored inline; poll/resume bookkeeping does not allocateDemonstrated, including nested state moved after suspension
Body allocationOrdinary Result with AllocError, propagated explicitlySpike uses a numeric status and tests injected failures; Plenty Result lowering unimplemented
Root admissionReserve slot/result/wake storage before transferring owner; return owner on failureOne fallible root allocation and fixed two-slot queue demonstrated
Queue growth / saturationExplicit fallible reservation or fixed capacity with a capacity errorFixed capacity demonstrated; no dynamic growth
Wake/registration/timer storageReserve fallibly before registration; one owned cleanup pathNot implemented; cannot assume that waking or cloning a waker is free
Erased coroutine or resultExplicit fallible boxing/storage APINot implemented; no implicit boxing fallback
Cancellation/dropNo allocator-dependent success path for mandatory synchronous releaseMemory-only resources demonstrated; native handles and user cleanup unaudited

The guarantee concerns coroutine machinery; caller expressions and user cleanup retain their own effects. Compile-time layout computation and compiler memory are outside the generated program’s allocation contract. Inline frames can be large even with no heap allocation. Set layout/nesting limits and diagnose excess rather than silently box. A runtime stack limit is not a recoverable heap allocation error. Recursive async calls need an explicit heap indirection or an initial rejection, as recursive generator layouts already do.

Cancellation is not asynchronous destruction

For the narrow candidate, dropping a suspended coroutine synchronously destroys its live child and owned locals in reverse acquisition order. It does not resume source statements, invoke arbitrary await, allocate an emergency cleanup task, or promise that a remote peer rolled back work. The spike uses exactly this rule. Scope cancellation can stop and destroy a suspended task on the executor thread; cancellation cannot interrupt a currently executing poll or a blocking foreign call. Long CPU work needs explicit cooperative checkpoints.

Only operations with a safe synchronous fallback can use that rule. Closing a local socket may be an acceptable fallback to skipping a graceful protocol shutdown; leaving a kernel operation referencing freed memory is never one. The relevant property is not just a destructor returning (): it includes in-flight registrations, external buffer ownership and the cost of unregistering. A potentially blocking destructor also defeats bounded cancellation latency. Do not generalize from the spike’s free() to all existing Plenty values.

An explicit await finish(owner) can attempt graceful shutdown and return its errors. The program still needs an admissible fallback if it is cancelled during that await. Initially exclude awaiting inside with, async context managers, async destructors and operations with mandatory suspending cleanup. This is a restriction, not a promise that all cleanup becomes simple.

For a protocol requiring completion, a scope/runtime must retain and drive the owner to a permitted terminal state, or return it to someone responsible. If that can suspend, synchronous scope exit cannot provide a nonblocking completion guarantee. Choices are explicit awaited closure, blocking exit with possible deadlock, or transfer to an explicitly owned supervisor. Each needs failure and shutdown rules; none belongs invisibly in drop.

This agrees with linear integration: affine abandonment cannot carry a future linear obligation. async with alone supplies a location for cleanup code, not an owner guaranteed to execute it. Initial async frames should exclude non-discardable values if the separate linear proposal lands. Cancellation shields can delay a request, but cannot prove termination, repair allocation failure, or outlive a borrowed resource.

Scope and error questions still open

A production scope needs rules for the first child error, siblings still running, cancellation result ownership, unclaimed results and multiple errors. Fail-fast cancellation can be useful, but aggregating errors must not introduce infallible allocations. A bounded result array, first error plus explicitly discardable later results, or a fallible collection step are distinct policies.

The experiment’s task cancellation returns a separate CANCELLED status and ordinary child errors keep their numeric code. In Plenty, await coroutine should return the coroutine’s declared output; awaiting a task handle may need an additional scheduler cancellation result. Do not silently inject cancellation into every function’s error set or pretend graceful cancellation is just a transport error. The spelling and exact ownership of those outcomes remain open.

Native async/await spike evidence

Companion to the unadopted proposal. Evidence collected on 2026-10-22 against local main at 9c994af0. This is a standalone compiler experiment under experiments/async-await, not a modification to the Plenty parser, checker, code generator or runtime.

What actually executes

A small Python program parses a restricted Python-shaped source grammar and emits C state machines. Python supplies an AST, not coroutine execution. A C11 compiler produces a native executable; a C harness polls it, moves suspended frames, injects allocation failures and checks cleanup. No Rust futures, pinning, GC, async library, hidden thread or manually substituted source result is used. The source really contains async functions and await expressions:

async def leaf(value: i64) -> i64:
    resource = acquire()
    result = await tick(value)
    return result + 1

async def parent(value: i64) -> i64:
    resource = acquire()
    first = await leaf(value)
    second = await tick(first)
    return second + 1

This syntax is accepted only by the spike. It is neither ordinary Python (the intrinsics and i64 are experiment-specific) nor current Plenty. Numeric error propagation is implicit in this small compiler; the proposed Plenty Result/? syntax is not implemented. There is no I/O backend: tick returns pending once and then its value. Real nonblocking I/O and registration lifetimes remain a separate adoption gate.

Each await has an initialization state and a resume state. Initialization runs once; pending returns preserve the child frame and all live locals. Ready results flow to the next statement in the same poll. Cleanup flags distinguish initialized resources/children from untouched fields; completion and error both release them. A child frame contains no pointer to its parent. The harness moves frames by transferring their bytes and invalidating the source, preserving owned heap handles. C does not enforce ownership; the restricted generator and trusted harness do. This is mechanical evidence, not a proof about arbitrary Plenty programs.

Reproduce and inspect

From the task/repository root:

python3 -B experiments/async-await/test_spike.py
SANITIZE=1 python3 -B experiments/async-await/test_spike.py
python3 -B experiments/async-await/lower.py experiments/async-await/program.async

The first two commands compile and execute temporary artifacts and report test status. The third prints generated C for inspection. No dependencies beyond Python 3 and a C11 compiler are required; CC selects the compiler. See the README for the exact subset and host/compiler allocation boundary.

Validation passed with the default C compiler and with Clang’s address, undefined-behavior and leak sanitizers. The default GCC sanitizer link failed because its runtime shared libraries were unavailable; Clang supplied a working alternative. LeakSanitizer required execution outside the sandbox’s ptrace wrapper. Both Python test methods passed, including twelve rejection subcases and all native harness assertions. Book link checking and mdbook build book also passed. The production compiler/runtime suite was not run: neither their code nor executable tutorial examples changed.

Observations and their limits

ExperimentObserved resultWhat it supports
Create/drop without pollingZero allocation attempts and no body effectsLazy inline construction for scalar captures
Nested parent(40)Two suspensions, then 42; two body resource allocationsAwait composition needs no additional frame allocations
Move while parent and child are suspendedResumes correctly; child then parent release exactly onceThis restricted pointer-free frame layout can move after suspension
Drop before start / at each suspensionZero surviving allocations; drop makes no allocation attemptsSynchronous cancellation works for the experiment’s memory-only owners
Child failureError code 71 survives parent propagation; both resources releasedError and completion cleanup paths can share a state-machine structure
Fail allocation 1, 2 or 3Admission failure returns intact frame; parent/child failures release acquired ownersAllocation failure need not abort or leak in these paths
Fail admission of a suspended frameByte-identical source still owns both resources; retry then cancellation releases everythingReservation-before-transfer works beyond unstarted frames
Two roots and a third submissionBoth progress; third gets capacity failure without allocation or transferBounded explicit scheduling can preserve failed-submission ownership
Close scope while pending and allocator disabledAll remaining resources/tasks released without allocatingA restricted affine scope can cancel synchronously
Twelve unsupported-source casesRejected before C compilationBorrowed parameters, recursion, loops, async context managers, escapes/copies and invalid calls are not silently accepted

On this x86-64 host, frame sizes were: leaf 56 bytes, parent 128 bytes, failing leaf 56 bytes, propagating parent 96 bytes; two-slot scope 96 bytes. The successful scheduled parent performs three allocations: one root frame and two explicit 16-byte body resources. Direct polling performs only the latter two. The generated parent retains separate storage for its two await sites instead of overlaying disjoint lifetimes. Frame-size reduction is possible but not demonstrated. These figures are layout observations, not throughput or memory comparisons with Rust, Python, threads or real Plenty.

What changed our assessment

The basic state machine and failure rollback are small when every suspended value is owned and destruction is synchronous. That makes a constrained language extension plausible. The experiment also exposes why the restriction matters: admitting even one child borrow into its parent would invalidate its simple relocation test. A stable allocation, offset-based references with careful external-pointer rules, or a pin-like invariant would then be necessary.

The substantial unanswered work is at the library boundary. A buffer-owning read API must return owners on failure and cancellation; a reactor must never retain a movable frame pointer; an in-flight native operation may outlive a cancel request. None of those problems is exercised by a synthetic tick. Likewise, memory-only drop cannot establish protocol completion. The result is a reason to evaluate owned-operation ergonomics, not evidence that we have eliminated the hard parts of asynchronous I/O.

Production work would touch parsing/AST, concrete coroutine identity and generic specialization, suspension-aware loans, return/error lowering, partial initialization and cleanup, native-effect certification, diagnostics/cache identity, runtime wake/task storage, every exposed async I/O adapter and the tutorial. Existing generator machinery may be reusable, but its absence of body ?, return values and nested await means this is not just a new keyword. The spike omits branches, loops, arbitrary captures/results and user-defined destructors; it estimates no calendar schedule or compiler performance.

Until a realistic owned I/O prototype and a comparison with current workers justify those costs, leaving async/await out remains the conservative choice.

Exploring Go-style tasks with Plenty’s ownership rules

Status: Not implemented; exploratory proposal with a standalone native spike. Lightweight tasks are neither adopted nor scheduled for Plenty. The backlog records the investigation; the concurrency reference describes the existing native workers. The production compiler, runtime and tutorial are unchanged. Keeping the current concurrency model is a valid outcome.

The question and provisional answer

Can ordinary functions suspend on communication, retain their local borrows, and clean up synchronously, without an exposed pinning API or async destructors? Scoped tasks on immovable, fixed-size stacks look like a plausible smaller model. The spike exercises actual stack switching, nested calls with stack-local references, bounded communication, cancellation and fallible admission. It does not establish a production implementation or the scalability normally associated with Go.

This is simpler at the function boundary than movable async frames: the call stack retains ordinary locals and nested calls; a callee can keep borrowing its caller’s local across a suspension. The simplification costs a stack reservation per task, fatal exhaustion unless another design is chosen, suspension effects that must still be checked, and a scheduler-aware I/O boundary. Full Go-style growable-stack M:N execution is a much larger commitment. The runtime choices separate these options.

What to borrow from Go

Go starts ordinary function calls concurrently, and its channels provide typed communication. Arguments are evaluated in the starting goroutine. The call’s return values are discarded by a go statement. These are language semantics, not a promise about which operating-system thread runs the function. Source: Go specification.

For Plenty, retain ordinary function bodies and explicit communication, while using a fallible library operation that returns a scoped result handle. Spawning must preserve ownership on failure. Starting a task cannot silently succeed when there is no stack or queue capacity; detached tasks would also conflict with the initial lifetime/cleanup contract. No go expression, unbounded scheduler queue, automatic GC, implicit channel allocation or process-wide hidden executor follows from this proposal.

Go’s scheduler distributes goroutines over worker threads, with separate goroutine, worker and processor state. Its runtime includes parking, syscall transitions and work distribution. That implementation is evidence of the work behind the surface, not a minimum dependency for a single-worker experiment. Source: runtime scheduler.

Candidate surface

Prefer library APIs and familiar with, explicit captures, Result/?, concrete jobs and consuming result handles. This sketch is proposed API spelling; neither std.task nor these error types exist:

import std.task as task

def answer(value: i64) -> Result[i64, task.Cancelled]:
    task.checkpoint()?
    Ok(value + 1)

def compute() -> Result[i64, task.ScopeError | task.StartError[()] | task.Cancelled]:
    with task.scope(max_tasks=8, stack_bytes=65536)? as scope:
        with scope.spawn(def once () -> Result[i64, task.Cancelled]: answer(41)).drop_job()? as work:
            return Ok(work.join()?)

The sketch omits typed channel construction; it is not a proposed rewrite of the existing std.thread or std.sync APIs. Names and whether scope/handle contexts should be nested remain open. Keep task types distinct until their ownership and wait behavior actually satisfy the native-thread contracts.

The parent evaluates captures and arguments first. scope.spawn(job) attempts admission, commits ownership only on success and schedules the body for later execution. Failure returns the unstarted concrete job, following consuming native jobs. An allocation-free .drop_job() conversion would discard that job deliberately. A scope reserves its task table and ready/wait bookkeeping fallibly. Capacity exhaustion is a separate Full(job) result; spawn_nowait is the initial policy. Waiting for capacity while holding a dependency can deadlock, so blocking admission needs an explicit later contract. A completed slot may be reused only after its result/handle lifetime is resolved; the spike does not implement reuse.

Channels have explicit positive capacity, fallible construction and owned messages. A successful send commits ownership exactly once. Full/nonblocking, closed and cancelled send errors return the unsent value without allocating; a parked send retains its value in its task until it commits or returns it. Wait nodes are reserved with admission, and park/wake/cancel never grow a queue. Dropping the last sender wakes receivers to observe closure; closing the receiver side wakes senders with their values. Cancellation is checked before a new transfer commits; an already committed transfer is not rolled back. Returning a cancelled unsent value must still respect linear obligations. Rendezvous and general selection are outside the first candidate. These are proposed contracts, not behaviors proved by the integer-only spike.

join() drives or yields to the owning scheduler until completion, then moves the result once. Returning a Result is ordinary task output, not a request to cancel siblings. Scope teardown joins all admitted children on every normal exit, including ?; unclaimed results are dropped once. Parent errors cannot erase child ownership or silently discard unresolved linear obligations. Existing linear integration remains a separate proposed contract.

Loans, suspension and migration

An ordinary stack remains at the same virtual address throughout task execution. Task-local loans and loans down its synchronous call chain may survive a yield without becoming movable self-referential frame fields. No public pin wrapper is needed. The runtime still promises address stability internally; hiding the promise does not remove its implementation obligations.

Initially accept wholly owned concrete jobs/results and prohibit references in messages, escaping callbacks and stored task handles. Within one task, preserve ordinary lifetime, exclusive-loan, owner-move and container-invalidation checks. Interleaving on one OS thread never authorizes two exclusive loans. Adding parent loans later should reuse scoped-worker reasoning: join before the owner dies, prevent conflicting parent accesses throughout the scope, and reject cycles. Do not infer those rules from the C experiment, which has no borrow checker.

Start with one scheduler bound to one native thread. Cross-thread task creation, handle transfer and migration are excluded. Owned inputs must still satisfy a defined task effect/eligibility check; existing native-worker certification does not prove that an operation is safe to suspend. A future M:N mode needs structural thread eligibility for captures, every live local, references, callbacks and destructors, and a contract for thread-local state and foreign handles. Merely making the initial closure transferable is insufficient.

Ordinary syntax can hide significant suspension. Infer a separate transitive may suspend effect for task operations and their callers; validate scheduler context at every reachable entry. A task-only helper must be rejected from an ordinary non-task entry. Calls through stored callbacks require evidence too. Reject suspension while a mutex guard, an uncertified foreign call, or a destructor is active. A pure helper can be called in either context. These checks may be a substantial compiler addition even without async/await keywords.

Cancellation and deterministic cleanup

Cancellation is a request, observed at task-aware waits and explicit checkpoints. It wakes parked channel/I/O waits and returns a typed cancellation error. The function then returns through ordinary cleanup and ? paths. Cleanup runs synchronously on that task’s retained stack before join releases the stack. Destructors cannot suspend, recursively drive the scheduler or allocate hidden task storage. Explicit fallible finishing operations handle protocol shutdown before ordinary destruction. A linear obligation cannot be abandoned merely because cancellation was requested.

Do not free a suspended live stack, asynchronously throw into arbitrary native code, use longjmp across destructors, or treat signal-handler unwinding as safe cancellation. A running task may ignore cancellation; scope exit can consequently wait forever. Fatal traps retain Plenty’s existing no-unwind behavior. Synchronous destruction also does not imply nonblocking destruction: a blocking close/drop path can stall this worker. Real adapters need that audit.

Go’s pipeline guidance explicitly teaches cancellation when downstream consumers stop early; convenient spawning and channels do not arrange application lifetime automatically. Plenty should make the scope boundary explicit and preserve its existing endpoint/join deadlock checks. Those checks are not a deadlock proof. Source: Go pipelines and cancellation.

Comparison with the async exploration

The independent async/await exploration describes restricted movable frames, with its ownership contract and native spike evidence. Both experiments are available for comparison; neither adds a production feature or depends on the other’s implementation.

QuestionRestricted movable async frames in that explorationFixed-stack task candidate here
Suspension syntaxExplicit await inside async functionsOrdinary calls; inferred suspension effects
References across suspensionInitially excluded, including hidden child borrowsTask-local call-chain borrows can remain at stable addresses
StorageConcrete frames inline; explicit fallible root admissionExplicit fallible task record and a whole stack reservation
DestructionSynchronous cleanup of initialized frame fieldsOrdinary synchronous stack cleanup after cooperative return
CancellationPending operation must support safe abandonmentTask must resume, observe cancellation and return
Existing blocking functionsNeed an owned nonblocking adapter or blocking poolAlso stall the worker unless adapted or offloaded
Runtime portabilityState machines can use ordinary native calling conventionsStack switching requires ABI/tooling/platform integration

Both candidates avoid exposing pinning by narrowing their contracts. Neither establishes general async-drop, cancellation latency, linear resolution, real I/O suitability or a concurrency feature ready for adoption. The async spike’s small frame layouts and this spike’s fixed stacks are observations of different workloads, not a fair performance comparison. A full Go emulation is not an established simpler alternative to either restricted candidate.

Decision still open

The experiment makes fixed-stack tasks worth comparing with the async candidate, especially when borrowed-buffer helper chains matter. Adoption needs the same bounded TCP/HTTP workload, cancellation and failed-start tests, a realistic stack budget, I/O registration-lifetime checks and compiler effect/loan experiments. Measure throughput, p95 latency, committed/resident/virtual memory, admission failure and cleanup work under equal limits. Those acceptance gates are recorded only in the backlog. Keeping native workers, adopting a narrow task library, choosing restricted async, or adopting neither all remain possible.

Stack, scheduler and allocation choices

Companion to the Go-style task exploration. All task designs here are candidates; today’s native workers remain the implemented contract.

Four different implementation commitments

ModelSimplificationCost and boundary
Existing native workers/executorsOS manages stacks/scheduling; ordinary synchronous code and checked ownershipEach running worker uses a native thread; executor queues jobs, rather than suspending arbitrary call stacks
Fixed-stack cooperative tasks on one workerStable stack addresses; no relocation maps; explicit bounded schedulingPer-task stack budget, no automatic preemption, task-aware blocking, fatal unchecked overflow
Fixed-stack tasks over several workersPotential task concurrency and parallelism without stack copyingCross-thread eligibility/migration, synchronization, wake races, TLS/FFI, work distribution, shutdown
Growable-stack M:N goroutinesSmall initial stacks and scalable ordinary call chainsCompiler stack checks, relocation/segmentation, failure semantics, syscall handoff, preemption, pointer maps and tooling

The one-worker experiment is stackful M:1 execution. It is not an M:N scheduler or a native-thread pool. Its stack switching shows that ordinary C calls can suspend below their callers, without manually translating every call into an async state machine. It demonstrates no million-task scalability, parallelism or end-to-end I/O speedup.

Stack choices and recoverable allocation

Go’s runtime checks stack bounds in generated code, allocates replacement stack storage, copies stacks and adjusts pointers. Its stack allocator has fatal OOM paths. Go’s stack machinery is integrated with its pointer metadata and runtime; removing GC does not supply the relocation machinery for free. Source: runtime stack implementation.

For Plenty, assess these independently:

  • Fixed guarded stacks: choose size and maximum task count explicitly. Allocate task records and mapped stacks before transferring the job; failed reservation is an admission error. This avoids later stack-growth allocation and keeps interior addresses stable. It wastes address space and commits pages as touched; a guard trap still terminates unless a separate safe failure path exists. Large stack frames can skip a guard without stack probes.
  • Reserved, incrementally committed stacks: stable virtual addresses can reduce initial resident cost, but later commitment can fail. Reserving address space or successful mmap does not guarantee physical memory on Linux with overcommit. Signal-driven commitment does not automatically yield a typed, recoverable allocation error or safe cleanup. Platforms also have different reservation/commit contracts.
  • Movable contiguous stacks: compiler-generated pointer maps and provenance rules must account for interior pointers, registers, channel wait records, callbacks and foreign pointers. Stable heap allocation alone does not repair stack pointers. Such metadata can exist without tracing GC, but writing it is new compiler/runtime work. Restricting every borrow across every possible growth point largely sacrifices the benefit over restricted async.
  • Segmented stacks: old addresses can stay stable, but calls/returns need an alternate ABI, segment metadata, foreign-call bridges and reclamation rules. Segment growth still allocates inside an ordinary call chain.

A recoverable stack-growth failure cannot simply appear as AllocError in an arbitrary function declared -> i64. Choices are to propagate a new effect and error through all possible growth points, reserve a compiler-proven complete budget, restrict call graphs/recursion, or retain fatal exhaustion. Each changes the source contract or implementation substantially. This proposal provisionally uses fallible explicit admission with fatal unchecked stack exhaustion, matching today’s native-worker exhaustion boundary. If the requirement is that all later stack capacity failures must be recoverable, this candidate is incomplete and should not be adopted on the present evidence.

Allocation ledger

Fallible allocation means all explicit recoverable resource requests are reported; it is not a promise that OS process termination can be recovered. The candidate must advertise the fixed-stack limitation directly.

RequestFailure ownership and recovery
Scope task table, ready queue and wait recordsFallible construction; no live children on failed construction; preallocate bounded bookkeeping
Task record, stack reservation and native context setupReturn complete unstarted job; unwind partial setup before admission; no body execution
Slot capacity exhaustedReturn Full(job) without allocating, distinct from resource failure
Bounded channel buffer and readiness registrationsExplicit fallible construction/registration, rollback on failure; no hidden allocation on park/wake/cancel
Task body allocationOrdinary typed Result; return through normal initialized-resource cleanup
Result storageReserve with admission or keep result in task cell; preserve concrete size/type and exactly-once transfer/drop
Optional blocking worker submissionExplicit bounded fallible handoff, preserving job/inputs on failure
Fixed stack exhausted / OS kills process under memory pressureFatal boundary; no claim of a returned job error or reliable cleanup

The spike accounts for queue, task record, stack mapping and body allocations. It does not implement real readiness registration, result objects, OS worker creation or operating-system physical-memory admission guarantees.

I/O, foreign calls and fairness

Today’s std.ready waits use a bounded native poll; today’s channel/thread/executor waits block native threads. Running these unchanged inside one cooperative worker would block every task on that worker. The native operations are not automatically task-aware because ordinary syntax looks similar.

A candidate task backend needs nonblocking handle operations, owned readiness registrations and cancellation that removes those registrations before the buffer/handle can drop. Borrowed stack buffers can remain stable while parked, but late completion must never access them after return or cancellation. A kernel operation that cannot be synchronously cancelled needs a separate owned operation/result lifetime or explicit completion; stackful syntax does not solve that problem. Partial read/write progress and typed transport errors must survive.

Initially reject blocking OS/FFI calls in task bodies through effect checks, or provide an explicit bounded native-worker offload with owned transferable data. An offload must be joined/completed before freeing captured resources; requesting cancellation does not terminate its foreign call. DNS, file I/O, TLS and user destructors need separate audits. Native thread TLS, errno, signal masks, stack bounds and thread-affine native handles need defined treatment before any migration; a scheduler cannot assume that saving CPU registers solves them.

Cooperative scheduling offers no preemption of CPU loops, arbitrary FFI or destructors. Task-aware channel waits and explicit checkpoints provide known suspension points; inference makes their reachability visible to the compiler. Document fairness and bounded work per turn. A later preemptive scheduler must define safe points, stack scanning and lock/runtime critical sections; forced cancellation still must not interrupt cleanup. Debuggers, profiler unwinding, sanitizers, guard pages, ABI register state and supported targets also need qualification. Linux ucontext is only an experiment backend, not a portability plan.

GC-free lifetime and scope exit

Stack and task records have one owning scope; channels transfer owned values; reference-counted shared owners follow today’s cycle restrictions. Ready/wait queues contain non-owning task identities whose validity is protected by the scope. No task survives scope completion. Drop order is explicit: request optional cancellation, resume children until ordinary return, transfer/drop results and captures, detach wait records, then reclaim task stacks and the scope. Do not destroy a scope while its scheduler context is still referenced.

A child waiting forever or ignoring cancellation prevents scope exit. The runtime can detect some states with no ready task and no external registrations, but cannot prove general deadlock freedom or safely free live stacks as a fallback. Joining parents/children and bounded channel dependencies retain the existing analysis problems. Smaller surface syntax changes none of these lifetime facts.

Native stackful-task spike evidence

Companion to the Go-style task exploration. The isolated source lives in experiments/stackful-tasks/ at the repository root. It is a C experiment, not accepted Plenty syntax or production code.

What actually executes

Linux/glibc getcontext/makecontext/swapcontext run two task call stacks on one native thread. Each task owns a 64 KiB usable stack, surrounded by two inaccessible pages, plus a separate heap task record. Ordinary C helper calls perform channel waits and nested cooperative suspension. The scheduler returns to its own stack between turns. No native thread is created; there is no GC, async library, frame relocation, custom forced unwinding or language compiler.

The scope supports two lifetime task slots and one explicitly allocated channel slot. It has preallocated task identities, implicit ready order by table scan, and states for ready, send wait, receive wait and done. A send on a full channel or receive on an empty channel parks, and communication wakes the opposite waiters. This is a tiny test scheduler, with no slot reuse, endpoint closure, selection or general-purpose channel API. Messages are integers; affine-value transfer, failed-send ownership, endpoint closure races and linear obligations are not tested.

Admission allocates a task record, maps the guarded stack, makes its usable pages accessible and prepares its native context. The input pointer changes ownership only after those operations succeed. The failure injector covers queue allocation, task-record allocation and stack mapping; real mmap, mprotect and context-setup failures are checked, but only allocation sites are injected. Capacity failure allocates nothing and retains the job. Stack mapping failure releases partial task storage, and the original job can be retried.

Cancellation marks a task and makes parked tasks ready. Channel operations observe the request and return normally. The cleanup worker also yields from a nested helper holding a reference to a caller’s volatile stack local, mutates that local on resumption, and checks the result before exiting. It explicitly releases initialized heap resources in reverse order on success, cancellation and body allocation failure. Destruction refuses to reclaim any unfinished stack. These C cleanup paths model the candidate control flow; they do not implement Plenty destructors, its borrow checker, linearity or Result lowering.

Reproduce

From the task checkout, with a C11 compiler and Linux/glibc:

python3 -B experiments/stackful-tasks/test_spike.py
CC=clang SANITIZE=1 python3 -B experiments/stackful-tasks/test_spike.py

The runner creates a temporary executable, compiles with -O2 -Wall -Wextra -Werror -pedantic, runs assertions with a timeout, and removes the executable. SANITIZE=1 adds UndefinedBehaviorSanitizer; it does not claim complete alternate-stack instrumentation or AddressSanitizer coverage. The bounded experiment is intended as a mechanically inspectable counterexample to needing movable frames for every suspension, not a benchmark.

Observations

Observed on x86-64 Linux/glibc:

CheckObserved result
1,000 values through capacity-one channelSum 500,500; 999 send parks and 999 receive parks
Reference to stack local across nested suspensionSame retained local is read/modified after resume
Task record / usable stack / full mapping1,016 bytes / 65,536 bytes / 73,728 bytes with 4 KiB pages
Queue / task-record / stack allocation failuresFailure returned; partial admission storage released; unstarted job retained and retried
Full task tableJob retained, allocation attempt count unchanged
Cancellation before body entryCaptured job dropped by scope; body never starts
Cancellation of parked sender and receiverTask resumes and returns cancellation before scope reclamation
Success / cancel / either body allocation failureInitialized resources released once in reverse order; tracked live allocations and mappings return to zero
No ready taskStep reports no progress; explicit cancellation wakes test waiters

Two 64 KiB stacks are enough for these tiny call chains. This is not a safe general default. At 10,000 tasks the demonstrated mapping size alone would reserve about 703 MiB of virtual address space, excluding task records, allocator metadata and other storage. This is arithmetic from the observed mapping size, not a scalability run or resident-memory measurement. Successful mapping does not guarantee later physical-memory availability. The guards are installed, but no overflow-recovery experiment is claimed; overflowing or skipping guards is outside the safe experiment contract.

The finite no-yield example returns without scheduling a peer; it exercises no preemption mechanism. There is no real blocking-I/O demo, readiness backend, fairness/latency measurement, syscall handoff, FFI migration, or scheduler performance comparison. The source’s explicit swap points already establish that a blocked native call cannot yield by magic.

What this changes in the design discussion

Actual stack retention makes an ordinary borrowed helper chain possible across suspension without a public pinning type. Synchronous cancellation needs resumption and return; stack retention does not provide safe arbitrary stack destruction. The experiment consequently supports a narrow scoped library candidate, while leaving the most consequential deployment questions open: stack budgets and exhaustion, real I/O lifetime, transitive suspension checks, thread eligibility, bounded cancellation and tooling portability.

The sibling async experiment avoids whole-stack reservations and demonstrates moving restricted owned frames, while prohibiting surviving references. This experiment retains references by making the stack immovable and paying for it. The experiments expose different costs; they do not establish a winner.

Runtime-independent networking protocols

Status: Proposed; not implemented as a general convention. The three-layer direction is agreed: codec, sans-I/O session, runtime driver. This proposal specifies the intended contracts, not new APIs available today. Existing TLS sessions and the resumable HTTP decoder demonstrate parts of the design; they do not imply that all current networking modules already use it. Remaining scope and priorities belong to B94 and B85 in the backlog.

Companion pages describe the session contract, runtime drivers, and WebSocket validation and acceptance. All interface sketches on these pages are hypothetical, not executable Plenty or final public declarations.

Problem and decision

Plenty supports both sequential networking and supervised actor networking. A protocol implementation should not need separate copies of its framing, validation, exchange state, close rules and retransmission logic for each. Generic socket wrappers help share code across TCP and TLS, but do not remove the assumption that a protocol operation can wait on its caller’s stack.

Adopt three layers for new stateful networking protocols:

  1. Codec: parse and serialize protocol representations incrementally.
  2. Session: own protocol state and advance it from explicit inputs without performing I/O or waiting.
  3. Driver: connect a session to a runtime, transport and application API.

Share the protocol semantics, not one universal execution model. Sequential callers retain convenient operations such as receiving a message; actor callers receive events and submit commands without blocking the stage. Both drive the same session implementation.

Existing foundations and boundaries

Existing componentWhat it establishesWhat it does not establish
std.tls.SessionSocket-free TLS state, explicit ciphertext input/output, plaintext operations and EOFA universal protocol interface or actor-independent configuration ownership
std.http.decoderIncremental request framing, consumed-byte counts, explicit EOF and bounded feedsA complete HTTP client/server session including response sequencing and lifecycle
std.actor.netBounded socket buffers, readiness, timers, nonblocking connection setupApplication-protocol grammar or session rules
std.http.websocketImplemented RFC 6455 behavior and a sequential API over generic transportsAn actor WebSocket adapter or a public socket-free WebSocket session

The actor HTTP server uses the resumable decoder. The worker HTTP server still uses stream-oriented request reading; sharing parsing rules is not proof that both servers share all protocol orchestration. Existing qualification remains necessary during any migration.

Responsibilities

Codec

A codec recognizes or produces wire representations. It handles such matters as lengths, field encodings, frame headers, masking and incremental UTF-8 validation where appropriate. It accepts partial input and preserves enough state to resume; it must not require a complete peer-controlled frame to be buffered before checking its advertised size.

A codec does not decide when a socket should be read, when an actor should run, or whether an application may send a message in the current session state. Keep codecs private unless a real independent consumer needs their API. Three architectural layers do not require three public modules.

Session

A session enforces protocol semantics across codec operations: legal message sequences, negotiated settings, fragmentation, acknowledgements, control responses, closure and protocol timers. It accepts application commands, received data, explicit EOF and supplied time. It produces application events, wire output, timer requirements and terminal status.

No session owns a socket, selector, actor context, thread, cancellation token or runtime task. It neither waits nor obtains the current time. It can allocate within documented limits; sans-I/O does not mean allocation-free or immutable. State transitions happen through bounded, nonblocking calls.

Driver

A driver owns transport progress, readiness registration, waiting, cancellation, application delivery and runtime shutdown. It translates network EOF into session EOF and runs protocol timers when requested. It retains partially sent output and resumes it without loss or duplication.

Connection establishment, DNS and TLS configuration are driver/composition concerns. Protocol-specific decisions such as an acknowledgement timeout or WebSocket close response belong in the session. Application idle policies, server admission and a caller’s operation deadline belong outside it.

Architectural dependency rules

The permitted direction is driver → session → codec. Codec/session code may use ordinary data structures, byte operations and bounded computation, but must not import transport, actor or blocking-wait facilities. Review actual calls as well as imports: a callback or helper that secretly reads a socket still violates the boundary.

Protocol layers compose through bytes and explicit state, not inheritance. For example, a WebSocket session can sit above a TLS session, with a driver moving plaintext and ciphertext between their bounded buffers. A simpler blocking driver can instead use the existing TLS transport. This does not require changing every layer to the same representation.

Do not add a universal Protocol, global event enum, runtime registry or standard protocol executor initially. TCP byte streams and UDP datagrams have different contracts. Multiplexed protocols may need stream identifiers, connection-wide flow control and endpoint-level state. Share small helpers only after independent drivers demonstrate identical mechanics.

Public API direction

Keep existing convenience modules and behavior while extracting their protocol logic. A proposed WebSocket session could live under std.http.websocket.session; an actor adapter could live under std.actor.http.websocket. These are candidate paths, not reserved names. The codec may remain an implementation detail of the session module.

The session API should be useful to custom event loops and tests, not a public hook exposed solely so another std module can compile. Its module reference must document progress, ownership, allocation, errors and limits. Check exact signatures against stdlib conventions before implementation; regenerate the signature index only when public declarations actually change.

An initial API can favor owned events and copying into caller-owned output buffers. This is easier to specify under current borrowing and actor rules than retained views or callbacks into a mutably borrowed session. A zero-copy API is not a prerequisite; copies and memory costs must still be measured.

Implementation policy

Implement small, straightforward protocol machinery in checked Plenty. Use an established native library where substantial complexity or security expertise justifies it, preferably through its sans-I/O boundary. TLS remains backed by its existing security implementation; this proposal does not call for rewriting TLS, certificate validation, cryptographic primitives or QUIC in Plenty.

A native wrapper must retain input/output bounds, allocator provenance, error causes and actor cleanup guarantees. A library that performs hidden I/O is not made sans-I/O by placing a different name around it. A library whose unbounded allocation or global state cannot meet Plenty’s contracts requires a separate justification or must not be adopted.

Alternatives

AlternativeUseful forWhy it is not the general answer
Generic Reader/Writer or transportSharing a sequential driver across TCP, TLS and memoryA blocked read or WouldBlock does not preserve an actor callback’s execution position
Shared codecs onlySimple framing and stateless request formatsStateful validation and automatic responses can still diverge between drivers
One dedicated protocol actor behind every APIActor-native servicesSequential callers acquire an implicit runtime, mailbox and failure model
Worker offload of blocking implementationsExisting libraries without incremental interfacesWaiting still consumes a worker; it is compatibility, not shared nonblocking protocol machinery
Coroutines or effectsPotential future sequential driver ergonomicsRequires suspension/ownership/cancellation machinery and does not remove backpressure requirements
One universal protocol interfacePotential common driver helpersPrematurely forces stream, datagram, multiplexing and external-service differences into one shape

The blocking-work proposal remains complementary. No async/await, stackful-task, effect, stored-reference or linearity feature is required for this design.

Scope and consequences

Use WebSocket to validate a complete shared session and both drivers. This is not approval to rewrite HTTP, introduce every common protocol, or change the actor scheduler. Existing modules migrate only with retained behavior and protocol-specific evidence. A future DNS, MQTT, SMTP or QUIC design should use the same separation while choosing its own session shape.

Benefits are a single location for protocol fixes, deterministic state-machine tests, reusable custom-runtime integration and independent runtime evolution. Costs are explicit state, more detailed progress contracts, bounded queues and some runtime-specific driver code. Shared protocol code does not guarantee equal throughput, equal memory use or identical application APIs.

Design references

  • Sans-I/O guidance: separation of network protocol behavior from I/O mechanisms.
  • h11: HTTP/1.1 as an I/O-independent event/state machine.
  • wsproto: a socket-free WebSocket implementation as a design precedent, not a dependency or proposed port.
  • Quinn protocol connections: explicit datagram, transmit and timer integration for a more complex protocol family.

These references inform the architecture; this documentation change imports no external implementation or data.

Session progress, ownership and resource contract

Companion to Runtime-independent networking protocols. Status: Proposed contract, not implemented APIs. The notation below describes operations, not final Plenty declarations. Priorities remain in the backlog.

Inputs and outputs

A stream session needs these conceptual operations. A concrete protocol may combine them when doing so leaves every progress and ownership rule explicit.

OperationInputObservable result
ConstructRole, configuration, limits and initial protocol dataValid session or a named configuration/allocation failure
FeedRead-only byte viewConsumed-byte count, optional event and reason advancement stopped
SubmitApplication commandAccepted command, partial byte acceptance, or refusal preserving ownership
Take outputCaller-owned mutable byte buffer and rangeCount of wire bytes transferred out of the session
Advance timeSupplied monotonic instantBounded timer transitions, possibly producing events/output
Finish inputExplicit stream EOFEnd-of-input transition or truncation failure
InspectNo external effectsPending work, next protocol deadline and lifecycle state

Read-only bytes use &[u8]; owned buffers use Bytes; caller output uses &mut Bytes and a half-open start, stop range. Counts and capacities use i64. Time uses Instant, Duration and Deadline, never integer milliseconds. No nonwaiting session operation takes a cancellation token merely to resemble a blocking API.

An event-oriented session may offer a separate event-draining operation rather than return one event from feed. Either arrangement must bound the number and size of events retained internally. Returning an unbounded list of all events from one input is not acceptable.

Progress must be unambiguous

A feed reports a count in 0..source.len(). The caller advances by precisely that count and retains the remainder unchanged. A short count does not imply EOF, and a call can consume bytes without producing an application event. An empty feed may expose already-pending work but never announces EOF.

Distinguish at least these stop conditions, whether through a result enum or separate documented queries:

  • Needs input: no internal work can advance without more network data.
  • Needs output space: pending output must be transferred before advancement.
  • Needs event delivery: the application must take a pending event.
  • Needs external result: a named operation is awaiting a supplied result.
  • Budget exhausted: work remains runnable; arrange a continuation.
  • Terminal: no future protocol work is possible.

A pending event can be progress even when the consumed count is zero. Conversely, 0 consumed with no event and no change in pending work is not permission to spin. A driver must wait for the identified dependency or yield after exhausting its budget. If multiple dependencies exist, report enough information to choose interests without repeatedly probing unchanged state.

Do not automatically reset a completed exchange if bytes for another exchange remain in the source. A protocol must specify when the application can admit the next exchange; HTTP pipelining is an example where this affects response order.

Input ownership and persistent state

feed borrows only for that call. Before returning, the session either consumes bytes into owned, bounded state or leaves them for the caller. It never retains a pointer into a caller buffer. Header parsing must not keep a view alive while a driver compacts, grows or reuses that buffer.

Initially favor owned byte events. An event moved out belongs to its recipient; the session cannot mutate it later. Ownership does not mean unrestricted actor storage: an owned event containing shared str or another actor-ineligible type may be usable only during a callback. The concrete reference must distinguish transient events from persistable events and explain any conversion required.

Construct session buffers in the connection actor’s heap, not outside it and then move them into a child. Immutable configuration, foreign handles and secret material need the same provenance review as actor TLS. Sans-I/O is not an exemption from actor heap rules.

Command admission and retry safety

An application command is not accepted until the session reserves the resources needed to retain it or has made the documented partial progress. Backpressure is normal flow control, not a malformed-peer error.

Prefer borrowing command payloads when copying/encoding during the call. For a partial byte-oriented submission, return the accepted count; the caller retains the unsent suffix. For an ownership-taking command that can be refused before use, return the original command with the refusal cause, following std’s returned-input convention. Do not silently discard a moved message because the output queue is full.

Admission must have one specified outcome on allocation failure: either the command is entirely unaccepted, or an explicit progress value identifies the accepted prefix. Never return a bare failure after accepting an unknown amount and invite the caller to retry the entire message. Session mutation that can fail midway needs transactional preparation or a documented terminal failure.

For complete-message operations, accepting means local ownership/encoding responsibility, not successful peer delivery. Drivers must not invent delivery acknowledgements from successful socket writes.

Wire output and partial writes

Separate output transfer from transport delivery:

  1. take copies/moves a bounded prefix into a driver-owned buffer and removes exactly that prefix from the session’s pending output.
  2. The driver owns those bytes until the transport accepts them or the connection terminates. A short socket write advances only its send cursor.
  3. The driver drains that cursor before sending later bytes on the same stream.

A temporary unwritable socket does not permit calling take again into the same storage and overwriting unsent data. A future borrowed-output/commit API could avoid a copy but needs its own borrow and failure contract; it is not required here.

A session cannot infer that the peer has received a close frame just because take returned its bytes. Where protocol state needs to distinguish queued output from output accepted by the transport, the driver must provide an explicit ordered flush milestone. No public API may claim graceful closure solely because the session’s internal output queue is empty.

Time and external dependencies

Supply monotonic now explicitly for time-sensitive transitions. The session returns an optional next protocol deadline; a timer notification means “advance using this instant”, not “assume the old deadline is still current”. Recompute the next deadline after every state change. Stale timer wakes must be harmless, and decreasing instants must produce a named invalid-time error rather than move protocol time backwards. Tests use supplied fake time.

Caller operation deadlines and cancellation remain in the driver. A keepalive, retransmission or close-handshake timeout belongs in the session; a server-wide drain deadline can force termination earlier without pretending the protocol completed gracefully.

Do not obtain entropy, resolve names, read credentials or invoke arbitrary application code inside a session transition. Supply data up front or request an explicit bounded external result. A request carries a checked identity so a late or duplicate result cannot satisfy a different operation. The requesting session specifies whether it can do other work while waiting. External failure preserves its cause and follows a documented retry/terminal policy.

Security-sensitive randomness must still come from the existing secure entropy facility. Supplying entropy explicitly makes tests reproducible; it does not justify a predictable seed or random stream in production.

Resource bounds and fairness

Each session documents separate caps for:

  • incomplete headers, frame payloads and assembled messages;
  • queued application commands and undelivered events, in bytes and count;
  • protocol output, including automatic control traffic;
  • codec scratch space and retained metadata;
  • active exchanges, streams or external requests where applicable.

The total bound includes codec state, session queues, driver buffers and transport/TLS buffers. Moving bytes between layers must not double-count them as free capacity or hide multiple full-sized copies. Document peak memory, not just the session’s most visible buffer. Validate configurations before use; never silently clamp a bad size or overflow a combined-capacity calculation.

Check advertised lengths before allocating or converting to a machine index. Queue saturation pauses progress; an oversized peer object produces a named limit error. Allocation failures are preserved in the module’s domain error as Allocation(AllocError).

A bounded input byte count is insufficient by itself. Bound parser scanning, event emission, internal transitions and output production per call; bound the number of calls and bytes copied per actor turn. Work left after a budget requires a scheduled continuation, not a wait for network readiness that might never recur. Limits must also cover many tiny control frames and fragmented messages, not just large payloads.

Reserve bounded space for mandatory control output where the protocol requires it. Define what happens if that reserve is exhausted: stop input, reject new application output or terminate according to the protocol. Never let a full application queue cause unbounded automatic replies or deadlock a required close response. For stream protocols, preserve wire order: a control response cannot be inserted into a frame already partly emitted.

EOF, errors and final states

Distinguish no bytes available, read-side EOF, orderly protocol closure, protocol failure, transport failure and administrative abort. A transport half-close does not automatically close the opposite direction; the protocol determines whether remaining output is useful. Dropping state performs resource cleanup only and must not wait to send a graceful close.

A session’s native error describes protocol/configuration/allocation/external input failures, not fictitious network failures. The driver preserves actual transport causes in its own error mapping. Keep existing public error variants where compatibility requires them; a new pure-session API need not inherit network variants that it cannot produce.

After a fatal parse error, do not resume from an untrusted frame boundary. A session may enter a bounded failing state to emit a protocol alert/close; the original error remains available while that output drains. It then becomes terminal. Terminal means no further output, event or timeout can advance it; failed-with-output-pending is not terminal. Document which inspection/draining operations remain permitted after failure and reject other operations with a named state error.

Blocking and actor protocol drivers

Companion to Runtime-independent networking protocols. Status: Proposed architecture, not implemented adapters. Drivers follow the session contract. Work and priority live in the backlog.

Shared mechanics, different execution

Both drivers must preserve the same input/output order and protocol state. They need not expose identical application methods or share all control flow. A sequential receive can wait; an actor receive reports an event and returns. Connection establishment, transport ownership and application dispatch belong in the driver, while frame validity and legal state transitions stay in the session.

Useful mechanics to extract after both implementations exist include cursor management, bounded buffer transfer and a single bounded advancement step. Do not hide waiting, actor contexts or runtime callbacks in those helpers. Keep protocol-specific decisions out of both drivers even if copying them would be shorter than extracting the correct session operation.

Blocking driver

A sequential operation conceptually does the following:

check caller cancellation and deadline before admitting a new command
submit the command, or remember its explicitly unaccepted suffix
repeat within a bounded advancement batch:
    deliver pending events relevant to the operation
    move session output into a bounded driver send buffer
    write pending transport output and retain any unsent suffix
    feed buffered input and retain any unconsumed suffix
    advance protocol time using supplied monotonic now
    retry admissible command progress
if operation is not complete:
    wait for useful transport readiness or the earliest relevant deadline
    perform bounded reads/writes, then resume advancement

This is an ordering sketch, not an instruction to drain every queue before checking another. Balance both directions: bidirectional protocols can deadlock if a driver insists on completing a large write before processing a peer’s control input. Conversely, do not read beyond bounded capacity merely because input is ready.

Use existing deadline-aware networking and cancellation mechanisms rather than polling or adding a hidden event runtime. The wait ends at the earliest of the caller’s deadline, the next protocol timer, and any driver shutdown deadline. If protocol progress requires simultaneous interests, use the existing readiness facilities rather than serial full-duration read and write waits. A generic blocking transport remains useful inside this driver where its capabilities actually suffice.

An operation completes according to its documented local semantics: command admitted, bytes flushed, message received or close handshake finished. These are different milestones. Output remaining after a receive operation must have a specified owner and flush policy; automatic pongs cannot remain queued forever because the application stopped calling receive.

Do not silently start a background thread to solve that problem. State whether the API makes progress only during calls, offers a drive/flush operation, or requires an explicit service loop for autonomous keepalive behavior. Preserve existing sequential API guarantees during extraction.

Cancellation and retry

Before admission, pre-cancellation leaves the session unchanged. Once any part of an operation has affected the wire, a timeout/cancellation cannot generally be retried as a fresh operation. The driver either exposes explicit resumable progress or terminates the connection according to its existing contract.

For WebSocket, preserve the current rule that cancellation mid-frame closes the connection rather than inviting an unsafe resend. Retaining internal state makes correct resumption possible in principle; it does not change a public method’s cancellation promise without a separate compatibility decision.

An abort discards remaining output and releases the transport. A graceful close is a separate bounded operation driven until its completion condition or a specified deadline. Neither destructor cleanup nor actor trap cleanup may wait for it.

Actor driver

A connection actor constructs its session and buffers inside its own heap. It owns the transport adapter, session, input cursor, output cursor and protocol timer state. It reacts to:

TriggerDriver action
Network readinessRun bounded transport progress, then bounded session advancement
Application message/commandAttempt admission without waiting and return an explicit result
Protocol timerAdvance with the current instant, ignore stale schedules, recompute deadline
ContinuationResume already-runnable bounded work without requiring new socket readiness
Application capacity restoredResume event delivery and any paused session/input progress
Stop/drain requestReject new work, drive bounded protocol closure, then force abort at the drain deadline

Readiness is evidence that an attempt may progress, not a promise that it will. WouldBlock is normal. Enable writable interest only while transport output actually waits; permanently watching a writable socket can spin the stage. When input/event capacity is exhausted, pause the appropriate intake and retain unconsumed input. Resume explicitly when capacity returns.

If a turn budget expires while buffered input or internal transitions remain, arrange a continuation through a supported actor wake mechanism. Do not wait for another readiness event: the data may already have left the kernel. Bound continuation scheduling so one connection does not fill its own mailbox with redundant wakeups. Timer/admission failure is a reported failure, not permission to drop a required wake.

Handlers run to completion. The actor driver must not call blocking DNS, HTTP, database, entropy acquisition or user callbacks with unspecified blocking behavior from its protocol advancement path. Use existing nonblocking facilities or an explicitly supported external-service integration; do not assume the proposed blocking-work pool already exists.

An actor’s bounded mailbox is not an automatic delivery guarantee. If command submission or event forwarding encounters a full mailbox, preserve ownership and report backpressure. Avoid forwarding every protocol event through another mailbox when direct bounded delivery to the owning actor is sufficient.

Layering over TLS

The straightforward choices are:

  • Blocking WebSocket → existing TLS transport → TCP.
  • Actor WebSocket → actor TLS → actor TCP.
  • Custom runtime → WebSocket session → TLS session → caller-owned socket.

Every intermediate layer can retain partial input/output. Pump the composed layers until an external dependency or work budget stops progress. Each layer must identify pending work without assuming that lack of new socket input means lack of plaintext or protocol events. Include plaintext, ciphertext and protocol buffers in the connection’s peak-memory accounting.

A TLS close notification is not a WebSocket close frame. Preserve the ordering of WebSocket close, protocol flush, TLS close and socket release, with a bounded fallback for peers that stop responding. TLS alerts and protocol close output may still need draining after an error, but the driver must retain the original cause and never wait indefinitely to deliver them.

Datagram and multiplexed protocols

Do not adapt UDP by concatenating packets into a stream. A datagram session receives one packet with source/destination metadata and produces packet output with its destination and relevant metadata. Define atomic packet admission, maximum datagram size, truncation handling and endpoint demultiplexing.

An endpoint may own routing state shared across several sessions. Multiplexed sessions can require per-stream credit, connection credit and reset events. Retransmission/congestion timers belong in that protocol implementation, not a generic driver. These distinctions are reasons to reuse the architecture rather than impose a universal session interface.

Platform and failure claims

A socket-free session can be portable without making its driver portable. Keep the existing actor platform limits explicit until native qualification proves otherwise. Do not broaden supported targets based only on session tests.

Returned errors, contained actor traps and process-fatal runtime failures remain different outcomes. Session extraction does not strengthen the actor sandbox. Test resource recovery after actor heap exhaustion and ensure native session state owns nothing in a discarded or unrelated heap. No hidden global protocol service should turn a connection-local failure into shared corrupted state.

WebSocket validation and acceptance

Companion to Runtime-independent networking protocols. Status: Proposed validation scope, not implemented session or actor support. This page defines the evidence needed to validate the architecture. The backlog, B94 with B85, owns implementation sequencing and priorities; the cases below are acceptance criteria, not a separate work queue.

Why WebSocket

WebSocket is a useful first complete session because it combines incremental framing with bidirectional commands, fragmentation, control responses, text validation and a closing handshake. It exercises more than a passive decoder without introducing QUIC’s transport/security complexity. Plenty already has an implemented sequential WebSocket API, which supplies a behavioral baseline and a migration obligation.

The initial scope is existing RFC 6455 client/server behavior over TCP and TLS, plus an actor integration. It excludes permessage-deflate, proxy tunnels, HTTP/2 extended CONNECT, new scheduling models and automatic migration of all HTTP machinery. No additional wire features are required to validate the three-layer design.

Codec responsibilities

The incremental codec handles frame headers, extended lengths, mask position across input slices and payload encoding/decoding. Validate reserved bits, opcodes, canonical length encodings and role-specific masking before accepting an invalid frame. Check frame and message bounds before allocating based on advertised lengths. Control frames remain at most 125 bytes and cannot be fragmented.

The codec can emit bounded payload chunks rather than assemble a complete message. Message limits are enforced across all fragments, including when the application consumes chunks and no single retained buffer is large. UTF-8 validation carries state across network fragments and continuation frames; validating each received byte slice independently is incorrect.

Session responsibilities

The session has explicit client/server roles and state for fragmentation, message size/text validation, automatic pong generation and closure. It rejects continuations without an open fragmented message, overlapping data messages, and data sends that are no longer legal after closure starts.

A conceptual input/output vocabulary is:

Inputs: received bytes, send text/binary, ping, begin close,
        supplied entropy result, input EOF, supplied time, output flush milestone
Outputs: message chunks and boundaries, ping/pong notices, peer close,
         failure/termination status, wire bytes, entropy request, next deadline

This is a contract sketch, not a commitment to these exact events. The blocking adapter can assemble chunks into its existing complete-message API. Actor applications should be able to stream within bounds instead of requiring a whole maximum-sized message in their heap. Do not expose a chunked text event as independently valid str when a UTF-8 scalar can span chunks; use bytes and message-level validation semantics.

Specify when incoming pings/pongs are observable, which replies are automatic, and what happens when the application is not draining events. A bounded control reserve must allow required replies without unbounded allocation. If continuing input would exceed that reserve, stop consuming input or terminate explicitly. No driver implements its own ping/pong or fragmentation policy.

Entropy without hidden I/O

Clients need unpredictable handshake nonces and fresh unpredictable frame masks. This must not become a hidden entropy read in the codec/session or a blocking call on the actor stage. The driver/external-service boundary supplies securely obtained nonce/mask material through a narrowly documented operation.

For each accepted outbound client frame, bind a fresh mask to that frame before its header or payload can be emitted. If entropy is unavailable, leave the command unaccepted or retain it in an explicitly bounded pending state. Retrying must not duplicate a frame, reuse a previous frame’s mask or consume unrelated entropy results. Bound pending requests and reject late/duplicate identities. Test providers may supply fixed material; production providers must use Plenty’s existing secure entropy facility.

Before shipping actor client support, demonstrate an actual nonblocking or explicitly off-stage entropy path. This proposal does not claim that the current entropy API already supplies one. If that requires runtime work, record and review the concrete capability separately; do not weaken the sans-I/O boundary or silently rely on the unimplemented general blocking-work pool.

HTTP upgrade boundary

Opening-handshake validation and generation are protocol logic. Socket reading, HTTP exchange ownership and actor dispatch are driver concerns. Reuse the same handshake validators/builders from both drivers; they may be helpers separate from the post-upgrade frame session, provided no protocol rules are duplicated.

Retain current checks for method/version/body, upgrade tokens, key/version, accept value, subprotocol and unsupported extensions. Preserve existing server refusal status and client error behavior unless a separate change explicitly justifies it. Do not accept unsupported extensions merely because their frames look superficially valid.

Upgrade transfers ownership of the transport and every already-read byte beyond the HTTP head. Feed that suffix exactly once into the WebSocket session. It may contain several frames or a partial frame, including when the HTTP response and first frame arrive in one read. HTTP must not continue parsing bytes after a successful transfer.

For actor HTTP, define a supported ownership/lifecycle handoff before exposing upgrades. The current actor server lists upgrades as remaining work; constructing a frame session alone does not implement that integration. Specify how watches, timers, shutdown ownership and connection accounting transfer, including failure between sending 101 and completing the handoff.

Closing and compatibility

Track local close queued/flushed, peer close received, transport EOF and timeout as distinct facts. Echo/response behavior, close-code validation and reason validation must follow the existing contract and RFC. A forced abort is not a successful close handshake. Simultaneous close must not generate endless responses, and a close timeout must not discard the original failure cause.

Keep std.http.websocket and std.http.wss source-compatible where practical. Preserve bounded frame/message behavior, refusal/rejection distinctions, cancellation behavior, server stop tokens, worker ownership and conditional TLS linking. Current whole-message buffering or exclusive borrowing may remain in the convenience driver even if the session supports finer-grained progress. Do not use extraction as an excuse to change these promises silently.

Reference/tutorial updates ship with the implementation: explain choosing the blocking or actor driver, show recommended error handling and shutdown, and include runnable examples for both. The session reference must explain custom runtime integration, progress and memory bounds. The proposal itself is not a substitute for those implemented-behavior pages.

Deterministic session evidence

Run the same wire corpus without sockets, threads, real clocks or production entropy. Include:

  • Every split point for representative headers, extended lengths, masks, control payloads, UTF-8 sequences and close frames; one-byte and coalesced feeds.
  • Fragmented text/binary messages with interleaved controls, invalid opcodes, bad masks, invalid UTF-8, invalid lengths and limits exceeded across fragments.
  • Empty feeds, EOF at each incomplete boundary, repeated inspection after terminal states and rejected operations after failure/close.
  • Zero/small output capacity, partial takes, partial command admission, saturated event/output queues and exactly-once retry behavior.
  • Stale/duplicate external results, unavailable entropy, stale timer wakes, decreasing time, simultaneous close and a peer that never completes close.
  • Allocation failure at each fallible growth/admission point, preserving ownership and reporting known progress or terminal failure.

Property tests vary byte segmentation, output capacity and legal pump scheduling while preserving input order. Equivalent schedules must have equivalent message and terminal semantics; chunk boundaries and batching may differ. Assert that reported consumption stays in range, queue bounds hold, output order is stable, and no no-progress state spins. Replayable traces contain inputs, commands, supplied time and test entropy, not secrets from production sessions.

Fuzz codec/session transitions and external-result ordering. Treat an unexpected trap, hang, bound violation or consumption inconsistency as a finding. Mutating only complete frames misses incremental-state defects. Independent RFC vectors and another implementation are needed as checks: both drivers can agree because they share the same bug.

Driver and composition evidence

Use one scripted peer corpus against both drivers and compare normalized application events, wire validity and terminal outcomes. Cover real TCP/TLS, partial writes, slow readers, read/write interdependence, EOF/half-close, cancellation, close deadlines and both upgrade directions. Include HTTP heads coalesced with WebSocket bytes and TLS records split at unrelated boundaries.

Actor tests must demonstrate bounded work per turn, progress without fresh network readiness after a budget yield, output/event backpressure recovery, correct timer rearming, full-mailbox handling, graceful/forced shutdown and unrelated-connection progress under load. Repeat handler traps and heap exhaustion; compare descriptor, live-heap and native-allocation baselines after cleanup. Verify transport ownership is recovered or closed exactly once on every upgrade-construction failure.

Retain existing HTTP, WebSocket, WSS and actor networking/TLS regressions. Measure throughput, latency, CPU and peak memory with matched payloads, connection counts, limits, TLS configuration and CPU allocation. Include idle connections, large/fragmented messages and control-heavy traffic. Report measured copy and buffering costs; do not claim an improvement from architecture alone. Actor platform support requires real platform evidence, not just portable session tests.

Acceptance and boundaries

The convention is validated when one complete WebSocket implementation supplies both blocking and actor drivers, with protocol rules only in the shared codec/session/handshake helpers; all documented progress/resource contracts have tests; existing convenience behavior is retained or explicitly reviewed; and reference, signatures and executable tutorials match the implementation.

The first extraction can be reviewed independently of the actor handoff, but it must not be called complete cross-runtime support until actor upgrades, client entropy and lifecycle integration are demonstrated. Any unresolved gate remains visible in the backlog and PR evidence. No production-readiness, zero-copy or cross-platform claim follows automatically.

Protocol specification

  • RFC 6455: opening handshake, framing, masking, control frames, UTF-8 and closing rules. No RFC code or data is imported by this proposal.

An HTTP server on supervised actors

Status: Implementation and qualification in progress (B85). This proposal records the intended second server; it does not declare the actor server production-ready. The existing worker server remains supported independently. The backlog owns remaining work.

Two supported servers

std.http.server keeps sequential handlers, bounded workers and its existing qualification. std.actor.http.server uses one supervised actor per connection, with bounded heaps, mailboxes and socket buffers. A contained handler trap loses that connection while unrelated connections and admission remain available. Applications choose the execution and failure model that suits their work.

The actor server should make ordinary bounded request/response handlers simple. A separate event handler interface handles streaming input/output, timer wakes, actor messages and shutdown without blocking the stage. Request heads are owned transient callback inputs; persistent application state obeys the existing actor heap rules. Factory and connection initialization run in the child heap. No compiler heap-boundary relaxation is part of this design. The separate blocking-work proposal describes bounded background workers for existing blocking clients, without blocking callbacks.

Protocol and lifecycle

Share HTTP line, field, target and framing rules with std.http. A resumable byte-feed decoder retains no shared strings across actor callbacks and consumes a bounded amount of input per turn. Partial bodies and output resume on readiness; waiting does not occupy a native worker. Plain TCP and TLS use the existing actor network layers. Bounds cover admission, headers, bodies, output, actor heap use, requests per connection, idle time, handshake, request phases and writes.

Shutdown stops admission, finishes bounded active responses during a grace period, closes idle connections and then forcibly retires remaining connection actors. A stop signal must not bypass the drain phase. Refusals preserve the existing bounded lingering-close policy where possible.

The first supported implementation uses one stage/thread and the existing 1024-live-heap process ceiling. Handlers run to completion and cannot safely perform blocking operations or unbounded CPU work. Stack overflow and internal runtime panics remain process-fatal; actor containment is not a sandbox. Shared foreign state and TLS configuration ownership need explicit repeated-failure checks. HTTP/2, HTTP/3 and multi-thread stages are separate scope.

Qualification and comparison

Production readiness requires retained parser corpus parity (including fragmentation, malformed framing, pipelining and trailers), real TCP/TLS clients, slow heads/bodies/readers/handshakes, bounded admission, repeated handler traps and allocation exhaustion, descriptor/heap baselines, graceful shutdown under load, reverse proxies, fuzzing that treats an unexpected contained parser trap as a finding, and documented platform limits.

Run the worker and actor servers on the same machine, compiler revision, requests, response sizes, concurrency and CPU allocations. Publish both an equal-core comparison and each model’s normal configuration where useful. Record throughput, latency, memory, failures and failure isolation. Simplicity, speed and resilience differences are hypotheses to measure, not promises inferred from the concurrency model. The benchmark must stay below each implementation’s configured connection limit.

Actor budgets and retained stacks

Status: Exploratory; no production scheduling change. A test-only native probe validates stack retention with the existing actor heap router and trap recovery. It does not insert compiler checkpoints or drive std.actor.Stage. The actor reference still specifies handlers that run to completion. Remaining implementation and qualification scope belongs to B85 in the backlog. See the probe evidence.

Question and provisional decision

Could compiler-inserted budget checks let an actor yield inside ordinary nested calls, without changing application methods into generators?

The mechanics are plausible with immovable, guarded stacks on one scheduler thread. A checkpoint decrements the active actor’s approximate work budget. On exhaustion it saves the actor’s execution and runtime state, switches to the scheduler, and later continues after the check. Helper calls keep their normal calling convention. The runtime must separately preserve state currently stored per native thread, and the stage must stop lending the same exclusive Context to overlapping handler activations. These are substantive integration changes.

This is a narrower investigation than general stackful tasks: actor state already has an isolation boundary, mailboxes remain bounded, and a suspended callback remains the actor’s only active callback. The older C task spike proves nested stack retention; the new Rust probe exercises actual actor trap recovery and allocation routing. Closed local PRs #639/#642 explored socket-parking fibers before the implemented actor API; they did not validate budget checks or the current stage’s ownership contract. No unmerged source from those PRs is used here.

Scheduling and compiler boundary

Checks belong on executed loop cycles and recursive call cycles, including tail calls. A check only at the outer actor entry misses an unbounded helper loop. Calls through callbacks and generators also need coverage; inlining, loop specialization, SIMD lowering and cached object reuse must preserve that coverage. Existing lowering has distinct loop, tail-call and callback paths, so adding one check to one loop emitter would not establish fairness.

An initial budget can count approximate work between checks. It is not a time limit: a single large operation may have substantial cost. Long runtime operations need chunking, their own safe checkpoints or worker offload. Measure both uncontended check overhead and peer progress under expensive operations before choosing a charging policy. The probe’s quantum of 17 is only a deterministic test parameter.

One possible implementation shares compiled helpers and makes a checkpoint a no-op outside an actor, while honoring an explicit non-suspending region inside one. Actor-specialized helpers could reduce unrelated overhead but increase code and cache variants. Neither option has been implemented or measured. Every checkpoint must be visible to code generation as an ordinary call with the appropriate memory effects; switching stacks does not relax aliasing rules.

Do not switch inside allocation, heap-to-heap message copying, stage queue mutation, foreign-library arenas, or synchronous destruction. Compiled code invoked by a runtime helper can reach another checkpoint: excluding only Rust function bodies is insufficient. Transitive compiler effects or a runtime non-suspending depth, with audited entry/exit and trap restoration, must protect these regions. Deferral can itself be unbounded unless the protected operation has a bounded cost or an offload path.

Actor and scheduler ownership

Today Stage.deliver passes &mut self._nodes[index] and &mut self._context to __actor_run. Its adapter arguments and result storage are slots in the caller’s native stack. Context owns the shared mailboxes, ready queue, selector, timer table, and current actor/spawn/stop fields.

A suspended callback cannot retain that exclusive context while the scheduler and another callback access it. Saving _self alone would repair the visible identity but leave overlapping exclusive loans. A candidate replacement is one context capability per activation, holding actor-local identity and pending commands, with stage access restricted to short non-suspending operations. Its interior access needs an explicit compiler/runtime ownership contract. A raw pointer hidden in the library is not sufficient proof of soundness.

Node slots, callback arguments/results, and context storage must stay at stable addresses until the callback finishes or traps. Scheduler dispatch frames must return independently of suspended actors. Keep suspended callbacks runnable even with empty mailboxes; enqueue new mail without starting another callback for that actor. Run spawn hooks, settlement and restart policy at their defined lifecycle points. Do not restart an actor or destroy its stack while a suspended activation still references its state.

Runtime, failure and resources

The probe places plenty_actor_run and its recovery frame on the actor’s own stack. This lets the existing nonlocal trap path return within that actor after several suspensions. The scheduler’s stack is separate. Each transition restores the intended allocation heap, recovery pointer, caller source location, drop queue and last-trap state. Arena calls and active destruction are excluded from probe checkpoints. Other TLS, callback, lock and FFI boundaries still need an audit; the demonstrated snapshot is not an exhaustive runtime contract.

The actor owning the stack also owns the trap boundary. Normal completion and cooperative cancellation return through ordinary cleanup; a contained trap abandons frames without destructors, following existing actor semantics, then discards the heap. Resuming a trapped or completed context is forbidden. Scheduler deadlines and cancellation can be examined between quanta, but delivering Mail.Stop does not interrupt a callback already in progress. Cancellation, stage teardown and any forced-discard policy need a separate defined contract; stack switching does not make arbitrary stack abandonment safe.

The experiment uses fixed stacks with inaccessible guards. Stack admission must be fallible and bounded in a production implementation, with a separately visible stack-memory cost. Virtual reservation is not resident usage or a guarantee of later physical memory. Guard pages do not prove that oversized frames cannot skip guards. Existing stack-overflow reporting assumes the native stack and its alternate-signal-stack reservation; alternate stacks require explicit bounds and signal-handler qualification. Overflow remains process-fatal under the current contract. Recoverable stack overflow is not established here.

The probe uses glibc swapcontext, which preserves more state than a minimal ABI switch and may involve a system call for signal masks. It is a correctness vehicle, not the selected production switch. A custom switch needs callee-saved register and floating-point/control-state tests, stack alignment, platform hardening, debugger/unwinder and sanitizer integration. The experiment qualifies only its host x86-64 GNU/Linux configuration; AArch64 and other targets are not covered. Thread migration is outside this candidate.

Blocking native operations still require nonblocking actor adapters or bounded blocking-work offload. A checkpoint cannot interrupt a blocked syscall or arbitrary native computation. Where possible, compiler checks should reject known blocking operations from actor-reachable code, including indirect calls, with explicitly audited offload as the escape.

Assessment

Retained stacks avoid a generator transformation throughout the call graph and support actor-local borrows across nested suspension. The actual heap router and trap boundary can coexist with suspension when their state is restored. This is enough to justify a compiler/stage integration prototype, without adopting a production scheduler or claiming BEAM-equivalent fairness. The ownership of the shared stage context is the largest architectural question exposed by this probe. The backlog records the remaining gates.

Actor stack-switching probe evidence

Companion to Actor budgets and retained stacks. Status: Test-only native probe; compiler insertion and stage integration are not implemented. Source: experiments/actor-stacks/probe.rs, included inside the runtime’s actor heap module only in x86-64 GNU/Linux test builds.

What executes

Two callbacks run on separate fixed 64 KiB stacks, with an inaccessible page at each end. A small round-robin harness alternates them whenever an explicit checkpoint exhausts a budget of 17. Checkpoints model intended compiler output; no compiler pass inserts them. Each callback enters the actual plenty_actor_run, so its production recovery record lives on its own stack. The probe uses the real routing allocator, heap limits and unwrap trap helper.

The harness saves/restores allocation routing, recovery, caller-location and last-trap TLS, plus the production drop queue. It refuses checkpoints during arena calls or active destruction. The drop queue remains empty in these fixtures; nonempty queue switching is not tested. Caller-location identities are distinct and checked after resumption. Retained heap allocations are checked for ownership and contents, and an oversized allocation returns null within each actor’s limit. A borrowed stack local survives nested helper suspension.

The harness uses stable raw-pointer records rather than keeping an exclusive Rust reference to scheduler storage across a switch. It does not model a sound replacement for Plenty’s shared Context, compiler adapter storage, mailbox delivery, readiness, timers, supervision or restart. It cannot establish the borrow checker’s acceptance of a new context design.

Reproduce

From the task checkout:

cargo test --locked -p plenty-runtime actor_stack_switching_probe -- --ignored --nocapture --test-threads=1

The probe is ignored during ordinary suites because it owns the process-wide actor heap table; run it alone. An external timeout is appropriate for detecting switch failures. Its scheduler also asserts a finite maximum turn count. The default fixture has finite loops; it does not execute an infinite loop.

Checks and interpretation

CaseAssertion
Budget disabledPeer first runs after the first actor completes all 10,000 iterations
Budget enabledPeer first runs after 17 iterations; each actor completes 10,000 iterations with 588 yields
Trap after resumptionFirst actor traps after three yields, returns Unwrap, runs no local destructor; peer completes and runs its destructor once
Cooperative cancellationScheduler requests cancellation after eight dispatches; both callbacks resume, observe it and return through cleanup before completing their loops
Allocation routing and limitsSmall allocations belong to the running actor before/after switches; oversized allocation is refused
ReclamationEach case discards both heaps and unmaps both stacks; live heap count, used bytes and committed heap bytes return to zero

These are deterministic work/progress checks, not throughput or latency benchmarks. The cancellation flag is an experiment-only mechanism, not Mail.Stop or a new public cancellation API. The trap fixture deliberately leaves its allocation and destructor behind; existing actor heap discard reclaims that allocation without unwinding the stack. Runtime panics are not recovered.

Observed on x86-64 GNU/Linux on 2026-11-02: all four cases passed in debug, release and debug with allocation-checks. The budgeted pair performed 588 yields each; the trap case returned Unwrap at iteration 52 while its peer completed 10,000 iterations. Cancellation returned both callbacks at iteration 68 with one destructor each. The oversized-allocation check calls Router directly because an unused standard Rust allocation can be optimized away.

Validation also passed the ordinary runtime suite (187 passed, 5 ignored), workspace/all-target Clippy with warnings denied, workspace formatting and an explicit rustfmt check of the included probe. Three networking tests initially failed under sandbox restrictions; the runtime suite passed when rerun with local networking available. Documentation validation passed source links, 151 script tests (3 skipped), and the complete site using mdBook 0.5.4 (1,444 default and 722 development HTML pages). No compiler integration suite was run: production code generation and executable Plenty examples are unchanged.

No stack admission fault injection, guard-page overflow, real socket workload, platform register conformance, signal delivery, blocked-native-call latency or complete sanitizer coverage is established. The stackful runtime uses glibc contexts, so it does not benchmark the cost of a future minimal register switch. Only a compiler prototype can validate coverage of optimized loops, recursion, tail calls, callbacks, generators and cached objects.

Blocking work for actors

Status: Proposed; not implemented. This is a design for the blocking-work part of B85, not a new scheduling or thread API available today. Priorities and remaining scope live in the backlog. The companion pages cover ownership and failure and runtime integration and qualification.

Purpose and existing limits

An actor HTTP handler should be able to call an existing blocking HTTP client, database driver or file operation without stopping unrelated connections. The callback submits work, returns to the stage, and later receives a completion event. Waiting occupies a worker thread, not the stage thread.

Today actor callbacks run to completion on one thread. Ordinary thread creation inside a callback traps; scoped threads also join before scope exit. Actor nodes cannot retain ordinary thread, channel, executor or shared-state handles. Wrapping spawn or a future in a Plenty class cannot bridge these constraints safely. The existing actor DNS lookup already demonstrates helper-thread completion independent of an actor heap, but is a specialized operation, not a general offload API.

Prefer actor networking and actor TLS for protocols that already have nonblocking state machines. Offload is a compatibility path for blocking APIs; it is not a promise that more threads improve throughput.

Use an explicitly created, bounded OS-thread pool, owned outside the stage and attached before the stage runs. No implicit global pool or thread per request. The first version accepts named, capture-free worker functions and owned Bytes request/result payloads. A typed error remains available; arbitrary closures, borrowed arguments and generic object-graph transfer are excluded.

The following is a contract sketch, not executable Plenty or final generic syntax. Suggested module: std.actor.blocking.

Limits(max_workers: i64, max_jobs: i64, max_input_bytes: i64,
       max_output_bytes: i64, max_worker_bytes: i64)
limits() -> Limits
open_pool(limits: &Limits) -> Result[Pool, BlockingError]
Pool.attach(stage) -> Result[PoolKey, BlockingError]

worker(input: Bytes, deadline: &Deadline, cancel: &CancellationToken)
    -> Result[Bytes, E]

Context.submit_blocking(pool: PoolKey, worker, input: Bytes,
                        deadline: &Deadline)
    -> Result[Job[E], SubmitError[Bytes]]
Job[E].take_now(context) -> Result[Option[Result[Bytes, E]], BlockingError]
Job[E].abandon(context) -> Result[(), BlockingError]

Pool is not actor state. PoolKey is a checked non-owning capability scoped to its attachment; Job[E] is a narrowly certified opaque actor handle. Neither grants a worker access to Context or the stage. An attach operation must not make Pool destruction race a live attachment. Final names and receiver syntax must be checked against stdlib conventions and the compiler’s generic callback representation during implementation.

submit_blocking never waits for a free worker or queue slot. It checks the pool/stage identity, input size and deadline, prepares an independent input snapshot, and atomically admits a job. Function eligibility is a compile-time check, not a run-time escape hatch. Admission errors return the original input in SubmitError[Bytes], with the reason (Capacity, Closed, InvalidPool, TimedOut, LimitExceeded or allocation failure). BlockingError embeds Allocation(AllocError) rather than an allocation union. Worker errors E retain their type and cause; they are not flattened into text.

An accepted job reserves one of max_jobs records until its worker resources and result are reclaimed, including after its caller abandons it. The bound covers queued, running and completed-but-unclaimed jobs, not just the waiting queue. Invalid sizes and unsupported configurations fail explicitly. Defaults must be documented and measured before adoption; the API does not derive a worker count from incoming traffic.

HTTP handler experience

Use the event HTTP interface:

  1. On request End, build a bounded Bytes description of the upstream request, submit a named worker and retain its Job[E] and application correlation ID.
  2. Return without joining, polling in a loop or waiting for the result.
  3. On Event.Actor(Mail.Blocking(job_id)) (proposed), call take_now for the matching job, then build the response from its result.
  4. On disconnect, request timeout or Stopping, abandon outstanding jobs. Apply the application’s existing HTTP error policy for upstream failure or capacity exhaustion; do not automatically retry a side-effecting operation.

The worker constructs its own client/configuration, decodes input, invokes the existing blocking fetch with the supplied deadline and cancellation token, and returns bounded bytes or a transferable typed error. Do not claim every existing client error is transferable: eligibility must be checked, with an explicit owned error adapter where needed. The streaming interface means asynchronous response timing; the initial offload API itself returns one bounded result, not an unbounded response stream.

A database wrapper follows the same pattern but must satisfy the driver’s thread-affinity and effect contracts. Persistent database sessions and connection pools need a separately designed worker-owned resource facility; an arbitrary connection cannot simply move out of an actor into successive pool threads.

Decisions and alternatives

A bounded pool amortizes thread creation and makes saturation visible. One OS thread per request allows incoming traffic to exhaust threads and stacks. Blocking join inside a callback defeats the purpose. Moving the blocking call to another actor on the same stage also stalls that stage. Supporting several stage threads would not make a blocking call cooperative and is independent.

Use byte payloads initially to avoid promising safe deep transfer of arbitrary actor-owned values. A named function avoids storing actor captures in a job. The compiler can reuse existing worker-effect checks, but must add an explicit actor-to-worker boundary rather than exempting ordinary spawn from its guard. Full typed requests, streaming results and retained worker resources require separate contracts. This proposal does not introduce async/await.

Blocking-work ownership and failure

This accompanies the proposed actor blocking-work API. None of these contracts are implemented yet.

Three allocation domains

An actor’s heap can disappear after any callback traps. Moving a Bytes header into a native job does not make its backing storage independent. Submission must copy the payload into a separately owned job allocation before publishing it to workers. The copy is explicit in the contract and bounded by max_input_bytes; failure leaves the original input recoverable. Metadata, cancellation state, wake handles and error storage must likewise contain no pointers into an actor heap.

Workers use ordinary thread-owned allocations under a proposed per-job allocation budget, never the actor heap table. The worker creates its own client, TLS configuration, request strings and any foreign handles. Native code must not temporarily switch a background thread into an actor’s heap. The existing single-owner actor heap rule remains intact.

Completion retains an independent result until the stage claims it. Only the stage thread copies the result into the receiving actor’s heap. Output bytes and typed errors together must fit max_output_bytes; transfer can also fail if that actor has insufficient space. Report a distinguishable transfer error when recoverable; an actor allocation trap must still retire the actor and reclaim the independent completion. No worker frees or reads retired actor storage, and no actor owns a pointer into a worker’s temporary stack.

max_worker_bytes requires real enforcement in the Plenty allocator path, including worker-created client state and reallocation. Merely limiting the returned body does not bound allocations made while fetching it. Reuse or extend allocation-failure propagation so budget exhaustion is a recoverable job failure; existing actor trap/OOM machinery is not a worker implementation. Audit the client’s header/body/decompression limits as well. Foreign allocations and OS thread stacks are outside this allocator budget and must be documented, bounded where the wrapped API permits, and included in qualification measurements.

Eligibility and API capability

The first version uses capture-free named functions. Apply the existing worker eligibility checks to the function, transitive calls, errors, results and destructors. Owned bytes do not make an otherwise forbidden foreign effect safe. Reject borrowed state, uncertified indirect effects, actor addresses used as cross-thread senders, actor heap operations, executor submission and worker-side future waits.

E must additionally be independently owned, bounded and transferable into an actor node under the actor type rules. Derive/check the transfer operations from concrete types; do not introduce an unchecked public marker or raw-pointer API. Restrict unsupported error types with a useful diagnostic. Account for the size of any owned error payload, and retain the cause through an explicit adapter. A byte-only input/result API does not silently authorize shared strings, locks, channels or arbitrary handles in actor nodes.

The only new actor-storable types are the certified job handle and pool key. A handle identifies a registry entry; it does not own a native thread or a shared allocation directly. Dropping it marks the waiter abandoned without joining. Trap retirement must do the same through stage bookkeeping, because user destructors do not run on contained traps. Pool shutdown must remain safe even if no user-level job handle destructor ran.

Completion identity and cleanup

Use the stage attachment identity, actor slot generation and job generation when routing a completion. A recycled actor slot, new stage instance or new HTTP request must never receive an old result. Validate the result’s concrete error/transfer type against the job record before decoding it.

Exactly one owner wins a claim-versus-abandon race. Claimed results are consumed once; failed transfer destroys the source once. Abandoned queued jobs are removed before execution when possible. Abandoned running jobs retain their slot and cancellation state until they finish, then destroy output and release resources without posting to the retired actor. Cancellation cannot promise that an accepted operation has had no external effects.

Cleanup of worker-owned values, including abandoned results and errors, runs on a worker/reaper path whose lifetimes and capacity are bounded by the pool. Unbounded user cleanup must not run on the stage thread. The first version’s payload/error restrictions must permit bounded transfer and cleanup on the actor side. Reclamation cannot require a new queue allocation or mailbox slot.

Deadlines, cancellation and reliability

Submission takes an absolute Deadline and copies its monotonic value. Queue waiting counts against it. Expired queued work must not start. The supervisor signals a worker-owned cancellation token when the job is abandoned, its deadline passes or the pool starts cancelling. A worker may inspect that token or pass it to a cooperating library. The actor does not retain a shared token in its node.

Stopping the wait is prompt; stopping a blocking operation is cooperative. A foreign call that ignores cancellation can continue after the HTTP request, actor and server have ended. Its slot remains occupied, so repeated timed-out requests cannot create unlimited replacement threads. The registry separates the actor’s terminal wait outcome from physical job completion: deadline expiry can notify the actor with BlockingError.TimedOut even while an uncooperative worker retains the job slot. No new per-expiry job record is allocated. Late worker output is discarded after that outcome wins.

Serialize outcome publication, deadline expiry and abandonment under registry synchronization. At publication, compare the monotonic clock with the copied deadline: a result published at or after expiry loses to timeout, even if the stage has not processed its timer yet. A result published before expiry remains a result if the actor observes it later. Explicit abandonment suppresses further user notification and makes an already-queued notice stale. Tests must cover these orderings with controlled time and barriers rather than scheduler luck.

An ordinary returned Err(E) is a job outcome. A late or abandoned result is reclaimed. Neither actor supervision nor thread isolation contains arbitrary worker traps, stack overflow, native crashes or process-wide lock corruption. The first version retains ordinary native worker fatal-trap semantics and must say so prominently: code that would be contained inside an actor may instead terminate the process when offloaded. Do not catch a runtime panic and report a successful containment guarantee. Use a separate process when that isolation is required. External database writes and HTTP side effects are never rolled back.

Blocking-work runtime and qualification

This records implementation guidance for the proposed actor blocking-work API, not a second work queue. B85 in the backlog owns scheduling and outstanding scope.

Runtime division

Implement policy, limits and the HTTP adapter examples in Plenty where practical. A small native boundary is necessary for thread creation, independently owned job storage, synchronization, allocator budgets, compiler-certified transfer and selector wake-ups. Ordinary Plenty code cannot safely construct these lifetimes from actor heaps and lexical thread handles. Reuse existing runtime thread and selector primitives where their contracts fit; do not add public raw std hooks.

Before coding, map the specialized DNS helper, actor retirement, selector handle tracking, native worker checks, callable lowering and allocator budget paths. The generic bridge is new work, not a wrapper around DNS’s private payloads. An existing executor cannot be assumed safe to retain or destroy in an actor. Sharing executor internals requires the same ownership and shutdown guarantees.

Bounded registry and wake protocol

Allocate max_jobs registry entries and completion capacity when opening the pool; validate overflow and allocation failures. Each entry has an identity, state, owner generation, deadline, cancellation flag, transfer descriptor and bounded payload storage. Reserve a small allocation-free infrastructure-error outcome per entry, so failure to allocate an ordinary output can still notify the actor. Opening a pool must roll back already-started workers on partial startup failure. Native synchronization must have short critical sections; never execute work, user cleanup, blocking I/O or wait for capacity under a stage-visible lock. No stage operation waits on worker completion.

Use the following lifecycle, with abandonment tracked independently once running:

Free -> Queued -> Running -> Completed -> Claiming -> Reclaiming -> Free
Queued -- cancelled before start --> Reclaiming
Running -- abandoned --> Running (no waiter) -- finished --> Reclaiming
Completed -- abandoned --> Reclaiming

A worker publishes its result before signalling the selector wake handle. The signal is a hint to drain the registry, not the sole record of completion. Coalesced or saturated wakes must not lose results. The stage drains a bounded number of completions per turn and schedules another internal turn if records remain; otherwise a completion flood can starve HTTP sockets and timers.

Deliver Mail.Blocking(job_id) through reserved control state, independent of ordinary mailbox capacity and application timers. One reserved notification per outstanding job is sufficient. Retry notification scheduling while the job is live if necessary; do not silently discard it on Full. take_now may see a pending job and return Ok(Nothing); repeated reads after consumption or explicit abandonment return a named error. Stale already-queued notices are harmless and never read reclaimed storage. Actor retirement cancels all its registry waiters.

Budget planning must count registry metadata, independent input/output copies, worker scratch allocations, actor-side result copies, thread stacks and native library allocations separately. max_jobs bounds retained input/output records; max_workers bounds concurrent scratch budgets and stacks. Queued input must not also acquire a full worker budget before execution. Measure transient transfer peaks and cleanup backlog, not just steady-state body sizes.

Ready results consume admission capacity until claimed or reclaimed, bounding slow consumers. Scope attachment to one stage initially; one pool shared by many stages needs additional fairness and wake-routing contracts. Permit separate pools for different services so a stuck database need not exhaust HTTP-client workers, with process-wide resource budgets remaining the application’s concern.

Lifetime and shutdown

The outer driver owns the pool and a stage attachment. An actor cannot close or join the pool. HTTP serving needs an explicit optional pool attachment accepted by event-server entry points or a server setup object; do not hide a pool in a capture that violates the current factory/node rules. Preserve existing server entry points and defaults. This integration/API shape is an implementation design gate, not an assumed existing hook.

Expose nonblocking admission closure and explicit drain-versus-cancel modes. During HTTP draining, application completion events continue alongside socket progress until the grace deadline. After that, retiring connection actors abandons their jobs and detaches their response destinations. The pool must outlive running jobs; stage storage and wake handles cannot be freed while a worker may still access them. Use independently retained completion state and an explicit detach handshake, not borrowed stage pointers.

Final pool shutdown/join runs outside callbacks, with deadline: &Deadline and cancel: &CancellationToken on a waiting API. If that wait times out, return an error while retaining the owner for retry; it does not authorize freeing running job storage. No implicit detached workers. Final owner destruction must join remaining workers and can therefore wait indefinitely for uncooperative calls; document this even when an explicit earlier wait was bounded. Do not promise a bounded whole-process shutdown for arbitrary blocking libraries. Releasing a stage attachment must only sever routing and signal cancellation, without joining.

Evidence required for adoption

Use a controlled local upstream and barriers/latches rather than relying on scheduler luck or an Internet endpoint. Tests should establish:

  • An HTTP request waits on a blocked worker while unrelated connections complete, timers fire and cancellation is observed by the stage.
  • max_workers bounds concurrent execution; max_jobs includes completed and abandoned-running work. Saturation returns original input without running it.
  • Actor traps, ordinary stop, disconnect, slot reuse and stage destruction before completion leave no dangling reads, misdelivery, lost cleanup or leaked heaps.
  • Full mailboxes, exhausted application timers, coalesced wakes and completion bursts preserve notification and fair progress without busy polling.
  • Deadline expiry before start, completion/abandon races and shutdown while queued, running or ready produce one outcome and one cleanup. A call ignoring its token holds its slot until the controlled upstream releases it.
  • Allocation failure at admission, worker execution, publication and actor result transfer is recoverable where specified and never publishes a partial job.
  • Compile-fail tests reject actor captures, borrowed payloads, forbidden effects, unsafe error types and ordinary thread spawning in actor callbacks. Valid fetch adapters pass the normal worker checks without an exemption.
  • Returned typed upstream errors survive transfer. Fatal worker behavior is qualified in a subprocess and documented independently of actor containment.
  • Repeated create/run/drain/drop cycles return threads, descriptors and tracked allocations to baseline; race/sanitizer checks cover abandonment and wake teardown.

An executable tutorial should implement an actor HTTP endpoint backed by a local blocking fetch, including capacity exhaustion, upstream failure, timeout and shutdown. Measure latency of unaffected requests, queue delay, throughput, CPU, thread count and peak memory against nonblocking actor I/O and the worker HTTP server. Report actual platform coverage; support must follow native thread and selector qualification rather than claiming portability from a Linux result.

Questions to settle during implementation

The safety contracts above are requirements; representation details still need prototypes: concrete generic error-transfer lowering, the bounded worker allocator, reserved completion-mail scheduling, the stage attachment lifetime, HTTP server pool injection, and errors/results that require thread-affine destruction. The initial version may narrow eligible errors further if safe bounded transfer cannot be established. Do not widen accepted types or drop allocation guarantees merely to make the first fetch example compile.

Keyword arguments, defaults, and static unpacking

Revised direction, 2026-10-17: Named arguments and fresh defaults is the preferred narrow proposal for B36. It separates named calls and defaults from splat unpacking and signature markers, and implements fresh fallible defaults. This older document retains the broader alternatives; its constant-only restriction and combined first stage are not the new direction.

Status: Partly implemented (B36). Named calls and fresh per-call defaults ship for named functions, methods and constructors. Splats, variadics and positional-only/keyword-only signature markers remain unimplemented. Signatures and calls owns the current contract, including callable and protocol restrictions. The present tense below records the older, broader proposed design; its constant-only defaults are superseded by fresh expression defaults.

Python’s keyword arguments, defaults, and */** unpacking make calls and literals easier to read and write. Plenty adopts them only where the compiler can resolve them completely. This is not a compatibility goal: a Python form that needs runtime arity checks, runtime key lookup, dynamic typing, or hidden allocation is rejected, not approximated.

In this role * and ** are unpacking syntax, not operators. Binary a * b and a ** b keep their arithmetic meaning.

Rules every form must satisfy

  • Static resolution. Every call is rewritten during checking into one positional call to a known declaration. Codegen and the runtime never see names, defaults, or unpacking at call sites.
  • No runtime call failure. Arity, names, and types are checked at compile time. No form can fail at run time because a sequence has the wrong length or a mapping lacks a key.
  • No hidden allocation. A form that allocates already has a Result type at that site (a collection display). A call never allocates to pass its arguments.
  • One rule, no exceptions. Each form has one meaning, and the conditions for accepting it can be stated in a sentence.

Keyword arguments

Any parameter of a named function, method, generic function, or class constructor can be passed by name unless it is positional-only:

def scale(value: i64, factor: i64, offset: i64) -> i64:
    value * factor + offset

scale(3, offset=1, factor=2)
  • Positional arguments come before keyword arguments.
  • Each parameter is bound exactly once. Unknown names and duplicates are errors.
  • Arguments are evaluated and moved left to right in source order, as in Python, not in parameter order. A call’s effects follow the text.
  • Expected-type inference works as it does for positional arguments: factor=2 against factor: u8 produces a u8.

Parameter names become part of a declaration’s public interface: renaming a pub function’s parameter can break callers. Positional-only parameters (below) let an author opt out.

Function values stay positional. Callable[[A], B] has no parameter names, and this proposal does not add them. A call through a function value accepts positional arguments only. A keyword call needs a statically known declaration.

Generated constructors take field names as keywords: Point(x=3, y=4). This needs no new mechanism, since the generated constructor’s parameters are the fields.

Builtins follow the same rules. Builtins with optional arguments, such as splitlines and open, become ordinary signatures with a keyword and a default.

Protocol requirements. The reference says implementation parameter names may differ from the protocol’s. Keep that. A generic body checks keyword calls against the protocol requirement’s names and markers, then reaches the implementation positionally. Implementation names never matter through a bound.

Positional-only and keyword-only markers

Python’s / and bare * markers are adopted unchanged:

def clamp(value: i64, /, low: i64, high: i64, *, wrap: bool) -> i64:
    ...

Parameters before / are positional-only. Parameters after * are keyword-only. Keyword-only parameters suit Boolean flags and options, where a bare positional True hides intent.

Default values

def split(text: &str, separator: &str = ",", limit: i64 = -1) -> Result[list[str], AllocError]:

A default is an allocation-free constant expression: a numeric, bool, or str literal, unit, Nothing, or a nullary enum variant that needs no allocation. A default cannot reference other parameters, globals, or calls. Inside these limits, the moment of evaluation (definition time or call time) has no visible effect, so Python’s shared mutable default problem cannot occur.

  • A positional parameter with a default cannot be followed by a positional parameter without one. Keyword-only parameters can appear in any order.
  • A parameter whose type mentions a type parameter cannot have a default. A default must check against a concrete type, never per specialization.
  • The compiler inserts omitted defaults during the positional rewrite.

Class field defaults use the same constant rule and become defaults of the generated constructor. They are currently deferred in the class reference and can land with this work or later.

Unpacking at call sites

*tuple expands a tuple value into positional arguments:

point = (3, 4)
distance(*point)

The operand’s static type must be a tuple, so its arity and component types are known. The tuple is evaluated once at its source position and consumed like a whole-tuple unpacking. Several starred tuples may appear in one call. No storage is allocated.

**record expands a class value into keyword arguments named by its fields:

def draw(x: i64, y: i64, *, color: Color) -> ():

draw(**origin, color=Color.Red)
  • The operand’s static type must be a class. Its visible fields must each match a parameter name. An extra field is an error, not an ignored value.
  • The record is consumed and its fields move into the call. Moving fields out of a class needs a whole-value destructuring rule that the language does not have yet. As a hard rule, ** rejects classes with a custom __del__, since their fields cannot be separated from the destructor.
  • Remaining parameters come from other arguments or defaults.

Rejected: *list, *generator, or any operand whose length is not in its type, used for fixed parameters. **dict in any position. Both need runtime checks that could fail during the call.

Variadic parameters: *args: T

def total(*values: i64) -> i64:
    mut sum = 0
    for value in values:
        sum = sum + value
    sum

total(1, 2, 3)

The extra arguments must all have one element type T. At each call site the count is known, so the caller places the evaluated arguments in a stack buffer in its own frame and passes a pointer and length. Inside the body, values is a read-only borrowed view of that buffer. The caller drops the elements after the call returns. No heap allocation is involved.

  • Generic *values: T infers T from all extra arguments. Under the existing inference rule, all evidence must agree.
  • A list can supply the whole variadic group, total(*numbers), by passing a borrowed view of its storage. Mixing it with other extra arguments, as in total(1, *numbers), is rejected because it would need a buffer of unknown size.
  • Heterogeneous variadics (*args of mixed types) need variadic generics and are out of scope.
  • Taking a variadic function as a Callable value is deferred until the view type has a stable spelling for callable signatures.
  • *args has no connection to C varargs, which stay excluded from extern def.

Dependency: a borrowed sequence view type. Plenty has no slice or span type yet, and a view should be designed for its own sake (string, list, and buffer views) before *args relies on it. A consuming variant, where the callee owns the elements, can follow once owned views exist.

No **kwargs parameters

Python’s **kwargs is a dict[str, Any]. It needs dynamic typing and allocates a dictionary on each call. Plenty does not add it.

Its main uses already have static alternatives:

  • Option bags: use keyword-only parameters with defaults, or take an explicit options class with field defaults: render(page, opts=Options(dense=True)).
  • Forwarding: a wrapper takes opts: Options and passes **opts to the inner call.

A future “keyword group” parameter that exposes a class’s fields as keyword-only parameters without constructing the class is possible. Defer it until a concrete forwarding need shows that the alternatives above are insufficient.

Unpacking in displays

Collection displays already return Result, so spreading into them adds no new failure path. A tuple display is inline and allocates nothing:

both = [*first, *second]?
members = {*left, *right}?
merged = {**defaults, **overrides}?
extended = (*pair, 3)
  • In a list or set display, *source accepts the owned sources that list[T].from accepts: a collection, range, or generator. The source is consumed. Use copy(source)? to keep the input, as update already requires.
  • In a dict display, **source accepts only an owned dict[K, V] of the same type. Later keys replace earlier ones, matching dict.update.
  • In a tuple display, *source accepts only a tuple, so the result arity is static.
  • Construction stays left to right. The first failure releases the initialized prefix and drops evaluated sources not yet transferred; later entries are not evaluated. Implementations may sum known source lengths to reserve once.
  • return *head, last and yield *head, last are tuple displays and follow the tuple rule.

Starred targets in bindings

first, *_ = triple ignores the remaining tuple components and drops their owners. It is static and allocation-free, so it is accepted.

first, *rest = triple with a named rest was rejected while tuples were heap records: rest would have been a new allocation inside a binding statement that has no place for ?. Tuples are now inline, so the form is allocation-free and the rejection needs revisiting.

head, *tail = items on a list is rejected permanently as a binding: the list’s length is not in its type. The checked form is a sequence pattern, where a length mismatch is an explicit branch, not a hidden failure:

match items:
    case [head, *tail]:
        ...
    case []:
        ...

tail would be a borrowed view. This depends on the same view type as *args and on sequence patterns, which match does not support yet. Mapping patterns with **rest are rejected.

Rejected Python forms

FormReason
**kwargs parameterDynamic typing and a per-call dictionary allocation
f(**mapping) with a dictKeys are checked only at run time
f(*items) into fixed parameters, for an operand whose length is not in its typeArity is checked only at run time
f(1, *items) into *argsNeeds a buffer of unknown size
Mixed-type *argsNeeds variadic generics
Defaults that are arbitrary expressionsHidden allocation and effects; Python’s evaluation-time trap
head, *tail = items on a listRuntime length check in a statement
case {**rest} mapping patternsRuntime key sets
Keyword calls through Callable valuesCallable types carry no parameter names

Suggested stages

  1. Keyword arguments, / and * markers, constant defaults, keyword constructors, and builtin special cases moved onto ordinary signatures. This stage is frontend-only.
  2. *tuple and **record at call sites, and *_ in tuple bindings.
  3. Spreads in list, set, dict, and tuple displays.
  4. A borrowed sequence view type, then *args: T.
  5. Sequence patterns with *rest, after views and sequence match.

Each stage updates the function reference, the tutorial lessons that teach calls, and its backlog entry (B36).

Open questions

  • The spelling of the view type that *args and *rest bind.
  • Whether **record should accept classes with __del__ once a whole-value destructuring rule exists.
  • Whether protocol requirements may declare defaults, or only names and markers.
  • Whether class field defaults ship in stage 1 or wait for a separate change.

Returned enums that carry references

Status: Phase 1 implemented (issue #520). Functions and methods can return Option[&T], Option[&mut T], Result[&T, E], and nested success payloads of these; callers match, propagate, unwrap, inspect, print, forward, or hold the result in an immutable local. The reference is authoritative: optional and fallible references. Phase 2 (mut holders, parameters) and phase 3 (pipelines) are not implemented, and standard library getters such as dict.get_ref remain follow-up work under B11. This page is the design record; examples below may predate the implemented diagnostics.

Problem

“Find it, or say it is absent” is one of the most common things a program does. In Plenty today a function cannot return a borrow that may be absent:

def find(items: &list[Item], name: &str) -> Option[&Item]:
#                                            ^ references cannot be stored in enum payloads

Every container that holds non-copyable values works around this, and each workaround costs something:

WorkaroundWhere it is usedCost
Callback getters, get_with(&key, &f) -> Option[R]std.cache.lru get_with, peek_withA callback per lookup; no ?, return or break from inside it; an extra type parameter in every signature
Trapping getters plus a membership testLruCache.get_ref/peek_mut, Heap.peek, graph.node/edgeTwo lookups (contains then the getter); a missing key is a crash rather than a value
Return an index, then index againstd.http.find, std.http.cookie.find, std.http.body.find, std.json.find, std.toml.findA second bounds check; the index is not tied to the collection, and private layouts (LRU slots) cannot expose one
Copying lookupdict.get, list.getRejects affine values (“get cannot return owned dictionary values”)
Rejected outrightnext(p), find and reduce on pipelines of &T items“references cannot be stored in a find result; map to an owned value first”

The lookup lesson currently tells learners “Optional borrowed lookup is not implemented”. std.graph traps on an invalid handle by calling itself with an out-of-range index, because a getter cannot report absence.

Plenty avoids lifetime annotations with two rules: a returned reference borrows from the first reference parameter (the origin), and references never enter aggregates. This proposal keeps both. It adds one narrow exception: an Option or Result may carry a reference as a value in flight, produced by a call under the origin rule and consumed locally, but never stored.

What exists today

Probes ran on local main at 13579f12 with a debug compiler (plenty --check).

ProbeResultConsequence
def find(xs: &list[Item], n: &str) -> Option[&Item]Rejected: references cannot be stored in enum payloadsThe motivating signature
-> Result[&str, i64], o = Some(&xs[0]), o: Option[&i64] = NothingSame rejectionConstruction and annotation share one barrier (Ty::contains_reference)
enum Found: Hit(&str)Rejected at the declarationUser enums use the storage-barrier check
-> (&str, i64), t = (find(&xs, "a"), 1)Rejected: references cannot be stored in tuplesTuples are aggregates; out of scope here
Callable[[&str], Option[&str]]Same rejectionCallable signatures share the type check
def pick[T](x: T) -> Option[T] called with &sRejected: reference type arguments are not supportedA type parameter is never a reference; generic user enums cannot carry one
d.get("a") with list[i64] valuesRejected: get cannot return owned dictionary valuesAffine values have no optional lookup
next(p), iter(&xs).find(f) with &str itemsRejected: references cannot be stored in a next/find resultPipelines hit the same wall
Return a &mut element on one path, xs.append(...) on the otherAccepted and runsLoans already end per path, so get-or-insert shapes work
def touch(xs: &mut list[i64]) -> &i64, then read xs while the result is liveAcceptedA shared result from an &mut origin holds a shared loan
Payload reference returned from match &node.nextAcceptedBorrowed-match payloads already satisfy the origin rule

Relevant implementation facts:

  • Representation. Wide values (enums, classes) and references are 128-bit slots. A reference’s low word is a slot address; its high word is a tag shift, non-zero only after projecting into a nested sum payload (src/codegen/references.rs). Enum tags stack in the high word: wrapping is (payload_tags << tag_bits) | tag, unwrapping shifts back (src/codegen/enums.rs, wrap_tagged, tagged_payload). There is no niche optimization.
  • Origins. The checker picks an origin only when the result type is Ty::Ref (src/frontend.rs, return_origin); check_return_reference compares each return path’s loan root with it, and call_reference_result gives the caller a child loan of argument 0’s loan (src/frontend/references.rs). The ownership pass itself only sees Loan/UseLoan/EndLoan ops, so it is type-agnostic.
  • Local loan carriers that are not references already exist. KeyView values and Iter[T] pipelines hold loans per slot through reference_locals; neither can be returned.
  • ? and builtins. ? consumes the operand and moves the payload out (lower_try). unwrap, unwrap_or, ok_or, is_some and friends are compiler builtins (src/frontend/collections.rs), not generic Plenty methods; neither ? nor unwrap forwards a loan today. There is no map or and_then on Option/Result, and no if let/while let form: match is the only pattern construct.

Precedents

LanguageOptional borrowHow it stays sound
RustOption<&T>, Result<&T, E>, any enumLifetime parameters, elided for one input reference; references may be stored anywhere a lifetime allows
SwiftNo borrowed optionals; Optional<T> copies or retainsReference counting; inout/borrowing parameters cannot be returned
Hylo (Val)No first-class references; subscript yields a projection for a scopeProjections cannot escape their scope

Rust’s lifetime elision rule (“one input reference: the output borrows it”) is Plenty’s origin rule. Rust needs annotations only for the cases this proposal continues to reject: several possible origins and stored references.

Proposal

The rule

A reference-carrying enum is an Option or Result type in which a variant’s payload is a reference, directly or through further Option/Result payloads: Option[&T], Option[&mut T], Result[&T, E], Result[Option[&T], E]. Its reference payloads behave exactly like a returned reference, and the value itself is never stored:

  1. Produced by a call or a local construction. A function or method may declare a reference-carrying enum as its return type. Every return path, including each ? that propagates, must originate in the first reference parameter under the existing returned-reference rule. Nothing and payload-free variants originate in nothing and are always accepted. A function without a reference parameter cannot declare one, and an &mut payload requires an &mut origin.
  2. Held by a local. The caller may bind the result to a local, which holds the origin loan until the last use of the local and of every reference extracted from it. Some(&place), Ok(&place) and Err(&place) may be bound to a local or returned; the binding holds that place’s loan, as r = &place does.
  3. Consumed by inspection. It may be matched, propagated with ?, tested (is_some, is_ok, …), unwrapped, printed, or passed straight to return.
  4. Never stored. It cannot enter a tuple, class field, list, dictionary, set, Box, closure capture, another enum value, a scoped worker, a channel, or a live state across yield. It cannot be a type argument other than the payload position of Option/Result, so list[Option[&T]], Generator[Option[&T]] and Box[Option[&T]] stay rejected.

Rule 4 is the existing “references never enter aggregates” rule with no exception: the enum is the reference’s wrapper, not a container it is stored in. A nested type such as Result[Option[&T], E] is written in one signature and produced as one value; wrapping an existing Option[&T] local in Ok(...) at a return is permitted for the same reason, and is the only place an enum value may appear inside another.

class Item:
    name: str
    tags: list[str]

def find(items: &list[Item], name: &str) -> Option[&Item]:
    for item in items:
        if item.name == name:
            return Some(item)
    Nothing

def main() -> Result[(), AllocError | IoError]:
    mut items = [Item("ada", ["x"]?)]?
    match find(&items, "ada"):
        case Some(item):
            print(item.tags)?      # `item: &Item`; `items` is borrowed here
        case Nothing:
            items.append(Item("ada", []?))?   # the loan ended at the match
    Ok(())

Which enums qualify

OptionForAgainst
Option onlySmallest change; covers get, find, firstFallible borrowed lookups (Result[Option[&V], E], a parser returning Result[&Token, ParseError]) remain impossible
Option and Result, nested in each other (recommended)Covers infallible and fallible lookups; both are builtin sums whose methods are compiler builtins, so no generic instantiation with a reference type argument is needed; ? already treats them speciallyTwo types to document instead of one
Any enum, including user enumsUniformUser enum payloads come from declarations or type parameters. A declaration naming &T would be the first user type with a reference field, which reads as “references can be stored”. A generic Lookup[&V] needs reference type arguments, which Plenty does not support and which would make every generic body a potential reference holder

Recommendation: Option and Result only. User enums keep the rule that declarations and type arguments never contain references. Revisit only with a concrete case that Result[Option[&T], E] cannot express.

Anonymous error sums (AllocError | IoError) are Result error types and may not contain references; an error that borrows its input would escape through ? into unrelated callers’ error unions.

Matching, ? and builtins

  • match by value on a reference-carrying enum binds payload names as the reference type itself (item: &Item, slot: &mut V). Each binding is a child of the enum local’s loan, like a borrowed-match payload. In an arm without payload bindings, the enum’s last use is the scrutinee, so the loan ends there; this is what lets the Nothing arm above mutate items.
  • match &o is rejected: a reference to a reference-carrying enum would be a reference to a reference, which Plenty does not have. The diagnostic says to match o directly. Matching a shared Option[&T] by value copies only the reference and leaves o usable.
  • ? returns early on Nothing/Err and yields the payload reference, which inherits the operand’s loan: item = find(&items, "ada")? is a reference binding from a call result. Propagation is a return path: in a function returning Option[&T], ? on Nothing returns Nothing, which borrows nothing, and the extracted reference must still reach any returned Some from the origin.
  • is_some, is_none, is_ok, is_err read the tag; they are uses of the value and extend its loan to that point only.
  • unwrap() yields the reference with the operand’s loan and traps on absence, the same as indexing.
  • ok_or(error) turns Option[&T] into Result[&T, E] with an owned error; the loan carries over.
  • unwrap_or(default) with a reference default is rejected. The result could borrow either the option’s origin or default, which is a conditional reference expression; those are unsupported, and supporting them only here would give one builtin a rule no function signature can state. The diagnostic suggests match.
  • Printing reads through the reference (Option[&str].Some("ada")). Equality is not extended in phase 1.

match is the only pattern form; if conditional forms such as if let are added, they bind payloads as match does. Should map/and_then combinators be added later, they follow from the signature rule: the receiver is the first reference-carrying parameter, so o.map(f) may return a borrow only through f’s own origin, its argument.

Err payloads

Recommendation: in Result, only the Ok payload may carry a reference. Error values are routinely widened into anonymous sums, printed by main, and propagated far from their origin; a borrowed error would make ? in an ordinary Result[(), IoError | E] caller fail with an ownership error. Errors that need to describe input should copy the relevant part, as they do today.

&mut payloads and aliasing

Option[&mut T] behaves as an exclusive reference. It is affine: a by-value match, ?, unwrap, passing it, or returning it moves it, and the extracted &mut T takes over the loan. While the value or its payload binding is live, the origin is exclusively borrowed, so two get_mut results from one dictionary cannot be live together (dictionary keys already protect the whole collection). A shared result from an &mut origin holds only a shared loan, as today: cache.get_ref(&k) may promote the entry through &mut self and still let the caller read cache.len() while the result is live.

Get-or-insert works without new rules, because loans already end per path:

def counter(counts: &mut dict[str, list[i64]], key: &str) -> Result[&mut list[i64], AllocError]:
    match counts.get_mut(key):
        case Some(found):
            return Ok(found)
        case Nothing:
            pass
    counts.insert(*key, []?)?
    Ok(&mut counts[key])

Loans, last use and temporaries

The loan model is the one returned references already use. A reference-carrying local is a loan holder like a reference local; payload bindings, ? results and unwrap results are child loans. The origin loan ends at the last use of all of them on each path. A precise field origin stays precise; indexed and control-dependent bodies protect the whole argument, as they do for &T results.

Temporaries follow the returned-reference rules unchanged. A method returning a reference-carrying enum needs a named receiver or a place in named storage. With a temporary shared argument the result is usable only within its full expression: print(lookup(&row(), "id").is_some())? works; binding the result is rejected.

Mutable locals

A mut reference-carrying local may be reassigned, which is how a search remembers its best match without an index:

def longest(words: &list[str]) -> Option[&str]:
    mut best: Option[&str] = Nothing
    for word in words:
        match best:
            case Some(current):
                if word.byte_len() > current.byte_len():
                    best = Some(word)
            case Nothing:
                best = Some(word)
    best

It follows reference rebinding: the first reference it holds fixes its first place, later Some(...) assignments must stay within that place, and assigning Nothing keeps the first place. A binding initialized with Nothing takes its first place from its first Some assignment. Recommendation: phase 2, after immutable locals have shipped, because it touches the rebinding checker.

Parameters, generics and Callable

  • Parameters. Recommendation: allow reference-carrying enum parameters in phase 2. They give optional borrowed arguments (def render(node: &Node, parent: Option[&Node])) and let helpers take a lookup result. Such a parameter counts as a reference parameter when choosing the origin, so def or_first(found: Option[&T], xs: &list[T]) -> &T may return only found’s payload. Passing a shared one copies it; passing an &mut one moves it.
  • Generics. Type parameters still never bind references, so Option[T] with T = &U stays rejected; a generic function must spell Option[&T]. Its body is checked per instance, as reference-returning generic functions are now. Generic class methods such as LruCache[K, V].get_ref -> Option[&V] need no copyable bound, so they work for every instantiation regardless of issue #515. That issue still matters for copying accessors; this proposal does not depend on it.
  • Callable. Callable[[&K], Option[&V]] is accepted, with the origin fixed by the first reference-carrying parameter, as for &V results.

Layout and code generation

No runtime cost and no new representation. An Option[&T] is one 128-bit slot, the same size as Option[i64] and &T:

ValueLow wordHigh word
&Tslot addresstag shift
Some(r)slot address(shift << 1) | 1
Nothing00
Ok(r) in Result[&T, E]slot addressshift << 1
Err(e)e’s word(e.tags << 1) | 1

This is the existing nested-tag scheme: wrap_tagged and tagged_payload already restore a payload’s high word exactly, so construction is a shift and an or, and a match tests bit 0. The shift needs at most 7 bits, so layout depth accounting charges a reference payload 7 tag bits within the existing 64-bit tag budget.

A null-address niche (the Rust layout) is rejected: the slot is 128 bits either way, so it saves nothing, and it would add a special case to every tag read (sum_tag, Unwrap, lower_try, widen_sum). Values are not stored, so no heap or owner-local storage is involved, and there is no cleanup: a reference-carrying enum is never managed.

Diagnostics

Each rejection names the value, the place, and the fix:

CaseMessage
Stored in an aggregateOption[&Item] carries a reference and cannot be stored in a list; match it and store an owned copy or an index
Class field or user enum declarationreferences cannot be stored in class fields; Option[&T] and Result[&T, E] may only be returned or held in locals
No reference parametera function returning Option[&Item] needs a reference parameter to borrow from
Wrong originthis Some borrows fallback, but Option[&Item] results borrow the first reference parameter, items
&mut payload from a shared originOption[&mut V] requires the first reference parameter to be &mut
Reference in ErrResult errors cannot carry references; copy the part of the input the error reports
match &oo already holds a reference; match o directly
unwrap_or(&other)unwrap_or cannot choose between two borrows; use match
Across yield, into a scoped worker, into a closurefound holds a borrow of items and cannot cross yield / be sent to a worker / be captured; match it first and pass the reference
Temporary originlookup(...) borrows a temporary argument; bind the argument to a local first
Type argumentreference type arguments are not supported; write Option[&T] in the signature (existing message, reused)

Teaching

  • Look up a value that may be absent replaces “Optional borrowed lookup is not implemented” with a short dict.get_ref section: get copies, get_ref borrows, and the dictionary stays borrowed until the payload’s last use. Readers already know element borrows from the preceding lessons.
  • Return a reference to a field gains “Return a reference that may be absent”, writing find with Option[&Item] and showing ? on it. This is the first point where readers write their own returning functions with references.
  • The reference gains a page after returned references, “Optional and fallible references”, stating rules 1 to 4, the layout, and the rejected cases. Returned references and borrowing link to it instead of saying references never enter enums.

The rule to teach is one sentence: a lookup can return Option[&T]; treat the result like the reference it holds, and never store it.

Standard library changes

Naming follows one rule across builtin and std containers (decided after this proposal was accepted; see copying and borrowed lookup): indexing and plain accessors trap; get and peek return a copy in an Option; get_ref, peek_ref, and get_mut/peek_mut return borrows.

ModuleTodayProposed
builtin dictget(k) -> Option[V] copies non-affine valuesAdd get_ref(k) -> Option[&V] and get_mut(k) -> Option[&mut V] for every value type
builtin listget(i) -> Option[T] copiesAdd get_ref(i) and get_mut(i); get_ref(0) and get_ref(-1) replace a separate first/last
std.cache.lruget_with, peek_with, trapping get_ref, peek_mutCopying get(&mut self, k) -> Option[V] (promotes) and peek(k) -> Option[V], possible now that methods are checked only when used (issue #515); get_ref -> Option[&V] (promotes), get_mut -> Option[&mut V], peek_ref -> Option[&V], peek_mut -> Option[&mut V]; remove the callback and trapping forms (issue #516)
std.graphnode/edge trap; callers test contains_node firstKeep trapping node/edge (handle indexing); add get_node_ref, get_node_mut, get_edge_ref, get_edge_mut
std.collectionsHeap.peek, HeapBy.peek trap when emptypeek_ref() -> Option[&T], and a copying peek() -> Option[T] for copyable T, matching pop() -> Option[T]
std.dequeNo access without removalAdd get_ref(i)/get_mut(i) if a use case appears
std.http, std.http.cookie, std.http.body, std.json, std.tomlfind(...) -> Option[i64]find(...) -> Option[&Header] (and the matching element type); callers currently index immediately
pipelinesnext, find, reduce reject &T itemsPhase 3: return Option[&T] borrowing the pipeline’s source

Not affected: regex Captures.get (copyable Match), env.get (owned text), and toml.Document.get, whose arena nodes are not Toml values and which therefore still builds a copy.

Std has no stability promise yet, so the replaced forms are removed in the same change rather than deprecated, keeping one obvious way.

Alternatives considered

AlternativeWhy rejected
Status quo with callbacksEvery container repeats the work; callbacks cannot ?, return or break; signatures gain R and F parameters
Membership test plus trapping getterTwo lookups per access; a forgotten test is a crash rather than a type error
Handles and indicesA second bounds check, no relation between index and collection, and impossible for private layouts and dictionaries
A dedicated optional-reference type (&T? or Ref[T])New syntax and a type learners must map back to Option; match and ? already handle Option
Any enum carrying referencesNeeds reference fields in declarations or reference type arguments; see Which enums qualify
References in all aggregates with lifetime parametersBreaks the no-annotation design goal and adds checker and learner cost far beyond these use cases. This proposal does not preclude a later B11 decision on stored references
Returning Option[&T] but forbidding local bindingsForces every lookup into a match scrutinee; ? could not bind its result

Implementation plan

  1. Types and checking. Add a carries_loan predicate (references and reference-carrying enums) and use it where origin selection, return_origin, check_return_reference, trailing-expression checks and call_reference_result now test Ty::Ref. Admit reference payloads in Option/Result resolution only for return types and local annotations; keep the storage barrier for every other position, with the messages above. Make ?, unwrap and ok_or forward the operand’s loan, and by-value matches create child payload loans. Charge 7 tag bits in layout depth. Tests: acceptance, every rejection row, loan ends per arm, &mut aliasing, temporaries, generic per-instance checking, and codegen of nested Result[Option[&T], E].
  2. Builtins and docs. dict.get_ref/get_mut, list.get_ref/get_mut; the new reference page; the two tutorial sections.
  3. Std adoption. std.cache.lru (issue #516), std.graph, Heap.peek, the find functions, each with its reference page and tests.
  4. Extensions. mut reference-carrying locals, reference-carrying parameters, then pipeline next/find/reduce over &T items.

Each step is usable alone; step 1 with Option alone would already serve user code.

Open questions

  1. Scope: Option only, or Option and Result? Recommendation: both, with nesting, because fallible borrowed lookups are common and the two share one implementation.
  2. May Err carry a reference? Recommendation: no; see Err payloads.
  3. Nesting Result[Option[&T], E], including Ok(found) at a return. Recommendation: allow. It is still one value produced by one call, and rule 4 continues to reject every stored position.
  4. Local construction (found = Some(&xs[0])). Recommendation: allow in phase 1; it is r = &xs[0] plus a tag. mut reassignment waits for phase 2.
  5. Reference-carrying parameters. Recommendation: allow in phase 2, with the parameter counting toward origin selection.
  6. Builtin names: get_ref/get_mut, or change get to borrow affine values? Recommendation: new names. One method whose result is a copy for i64 and a borrow for list[i64] would hide which one the reader has. Decided: get/peek copy and get_ref/peek_ref/*_mut borrow, in every container, including std.cache.lru.
  7. Trapping accessors: replace or keep beside get_*? Recommendation: change Heap.peek to return Option; keep graph.node/edge trapping like indexing and add get_node_ref/get_edge_ref; remove the LRU callback and trapping getters.
  8. unwrap_or with a reference default. Recommendation: reject, as conditional reference expressions are.

Typestate and owned transitions

Status: Investigation complete. This explores existing facilities and limits, not a new language feature. Moves and destruction define current behavior. Scheduling belongs in B72.

The owned response experiment refreshes this assessment, with native cleanup/retry checks and compiler rejections. Explicit owned receivers now support method-shaped transitions, and field moves out of owned bindings are implemented (issue #534).

The separate linear-obligations proposal (issue #149) addresses non-abandonment of an owner, including retry errors. Typestate can remain affine when abandoning a state is harmless. See the integration assessment for a protocol example combining state transitions with obligations, worker effects, and execution responsibility.

What Plenty can express now

Typestate makes an operation available only for an object in a particular state type. Distinct nominal classes plus consuming free functions express a basic form today: prepare(draft: Draft) -> Ready moves the draft, and send(ready: Ready) cannot accept an unprepared draft. The ownership checker prevents reuse of the consumed binding. Private fields/constructors and module visibility must prevent callers fabricating the supposedly validated state.

A small Draft to Ready probe type-checked with the available local debug compiler during this assessment. This was a smoke check of that binary, not a fresh build or an exhaustive typestate conformance test. The design basis is the current ownership, classes, and function contracts.

Ordinary methods borrow their receiver by default. Explicit self: Class or self: Enum uses the same by-value rules as a function argument. For affine states, a successful call or an error both consume the previous binding; a retryable error must explicitly carry the owner back. Copyable enums still copy. There are no specialized method declarations for Connection[Open] versus Connection[Closed]; distinct classes remain the conservative starting point.

Generic instance methods are now checked only when used (issue #515), so an unused method need not work for every argument type. This improves flexibility but does not add explicit state-specialized method sets. Owned receivers are an independent feature, documented with method contracts.

Moving an entire owner is supported, and so is moving an affine field out of an owned binding (issue #534; see field moves). A transition that rebuilds a wrapper around an owned connection can move the connection field out of the consumed state. A class with __del__ still needs an Option plus replace, an enum whose payload can be consumed, or API restructuring. That friction is relevant to whether a typestate design is pleasant, not a reason to weaken initialization guarantees indiscriminately.

The HTTP client now demonstrates the API-restructuring option (issue #134, commit 2772825). run returns the response head and an Incoming body owner as tuple components. fetch can consume the body and move the head into its result without constructing a fallible placeholder; send assembles the public Response only when needed. attempt likewise carries its head separately. This removes the observed placeholder work using existing tuple moves, without adding partial class moves or compile-time response states.

Actual stdlib use

Area at baseline bef41b1Existing approachAssessment
http.client.Response, Client.finishResponse owns the connection; finish consumes itOwnership expresses a lifecycle boundary, but body completion is checked at runtime
http.server.Exchange.start/write/endOne class with integer state, remaining length and flagsNot compile-time typestate; invalid order returns an error
http.BodyReaderRuntime state machine over input-dependent chunk framingRuntime states are natural; enums can improve internal invariants
json.DecoderRuntime token/Unicode/container statesArbitrary input cannot reasonably become a compile-time sequence of state types
json.Builder.FramePayload enum for array/object accumulationGood state representation, distinct from public typestate
db.sqlite.Transaction.commit/rollbackMutable methods; a busy commit leaves the transaction retryableResource ownership with dynamic lifecycle, not consuming state transitions

The stdlib therefore uses ownership more broadly than typestate. An enum eliminating an invalid combination of internal fields and a public API rejecting an invalid operation sequence at compile time are complementary techniques.

Useful experiments and tradeoffs

For server responses, compare an API where beginning a response consumes an unanswered handle and returns a writer, and ending consumes that writer. This can rule out writing before start or ending twice. Runtime checks still enforce declared content length, I/O failures, and cancellation. Separate size/chunked writers only if they simplify callers rather than multiplying types and generic parameters. Existing handlers borrow Exchange, so a consuming design also requires deciding how request-body access and handler ownership interact.

For builders, use distinct stages only for meaningful required choices. Optional configuration normally suits a single options class with validation. An explicit constructor establishing JSON decoder invariants is preferable to a 28-stage builder mirroring fields.

Failure ownership is part of every transition’s contract. A failed consuming transition can destroy the resource, or return the still-valid owner alongside the error for retry. SQLite’s busy-commit behavior demonstrates why blindly changing a mutable method to a consuming function can lose useful behavior. Cancellation and partial external effects likewise need explicit outcomes.

Avoid placing every connection phase in a distinct public type if callers must immediately put those types back into an enum for queues, pools, or runtime routing. Compare end-to-end handler code, not just the impressive compile-time rejection of one small example.

Validation before adoption

For each experiment, demonstrate one useful compile-time rejection, legal retry after failure, preservation of partial-progress reporting, and exactly-once cleanup. Check that public construction cannot forge states. Measure new public types, caller branching, placeholder replacements, and code size. Keep runtime enums for input-driven parsers; prototype typestate at coarse ownership boundaries. No blanket stdlib conversion is proposed.

Linear obligations

Status: Not implemented; proposed design for review. Issue #149, tracked under B72. No syntax or standard-library conversion is approved. The current contracts remain ownership, destruction, and error propagation.

This proposal defines an opt-in ownership obligation. Its aggregate and error rules are part of the same semantic core. A separate scope-resolution proposal adds automatic synchronous finishing. The integration contract compares proposal boundaries, effects, typestate, and future concurrency work. All code fences in these pages are pseudocode, including names, annotations, signatures, and outcome types; none are runnable Plenty examples.

Purpose and guarantee

An affine owner may be consumed at most once and may be discarded. A linear owner must be consumed exactly once on an ordinary completed execution path: resolve its obligation through a permitted terminal operation, or transfer it to another owner that inherits the same requirement. Reads, borrows, and intermediate operations are not consumption. They may happen repeatedly.

This makes obligations such as responding to a request or finishing a local transaction visible across calls and storage. “Run the entirety of a state machine” means not abandoning an outstanding obligation before an allowed terminal outcome. It does not mean visiting every state, forcing success, or proving termination. A retry loop may run forever while retaining its owner.

The goals are to prevent silent abandonment, preserve explicit failure ownership, and let ordinary affine programs retain their behavior. Non-goals are async destruction, general effect syntax, state-specialized methods, blanket stdlib migration, or stronger unwinding. Traps and process aborts remain outside the guarantee. It cannot establish fairness, deadlock freedom, successful I/O, knowledge of remote state, or recovery after a crash.

Baseline checked for this proposal

The reference and implementation were inspected at local commit 82544a4. The source locations below are audit pointers, not additional specifications.

Current behaviorEvidence and consequence
Classes, collections, captured closures, and aggregates with affine components move; ordinary scope exit may destroy themOwnership reference above; src/ownership.rs. Moving already transfers ownership, but does not forbid abandonment.
Cleanup covers return, ?, break, continue, replacement, and expression temporariesDestruction reference above. __del__ cannot return a recoverable error or yield; suspended generator drop does not resume its body.
? may erase an error into Failure.Unspecified, destroying the originalError reference above; src/sum.rs. The unused-Result diagnostic does not cover unused bindings or wrappers and permits explicit drop.
Owned with calls borrowed __exit__ -> (), then drops its managerResource scopes; src/frontend/contexts.rs. This is not yet a consuming resolver.
Worker checking follows reachable native effects, concrete callbacks, and destructorsEffects; src/effects.rs, src/threading.rs. It rejects uncertified indirect/foreign calls, including through stdlib helpers.
Dropping a future does not cancel or wait; its worker retains execution responsibilityFutures, executors; plenty-runtime/src/executors.rs, especially run_worker, stop, and destroy_job. Cancelled captures and unclaimed results can still be destroyed.
SQLite commit/rollback borrow the transaction; a busy commit permits retrystdlib/std/db/sqlite.plenty, Transaction; typestate exploration. Existing fallback rollback is not a new linear contract.

Proposed ownership rules

Introduce a discardability capability independent of ordinary copying. Discardable[T] below is a schematic capability name, not chosen syntax. Existing types remain discardable unless they contain an obligation or opt in to nominal linearity. A linear value cannot be implicitly copied or duplicated by copy; explicit duplication of a resource must create separately specified obligations. A borrow is not an owner and does not acquire this obligation.

The checker tracks live obligations through its ownership control-flow graph. An owned argument transfers responsibility before the callee runs; the callee’s body must itself resolve or transfer it. Returning an owner is a valid transfer, including in an error. Passing it to an empty function is not an escape hatch: that function’s exit is rejected. Tail calls must resolve or transfer all other caller obligations before entering the callee.

SiteProposed rule
End of block/function, explicit return, or ? failureEvery owner in the scopes left must be resolved or transferred. Check the error path separately from success, including pending evaluated operands.
Branch joinCheck each outgoing path. An obligation cannot disappear on one path; represent conditional remaining ownership explicitly with a sum. Existing moved/possibly-moved availability still governs subsequent uses.
Loop backedge / continuePreserve the existing outer-owner availability rule. Resolve iteration-local obligations before leaving their scope; an outer obligation can remain in its slot across iterations, or be transferred and replaced with a new valid owner.
break and zero iterationsCheck the exited scopes and each resulting join. Do not infer that a collection was drained merely because its loop ended.
Ignored result, wildcard binding, expression temporaryReject implicit destruction of an obligation, even when no named binding exists. Observing a temporary does not consume it.
Assignment or replacementPlain assignment cannot destroy an old obligation. An exchange returning the old owner may transfer it; the returned owner must then be handled. Check failure while evaluating the replacement first.
Explicit drop, aggregate destruction, fallback __del__Require discardability. Memory reclamation alone is not an authorized protocol finish.
Ordinary allocation/call failureResponsibility stays with a live caller owner, returns in the error, or has already been resolved under an explicit contract. It cannot vanish during rollback of partially constructed values.
Fatal trap, abort, or nonreturning computationNo unwinding or liveness promise is added. A function which never returns has not completed an ordinary path.

Borrow checking continues to forbid consumption while a conflicting loan is live. A mutable borrower cannot silently remove the referent’s obligation; ordinary replacement must leave a valid owner. Resolving a borrowed value into a different state requires an explicit checked state/storage contract, not an assumption that existing &mut methods can consume their receiver.

Construction and terminal authority

Nominal opt-in must carry an abstract obligation, not merely rely on having a linear field. Otherwise unpacking an all-integer Transaction could silently turn it into ordinary values. Conceptually it has a noncopyable seal whose ownership follows transitions; the seal need not occupy runtime storage.

Seals and registry-authorized terminals are this proposal’s conservative design candidate, not prerequisites of every linear type system. An alternative could let each type’s defining module declare terminal operations as part of its trusted abstraction contract. That is more extensible but expands the authority to assert meaningful completion. The registry approach makes the initial trust boundary auditable at the cost of limiting new standalone resource protocols until a trusted-extension mechanism is designed.

Safe construction introduces that seal only through the type’s authorized constructor. Private fields/constructors prevent fabricated protocol states. Representation unpacking transfers every field and the seal; it does not resolve the seal. A safe transition passes it to its successor. An ordinary consumer, including one in the defining module, cannot discard it.

Terminal elimination needs a small, explicit trusted boundary. Initially the compiler/runtime contract registry may authorize particular constructors and terminal primitives, with an ownership/effect contract for each outcome. Safe wrappers must call those primitives or transfer the seal; an empty wrapper is rejected. Merely naming a function commit, marking it consuming, or adding an empty destructor grants no terminal authority. There is no safe general forget, user-declared empty eliminator, or FFI cast that removes an obligation.

This cannot prove the external truth of a primitive’s claim: the implementation of “response sent” or “local transaction released” belongs to the trusted resource contract. Incorrect trusted annotations can invalidate the guarantee, just as incorrect native ownership metadata can today. Review and test that boundary explicitly. A user-extensible trusted-declaration mechanism remains open; this proposal does not invent a general unsafe language or let arbitrary safe libraries mint terminal witnesses. Ordinary libraries can compose existing certified obligations without new authority.

Field extraction is a real dependency for ergonomic wrappers. Moving a field out of an owned binding is now implemented (issue #534; see moving fields), except for classes with a custom destructor and through references. All-fields deconstruction is not implemented; the smallest candidate remains consuming deconstruction within the declaring module, preserving any nominal seal and allowed only when no custom destructor must still observe the whole. General partial initialization, partial moves through borrows, and destructors on half-moved objects are outside this proposal. Whole-tuple moves and consuming enum matching can support early experiments without that extension.

Implementation and conformance outline

First derive discardability and obligation containment recursively in type metadata, then attach outstanding-owner information to ownership checking and every cleanup edge. Add capability checks at generic instantiations and native operations before allowing linear values into existing containers or workers. Validate return/error ownership and terminal annotations before lowering cleanup. Only proven discardable storage may reach ordinary generated drop glue. Audit temporary, constructor, callable, and allocation rollback paths as well as visible drop calls; suppressing a destructor would merely leak an obligation.

This ordering describes semantic dependencies, not a second work queue; B72 remains the planning record. Runtime resource conversions need their own API decisions and are not included in this documentation change.

Positive conformance caseNegative case / diagnostic requirement
Borrow a transaction repeatedly, then finish itExit with it live: identify the acquisition, remaining obligation, and exact exit.
Return the owner or transfer it to a checked helperEmpty helper: report its owned parameter and unresolved function exit.
Resolve on both branches, or return remaining ownership in a sumResolve only on one path: show the unresolved predecessor, not merely “unused value.”
Retain an outer owner through a retry loop; finish iteration-local ownerscontinue leaks a local or moves an outer owner without replacement.
Exchange an owner and resolve the returned old valueOverwrite, ignored temporary, or drop: identify the implicit/explicit discard.
Pass the same seal through a valid state transitionReuse after move, fabricate a state, or discard a deconstructed seal.

The companion pages add error, aggregate, scope, and worker cases. Native tests must inject allocation and submission failures and count transfers/terminal calls, including cancelled work, so every acquired seal is either returned or resolved once. Compile-fail tests must cover implicit cleanup as well as source operations. These are future conformance requirements, not tests run for this proposal.

Alternatives and open decisions

An expanded unused-value warning is cheaper but cannot enforce obligations across wrappers and calls. Affine RAII remains preferable when silent fallback cleanup is an adequate resource contract. Requiring all owners to be linear would impose unnecessary handling on ordinary containers and existing programs. Typestate alone and a combined effects redesign are assessed in the integration contract.

Proposed decisions are opt-in linearity, recursive non-discardability, checked transfer, and explicit terminal authority. Open design choices are declaration and capability syntax, how trusted contracts may be extended, the smallest consuming-deconstruction surface, and how much path-sensitive empty-container proof to expose. None may weaken error-path or aggregate preservation. Async completion and general effect polymorphism are explicitly deferred.

Linear values, aggregates, and errors

Status: Not implemented; companion to linear obligations. Issue #149, B72. This page is part of the core ownership proposal, not an independent feature. The current sums and ownership contracts remain authoritative. All examples are speculative pseudocode.

Wrapping preserves responsibility

Discardability is structural and conservative: a tuple, enum, class, box, collection, closure, or error that can own a linear value cannot be freely discarded. A nominally linear wrapper additionally retains its own seal. Wrapping, allocating, or erasing representation must not make an obligation disappear. Recursive types need a fixed-point containment calculation.

OwnerPermitted handling / required restriction
TupleMove as a whole or unpack every component; ignored linear components are errors.
Enum / Option / ResultMatch by consumption and handle each payload. A payload-free arm can finish normally after the match proves that arm.
Class / boxTransfer whole owner, or use checked consuming deconstruction. A custom destructor is not permission to drop linear fields.
List / dictionary / setInsert and remove by transfer with ownership-preserving failure. Clear, eviction, replacement, duplicate-key/value rejection, and container drop require discardable discarded items or an explicit resolver.
Capturing closureCapturing by move transfers the obligation. A one-shot call must resolve/return it; destroying an uncalled closure is rejected. Borrowed captures keep existing lifetime rules.
Reusable closureInvoking it does not consume its environment. A linear capture remains owed even after calls; an explicit final consuming operation would be needed.
Lazy generator / erased callableReject linear captures, retained locals, or erased obligations until their destruction and suspension contracts preserve them. The current generator drop does not run the rest of its body.

Option[Transaction] is conservatively non-discardable even if a particular value is Nothing. Consuming matching proves the payload-free branch locally; it does not grant the entire type discardability. Similarly an empty linear collection needs a checked consuming empty-finalization operation, rather than a rule that drops arbitrary list[Transaction]. Returning an iterator’s unconsumed remainder preserves ownership; early break from an iterator that silently destroys the remainder must be rejected. Borrowed iteration is useful without consuming anything, and consequently does not finish the collection.

Existing generic bodies that may discard a T need a Discardable[T] requirement. Transfer-only identity, pairing, and error forwarding need none. Instantiations with ordinary existing types retain today’s behavior. Initially the existing specialization machinery can check the requirement at use and point to the discarding body; explicit bound syntax and declaration-time generic checking are separate ergonomic decisions. Containers themselves need not require discardable elements: only operations which destroy elements do.

This affects builtin APIs, not just user-defined generic functions:

  • unwrap_or(default) discards the default when a present payload wins; the existing signature needs discardable T. Use an explicit match that handles both owners, or a new API returning the unselected alternative.
  • ok_or(error) drops the error on Some, so requires discardable E; it may still transfer a linear T into Ok. Construction of a temporary option or error follows the same failure rules.
  • is_ok() and similar observers do not discharge obligations. Observing a linear temporary and then destroying it is an error.
  • map_error must check the mapper’s handling of the owned error. Mapping a live-transaction error to an ordinary code cannot silently erase its payload.

Allocation and partial construction

A consuming allocating API cannot simply return AllocError after losing its input. It must reserve capacity before transfer, preserve caller ownership via an explicitly checked operation, return the owner in the failure payload, or invoke an authorized resolver under a documented policy. An ordinary owned call transfers its arguments: restoring a moved caller binding is not implicit.

For example, speculative insert_owned(list, item) may return either the updated list or (list, item, AllocError). An insertion through a borrowed list can return the uninserted item on failure. Dictionary insertion must return or resolve a replaced value and any unused incoming key. Constructors must account for earlier initialized fields when a later field expression propagates error. Closure allocation failures must return owned captures or occur before transfer. Hidden builders and argument temporaries are subject to the same rules.

For an existing native operation whose failure path releases an argument, reject linear instantiation until its contract is extended. “The allocator failed” is an ordinary error, not permission to abandon protocol state.

Transaction ownership on failure

Consider the intentionally rejected pseudocode:

tx = begin_transaction(connection)?
insert(&mut tx, record)?       # error returns while tx is still owed
commit(tx)?

Handle insertion failure by explicitly rolling back, returning an error that owns tx, or entering a checked resolution scope before the fallible operation. Acquisition must also specify ownership on failure: a consumed connection is returned or terminally closed; a borrow stays with its owner. No hidden success-only ownership contract is sufficient.

Commit APIs have two different valid shapes:

commit_local(tx: Transaction) -> Result[Receipt, TerminalCommitError]
# Every result has resolved the local obligation; no retry owner survives.

commit_retryable(tx: Transaction) -> Result[Receipt, CommitError]
CommitError = Retry(Transaction, DbError) | Finished(TerminalCommitError)
# Retry retains the live transaction; Finished does not.

These are alternative contracts, not two names for the current SQLite method. The existing typestate assessment explains why SQLite’s busy commit must remain retryable. Its current mutable method already preserves the transaction; a consuming redesign must carry it in the retry result. The error enum above is itself non-discardable.

match commit_retryable(tx):
    Ok(receipt): return Ok(receipt)
    Err(Retry(tx, cause)):
        return Err(NeedsDecision(tx, cause))   # caller now owns the obligation
    Err(Finished(cause)):
        return Err(TerminalFailure(cause))

A matching Result error type may propagate this obligation with ?, provided other live locals are handled. Propagation to current Failure.Unspecified must be rejected when it would destroy an obligation. A diagnostic should name the retained transaction, the error conversion, and the original error type; suggest preserving the typed error or resolving its owner first. Matching a terminal arm can still permit ordinary error erasure afterward.

Rollback is not automatically infallible. A retryable rollback must likewise return ownership; an authorized local-release operation can instead finish with a reported failure. “Local handle released” and “remote rollback confirmed” are different contracts. For a remote transaction with lost acknowledgements, an UnknownOutcome may be a terminal local outcome if the API explicitly permits it. If reconciliation is required, it must carry a live recovery obligation. The type system cannot invent knowledge the protocol lacks.

Conformance probes

Accept typed retry errors passed through several helpers, complete enum matches, owned removal with remainder returned, and allocation failure returning every input. Reject nested wrappers dropped through a generic helper, a discarded closure containing a transaction, wildcard patterns hiding obligations, replacement of a linear dictionary value, and partial iteration losing its remainder. Test both unwrap_or branches and ok_or’s unused error path.

Failure injection must cover every allocation point after the first transfer, including partially built objects and closures. Diagnostics should trace the containment path (for example SubmitError.job.capture.tx) and the cleanup edge, rather than merely saying a large aggregate is noncopyable. Discardability is the missing capability; copying and borrowing are separate concerns.

Synchronous resolution scopes

Status: Not implemented; separate proposal depending on linear obligations. Issue #149, B72. Current resource scopes and destruction do not provide the consuming-finalizer contract proposed here. All examples are pseudocode.

Why a separate extension

Manual transfer or resolution is a complete synchronous linearity rule, but a transaction around several fallible calls needs repetitive handling of every early exit. A manager can take responsibility, lend access to the body, and run an authorized resolver before the scope returns. This is cleanup policy on top of ownership; neither a new effect language nor async cleanup is necessary.

Python context managers illustrate the related control-flow guarantee: following successful __enter__, ordinary body exits invoke __exit__. That mechanism alone neither checks non-abandonment across calls/storage nor requires every resource to enter a context. It does not statically stop escaped aliases, promise cleanup success, or survive abrupt process termination. Plenty already adds borrowing restrictions; linearity would additionally check the manager’s own responsibility.

Proposed narrow contract

A resolution scope owns a hidden linear manager and registers a consuming exit operation before lending body access. On each ordinary exit, body-local owners finish first, borrowed access ends, then the manager is consumed exactly once. The operation must resolve or transfer its payload on every returning path. Its normal receiver is owned, unlike today’s borrowed __exit__.

with resolving(tx, release_local_transaction) as access:
    insert(access, record)?
    update(access, other_record)?
# every ordinary body exit consumes the manager and resolves tx

The illustrative resolver is explicitly a local-release policy that may attempt rollback but promises no remote success. There is no implicit commit-on-success policy. A library can offer a clearly named rollback/local-release guard; the language should not guess the business meaning of normal scope exit.

Acquisition and registration form one ownership contract: failure returns the unregistered owner, or occurs before transfer. Once registered, even a failed entry operation must return the owner or run the resolver. Do not copy today’s successful-entry rule in a way that leaks an already-owned obligation. Nested scopes finish in reverse order. If a later manager acquisition fails, earlier registered managers still resolve, and the failed acquisition’s returned owner must itself be handled.

The body receives a loan bounded by the manager. It cannot move the manager, replace its unresolved payload, retain the loan past exit, or return references to owned manager storage. Return values may be prepared before cleanup only if they obey these lifetime rules. A body-local linear owner is not automatically the manager’s responsibility unless explicitly transferred into it.

An explicit consuming finish inside the body needs a checked manager-state transition that proves the exit operation has nothing left to resolve. The first design may reject this and require manual ownership for commit/retry flows. Assuming that a boolean finished flag makes a linear field disappear would undermine the core rule. Similarly, borrowed with &mut existing cannot consume the caller’s owner; adapting it requires a separate state contract.

Worked example: cancel_on_exit()

token.cancel_on_exit(), implemented for issue #355, is an instance of the narrow unit-exit form specialized to one resolver. Its manager owns a share of a CancellationToken; its exit operation is cancel(), which is synchronous, infallible, returns unit and allocates nothing. Every ordinary exit, including ?, return, break, continue and a failed later manager in the same header, runs it exactly once, in reverse manager order:

with spawn(serve, listener, stop.share())?, stop.cancel_on_exit():
    prepare()?
# every exit cancels stop, then joins the task

The compiler check that pairs with it is the cancellation rule: when a function later cancels a token, a join or pool shutdown that waits for work holding it must come after a cancel on every path. The guard satisfies that rule by construction. It does not need linearity: the obligation is local to one function and its exits, and cancel() cannot fail, so there is no unresolved owner to return. It shows the shape a library resolver could take once consuming exits exist, and why the infallible case is the first one to make automatic.

Failure determines the available surface

The narrow automatic-exit form requires a synchronous resolver that returns unit after reaching an allowed terminal local outcome. It may block, and it may internally retry. If the only correct resolver returns a still-live owner on failure, this form is unavailable. Swallowing that failure is not resolution.

Today’s unit-returning __exit__ is sufficient to express an infallible side effect, but not sufficient to prove that a borrowed manager’s linear payload was consumed. A dedicated consuming-finalizer lowering (or a later verified state-changing exit contract) is necessary. The hidden owner must not subsequently reach ordinary drop with a live seal. This extension does not change all existing managers or reinterpret existing __exit__ hooks.

Fallible finishing should initially remain explicit. A future combinator can make error precedence and retained ownership visible instead of silently overriding a pending return or propagating an exit error through today’s with:

resolve_scope(owner, body, resolver) -> ScopeOutcome[BodyValue, BodyError, R]
ScopeOutcome = Resolved(Result[BodyValue, BodyError], ResolutionReport)
             | Unresolved(Result[BodyValue, BodyError], R, ResolveError)

Unresolved retains the owner R; the whole outcome is non-discardable. Both body failure and resolution failure survive. If the body value/error is itself linear, it remains owed in either branch. If allocation is needed to store an outcome, its failure contract must retain these values too. The body borrows the owner and returns a result to the combinator; it cannot return a borrowed view of that owner. This sketch does not introduce nonlocal callback returns.

This explicit outcome is the recommended direction for recoverable resolution failure, but callback syntax and a library implementation are open. A future fallible with syntax would need equally explicit rules for simultaneous errors, prepared return values, nested manager failures, and unresolved owners. It is not a prerequisite for core linearity or the narrow unit-exit form.

Destructors, effects, and execution limits

An ordinary __del__ remains fallback memory/resource cleanup. It cannot be used to launder a linear field into discardable storage. After terminal resolution, remaining ordinary fields may be destroyed in their existing order. This avoids requiring recoverable errors, suspension, or a stronger destructor model merely to express meaningful finishing.

Resolver calls, including compiler-inserted ones, belong in the existing reachable-effect graph. A resolver forbidden on a worker makes that scope ineligible there; see the integration example. Opaque higher-order resolver effects remain subject to current certification limits. There is no generic “cleanup is always allowed” exception.

No bounded completion is promised: scope exit can wait indefinitely. An immediate timeout return and completed cleanup cannot both be unconditional. Transferring responsibility to a longer-lived scope is possible only if it owns all needed resources and retains no loans whose owners have already returned. Async cleanup expressibility and the runtime obligation to drive it need a distinct proposal; neither Python async with nor a hypothetical async destructor alone supplies that obligation.

Conformance and unresolved choices

Test normal return, ?, break, continue, nested acquisition failure, and explicit body failure with a counted resolver: exactly one permitted resolution for each registered owner, body loans ended first, and unchanged cleanup order for ordinary values. Reject escaping loans, destroying an unentered manager, calling an empty resolver, consuming through a borrowed manager, and a resolver returning while still owning its payload. Verify worker diagnostics include inserted exit calls and their origin scopes.

The consuming-exit declaration surface, safe explicit finish within a scope, and the fallible outcome combinator remain open decisions. The proposed defaults are explicit policy, no implicit commit, no ignored recoverable resolution error, and no async requirement.

Linearity, typestate, and execution responsibility

Status: Not implemented; integration assessment for issue #149. Planning remains in B72. This page connects linear obligations, resolution scopes, and the existing typestate exploration; it does not approve an effects or concurrency redesign. Examples below are speculative pseudocode.

Recommendation and boundaries

Use small proposals with explicit interfaces: a linear ownership core (including its inseparable aggregate/error rules), a separate synchronous resolution-scope extension, and this integration contract. Keep the existing typestate exploration independent. No new effect syntax is required for concrete synchronous consumers. Propose effect-polymorphic resolution or suspension only when a concrete use cannot fit current certification or execution ownership.

Organization consideredAssessment
One linear-types proposal, existing typestate exploration, narrow effects sectionViable for manual synchronous resolution. Automatic scope policy and future execution ownership would make the document carry separable decisions.
Separate linearity/typestate/effect extensions with a small common contractPreferred where there is an actual extension. Keep typestate’s current assessment; split scope policy now. Do not manufacture an effects extension when reachable-call checking suffices.
Combined linearity, typestate, effects, and concurrency proposalShared examples help, but there is no demonstrated semantic need for one adoption unit. It would force syntax/runtime decisions that synchronous linearity does not depend on.

The pages form independent review units where possible, not claims of complete implementation independence. Aggregate cleanup and failure ownership cannot be deferred from the linear core: a single discarded wrapper defeats the guarantee. Scope resolution depends on core preservation and terminal authority. Worker support depends on auditing runtime discard paths. Async completion depends on a responsible execution owner and suspension rules; it is not a core prerequisite.

Responsibility and dependency matrix

ResponsibilityImplemented baselineProposed owner / dependencyDeferred
Ownership and permission to discardAffine moves, borrows, destructionLinear core adds non-discardability, seal transfer, and checks at every cleanup edgeGeneral user-defined trusted terminal declarations
Legal state transitionsNominal classes, consuming functions/methods, runtime enumsExisting typestate exploration; linear states additionally preserve sealsState-specialized generics
Permitted effects / worker eligibilityReachable native-effect walk, concrete callable/destructor checksReuse it for terminal calls and inserted resolvers; no stdlib exemptionGeneral indirect-effect certification / effect polymorphism
Cleanup policyDeterministic fallback drop, unit borrowed __exit__Separate consuming scope resolver; policy explicitly namedFallible exit syntax and async destructors
Execution responsibility / suspensionNative worker retention, joins, cooperative cancellationInitially restrict discard-capable APIs; new completion contract needs runtime ownership auditSuspended computation ownership and driving async cleanup
Failure ownershipResult payload transfer, lossy Failure erasure, owned submission errorsLinear core preserves live owners in errors and allocation rollback; each API defines terminal versus retry outcomesAutomatic recovery policy and remote-state certainty

Counterexamples that test the split

A synchronous transaction with commit(tx) and rollback(tx) can carry a linear obligation using one nominal type and consuming free functions. It needs neither state-specialized methods nor effect annotations. Conversely, prepare(Draft) -> Ready is useful affine typestate even if discarding a draft or prepared value is harmless. Linearizing every state would add obligations without a corresponding product requirement.

Effects determine where an operation may execute, not whether it eventually executes. A worker-certified rollback may block forever; a locally permitted main-thread-only resolver can be meaningful even though worker certification rejects it. A function marked consuming is still not a terminal operation unless its ownership contract accounts for the seal and all failure paths.

Linear typestate with explicit errors

Consider a protocol whose contract requires a response or completed local cancellation after acceptance:

send(Request) -> Result[InFlight, SendError]
SendError = NotAccepted(Request, Cause) | Uncertain(InFlight, Cause)
receive(InFlight) -> Result[Response, ReceiveError]
ReceiveError = Retry(InFlight, Cause) | Finished(TerminalFailure)
request_cancel(InFlight) -> Result[Cancelling, CancelStartError]
CancelStartError = Retry(InFlight, Cause) | Finished(TerminalFailure)
finish_cancel(Cancelling) -> Result[Cancelled, CancelFinishError]
CancelFinishError = Retry(Cancelling, Cause) | Finished(TerminalFailure)

Here Request is affine before acceptance; InFlight and Cancelling are linear and carry the same obligation through transitions. Response, Cancelled, and TerminalFailure are ordinary values because this example’s contract considers their outcomes terminal. Another protocol may require a linear response body or recovery token instead. Uncertain acceptance cannot be reported as definitely unstarted when work may already exist remotely.

Typestate rejects calling finish_cancel on InFlight or receiving after the handle becomes Cancelling. Linearity rejects simply returning after a cancel request or discarding Retry(Cancelling, cause). Either mechanism alone misses the other’s error: an affine state may be abandoned; one linear runtime-state handle may still reject operation order only at runtime. Together they prevent abandonment and statically restrict these transitions, without proving that finish_cancel ever returns or that a remote peer received the cancellation.

Constructors must not let clients forge accepted or cancelled states. Moves prevent reuse of old handles, and terminal authority defines allowed outcomes. Use consuming free functions in experiments; current ordinary methods borrow receivers. For queues and runtime routing, an enum owning either linear state may be simpler than carrying state parameters through every helper. Input-driven parsers generally still suit runtime enums. See the existing typestate page for HTTP response lifecycles, field-extraction friction, and SQLite retry behavior.

Worker effects and hidden cleanup

Suppose an explicitly chosen resolver restores a process working directory:

restore(scope: DirectoryObligation) -> ():
    while True:
        match std.env.set_current_dir(scope.saved_path):
            Ok(()):
                authorized_restored(scope)
                return
            Err(cause):
                record_error(cause)           # retain scope and retry

worker(owner: DirectoryObligation):
    with resolving(owner, restore):
        compute()

This worker must be rejected: its required exit reaches a process-global effect. The example deliberately retries restoration failures and may never return; it does not silently treat failed restoration as success. The current effect checker already follows ordinary calls and destructors; generated consuming resolver calls must be visible in that same graph. A diagnostic should identify the submission/spawn, captured obligation, resolution scope, and call chain to set_current_dir, with the existing reason that it affects every thread. Report an ownership error as well if a proposed alternative merely drops the owner to avoid the forbidden call.

Do not classify all linear owners as main-thread-only. In principle a worker can compute while borrowing the data and leave resolution with a parent, or return the obligation through an ownership-preserving join protocol. Reject at worker certification when the reachable body/required cleanup needs the effect, and at transfer when the chosen worker API cannot preserve ownership on all paths. For the initial core, existing APIs that can discard such owners are ineligible regardless of whether the particular body appears safe.

Concrete resolvers with known bodies need no new effect annotations. A generic resolver specialized to a concrete callable can use existing certification only where the resulting calls are inspectable. Current indirect and boxed callable restrictions still apply; an erased resolver or unresolved higher-order call cannot claim worker safety just because its type says it consumes an owner. Such use needs either a statically inspectable adapter or a separately justified effect-contract extension. The limitation must appear at the requiring worker use and the hidden resolver call, not as an exemption for library cleanup.

Native workers and future completion

Current Future is affine. Dropping it neither cancels the job nor waits: accepted running work retains a worker owner and proceeds. Executor shutdown joins running workers, and ordinary shutdown drains accepted queued jobs. Cancellation of running work is cooperative; setting a token does not force termination or change a future’s result by itself.

These facts are useful execution machinery but do not make every stored value linear-safe. Preserve today’s APIs for discardable data. The first linear core should reject linear captures/results in existing thread, executor, channel, and lazy-frame paths that can destroy them, until a path-complete contract is implemented. This is a proposed restriction for new types, not a claim that today’s compiler knows Discardable.

Existing pathObligation-preserving extension needed before admitting linear values
Thread-start failure destroys moved inputsReturn complete inputs or invoke an authorized failure resolver.
Executor submission failsSubmitError[F] already returns the complete job; preserve its captures, reject lossy erasure/drop, and audit allocation before transfer.
Queued job cancelled / shutdown(True)Current cancellation skips execution and later destroys captures. Return them to a responsible owner or run a certified resolver; marking the future done is not resolution.
Job runs normallyIts body and reusable-environment cleanup must resolve/transfer every capture, on success and failure.
Public future dropped / result unclaimedCurrent runtime may destroy the result. Require a new non-discardable result handle or an explicit runtime-owned result resolver.
Scope exits / executor shuts downJoining proves workers returned, not that a returned obligation was handled. Retained result storage needs a continuing owner.
Channel disconnect, queued-message cleanupReturn or resolve linear messages; destroying a queue is not enough.

A possible Completion[T] would specify those responsibilities explicitly; it must not silently redefine Future[T]. A joinable synchronous implementation could precede async/await. Submission failures preserve the job; cancellation must return it or complete its resolution; terminal results transfer to the consumer or an authorized resolver. All hidden resolvers require effect checks on the thread where they actually run, and must run without scheduler locks.

In future async work, moving a completion into .await merely transfers the obligation to the suspended frame. Dropping that frame cannot abandon it. A responsible scope/runtime must continue driving the computation or its cancellation to an allowed terminal outcome. async with or an async destructor provides a place to write suspending cleanup; neither ensures someone keeps executing it. The separate async/await exploration now examines suspension and execution ownership, but its affine spike does not support linear values or establish non-abandonment. Such syntax still needs a complete obligation-preserving runtime contract before claiming that guarantee.

A timeout can return immediately only by retaining responsibility elsewhere or returning the live owner. The responsible scope must retain all required owned resources and outlive any retained loans; a background cleanup cannot use a borrow into a caller that already returned. Completion can still wait forever, deadlock, or fail remotely. No fairness, termination, crash recovery, or bounded cancellation guarantee follows from linear ownership.

Integration checks before adoption

Demonstrate a synchronous linear transaction without new typestate/effect syntax, and an independently useful affine typestate API. Test the protocol above for illegal order, dropped retry states, and legal ownership-preserving retries. Worker compile-fail cases must include direct consumers, helper chains, inserted exit calls, and uncertified indirect resolvers. Positive worker tests must use permitted effects and an audited transfer protocol.

For every new worker protocol, inject start/submission failure, cancel queued work, drop the result consumer, exercise shutdown with pending/running work, and verify captures/results are resolved or returned exactly once. Include timeouts that return a live owner and reject a scope escaping required loans. Do not use a successful join or a cancellation flag as evidence of protocol completion. These checks bound any later implementation; this proposal adds no runtime or language behavior.

Protocol method bodies: defaults, final methods, and private hooks

Status: core design agreed; not implemented. This is the implemented protocol methods proposal: “implemented” means a method with a body, not a feature that has shipped. The compiler still requires protocol methods to have pass bodies; current protocols remain authoritative. Future prioritization belongs to B29.

Start with the agreed rules and short programs. The feasibility comparison below and implementation evidence record the investigation against d39e6ec on 2026-10-20. Alternatives such as automatic method injection were considered, not adopted.

Assessment: shared bodies with explicitly selected protocol calls are a small-to-medium frontend feature. Convenient calls through a protocol bound are a medium feature. Automatically adding methods to every structurally matching object, especially with private hooks, is a substantially larger language-design change. None inherently requires dynamic dispatch, an object wrapper, a new runtime representation, or a different borrow checker.

Those estimates concern sharing and lookup alone. The subsequently agreed definition-time contract checking adds substantial frontend work; the complete design with that checking and private-hook grants is not a small syntactic change.

This investigates sharing implementations. The earlier library-owned operations assessment addresses preventing callers from bypassing an algorithm; it remains relevant, because ordinary overridable defaults do not provide that guarantee.

The follow-up final-method and visibility design records the agreed core semantics: defaults honor overrides consistently, final methods reject same-named inherent methods at the structural boundary, and private hooks use member-specific grants while preserving module privacy. Three short programs illustrate the rules. The final-name check is a modest addition once provided-method selection exists; hidden external hooks require additional access-control machinery. Some specification work remains, and no compiler implementation is approved here.

Protocol bodies must also be checked against their declared contract at the definition, rather than accepted because a concrete class has extra members. This agreed requirement increases the scope beyond a generic-function lowering: the existing per-instance checker is not sufficient for a complete implementation.

One example, three distinct choices

Hypothetical syntax, not executable Plenty:

protocol DocumentStorage:
    def write(self: &mut DocumentStorage, document: &str) -> Result[(), IoError]:
        pass

    def save(self: &mut DocumentStorage, document: &str) -> Result[(), IoError]:
        validate(document)?
        self.write(document)?
        record_success()
        Ok(())

FileStorage and DatabaseStorage would supply write. The application wants to call the shared save. Treating an AST body consisting solely of pass as a requirement is straightforward; an intentional no-op implementation would need an explicit unit-returning body. Comments do not change the AST body.

There are three independent questions:

  1. Sharing: where may the application call the supplied save?
  2. Replacement: may a concrete type supply its own save instead?
  3. Access: may the application call write directly?

Implementing the first does not imply either answer to the other two. In particular, the example above deliberately uses ordinary accessible hooks. Cross-module private hooks need a separate access design.

Options and relative cost

All new spellings below are illustrative.

DesignExample callScope and assessment
Existing bounded functionsave(&mut files, document)Works today. Sharing has no language implementation cost.
Protocol-owned operationDocumentStorage.save(&mut files, document)Small-to-medium. Explicitly selects the protocol body, lowers to a generic function, and adds no inherent method.
Default selected through a boundstorage.save(document) inside S: DocumentStorage codeMedium. Needs protocol-aware method resolution, missing-default handling, override rules, and receiver provenance through specialization.
Explicit adoption on a typeAn illustrative use DocumentStorage declaration in FileStorageMedium-to-large. A local place to opt into methods and diagnose conflicts, but introduces a new relationship beyond structural matching.
Automatic structural extensionfiles.save(document) wherever a matching protocol is visibleLarge relative to the above. Needs candidate discovery, overlap rules, inference limits, and a decision about whether imports change available methods.
Final provided methodReject a matching object with a same-named inherent methodModest addition to provided-method selection: check the final-name restriction at conformance and consistently select the protocol body.
Private hooks across modulesAccess controlled separately from structural matchingSeparate substantial feature. Needs explicit authority, cross-module rules, and guarantees that survive every call path.

These are engineering judgments from source inspection and experiments, not calendar estimates or measurements of a completed implementation. A narrow compiler spike would be small; a supported feature includes diagnostics, modules, generic combinations, tooling, and regression coverage.

The protocol-owned operation is closest to ordinary generic-function sugar:

# Conceptual lowering of DocumentStorage.save:
def __document_storage_save[S: DocumentStorage](storage: &mut S,
                                               document: &str) -> Result[(), IoError]:
    validate(document)?
    storage.write(document)?
    record_success()
    Ok(())

Here DocumentStorage checks only its required hooks; implemented operations do not require same-named inherent methods. The generated name is an internal detail. Calls must preserve the protocol definition’s module visibility.

This design has deliberately fixed selection: DocumentStorage.save(...) calls that body even when the object has an inherent save. It is a provided operation, not a fallback override. It avoids global method searching and preserves the existing rule that importing a protocol does not change an object’s inherent methods.

It does not, by itself, give the application the convenient files.save(...) spelling. A bounded-call design could add that later, but must deliberately choose whether it denotes the fixed operation or an overridable default.

The important design traps

Defaults are not a validation boundary. If a concrete save replaces the default, it can omit validation. Explicitly selecting a fixed protocol body protects that call’s algorithm; it does not prohibit calling a public write or a separate concrete save. An owning wrapper remains the current-language way to expose only the intended operation.

Structural compatibility is not implementation selection. Two protocols may both require write but provide different save algorithms. A concrete object can match both without selecting either. A qualified call is unambiguous. For bound-based calls, the bound should identify the protocol; searching every matching protocol makes imports and unrelated declarations affect lookup.

Inherent overrides need exact rules. If defaults are replaceable, decide whether a same-named but incompatible method is an error or is ignored. Prefer an error over silently choosing the default. Also specify calls between defaults: does save_twice invoke an override of save, or its original protocol body? Do not let these answers depend accidentally on specialization order.

Bodies must be checked against the contract. Existing generic bodies are checked per concrete instance and may use methods not listed in their bound. That behavior is not acceptable for provided protocol bodies in the agreed design. Abstract checking is required, including through aliases, helpers and closures; a syntactic scan for self.foo is insufficient. This is additional frontend work beyond the demonstrated generic-function lowering.

Inference needs real evidence. Required concrete signatures can determine protocol parameters. A supplied body cannot become circular evidence for its own signature. If a protocol parameter occurs only in a provided operation, require explicit arguments or other independent evidence. Avoid searching other protocol defaults to manufacture conformance.

Privacy is lexical today. A library-defined shared body cannot access an external class’s private method. Automatically granting that access because a public protocol happens to match would cross an existing encapsulation boundary without the class author’s consent. If pursued, access grants should identify the protocol/member explicitly and be distinct from declaring conformance. “Private to the class’s module” also differs from “callable only by protocol bodies,” which the original request might require.

Precedents

Python’s structural protocols provide a particularly relevant warning: implicit structural matching does not supply default implementations; explicit subclassing does. Its existing inheritance mechanism selects the shared code. Python typing specification.

Rust permits trait bodies as defaults that implementations may replace. Individual trait items do not have independent visibility qualifiers. Rust Reference.

Swift protocol extensions supply defaults; a conforming type’s implementation of a requirement takes precedence. Constrained extension overlap also requires selection rules. This is useful precedent for both the feature and its semantic cost, not a lookup model Plenty can copy unchanged. Swift language guide source.

Suggested direction

The selected direction uses overridable defaults and @final bodies, selected through a bound or protocol qualification. Both forms honor overrides for non-final methods, including calls inside other protocol methods. Objects do not acquire inherent methods automatically. Matching requires signature and visibility equality, and cross-module private hooks require a @protocol.Protocol annotation on the hook method; module privacy otherwise stays unchanged. Final methods are annotated @final.

The earlier qualified-call alternative that always invokes the original body is not selected. The design page separates agreed rules from the remaining specification work. Definition-time checking against the declared protocol contract is mandatory.

The implementation map and probes show what already works and where the remaining compiler work lies. No new feature is scheduled by this sketch.

Final methods and hidden hooks

Status: core design agreed; not implemented. This refines the feasibility sketch using @final and @protocol.Name method annotations. Short programs explore the resulting programming model. Future prioritization remains in B29.

Separate implementation policy from call visibility

The agreed design has three implementation policies:

DeclarationRequirement on a matching class
def operation(...): passSupply an inherent method matching the signature and visibility.
def operation(...): <body>Supply an override matching the signature and visibility, or use the protocol body.
@final above def operation(...): <body>Do not declare an inherent method of that name; use the protocol body.

Visibility is independent: pub permits application calls through the bound; a non-public hook is available to authorized protocol code. A final method requires a body; @final on a pass body is contradictory and is rejected. An explicit unit-returning body can represent an implemented no-op.

@final is written on the line above the method, like Python’s typing.final. final communicates that an implementation cannot be replaced; fixed would emphasize ownership of the algorithm, but resolves no additional semantic issue. Method annotations defines the @ syntax.

Enforce final at the structural boundary

Matching S: DocumentStorage requires:

  • Every required method exists with exactly matching signature and visibility, and any necessary access grant.
  • Every supplied override matches the protocol signature and visibility exactly.
  • No inherent method has the name of a final protocol method.

Check the entire contract even if the generic function never calls save. Report the failed bound at its use, with notes identifying both the final declaration and the conflicting concrete declaration.

The class remains valid on its own. It simply does not satisfy this protocol. No global scan of all classes and protocols is needed. Explicit DocumentStorage.save(&mut value, ...) enforces the same complete contract rather than offer a way around conformance.

Name-only conflict detection follows the agreed rule: a same-named inherent method fails the match, including an incompatible signature or private method. There is no visibility-based exception to the final restriction. An internal class edit can therefore break conformance. Field/method name collisions still need an explicit rule before implementation.

Exact signature matching uses Plenty’s existing structural signature rules, including receiver borrowing, type parameters and return type; parameter names need not be identical. Visibility is an additional equality requirement.

Once protocol-aware selection exists, this final-name check is a relatively small addition to conformance checking. It does not require the broader access grant machinery below. It also does not resolve how calls find protocol methods.

A coherent initial lookup rule

Select methods through an explicit bound and through protocol qualification, without automatically adding methods to concrete objects:

def persist[S: DocumentStorage](storage: &mut S, text: &str):
    storage.save(text)       # Final protocol implementation.

DocumentStorage.save(&mut files, text)  # Same contract and selection.
files.save(text)             # No injected inherent method.

For an ordinary default, calls through the bound choose the compatible inherent override if present. For a final method they always choose the protocol body. Calls from a final body to a replaceable method follow the same rule. A qualified call selects the same operation, including its override, to avoid two meanings of the public API. Explicitly invoking the fallback body would need a separate feature; it is outside the agreed design.

The earlier alternative where qualification always selects the original protocol body was not selected. Fixed selection is expressed with final; ordinary implemented methods provide overridable defaults.

A class author supplies hooks and optional overrides. Protocol code operates on that existing object, subject to normal ownership and borrowing. It adds no fields, ownership, allocation, or runtime wrapper. However, methods available through a bound can differ from those available on the concrete object. This needs teaching and completion support; the examples make the difference visible.

What non-public means

The desired interface is:

pub protocol DocumentStorage:
    def write(self: &mut DocumentStorage, document: &str) -> Result[(), IoError]:
        pass

    @final
    pub def save(self: &mut DocumentStorage, document: &str) -> Result[(), IoError]:
        validate(document)?
        self.write(document)?
        record_success()
        Ok(())

write is the hidden customization hook, and save is the public entry point. Three permissions must be distinguished:

  1. Calling write through a DocumentStorage bound.
  2. Calling the concrete object’s inherent write.
  3. Letting a protocol defined elsewhere call that concrete private method.

Visibility matching requires the concrete write to be non-public too; a public implementation fails conformance. Existing module privacy remains: code in the class’s defining module can still call its private methods. Class-only privacy and literal “only protocol bodies may call this method” were considered and not selected.

For the third, structural matching alone should not authorize access to unrelated private members. Otherwise application code could declare its own protocol with a private write requirement and a public forwarding method, then use matching to obtain the same access. This would defeat existing module privacy.

The agreed class-side declaration annotates the hook itself:

pub class FileStorage:
    @protocol.DocumentStorage
    def write(...):
        ...

This grants methods defined inside the named protocol access to this specific hook. It grants no access to other private methods or fields. A hook used by two protocols carries two annotations, one per line, in any order.

The compiler checks the annotation where the class is defined. The named protocol must declare a non-public requirement with the method’s name, and the method must match that requirement’s signature and visibility exactly; otherwise the annotation is an error at the method. This reports a mismatched hook at its declaration instead of at a later bound. The annotation grants access only: it does not declare conformance, and classes without private hooks still match structurally with no annotation. Any method defined inside DocumentStorage, including a later addition, may call write. Unrelated protocols with identical signatures receive no permission.

The grant is lexical, not inherited from the caller. A standalone helper in the protocol’s module does not receive it merely because a protocol method calls the helper. Keep a hook-calling helper as a private implemented protocol method when it needs the grant. Ordinary functions in the concrete class’s module still have their existing module access. If the class and protocol share a module, that ordinary access remains available independently of the grant.

Broad grants to any protocol, module-wide switches, and a grant to all of one protocol’s requirements were considered and not selected. Member-specific permission makes expansion deliberate. An earlier class-level allow DocumentStorage.write declaration was replaced by the method annotation, which keeps the grant beside the hook and needs no repeated method name.

The grant is the one place a class must import a protocol: its module imports the protocol’s module. Import cycles are compile errors that name the dependency chain, and imports run no code, so Plenty has no partially initialized modules. A cycle would need the protocol’s module to import the class’s module. Protocol bodies are checked against the protocol’s own contract, so they never name a concrete class. Keep the contract in a module independent of concrete backends; anything that names a backend, such as a default factory, belongs in a third module. Public hooks still need no import.

No access scheme prevents an implementation from exposing a separate operation or performing its own I/O; the guarantee concerns the declared interface.

Method annotations

@ introduces a compiler annotation on the line above a def. The set is closed: @final on protocol methods and @protocol.Name on class methods, where Name resolves as a protocol in the class’s module (for example @protocol.DocumentStorage or @protocol.library.DocumentStorage). Any other annotation is an error. Annotations change no signature, body or call; they state facts the compiler checks, so their order does not matter.

final and protocol are reserved words, so annotation names cannot collide with user declarations. User-defined decorators are deferred and may never be added: generic wrappers need variadic generics (B36), Plenty runs no code at import, and a wrapper closure could allocate and change the signature. A later decorator design must keep the wrapped signature unchanged and must not alter the meaning of these annotations.

The annotation mechanism now exists, with one member: @copy on class declarations (copy classes, issue #535). The parser reads each @name alone on its line against the closed set and rejects unknown names, arguments, repeats, and annotations where no declaration accepts them. Names after @ are matched apart from identifiers, so @copy reserves nothing and the builtin copy(value) keeps its name; @final and @protocol.Name likewise do not need their names reserved. Adding them means adding members to that set and accepting annotations before def in protocol and class bodies, which the parser rejects today.

Check provided bodies against the contract

Agreed requirement: check protocol bodies against their declared contract, without depending on extra members of any particular implementing class. Every method or operation used on the implementing type must be justified by that contract and the ordinary operations available for that type parameter. A call to an undeclared self.bonus() is a definition-time error even if every currently used class happens to have a compatible bonus.

This applies to default and final bodies alike, including aliases, nested closures, and calls through helpers. Helpers must have sufficient declared constraints for the call to be checked; unconstrained per-instance checking must not become an escape from the protocol contract. Member access grants authorize access to declared hooks; they do not add undeclared members.

Report contract violations at the protocol definition. Concrete specialization can still perform lowering and concrete checks, but cannot make an invalid protocol body valid by discovering additional methods or fields on the object. This requires abstract contract checking beyond today’s per-instance generic body checking. A syntactic scan for self.method is insufficient.

Whether to extend the same discipline to all ordinary generic functions is outside this decision. The implementation must nevertheless enforce it across helper calls originating in a protocol body.

Remaining specification work

The core behavior above is settled. This is not yet a complete implementation specification or approval to implement the feature.

Compatibility: today’s protocol requirements use bare def and remain callable from bounded generic functions. Reinterpreting every bare def as a hidden hook requires a deliberate stdlib, example, and tutorial migration. Adding a final member can also invalidate existing structural matches.

Abstract checking machinery: specify how required, default and final methods, method-local bounds, helper contracts, and definition-site access are represented during checking before a concrete implementing type is known. Contract checking is required, not an optional later enhancement.

Inference and overlap: identify the protocol from the bound, not from the concrete receiver alone. Two bounds on different parameters can refer to the same concrete type and select different provided bodies. Combined bounds remain outside this design; qualification selects a protocol without global discovery. Rules for underdetermined type arguments and receiver aliases need precise specification and regression cases.

Access and collision edge cases: specify access in closures lexically nested inside protocol methods, and fields sharing a provided method name. These should be resolved explicitly rather than left to accidental behavior of generated functions.

State and validity: final freezes an algorithm, not its results. Hook contracts still define partial failure, side effects, and reentrancy. Persistent accounting/state must live in the object or a wrapper; a protocol supplies no new storage.

The @final and @protocol.Name spellings are agreed. Diagnostic wording can be decided without reopening dispatch.

Short programs with final protocol methods

Hypothetical Plenty. These complete sketches illustrate the agreed core rules; the current compiler does not accept them. The research harness runs handwritten current-language equivalents of their algorithms, not the new dispatch or conformance rules.

The examples keep hooks public to isolate implementation selection. The private-hook variation at the end adds the separate cross-module access question.

Saving a document: required hook, final operation

The protocol author supplies validation once:

enum SaveError:
    Empty

protocol DocumentStorage:
    pub def write(self: &mut DocumentStorage, document: &str) -> Result[(), SaveError]:
        pass

    @final
    pub def save(self: &mut DocumentStorage, document: &str) -> Result[(), SaveError]:
        if document.byte_len() == 0:
            return Err(SaveError.Empty)
        self.write(document)?
        Ok(())

class Memory:
    writes: i64

    pub def write(self: &mut Memory, document: &str) -> Result[(), SaveError]:
        self.writes += 1
        Ok(())

def persist[S: DocumentStorage](storage: &mut S, document: &str) -> Result[(), SaveError]:
    storage.save(document)

def main() -> Result[(), IoError]:
    mut memory = Memory(0)
    match persist(&mut memory, ""):
        case Ok(_):
            print("unexpected")?
        case Err(_):
            print("empty rejected")?
    match persist(&mut memory, "hello"):
        case Ok(_):
            print("saved")?
        case Err(_):
            print("unexpected")?
    print(memory.writes)?
    Ok(())

Expected output: empty rejected, saved, 1. The class author writes only storage behavior. The application uses persist, which explicitly names the contract and can call its final save.

If Memory adds its own save, the call to persist is rejected, even if that inherent method validates correctly. It is the attempted structural match, not the class declaration, that violates this candidate’s contract.

DocumentStorage.save(&mut memory, "hello") would be the qualified spelling. memory.save("hello") does not gain a new inherent method. Calling memory.write("hello") remains possible in this public-hook example.

Scoring: final operation calls a replaceable default

protocol Scored:
    pub def base_score(self) -> i64:
        pass

    pub def score(self) -> i64:
        self.base_score()

    @final
    pub def eligible(self) -> bool:
        self.score() >= 10

class Basic:
    points: i64

    pub def base_score(self) -> i64:
        self.points

class Bonus:
    points: i64

    pub def base_score(self) -> i64:
        self.points

    pub def score(self) -> i64:
        self.points + 5

def qualifies[S: Scored](item: &S) -> bool:
    item.eligible()

def main() -> Result[(), IoError]:
    basic = Basic(7)
    bonus = Bonus(7)
    print(qualifies(&basic))?
    print(qualifies(&bonus))?
    Ok(())

Expected output: False, True. The final threshold algorithm calls the selected score: the protocol fallback for Basic, the override for Bonus. It does not freeze every method reachable from its body.

From the class author’s perspective, Bonus.score is still an ordinary method. It may be called directly. Basic has no inherent score; uniform application code uses a bound or Scored.score(&basic). This asymmetry is the main ergonomic cost of avoiding automatic injection, and should be evaluated with users.

Same object, two protocols, different provided behavior

protocol Celsius:
    pub def celsius(self) -> i64:
        pass

    @final
    pub def display(self) -> i64:
        self.celsius()

protocol Fahrenheit:
    pub def celsius(self) -> i64:
        pass

    @final
    pub def display(self) -> i64:
        self.celsius() * 9 // 5 + 32

class Sensor:
    reading: i64

    pub def celsius(self) -> i64:
        self.reading

def show[T: Celsius, U: Fahrenheit](left: &T, right: &U) -> Result[(), IoError]:
    alias = left
    print(alias.display())?
    print(right.display())?
    Ok(())

def main() -> Result[(), IoError]:
    sensor = Sensor(20)
    show(&sensor, &sensor)?
    Ok(())

Expected output: 20, 68. Both generic arguments are concretely Sensor, but their bounds select different implementations. The alias must retain the left parameter’s protocol context. Neither default is installed globally on Sensor, and neither protocol overrides the other.

This is why simply registering a synthetic Sensor.display when first encountering a bound would be incorrect. It would make later calls depend on which protocol was instantiated first.

Making the storage hook private across modules

This variation is a syntax sketch, not another complete runnable program:

# library.plenty
pub protocol DocumentStorage:
    def write(self: &mut DocumentStorage, document: &str) -> Result[(), IoError]:
        pass

    @final
    pub def save(self: &mut DocumentStorage, document: &str) -> Result[(), IoError]:
        # Validate, call self.write(document), record success.
        ...

# backend.plenty
from library import DocumentStorage

pub class FileStorage:
    @protocol.DocumentStorage  # Agreed access-grant syntax; not implemented.
    def write(self: &mut FileStorage, document: &str) -> Result[(), IoError]:
        # Perform the actual file operation.
        ...

# application.plenty
def persist[S: DocumentStorage](storage: &mut S, document: &str) -> Result[(), IoError]:
    storage.save(document)   # Allowed.
    # storage.write(document) would be rejected.

Here the class author explicitly permits the real library’s protocol to call write; the application cannot create an equivalent protocol to acquire that permission. The compiler checks the annotation where FileStorage is defined: DocumentStorage must declare a non-public write with the same signature. No separate implementation block is required. Ordinary module privacy still lets code inside backend.plenty call its own hook. All methods defined in DocumentStorage may use the grant, but standalone helpers in library.plenty do not inherit it. Class-only privacy was not selected.

The imports run one way: backend.plenty imports library.plenty, and the application imports both. library.plenty never names FileStorage, because protocol bodies are checked against the protocol’s own contract.

Protocol body implementation map and probes

This supports the feasibility sketch. Source locations and results describe base d39e6ec, not a compiler with defaults. The experiments use handwritten generic-function lowerings; they do not implement new syntax, method selection, final-name checks, or private access grants.

Existing compiler machinery

AreaCurrent implementationImplication
Parsingsrc/frontend/protocols.rs, Parser::protocol_declUses the ordinary function parser and stores Vec<Function>, including bodies. An explicit guard rejects any body other than one Pass. Recognition is easy; removing the guard alone does not compile or register bodies.
Module resolutionsrc/frontend/modules.rs, protocol loop in resolveAlready walks each requirement through Scope::function and validates public signature types. Generated bodies must retain this definition-site namespace and access context, rather than borrowing the concrete class’s privileges.
Conformancesrc/frontend/protocols.rs, checkEvery method is currently required. Provided operations need separation from requirements; overridable defaults additionally need missing-method fallback and compatible-override checks. Include generic requirements and the bytes special case.
Inferencesrc/frontend/generics.rs, Engine::infer_boundsReads concrete inherent signatures for protocol requirements. Must avoid trying to read a nonexistent inherent default or inferring its parameters from itself.
Specializationsrc/frontend/generics.rs, prepare and Engine::specializeGeneric templates, type substitution, cached instances, and specialization budgets already exist. A protocol body can become a template parameterized by implementing type, protocol arguments, and method-local parameters.
Callssrc/frontend/classes.rs, class_methodDerives the concrete owner/method symbol, checks access, then selects concrete or generic inherent methods. No general protocol-default fallback exists.
Accesssrc/frontend/modules.rs, check_member_atPrivate access compares the member’s owner module with a source token. There is no protocol/member access-grant model.
Toolingsrc/frontend/ide.rs, protocol declarations and method_callNavigation needs the selected protocol body or override, not an invented concrete member. A feature must preserve source locations in generated templates and diagnostics.

For explicit protocol operations, introduce distinct callable symbols, lower their bodies into existing generic templates, and resolve the qualified protocol call to that symbol. Keep conformance checks independent of compiling the body. Ordinary static calls can then flow through the existing IR/backend. There is no inherent need for protocol values, witness objects, vtables, boxing, or additional heap allocations. These are architectural conclusions, not benchmarked performance claims.

For bound-based syntax, more information must survive specialization. Engine::specialize substitutes concrete types and clears function type parameters. protocol_instances records a function-level flag used for restrictions on named/default arguments; it is not a mapping from individual receivers to protocol evidence.

Consider left: &T with T: P and right: &U with U: Q, instantiated with the same concrete type. Choosing a default from the concrete type alone cannot distinguish left.operation() from right.operation(). Aliases, returned references, and captured receivers must preserve whichever selection semantics the design promises. One possible approach is to resolve calls to explicit protocol operations before erasing that information, but this needs more than a textual receiver-name rewrite.

Reproducible experiments

From a checkout containing the script, build its compiler and run:

cargo build --locked --bin plenty
python3 -B scripts/probes/protocol_methods.py target/debug/plenty

The script creates isolated temporary modules, disables the compiled-function cache, and checks exact output or an expected diagnostic. It is a research harness, not a new language conformance suite.

ProbeObserved result
Library save and save_twice, external mutable backend, application in a third moduleRuns; output -1, 3, 10, -999. Negative input is rejected before writing; nested shared calls update state; an inherent save remains a distinct operation returning -999.
Same backend with non-public writeRejected as private to its defining module.
Application imports library-private validation helperRejected as private.
A protocol method with a real bodyRejected by the existing default-implementation guard.
Application calls memory.save when only the library function existsRejected as an unknown method.
Parameterized source protocol, concrete error inference, enum implementationRuns; output 42.
Required generic method with its own protocol bound, called twice by shared codeRuns; output 14.
Shared generic function forwards a hook’s returned referenceRuns; output 8.
Mutation while that returned reference remains liveRejected by borrowing checks.
Generic body calls a concrete method absent from its declared protocolRuns; output 99, confirming per-instance checking rather than abstract contract checking.
Final storage candidate lowered to a generic functionRuns; output empty rejected, saved, 1; validation prevents the first write.
Final eligibility body with default and overridden score, lowered to two selected bodiesRuns; output False, True. Selection is handwritten, not inferred by a new compiler rule.
Two protocol operations on the same concrete sensor, including a receiver aliasRuns; output 20, 68; the lowering explicitly preserves the two selections.

All 13 probes passed with the compiler built from the exploration worktree. Eight executed native programs and five checked expected compile failures. No compiler/runtime code was changed. These probes establish reuse of existing generics and ownership behavior; they do not establish that adding fallback lookup is a small patch.

The last three correspond to the candidate programs. Their handwritten lowering tests the algorithms and borrowing but cannot validate automatic selection, alias provenance, or rejection of final-method conflicts. Those remain compiler work. The cross-module private-hook sketch is not executed.

The accepted behavior now requires definition-time contract checking. The probe that accepts an undeclared extra demonstrates a limitation of reusing today’s generic checker, not behavior permitted in a future protocol body. The successful lowerings establish executable algorithms and ownership behavior, but do not validate this additional checker requirement.

Remaining validation if implemented

An implementation would need focused tests for qualified and bound-based calls, imported aliases, two matching protocols, incompatible overrides, calls between provided methods, recursion and specialization reuse/limits, parameter inference without circular evidence, generic data and enum receivers, bytes, mutable receiver reservations, returned loans and temporary receivers, closures, diagnostics, and navigation. Changing body selection must also invalidate the appropriate compiled artifacts.

New syntax should preserve existing rules for method-local parameters, parameter defaults and positional calls unless those changes are separately designed. An all-provided protocol and mutually recursive defaults need explicit rules: accepting their structural shape does not prove their algorithm terminates. Do not recursively compile bodies merely to decide conformance.

The body-lowering experiment did not test generated compiler symbols, actual default dispatch, abstract body checking, access grants, compile-time overhead, or binary-size changes. Full workspace tests and Clippy were not run for this documentation/Python-harness change; they are required for a Rust implementation.

Boxed protocol objects

Status: implemented with conservative erasure limits. Explicit Box[P] owners support shared/mutable dispatch, fixed protocol arguments and generic consumers of the same protocol. The implemented contract is authoritative. This proposal retains the research and design rationale. Shared cannot contain protocol objects, and construction rejects payloads concealing executors, futures, channel endpoints or erased callbacks. Returned borrows, consuming requirements, method-local generics and native transfer remain unsupported. Generator-frame/concrete-closure signature types and recursive boxed interface signatures are also rejected. See the retained measurements.

The investigation used local main at bb97038b on 2026-11-03. It found a credible implementation path through the existing boxed-callable machinery, not a working implementation of boxed protocols. Read the public contract, implementation design, research evidence and runnable probes, and qualification criteria for the details.

Intended behavior

A program should be able to store instances of different classes and enums, iterate over them, and call the methods declared by a common protocol without matching or casting. Every method on an object must reach the same underlying instance, including changes made through mutable methods.

Complete example of the implemented interface:

protocol MyProtocol:
    def hey(self) -> i64:
        pass

class Counter:
    value: i64

    def hey(self) -> i64:
        self.value

enum Message:
    Ready
    Count(i64)

    def hey(self) -> i64:
        match self:
            case Message.Ready:
                0
            case Message.Count(value):
                *value

def main() -> Result[(), AllocError | IoError]:
    mut items: list[Box[MyProtocol]] = []?
    items.append(Box[MyProtocol](../Counter(10))?)?
    items.append(Box[MyProtocol](../Message.Count(20))?)?
    for item in &items:
        print(item.hey())?
    Ok(())

Expected output is 10 then 20. The list stores one common owner type; each owner retains its concrete payload and the methods needed to operate on it. The collection does not need a declaration listing every implementation.

Dictionary values work the same way, with ordinary supported keys:

items: dict[str, Box[MyProtocol]] = {
    "counter": Box[MyProtocol](../Counter(10))?,
    "message": Box[MyProtocol](../Message.Ready)?,
}?
for name, item in items.items():
    print(name, item.hey())?

Protocol boxes are not dictionary keys or set elements. See the current heterogeneous collection lessons for implemented enum and anonymous-sum alternatives.

Why Box is explicit

The proposed spelling is list[Box[MyProtocol]], not list[MyProtocol]. MyProtocol remains a compile-time contract rather than a standalone stored value type. The Box boundary supplies owned storage for an unknown concrete layout, while Box[MyProtocol](../value)? exposes allocation and its failure.

This is a deliberate language choice, not a mathematical requirement. An unboxed-looking protocol value could be defined as an implicitly boxed owner, a borrowed view, or a fixed-capacity inline container. Each would need a different storage and lifetime contract. None is implied by this proposal.

Ordinary Box(value)? continues to produce Box[T] for the concrete T. An annotation or collection insertion does not silently allocate or erase it. The explicit protocol argument selects erasure. Direct P, list[P], and bare &P remain unsupported as value types in the initial design; functions can accept &Box[P] or an existing generic T: P bound.

Support one protocol per box, shared and mutable receiver methods, fixed protocol type arguments, owned storage in ordinary aggregates, and existing generic consumers of that same protocol. Require every method to meet the boxing eligibility rules. Keep structural conformance and explicit allocation; introduce neither inheritance nor registration of implementations.

The initial contract excludes method-local generics, implementing-type occurrences outside the receiver, consuming receivers, returned borrows, downcasting, protocol-to-protocol conversions, copying, equality, and native worker/channel transfer. These restrictions do not reduce what protocols can express in today’s static generic calls.

Alternatives and tradeoffs

RepresentationBest fitConsequence
list[Item] with an enum wrapperA known, finite set of alternativesExhaustive matching and inline payload storage; adding a variant changes the wrapper.
list[A | B]A small explicit set, dispatched by matchingNo wrapper declaration; arbitrary user protocols are not forwarded today.
list[Box[P]]Implementations supplied independently of the collectionExplicit per-object allocation and indirect method calls; consumers know only P.
list[BoxedCallable[[], i64]]One reusable operation per objectWorks today with an owned closure adapter; it does not provide several named methods over one mutable instance.

An automatically generated enum of every implementation in the program would make the set depend on global discovery. Its layout, matching, compilation and module behavior need a separate design; it is not the mechanism proposed here.

A separate name such as BoxedProtocol[P], or a dyn P spelling inside Box, could emphasize erasure. Prefer the requested Box[P] spelling, provided the compiler and documentation clearly distinguish its restrictions from concrete Box[T]. In particular, an erased payload cannot be extracted by value.

Feasibility assessment

This is a moderate, cross-cutting compiler feature. Type checking, generated adapters, indirect calls and concrete destruction already have useful precedents. Existing list/dictionary storage should accept a fixed owner representation without a new collection implementation.

The difficult work is preserving borrowing and effect guarantees after the concrete type disappears from the caller, not allocating memory or selecting a function pointer. The estimate is architectural judgment, not a calendar estimate or a measured prototype result.

This is independent of protocol method bodies. Defaults, final methods, private-hook grants, and abstract checking of supplied bodies need their own implementation. Boxed objects can initially dispatch the accessible inherent methods required by protocols with pass bodies.

Boxed protocol object contract

Implementation note: the current reference describes delivered behavior. This page preserves the original design and qualification plan; research-era implementation statements below are historical. This is the proposed public contract for boxed protocols. It is not implemented. All new syntax in this page is illustrative.

Identity and construction

Box[P[Args...]] is an affine owner of one concrete value satisfying P[Args...]. Protocol identity includes its resolved declaration and fixed type arguments; import aliases do not create new identities. Distinct protocol declarations remain distinct boxed types even when their signatures match.

Box[P](../../boxed-protocols/value) consumes an owned concrete value, verifies conformance, and returns Result[Box[P], AllocError]. It attempts one allocation for the box, including for a zero-field class or payload-free enum. Method tables are static compiler output, not a second allocation per instance. Construction of the input and operations within its methods retain their own allocation costs.

Allocation failure destroys the consumed value, as ordinary Box(value) does. If the box is constructed successfully but insertion into a collection fails, the collection operation’s existing ownership contract applies to that box. The proposal must not promise that a failed insertion returns its argument. Programs preserving existing state should allocate detached replacement values before moving state into them, as taught in recursive insertion.

There is no implicit concrete-to-erased conversion on assignment, argument passing, return, or collection insertion. Box[P](../../boxed-protocols/value) is the explicit boundary. Allocation-free conversion from an existing Box[T] is outside the initial scope: do not accidentally unbox/rebox it via today’s expected-type coercions. Initially reject boxed inputs to this constructor and suggest boxing the concrete owner directly. Moving an existing Box[P] needs no constructor and performs no allocation.

The admitted concrete sources are ordinary storage-safe classes and enums, including specialized generic instances, and the existing bytes conformance case. Do not introduce new builtin conformances as part of boxing. Existing storage barriers still reject borrowed fields/captures, generators and other values that cannot enter a heap owner. An arbitrary anonymous sum does not gain protocol forwarding merely because it is boxed.

Eligible protocols

A protocol is eligible when every requirement can be called with a fixed signature after substituting its protocol arguments and erasing its receiver. Check eligibility at a boxed-type use, not at protocol declaration: a protocol that is ineligible for boxing remains valid for static generic use.

RequirementInitial rule and reason
def hey(self) -> i64Supported; bare self is a shared borrow.
def set(self: &mut P, value: i64) -> ()Supported; the object must be exclusively borrowed.
Ordinary owned and borrowed arguments, including slice viewsSupported under existing call and storage rules; arguments are not retained by erasure.
Concrete owned results, including Result[T, E]Supported; exact fallibility and payload types are part of the signature.
Protocol-level parameters, such as Readable[i64]Supported once every argument is fixed; they determine one concrete table shape.
Method-level parameters, such as put[T](../../boxed-protocols/...)Rejected; an ordinary finite table cannot supply every future specialization.
Protocol name outside the receiver, including nested occurrencesRejected; that name denotes the hidden implementing type under current protocol rules.
Owned receiver self: PRejected initially; consuming an erased payload requires a distinct transfer/cleanup contract.
Returned references or views, including Option[&T] and Result[&T, E]Rejected initially; dispatch must retain the origin loan before these are admitted.

For example, compare(self, other: &P) requires another instance of the same hidden concrete type. Two Box[P] values need not contain the same type, so replacing that parameter with another protocol box would change the contract. Similarly, returning P means returning the concrete implementing type today; it must not silently become an allocating boxed result. Detect these uses through aliases and nested signatures, before replacing the receiver type.

The current std.fmt.Formattable protocol has a method-local writer parameter. It is therefore ineligible for this initial boxed representation. An erased writer interface or a different reporting contract would be a separate design; this proposal does not automatically create a general boxed error type.

Reject the entire boxed protocol if any requirement is ineligible; do not make the callable method set depend on which methods a particular program uses. Diagnostics should identify the protocol, offending requirement, and reason.

Calls, mutation and loans

object.method(args) selects a method declared by the boxed protocol. The compiler knows its signature; only the implementation address varies at runtime. Concrete fields and additional inherent methods are not accessible through the box. No runtime name lookup or conformance check is needed on a call.

Shared calls borrow the owner, including through &Box[P], a list element, a dictionary value or a class field. Mutable calls require &mut Box[P] or an exclusively accessible mutable owner. Multiple methods share one payload:

protocol Counter:
    def increment(self: &mut Counter) -> ():
        pass

    def current(self) -> i64:
        pass

def advance(items: &mut list[Box[Counter]]) -> Result[(), IoError]:
    for item in items:
        item.increment()
        print(item.current())?
    Ok(())

The read observes the increment made to that same instance. This must not be implemented as separate independently captured copies for each method.

Loans remain rooted in the owner and its collection/field projection. A live borrow prevents conflicting mutation, replacement, removal, moving or dropping the owner, and invalidating collection changes. Preserve ordinary argument evaluation and receiver reservation rules: erasure grants no extra aliasing. Initially make no field-precision claim across indirect calls.

Read-only calls on temporary owners retain them until the full expression ends. Mutable calls follow existing boxed-receiver place requirements. Calls, owner moves, and compiler-provided cleanup add no allocation; user methods and destructor hooks retain their own behavior and effects.

Integration with generic consumers

Recommend that Box[P] satisfy the same P bound through its exposed methods:

def greet[T: MyProtocol](value: &T) -> i64:
    value.hey()

def use(value: &Box[MyProtocol]) -> i64:
    greet(value)

Here T is Box[MyProtocol]; specialization compiles a forwarding indirect call, not a new specialization for each hidden payload. Protocol arguments come from the box’s identity, including when generic inference needs them. Moving a box into a generic owner parameter still transfers its owner normally.

This is deliberately restricted to the identical resolved protocol and arguments initially. Matching another protocol from an exposed method subset, combining protocols, and converting between erased protocols need separate rules. Do not discover extra capabilities by examining the hidden payload.

Current generic bodies are checked per concrete instance. A body that relies on extra inherent methods may work for a concrete class yet fail for Box[P], which exposes only P. Supporting boxed objects does not require broadening that visible interface or silently implementing abstract generic-body checking.

Visibility and other operations

Use the existing protocol conformance visibility boundary, with the protocol’s defining module as the bound site. Generated adapters must not gain private access merely because they are emitted beside a concrete method. Public erased signatures must expose accessible protocols and signature types. A library may hide its concrete implementation behind a public Box[P] without exposing that implementation’s fields or type name.

Support owners in lists, dictionary values, class fields, enum payloads, tuples, Option, Result, parameters and factory returns, subject to ordinary aggregate restrictions. Dictionary reading must preserve ownership: use borrowing/index operations supported for affine values rather than implying get copies a box.

Do not support extraction of an erased payload with *object, bare &P or &mut P values, downcasts, type inspection, implicit copying, copy, equality, equality-based membership, hashing, or use as keys/set elements initially. Existing Box[T] behavior remains available for concrete T. Proposed automatic formatting is an opaque marker such as <boxed protocol MyProtocol>; it must not inspect private fields or invoke arbitrary formatting methods implicitly.

Reject native worker/channel transfer, including through nested containers. Erasure hides both payload eligibility and indirect method/destructor effects. Also apply conservative shared-owner cycle checks: mutable storage of an erased object must not conceal a back-reference that existing Shared checks would reject. Actor/task handoff and allocator provenance need explicit qualification; support for local calls does not certify them.

Boxed protocol implementation design

Implementation note: the current reference describes delivered behavior. This page preserves the original design and qualification plan; research-era implementation statements below are historical. This supports the boxed protocol proposal. The source map describes base bb97038b; representation and IR names below are suggestions, not implemented APIs. Scheduling belongs to B29 in the backlog.

Reuse a concrete owner with an erased call interface

The existing BoxedCallable separates three facts that protocol objects also need: a common public type, a concrete owned payload, and an adapter that knows the payload’s actual layout. For protocols, replace the single call adapter with a static table containing one adapter per eligible method.

A candidate representation is:

Box[P] owner
    |
    v
heap record
    ownership/allocation header
    concrete payload descriptor ------> layout, relocation and destruction
    protocol method-table pointer ----> method 0 adapter
    payload storage                    method 1 adapter
                                       ...

The public type has a fixed owner representation regardless of the payload’s size. Every table entry receives the same payload storage address. Adapters recover the concrete receiver representation and invoke the selected inherent method. Neither table lookup nor receiver adaptation allocates or copies the payload. Keep the descriptor needed for destruction even if the protocol has no cleanup-related method; lifecycle hooks are not protocol requirements.

plenty-runtime/src/aggregates.rs already has a Boxed header with a concrete descriptor, a word currently called padding, and a payload slot at byte 32. Boxed callable construction stores its adapter in that word. A protocol-table address could occupy the analogous word for a distinct erased owner kind. This suggests a compact implementation, not a frozen ABI or permission to reinterpret a callable as a protocol object. Validate alignment, relocation, destruction and target layouts before committing to offsets.

The existing allocator and descriptor-driven iterative box cleanup are the preferred foundation. Do not replace them with a new reference-counting or recursive destruction system. Explicit copying of an ordinary box must never become an accidental way to copy a protocol object.

Frontend and type identity

Introduce an explicit internal type, illustratively Ty::BoxedProtocol, whose identity carries the resolved protocol and instantiated method signatures. Keep the concrete payload type on construction/adapters, not in the erased public identity. This avoids pretending that P has an ordinary inline size or passing it through operations that assume every Ty::Box(T) can yield T.

Resolve Box[P[Args...]] through protocol-aware type resolution before the ordinary concrete Box path tries to resolve its content as a value type. Preserve imported aliases, visibility, generic substitution and canonical identity. A protocol name alone must not start resolving as a stored value. Explicit Box[P](../../boxed-protocols/value) construction also needs frontend handling; the current Box(value) builtin merely infers its concrete content type.

Perform eligibility checking after resolving protocol arguments, while retaining the implementing-type marker needed to detect forbidden non-receiver uses. Then check the concrete input with the existing structural conformance rules. Keep receiver mutability, exact signatures and module access checks; method names alone do not establish conformance. Use positional method arguments initially, following current protocol-bound call restrictions, rather than inventing a keyword/default mapping from implementation parameter names.

Register adapters and the table for each canonical (concrete T, protocol P, protocol arguments) combination. Repeated construction should share emitted code and data. Cache emitted artifacts separately from access decisions: visibility must still be checked in each appropriate source context. Materialize every required concrete method, including methods of generic classes/enums, so no table entry points to an uninstantiated definition.

For T: P instantiated with Box[P], synthesize the exposed signatures with the receiver replaced by a borrow of the owner and forward method calls through the table. Inference should read the box’s fixed protocol arguments, not inspect hidden concrete types. Do not loosen generic matching or protocol identity to make this case fit by accident.

Typed IR and native lowering

Use explicit typed operations, illustratively ProtocolBoxNew and ProtocolBoxCall, rather than smuggling an erased owner through concrete-box operations. The IR checker must independently validate input ownership, protocol identity, method index, receiver mutability, arguments, results and adapter signatures. Construction consumes its concrete input and produces the exact allocation-error result.

Calls load the table and statically selected method entry, then use the existing indirect-call ABI machinery. The code generator may erase the receiver address at the ABI boundary, as boxed callable lowering does, but the typed IR and loan checker must retain the real shared/exclusive owner contract. Shared-call BoxedCallable code is a precedent, not a drop-in mutable-call implementation.

Emit tables with native data/function relocations. Track adapter addresses and their callees in reachability analysis, including executable pruning on Windows. The compilation cache must include protocol identity, signatures, table order, receiver mode and selected implementation bodies. A method reorder or signature change must not reuse a caller expecting the old slot layout. Use deterministic table order, initially declaration order; no stable external table ABI is promised. Foreign export/import of these owners is outside the initial scope.

Integration map

AreaExisting locationsRequired integration
Types and constructionsrc/frontend.rs, src/op.rs, src/boxed.rsDistinguish boxed protocols from concrete box extraction and type identity.
Protocols and genericssrc/frontend/protocols.rs, src/frontend/generics.rsEligibility, conformance, fixed-argument inference, adapter specialization.
Method dispatchsrc/frontend/collections.rs, src/frontend/classes.rsResolve methods of erased owners through fields, references, collection projections and temporaries.
Callback precedentsrc/frontend/callables.rs, src/codegen/closures.rsGenerated adapters, fixed-signature indirect calls, fallible owner construction.
Independent verificationsrc/op/check.rsConstruction/call signature checks and receiver ownership checks.
Ownership and effectssrc/ownership.rs, src/ownership/calls.rs, src/ownership/futures.rs, src/frontend/provenance.rsPreserve loans and treat unknown indirect effects conservatively.
Runtime storageplenty-runtime/src/aggregates.rs, src/codegen/metadata.rsConcrete drop/relocation descriptor plus static method-table identity.
Reachability and artifactssrc/codegen/reachability.rs, src/threading.rs, codegen cache/dump pathsRetain adapters, serialize identities and reject uncertified transfer.
Shared ownerssrc/frontend/shared.rsPrevent hidden payloads from bypassing cycle and executor/future ownership checks.
Modules and toolingsrc/frontend/modules.rs, src/frontend/ide.rsPublic-interface validation, diagnostic blame and navigation to protocol requirements.

All type walkers need an audit: storage barriers, contained owners, copying, equality, formatting, generic substitution, references, checked effects, serialization and FFI eligibility. An exhaustive type-enum match helps find sites, but a wildcard arm can silently apply an unsafe default.

Safety boundaries that erasure must preserve

Conformance proves signatures, not effects. A method may call back into user code, allocate, wait, acquire a lock or access another owner. Existing indirect call analysis is conservative; a protocol method table must not accidentally look like an effect-free primitive. Audit lock-held waits, live future/task obligations and reentrant arguments as well as ordinary borrow conflicts.

The existing BoxedCallable rejection at worker/channel boundaries is a useful initial policy. Hiding a non-transferable object inside a list or a class must not make it transferable. A future transferable protocol owner would need certification at construction for payloads, reachable methods and destructors; the presence of a matching protocol is not that certification.

Shared ownership has a second erasure hazard: a payload stored behind mutable state can contain a share of its enclosing owner. Existing Shared traversal already treats erased callbacks conservatively across such edges. Extend that policy to protocol objects and audit hidden executors/futures too. Do not claim that the ordinary box descriptor alone makes these static checks unnecessary.

Returned borrows are plausible later because Plenty ties a method’s reference result to its receiver. They still require forwarding the caller’s owner loan through an unknown implementation, including optional/fallible borrows and temporary receivers. Reject every reference-carrying output in the initial contract rather than copying only the direct-reference guard from boxed callables. Borrow-carrying output must be recognized structurally.

Relationship to protocol bodies

The protocol-body proposal specifies defaults, final operations and private-hook grants. It is not a prerequisite here. The initial table contains concrete inherent implementations of pass requirements.

If both features ship, table construction should use the same resolved selection rules as static bound calls. Default calls must honor overrides consistently; final methods and access grants must retain their declared authority. Avoid a second, independent method-selection algorithm in the backend. Abstract checking of supplied bodies remains part of that proposal, not a capability established by the single-method erasure probe.

Boxed protocol research evidence

This records the investigation behind boxed protocols. The inspected baseline is local main at bb97038b, dated 2026-11-03. The local debug compiler reported plenty 0.8.8 (x86_64-unknown-linux-gnu). No protocol-object implementation or performance measurement was produced by that original investigation. The current implementation is described in the reference; the harness now tests protocol objects by default. Use --baseline with the research-era compiler to reproduce the historical rejection below.

Implementation measurements

The implementation was measured on x86_64 Linux with optimized native output (--release). The retained scripts/probes/boxed_protocols_bench.py constructs 20,000 alternating one-field/two-field objects, performs 200 shared-read passes (4 million calls), and destroys the list. Each phase uses std.time; the table reports medians of five runs in milliseconds, excluding process startup. Construction includes list growth. Every representation checks the same checksum.

RepresentationConstructionShared readsCleanupExecutable bytes
inline enum1.8707.7310.272183,896
concrete box2.3457.5220.587184,712
boxed callable3.50122.2130.903186,344
boxed protocol2.02416.2100.689185,112

The protocol representation trades indirect-call cost for an interface that can accept independently supplied implementations. In this workload its shared-read phase takes about twice the inline-enum/concrete-box time, and less than the boxed callback adapter. These are local microbenchmarks, not general speed promises: payload sizes differ between representations, methods do very little work, and host load changes the results. Mutation performance, large method tables and other native targets were not measured. Native execution qualification here is x86_64 Linux; no macOS, Windows or AArch64 run is claimed.

The harness also reports one cold compiler invocation per representation and linked executable size; compiler timings are not repeated measurements. Run it with a compiler built from this checkout:

python3 -B scripts/probes/boxed_protocols_bench.py target/debug/plenty

Reproducible compiler probes

The retained harness is scripts/probes/boxed_protocols.py. It writes isolated temporary programs, disables the compiled-function cache with PLENTY_CACHE=off, executes positive cases and checks expected diagnostic text in negative cases. It requires no third-party Python packages.

From the checkout whose behavior is being investigated:

cargo build --locked --bin plenty
python3 -B scripts/probes/boxed_protocols.py --baseline target/debug/plenty

The investigation ran the harness with the existing debug compiler in the primary checkout; no compiler/runtime sources differ between that baseline and this proposal branch. All four probes passed: two native programs and two expected type-checking failures. The rejection probes characterize this baseline, not permanent requirements; update them when B29 implements boxing.

ProbeObserved resultWhat it establishes
Declare items: list[Box[MyProtocol]] = []?Rejected: unknown type MyProtocol (the diagnostic quotes the name with backticks).Protocols currently do not resolve as boxed value types.
Iterate over list[Box[A]] and call item.hey()Runs; prints 10.Concrete boxes already support method calls through borrowed list elements.
Append Box[B] to that list[Box[A]]Rejected: expected Box[A], got Box[B].Heap storage alone does not erase the concrete type.
Adapt a class and an enum satisfying MyProtocol into boxed callbacksRuns; prints 10 then 20.Existing generic conformance, owned capture and callback erasure compose for one shared method.

Both implementations used by the probes:

protocol MyProtocol:
    def hey(self) -> i64:
        pass

class A:
    value: i64

    def hey(self) -> i64:
        self.value

enum B:
    Value(i64)

    def hey(self) -> i64:
        match self:
            case B.Value(value):
                *value

The positive erasure probe adds this current-language code:

def erase[T: MyProtocol](value: T) -> Result[BoxedCallable[[], i64], AllocError]:
    callback = def [value]() -> i64:
        value.hey()
    BoxedCallable(callback)

def main() -> Result[(), AllocError | IoError]:
    mut items: list[BoxedCallable[[], i64]] = []?
    items.append(erase(A(10))?)?
    items.append(erase(B.Value(20))?)?
    for item in &items:
        print(item())?
    Ok(())

The protocol requirement is checked when specializing erase. The resulting callbacks have the same public signature despite different concrete captured values. Calling the callback reaches the appropriate hey implementation.

This does not test multiple named methods, mutable erased receivers, protocol-table generation, Box[P] satisfying a generic bound, destruction under injected allocation failure, or returned borrows. Separate callbacks capturing separate objects would not establish that several methods operate on the same state. Existing BoxedCallable also rejects mutable and consuming callbacks; the successful probe does not remove that limit.

Source findings

Finding at the baselineSource evidenceConsequence for the proposal
Protocols are stored as required method declarations.src/frontend/protocols.rs, Protocol and Parser::protocol_declReuse declarations and exact conformance checks; do not assume a runtime protocol type already exists.
The protocol’s own name substitutes to the implementing type.protocols::check inserts the concrete type into its alias environment.Preserve a marker long enough to reject non-receiver occurrences before erasure.
Protocol conformance checks accessible methods and receiver/signature equality.protocols::check, including generic-requirement matchingBoxing must not weaken static guarantees or bypass module access.
Ordinary Box[T] requires a resolved concrete content type.src/frontend.rs, TypeRef::resolve and the Box constructor pathBoth annotation resolution and explicit protocol construction need new handling.
The IR has Box, BoxedCallable and Callable, but no protocol-object type.src/op.rs, Ty, BoxedCallableNew, BoxedCallableCallA distinct erased type/operation path is the safer starting point.
Boxed callbacks generate concrete receiver adapters.src/frontend/callables.rs, Lower::boxed_callableMulti-method objects can reuse the adapter approach without allocating one callback per method.
Calls use a hidden environment address and a fixed public signature.src/codegen/closures.rs, lower_boxed_callable_callThere is already native indirect-call support; the proposed extension selects an entry from a static table.
The allocated box retains its concrete descriptor.plenty-runtime/src/aggregates.rs, Boxed, try_box_new, box_destroy, operation 128Erased callers need not know layout to preserve correct relocation and destruction.
Metadata emission supports function-address relocations.src/codegen/metadata.rs, defineMethod tables can be immutable native data with adapter addresses.
Boxed callable eligibility and call signatures are independently checked.src/op/check.rs, BoxedCallableNew/BoxedCallableCall armsNew protocol operations need equivalent validation, especially for mutable receivers.
Erasure does not certify worker eligibility.src/threading.rs, Ty::BoxedCallable rejectionPreserve conservative nested transfer restrictions until a capability contract exists.
Shared-cycle checks treat mutable paths containing erased callbacks specially.src/frontend/shared.rs, rejectionHidden protocol payloads require the same safety audit.
Indirect calls affect ownership/effect summaries and function reachability.src/ownership/calls.rs, src/ownership/futures.rs, src/codegen/reachability.rsA callable table is not merely a runtime representation change.

The box layout contains a word used for the boxed callback’s adapter; its payload slot begins at byte 32 in the inspected implementation. Those are current source facts, not language promises. The suggestion to store a protocol-table pointer there is an inference requiring its own runtime and target tests.

Existing tests and local design history

tests/test_boxed_callables.rs already covers heterogeneous collections and factory returns, generic callable constraints, shared calls, owned/borrowed arguments, inline results, rejection of unsupported modes, ownership and destruction. Runtime tests in plenty-runtime/src/tests.rs include callable_boxes_preserve_adapter_concrete_descriptor_and_relocated_captures and the allocation-checks test failed_callable_box_drops_the_concrete_environment_once. These tests were inspected, not rerun as part of this investigation; their existence is a useful implementation template, not new validation evidence.

Local issue #25 and merged local PR #39 record the shipped BoxedCallable work. The issue is closed; its resolution names merge commit b25a2309368070c90fb69a42a4ccbc1480857717 and implementation commit bc6b1f79d7bb4959671fde60c75c4ff46d91bdf5. Its description and PR explain explicit allocation, concrete descriptor retention, and deliberately excluded mutable/consuming/borrowed/worker-crossing modes. These numbers refer to the shared local issues.db, not remote records.

The original protocol design explicitly deferred existential interface values and called out method tables, ownership and restrictions on implementing-type/generic signatures. The error-composition investigation separates static bounds, finite error sums and erased error objects; allocation while handling allocation failure remains a particular concern.

The boxed callable reference is authoritative for the shipped erasure contract. The protocol reference confirms that arbitrary anonymous sums do not forward user protocols, while std.fmt.Formattable has specific sum forwarding. Its writer-generic method is documented in reporting. Do not infer general protocol-object support from that special case.

Semantic search was checked fresh against the primary checkout before drafting. Related hits were verified against current source or local records. Some old HTTP route discussions predate enum methods; their historical limitations are not evidence that current enums lack methods. The native class/enum probe above establishes the relevant current behavior directly.

Confidence and limits

The feasibility conclusion is supported by current type checking, successful single-method erasure, and inspected ownership/runtime mechanisms. It does not establish a finished mutable protocol-object ABI, safe capability transfer, cross-platform qualification, code-size impact, or dispatch performance. The qualification criteria distinguish those obligations from the four baseline probes. No compiler/runtime feature has shipped in this work.

Boxed protocol qualification criteria

Implementation note: the current reference describes delivered behavior. This page preserves the original design and qualification plan; research-era implementation statements below are historical. These are acceptance criteria for the boxed protocol design, not a second work queue or a claim of implementation. B29 in the backlog owns scheduling. The research evidence records what was actually run.

Suggested implementation sequence

An internal spike can start with one shared receiver method and two concrete payload layouts. Prove construction, list iteration, indirect calls and correct drop before generalizing to several methods. Reuse descriptor-driven storage and generated adapters rather than building a second owner runtime.

Then implement eligibility, parameterized protocol identity, exact conformance, generic-bound integration and mutable receiver loans. A shared-only spike is useful evidence but does not meet the proposed shared-and-mutable contract. Complete type walkers, effects, visibility, diagnostics and tooling before presenting the feature as supported.

Keep these stages inside one agreed implementation scope or explicitly narrow the public contract before shipping a subset. Do not silently omit a method from the table because its ownership handling is unfinished.

Required behavior and regression coverage

AreaAcceptance evidence
Heterogeneous storageA class and an enum with different layouts share a list, dictionary values, fields, enum payloads and factory result type. Borrowed iteration calls the correct implementation without matching.
Shared stateAt least two methods, including a mutable method, operate on one instance. Repeated calls and moving the box preserve that state. Shared references cannot call mutable methods.
Generic useBox[P] works through T: P, fixed protocol arguments infer correctly, aliases preserve identity, and extra concrete methods remain inaccessible.
ABIScalar/unit/inline aggregate results; owned and reference/view arguments; fallible results; receiver modes; parameter count limits and argument evaluation order.
LoansNested projections, mutable list iteration, dictionary get_mut, temporaries, reentrant argument evaluation, and rejection of moves/removals/replacement during live loans.
Lifetime and destructionCollection growth, reorder, extraction, replacement, early return and ordinary drop destroy each concrete payload exactly once, including custom destructors and nested owners.
Allocation failureInject failure at box construction and collection growth/insertion; verify the existing consumed-input contract, no leaks/double drops and valid remaining owners.
Eligibility diagnosticsReject generic requirements, consuming receivers, all returned-borrow forms, and implementing-type occurrences outside the receiver, including through aliases and nested types. Existing static generic uses remain valid.
PrivacyCross-module public protocol/private implementation, inaccessible required methods, hidden signature types, imported aliases, and no capability gained from adapter emission context.
Unsupported operationsReject bare protocol values, implicit erasure, boxed-input construction, extraction, copying/equality/hashing, keys/set elements, downcasts and protocol conversions. Nested aggregate operations obey the same restrictions.
Capabilities and effectsReject uncertified worker/channel transfer at every nesting level. Audit Shared cycles, hidden executor/future ownership, indirect blocking effects, locks and actor/task handoff.
Artifacts and toolingRetain every adapter and destructor through pruning/linking; correct IR dumps, diagnostics/navigation, deterministic table layout and cache invalidation after protocol or implementation changes.

Independent IR tests should reject malformed protocol calls even if normal frontend lowering would not emit them. Runtime tests should protect the owner header, table address, payload relocation and destruction invariants, mirroring the concrete descriptor tests for boxed callables.

Performance questions

Compare equivalent enum-wrapper, concrete boxed, boxed callback and boxed protocol workloads. Keep payloads, allocation counts and method work comparable; an inline enum versus one allocation per object measures storage as well as dispatch. Separate construction, iteration, mutation and destruction costs.

Measure payload-size variation, homogeneous versus mixed implementations, method count and number of concrete implementations. Record native code/table size and compilation time as well as runtime and allocation counts. Verify that moves, method dispatch and compiler-provided cleanup introduce no extra allocations. Do not promise a precise instruction count or overhead before measurement; optimizers may remove some indirect calls in particular programs.

Documentation and release qualification

On implementation, update protocol and box references, ownership/returned-borrow limits, generic conformance, architecture/support and runtime representation notes. Update broad statements such as “no dynamic dispatch” to distinguish static generic calls from explicitly erased owners. Teach runnable list and dictionary-value examples after classes/enums, retaining enum wrappers as the clear finite-alternative option. No new heterogeneous key/set lesson is implied.

Use the repository’s required formatting/lint and compiler/runtime test commands, including allocation-checks coverage; test executable lessons and all docs CI checks. Qualify native object relocations and calls on supported targets before claiming portability. Keep proposal status, delivery records and B29’s remaining scope aligned with the exact subset that ships.

Decisions to revisit only with evidence

The recommended initial contract makes concrete choices: explicit Box[P], shared/mutable methods, no consuming/returned-borrow methods, and no certified thread transfer. Implementation may uncover reasons to revise those choices; record such revisions before relaxing a safety boundary.

The exact internal layout, table-sharing cache and diagnostic wording remain implementation choices. Bare borrowed protocol views, allocation-free erasure of existing boxes, cross-protocol conversion, method-generic dispatch, downcasting, copying and erased error reporting are outside this scope. Their absence does not block iteration and ordinary protocol method calls.

GCC backend (plenty_codegen_gcc)

Status: Not implemented (B20). The compiler has no --backend option. This page is the only record of the GCC backend design. API facts were checked against the gccjit 7.1.0 crate source and the libgccjit 17 documentation on 2026-10-07.

Out of date since then: line numbers, file sizes, and test counts refer to the tree on 2026-10-07. The Cranelift backend has grown from about 2,840 lines in five files to about 4,900 in src/codegen.rs and fifteen submodules, plus src/tail_abi.rs. A revised plan must cover what was added: parallel function emission (src/codegen/output.rs), which libgccjit’s global lock constrains; closures, threads, channels, executors, exports, and foreign calls; and inline aggregate values. Native emission supports only x86_64-unknown-linux-gnu (B18).

This backend design is separate from the native-platform assessment: another code generator does not by itself port the runtime, standard library, or foreign-call contracts.

Goal

Add a second AOT backend that uses GCC, through libgccjit, to make native code. Cranelift stays the default backend. The model is rustc_codegen_gcc.

                              ┌── codegen_cranelift ──> CLIF ──────┐
Op ── op::check ── codegen_ssa┤                                    ├──> .o ──cc──> executable
                              └── codegen_gcc ──> libgccjit trees ─┘

Why

  • GCC’s optimizer can make faster code, and with -Os, smaller code.
  • GCC supports more CPU targets than Cranelift.
  • It does not change runtime memory use. That comes from the value and heap design, not the backend.

Cost

  • System dependency. libgccjit.so must be installed to use the backend. Use the crate’s dlopen feature (see below) so the default build and the default binary do not need it.
  • One target per install. One libgccjit build makes code only for the target GCC was built for. To cross-compile, you need a separate libgccjit for each target.
  • Slower compiles, more compiler RAM than Cranelift.
  • Two backends to maintain. Every new Op needs two lowerings, unless you first add a shared lowering layer (see Phase 1).
  • License. The gccjit crate and libgccjit are GPL-3.0. See “License”.

License

Not legal advice. This is the common reading.

Plan: license Plenty as MIT OR Apache-2.0. The repo has no license file now.

What works:

  • All of Plenty’s own source can be MIT OR Apache-2.0. Both licenses are compatible with GPL-3.0, so GPL code can sit beside them.
  • User programs are not affected. The runtime that is linked into compiled Plenty programs is Plenty’s own MIT/Apache code. Machine code that GCC makes is not covered by GCC’s license.

What does not work:

  • A distributed plenty binary built with gcc-backend includes the gccjit crate (GPL-3.0). That binary must be distributed under GPL-3.0 terms, including an offer of its source.
  • dlopen does not fix this. It removes the link-time need for libgccjit.so, but the crate’s Rust code is still compiled into the binary.

Options:

  1. Keep gcc-backend off by default (recommended). Default binaries stay pure MIT/Apache. Only builds with the GCC backend carry the GPL. Say this in the README.
  2. Write your own bindings for the about 40 libgccjit functions you need, instead of using the GPL crate. libgccjit itself is still GPL-3.0, and the FSF treats dynamic loading as a combined work. This option is weaker than it looks.
  3. Emit C and run gcc as a separate program. This avoids the question completely, because Plenty does not link GCC. It is not the rustc_codegen_gcc model.

Setup:

  • Set license = "MIT OR Apache-2.0" in Cargo.toml and plenty-runtime/Cargo.toml.
  • Add LICENSE-MIT and LICENSE-APACHE at the repo root.

How rustc_codegen_gcc does it (the model)

  • rustc_codegen_ssa holds the shared lowering. It calls builder traits (BuilderMethods, ConstMethods, …).
  • Each backend (_llvm, _cranelift, _gcc) implements those traits.
  • The _gcc crate uses the gccjit Rust crate (bindings to libgccjit).
  • It uses a patched GCC fork. Plenty does not need the fork. See “The gccjit crate” below.

The gccjit crate: facts that save time

Crate: gccjit = "7.1.0" (depends on gccjit_sys 4.1.0). Repo: https://github.com/rust-lang/gccjit.rs. Docs on docs.rs are built with the master feature, so docs.rs shows functions that you cannot use.

Features:

FeatureUse it?Why
masterNoNeeds the rustc_codegen_gcc GCC fork. Gates try/catch, attributes, get_target_info, new_temp, global_set_readonly, vector ops, get_error_count. Plenty needs none of these
dlopenYesLoads libgccjit at run time with gccjit::load(c"libgccjit.so.0"). Without it, the crate links -lgccjit, and every build of plenty needs libgccjit installed

Every API this plan needs is available without master: Context::compile_to_file(OutputKind::ObjectFile, ..), RValue::set_require_tail_call, Block::end_with_switch, Context::new_case, Function::get_address, Context::get_builtin_function, LValue::global_set_initializer (bytes), LValue::global_set_initializer_rvalue, Context::new_struct_constructor, LValue::set_alignment, Context::add_command_line_option, Context::dump_reproducer_to_file.

Linkage maps directly (gccjit::FunctionType):

Cranelift LinkageUsed forFunctionType
Exportplenty_mainExported
Localuser functions, resume bodiesInternal
Importruntime helpersExtern

Error handling gotchas:

  • In debug builds (this includes cargo test), the crate panics after any call if get_last_error() returns text. Without master, it cannot read the error count, so it panics on warnings too. Do not emit code that makes GCC warn. If you must, wrap the GCC backend in std::panic::catch_unwind and turn the panic into a Result error.
  • In release builds, it does not panic. Check ctx.get_first_error() after compile_to_file, or a failed compile is silent.
  • Any call that returns NULL panics in all builds (panic_on_null).

libgccjit rejects unreachable blocks by default. The Cranelift lowering makes blocks that have no predecessor (for example the block after a TailCall or after a trap). Call ctx.set_allow_unreachable_blocks(true), or do not create those blocks.

Optimization levels: OptimizationLevel has None, Limited, Standard, Aggressive (-O0 to -O3). There is no -Os variant. Use ctx.add_command_line_option("-Os").

Concurrency: libgccjit holds a global lock while it compiles. Parallel tests that use the GCC backend run their GCC compiles one at a time. They are correct, but slower.

Assembler: libgccjit writes assembly and runs the assembler. as (binutils) must be on PATH. Plenty already needs cc, so this is normally true.

Minimum libgccjit version

The libgccjit docs list features by ABI tag, not by GCC version. The tags this plan needs:

ABI tagAddsNeeded for
LIBGCCJIT_ABI_3end_with_switch, new_caseGenerator resume dispatch
LIBGCCJIT_ABI_6set_bool_require_tail_callMandatory TCO
LIBGCCJIT_ABI_9function_get_addressResume and drop callbacks
LIBGCCJIT_ABI_14global_set_initializer (bytes)String literals
LIBGCCJIT_ABI_19global_set_initializer_rvalue, struct/array constructorsMetadata blobs with pointers
LIBGCCJIT_ABI_20INT128_T / UINT128_T, type_get_sizeInline sums (I128)
LIBGCCJIT_ABI_24lvalue_set_alignmentset_align(8) on data

So the minimum is LIBGCCJIT_ABI_24. With dlopen, gccjit::load fails if the library lacks any symbol that the non-master crate binds, so the crate may need a newer tag than this. Check what is installed:

objdump -T /usr/lib64/libgccjit.so.0 | grep -o 'LIBGCCJIT_ABI_[0-9]*' | sort -uV | tail -1

Current state (what the GCC backend must match)

The Cranelift backend is about 2,840 lines:

FileLinesContents
src/codegen.rs2029Driver, runtime declarations, functions, Lowerer
src/codegen/enums.rs313Sum types
src/codegen/generators.rs231Resumable bodies
src/codegen/collections.rs134Collection ops
src/codegen/metadata.rs133Static type metadata blobs

Where to look first:

WhatLocation
Entry point for all AOT buildscompile_ops_to_executable, src/codegen.rs:156
Object emission, passes 1–3compile_to_object, src/codegen.rs:212
ISA flagshost_isa, src/codegen.rs:273
Runtime helper table (34 imports)Runtime / declare_runtime, src/codegen.rs:293, :346
User function signatureuser_fn_signature, src/codegen.rs:528
Plenty type to machine typeclif_type, src/codegen.rs:836
Trap sequencetrap_if, src/codegen.rs:1477
Tail calllower_tail_call, src/codegen.rs:1858
Resume signature, func_addrsrc/codegen/generators.rs:12, :72
Data with pointersdefine, src/codegen/metadata.rs:9

Features it uses, and the libgccjit equivalent:

Cranelift featurelibgccjit equivalentNotes
Object file outputcompile_to_file(OutputKind::ObjectFile, ..)Link step with cc stays the same
Runtime calls (SystemV)new_function(.., FunctionType::Extern, ..)Same plenty-runtime archive
Compile-time stack of SSA valuesStore each pushed value in a localSee “rvalues are trees”
CallConv::Tail + return_callnew_call(..) + set_require_tail_call(true), then end_with_returnHighest risk. See Risks
Multiple return values (sig.outputs is a list)One struct type per distinct output list; return the structC has one return value. Cache struct types by output list
VariableFunction::new_localDirect match
Block params at match joinOne local per param; assign before end_with_jumpGCC’s SSA pass removes them
brif, jumpend_with_conditional, end_with_jumpDirect match
Generator resume dispatchend_with_switch + new_caseContinuation index becomes a case value
func_addr (resume, drop_callback)Function::get_address
*_overflow checked arithmeticget_builtin_function("__builtin_add_overflow") etc.Type-generic. If it fails, see Risks
trap after trap helperCall helper, then __builtin_unreachableHelper does not return
selectTwo blocks and a localNo ternary rvalue without master
fcvt_to_sint_sat / _uint_satExplicit NaN check, compares, and clampsA plain C cast is undefined when out of range
I128 (inline sums)new_c_type(CType::UInt128t)Must match Rust u128 ABI in the runtime
Stack slotsLocal array or struct + LValue::get_address
Static string datau8 array global + global_set_initializer
Data that points to other dataStruct global + new_struct_constructor + global_set_initializer_rvalueSee “Metadata blobs”
set_align(8)LValue::set_alignment(8)
is_pic, preserve_frame_pointersadd_command_line_option("-fPIC")Frame pointers are a Cranelift tail-call need only. Confirm in Phase 0 that the object links as a PIE

rvalues are trees, not SSA values

In Cranelift, a Value is computed once. You can use it many times.

In libgccjit, an rvalue is an expression tree. If you use the same rvalue twice, GCC computes it twice, and side effects (such as runtime calls) happen twice. Rule for the GCC backend: assign each pushed value to a new local, and push the local. This is how rustc_codegen_gcc handles it too. GCC’s optimizer removes the extra locals.

Metadata blobs

metadata.rs makes each blob as bytes plus a list of (offset, DataId) pointer slots. To build the same blob in libgccjit:

  1. Sort the pointer offsets.
  2. Make a struct type. Bytes between pointer slots become u8[N] fields. Each pointer slot becomes a void * field.
  3. Build a new_struct_constructor with array constructors for the byte fields (one rvalue per byte: a string literal cannot fill a char array) and get_address of the target global for each pointer field.
  4. Set alignment 8.

Phase 1 should make the bytes-plus-slots form the backend-neutral output of metadata.rs, so both backends use the same layout code.

Undefined behaviour

GCC optimizes on C undefined behaviour. Cranelift does not. Rules:

  • Do wrapping arithmetic on unsigned types, then cast back.
  • Check shift amounts before you shift.
  • Add -fno-strict-aliasing with add_command_line_option. The runtime and the generated code read the same memory through different types.

Phases

Phase 0: spike (small)

Goal: prove the toolchain works before you change the design.

  1. sudo dnf install libgccjit libgccjit-devel. Neither is installed now. System GCC is 16.2. Run the objdump command above.
  2. In a scratch crate with gccjit = { version = "7.1", features = ["dlopen"] }:
    • write an object file with one exported plenty_main,
    • call one runtime helper (for example plenty_print_str),
    • link it with libplenty_runtime.a through cc. Extract the archive with plenty --emit-runtime DIR.
  3. Test mandatory tail calls, at -O0 and -O2:
    • self tail call, mutual tail call,
    • a callee with more arguments than the caller (more than 6 integer arguments on x86-64, so some go on the stack),
    • a callee that returns a struct larger than 16 bytes (returned in memory),
    • a callee that takes or returns unsigned __int128.
  4. Test __builtin_add_overflow through get_builtin_function on u8, i32, and i64.
  5. Test a global whose initializer holds the address of another global.
  6. Build in debug mode and confirm that no step panics on a warning.

Exit: a short note on which features work, and which need a fallback.

Phase 1: backend seam (large, no behaviour change)

Goal: split the shared lowering from the Cranelift-specific code, like rustc_codegen_ssa.

  1. Move backend-neutral work out of codegen.rs:
    • collect_user_fns, collect_str_ids, needs_local_addresses,
    • the runtime helper table (names and signatures as plain data),
    • metadata blob layout (bytes plus pointer slots).
  2. Define builder traits for what Lowerer needs. Start from the feature table: constants, arithmetic, checked arithmetic, compares, casts, calls, tail calls, blocks, branches, switch, locals, memory load/store, data addresses.
  3. Make Lowerer generic over the builder. Implement the traits for Cranelift.
  4. All existing tests must pass with no changes.

Design notes:

  • gccjit handles carry a 'ctx lifetime (RValue<'ctx>, Block<'ctx>). Give the traits a lifetime parameter, as rustc_codegen_gcc does with CodegenCx<'gcc, 'tcx>. Do this in the first version of the traits, or you must change every signature later.
  • Model “value” in the trait as something that a backend can read many times with no side effects. Cranelift returns its SSA Value. GCC returns a local.
  • Model block params as “declare join values, assign, jump”. Cranelift maps this to block params; GCC maps it to locals.

This is the largest phase. It is also useful on its own: it makes new Op lowerings easier to write and review.

Alternative: skip Phase 1 and copy Lowerer into a GCC version. This is faster to start, but every later Op change needs two edits that can drift apart. Use this only if you want a throwaway experiment.

Phase 2: GCC backend, core (medium)

Put it in src/codegen/gcc/ behind a Cargo feature gcc-backend (optional dependency on gccjit with dlopen).

Order of work, smallest first:

  1. Integers, booleans, Display of numbers, plenty_main, exit status.
  2. User functions, calls, multiple returns, tail calls.
  3. match, patterns, join locals.
  4. Strings and static data.
  5. Checked arithmetic and traps.
  6. Floats and saturating casts.

Crate layout note: a separate plenty-codegen-gcc workspace crate would need Op and Ty from plenty, and plenty would need the backend. Cargo does not allow that cycle. Use a module behind a feature now. Split out a plenty-core crate later only if you need it.

Phase 3: GCC backend, full coverage (medium)

  1. Sum types (enums.rs), including I128 inline sums.
  2. Collections.
  3. Metadata blobs with pointers.
  4. Generators (switch-based resume, get_address for callbacks).

Phase 4: CLI and tests (small)

  • CLI: --backend cranelift|gcc. Default cranelift. If the binary was built without gcc-backend, or gccjit::load fails, give a clear error.
  • Tests: all 20 test files that compile programs reach compile_ops_to_executable (src/codegen.rs:156), most through tests/support/mod.rs. Read a PLENTY_BACKEND env var there, and one change runs the full AOT suite on GCC: PLENTY_BACKEND=gcc cargo test --features gcc-backend.
  • Run the GCC suite at -O0 and -O2. Undefined behaviour bugs show only at -O2.
  • CI: add one job that installs libgccjit and runs the command above.
  • Docs: update the build table in CLAUDE.md and the toolchain reference.

Phase 5: options (optional)

  • -O0 / -O2 / -Os flag. Use set_optimization_level for -O0–-O3 and add_command_line_option("-Os") for size.
  • Debugging: dump_reproducer_to_file(path) writes a C program that rebuilds the same context. set_dump_code_on_compile(true) prints the assembly. set_dump_initial_gimple(true) prints GCC’s view of your trees. Put these behind an env var such as PLENTY_GCC_DUMP.

Risks

RiskImpactMitigation
GCC cannot always do a required tail call. On x86-64 SysV, a callee with more stack arguments than its caller fails. Cranelift’s Tail convention handles this; the C ABI does not.Compile error for valid Plenty programsPhase 0 test. If it fails: pass arguments through a pointer to a frame struct when they do not fit in registers, or turn tail calls into a loop with a dispatcher (trampoline)
Tail calls to functions with multiple returns. A struct return larger than 16 bytes goes through a hidden pointer, and GCC may refuse the tail callCompile errorPhase 0 test. If it fails: pass the caller’s return pointer explicitly as a parameter
__builtin_*_overflow does not work through get_builtin_functionNo checked arithmeticUse the typed builtins (__builtin_sadd_overflow, __builtin_saddl_overflow, __builtin_uaddll_overflow, …). For 8- and 16-bit types, compute in 32 bits and check the range
Installed libgccjit is older than LIBGCCJIT_ABI_24No metadata alignment; gccjit::load failsCheck with objdump in Phase 0. Show the required tag in the load error
Debug-build panics on GCC warningsTest crashes with a GCC messageFix the code that causes the warning; catch_unwind at the backend boundary
Undefined behaviour in generated treesWrong code only at -O2Rules above; run the suite at -O0 and -O2
I128 ABI mismatch with the Rust runtimeCorrupt sum valuesPhase 0 test with a runtime call that takes and returns u128
Two backends driftTest failures on one backend onlyPhase 1 seam; run all AOT tests on both backends

Effort

PhaseSizeNote
0 SpikeSmallA few evenings. It decides if the plan works
1 SeamLargeTouches all 2,840 backend lines; no new features
2 CoreMedium
3 FullMediumGenerators and metadata are the hard parts
4 CLI/testsSmallOne env var covers the whole suite

The total is about the same as writing the Cranelift backend again. Phase 1 returns some of that cost, because each later backend or Op is cheaper.

Open questions

  1. Do you want Phase 1 (shared lowering), or a separate copy of Lowerer for an experiment?
  2. Which license option (see “License”) do you want for GCC-enabled builds?
  3. Is cross-compilation a goal? If yes, the “one target per libgccjit” limit matters, and you need a separate libgccjit for each target.
  4. Should the GCC backend be in the default CI, or in an optional job only?

User-defined ordering

Status: Not implemented. Classes and enums have structural == and no ordering. The current contracts are classes, enums, tuple ordering, list sorting and std.collections. Remaining work belongs in B37. Equality is shared with the sibling hashing proposal.

Timsort is a separate algorithm and stability proposal. A comparison hook determines the order; it does not choose the sorting algorithm, require workspace allocation, or guarantee stability.

Problem

Ordering accepts integers, floats, str, and tuples of those. Every other type is rejected: a < b on a class reports “operator < does not accept”, and list.sort(), sorted, Heap[T], and the std.math helpers min and max inherit that limit. The workarounds are per call site:

  • items.sort_by_key(key) with a key function returning an orderable tuple;
  • HeapBy(before) with a stored comparator;
  • an ad hoc method such as as_nanoseconds() in std.time, which says “Ordering has no operator”.

A key cannot be a reference, so a key holding a str field copies the string on every comparison. A type that should have one natural order states it again at every sort, heap, and comparison, and the statements can disagree. Classes are not orderable tuple components, so (priority, task) cannot be ordered when task is a class.

Only __init__, __del__, __enter__, and __exit__ are recognized hooks today; any other dunder is an ordinary method.

Recommendation

A class or enum defines its order with one three-way method:

class Version:
    major: i64
    minor: i64
    label: str

    def __cmp__(self, other: &Version) -> Order:
        order = self.major.compare(&other.major)
        if order != Order.Equal:
            return order
        order = self.minor.compare(&other.minor)
        if order != Order.Equal:
            return order
        self.label.compare(&other.label)

Order is a prelude enum with nullary variants Less, Equal, and Greater. Every orderable built-in value gains x.compare(&y) -> Order, using the existing total sorting order. A type with __cmp__ is orderable everywhere an orderable built-in is: all six comparison operators, sort, sorted, keys of sort_by_key, Heap[T], tuple components, and generic bodies that use <.

Rules

  1. The signature is exactly def __cmp__(self, other: &T) -> Order, where T is the declaring type (for a generic type, the same instance). It has no type parameters, defaults, or Result. Classes and enums may define it; anonymous sums may not.
  2. All six operators derive from it. a < b is __cmp__(&a, &b) == Less; a <= b is != Greater; a > b and a >= b are the mirrored tests; a == b is == Equal; a != b is != Equal. Individual operators cannot be defined.
  3. Defining __cmp__ replaces structural == and != for that type. There is one source of truth for equality and order, and they cannot disagree.
  4. __cmp__ is not called by name. a.compare(&b) is the spelling, for user and built-in types alike. This follows the existing rule that rejects explicit __init__/__del__ calls.
  5. Like __del__, the hook belongs to the type: it applies wherever the type is visible, independent of the method’s own visibility.
  6. The order must be total and consistent: antisymmetric, transitive, and stable for unchanged values. The compiler cannot check this. A violation gives an unspecified order, as an inconsistent HeapBy comparator does today, but cannot break memory safety.
  7. Its body may perform only computation, as the worker effect walk classifies it: no I/O, clock, entropy, or process effects.

Alternatives compared

(a) six dunders(b) one __cmp__(c) derived(d) comparators only
Methods writtenUp to six, or a total_ordering ruleOneNoneOne per call site
Operators can disagreeYesNoNoNot applicable
== consistent with orderBy conventionBy constructionBy constructionNo relation
Lexicographic compositionTwo calls per field (a < b, b < a)One call per fieldGeneratedKey tuples
Field order is semanticNoNoYesNo
New surfaceSix hook namesOne hook, Order, compareOpt-in syntax or noneNone

(a) Python’s six rich comparisons (__eq__, __ne__, __lt__, __le__, __gt__, __ge__). Python lets each disagree and returns NotImplemented for mixed types; functools.total_ordering fills in the missing ones. A Plenty version would need a rule for which subsets are legal, a consistency rule it cannot check, and checker work for six signatures per type. Sorting needs only <, but composing an order over fields needs equality as well, so a tuple (priority, task) would call __lt__ twice per tied component. (CPython’s C API combines these into one tp_richcompare slot with an operator argument, which PyO3 exposes as __richcmp__; that is a dispatch detail, not a Python-level method.)

(b) One three-way method, as in Rust’s Ord::cmp, Python 2’s __cmp__, and C++’s operator<=>. One method answers every operator and composes lexicographically in one call per component. The cost of deriving an operator is one compare of the returned tag (see Costs).

(c) Derived structural ordering, lexicographic over fields in declaration order like tuples. Automatic ordering would make every class orderable, including types where order is meaningless (instants from different clocks, handles), and would make reordering fields a silent behavior change. Opt-in derivation needs attribute or decorator syntax that Plenty does not have, and B45 already declines automatic derive for formatting. Defer it; a later derive, if any, generates an ordinary __cmp__.

(d) Comparators only (status quo). It keeps the language unchanged, but leaves <, min, max, Heap[T], and tuple components unavailable, repeats the order at each use, and makes str keys copy per comparison. Keep sort_by_key and HeapBy for orders other than the natural one.

Total and partial order

Plenty has one total order for sorting and tuple operators, and IEEE behavior for scalar float operators. Rust splits PartialOrd from Ord so that floats can have operators but not sort; learners then meet total_cmp anyway. Plenty should not add that split. __cmp__ returns one of three answers, so every user order is total. x.compare(&y) on floats uses the sorting order: -0.0 before 0.0, every NaN equal to every other NaN and after all numbers. A class wrapping an f64 and comparing with compare therefore has NaN == NaN under its own ==, matching its order. That differs from scalar f64 ==, deliberately: the class’s equality follows its declared order.

There is no Unordered variant. A type with values that genuinely do not compare (instants from different clocks) should not define __cmp__.

Equality

Classes and enums currently use structural equality, which the hashing proposal also depends on: a hash must agree with ==. Rule 3 makes __cmp__ the definition of equality for an ordered type. The hashing proposal admits only structural equality and hashing, so a type that defines __cmp__ is not a key type: its == is user code that a structural hash cannot follow. To key a dictionary by such a value, key on a projection of the fields __cmp__ compares. A type without __cmp__ keeps structural == unchanged and stays eligible as a key.

Making __cmp__ define == also lifts a limit: structural equality rejects recursive data, but a written __cmp__ may recurse explicitly through Box[T] fields, giving such types both equality and order.

Use sites

Operators and generic code. Generic bodies are checked per concrete instance (protocols), so std.math.min, max, std.test.less, and Heap[T] accept ordered user types without signature changes. min_all/max_all still require copyable elements, and class instances are affine unless declared @copy, so they remain unavailable for other classes.

Protocols. No Ordered protocol is needed for generic bodies. Protocols constrain only classes, enums, and bytes, so a protocol could not admit i64 today. A user protocol may still require __cmp__ to restrict a parameter to ordered user types; whether built-in orderable types should satisfy such a protocol is an open question.

Sorting. items.sort() and sorted(items) accept element types with __cmp__; sort_by_key accepts keys of such types. The sort itself allocates nothing; allocation inside __cmp__ is the type’s own, as for key callbacks.

Tuples. A tuple whose components are orderable, including ordered user types, is orderable. Heap[(i64, Task)] orders by priority, then by Task.

Dictionaries and sets. Unchanged here; see the hashing proposal.

Ownership

Both operands are shared borrows: self and other: &T. A comparison never moves, copies, or drops an operand. a < b on owned values borrows both for the call, as tuple comparison observes them today; comparing &T values passes the references through. Because the receiver is shared, __cmp__ cannot mutate either value, so a sort cannot change the elements it is ordering. The list remains exclusively borrowed by sort; the hook receives element loans derived from that borrow, as sort_by_key callbacks do.

Errors, allocation, and workers

__cmp__ cannot fail: it returns Order, not Result, so ? is unavailable inside it. Allocation in Plenty is always fallible and visible as a Result, so a comparison that allocates must handle the failure locally. Traps (overflow, indexing) remain traps and abort the sort like any other trap.

Rule 7 restricts the body to computation. This keeps comparisons pure for learners, lets sort keep its documented contract apart from the hook’s own allocation, and makes every ordered type usable from workers. It reuses the existing closed-world walk in src/effects.rs and threading::check, which already certifies every reachable body; no new eligibility mechanism is needed. Whether to also forbid allocation is an open question.

Costs

Measured today

A probe on this branch (--release, Cranelift 0.131, 1,000,000 pairs, single core, median of seven runs, construction subtracted; checksums equal):

SortPathTime
list[i64].sort()Runtime, Rust sort_unstable_by over scalar rows78 ms
list[(i64, i64)].sort()Runtime heapsort, descriptor-interpreted compare1,444 ms
list[Point].sort_by_key(key)Plenty-source heapsort, runtime OrderCompare per comparison4,142 ms
list[Point] heapsort in Plenty, direct cmp(&Point, &Point) -> i64Models the hook lowered through a generated sort2,473 ms

perf attributes the last row’s time to runtime calls for element access and row exchange (plenty_collection 48%, Payload::relocate 15%, plenty_retain/plenty_release 19%, collection_at 7%). The comparison function itself is 4%; replacing the call with inline field compares did not improve the time. The tuple sort spends 27% in the runtime’s interpretive compare (plenty-runtime/src/aggregates.rs), which dispatches on the descriptor kind and recurses per field on every comparison. Sizes: the direct comparison function is 400 bytes, a specialized Plenty heapsort 2.8 KB, and a sort_by_key instantiation 3.7 KB.

The comparison call is not the bottleneck; how the sort reaches the elements is.

Issue #331 later moved sort_by_key into the runtime: it sorts rows with the wide-row introsort and calls a generated native comparator, with no per-access runtime call or reference count change. Natural tuple sorts resolve field offsets once per sort, and moved flat records repair their address without a descriptor walk. On a contended host (median of three paired runs, 1,000,000 elements, sort time only) list[(i64, i64)].sort() went from about 725 ms to 265 ms and list[Point].sort_by_key with an i64 key from about 3,090 ms to 330 ms, against 74 ms for list[i64].sort(). Option 1 below can therefore reuse that row path.

Dispatch

The hook is resolved statically: a < b lowers to a direct call of the class’s mangled method, as with lowers to Op::Call of __enter__ in src/frontend/contexts.rs. There is no vtable or runtime lookup. Plenty does not inline calls. Cranelift 0.131 offers Context::inline with an embedder-supplied Inline policy, but the compiler does not use it, so every user comparison costs one call. A probe returning a three-variant nullary enum compiled to a register return with no runtime call; a less wrapper around it was 39 bytes. Deriving < from __cmp__ therefore costs one integer compare on the returned tag, versus a direct __lt__: no measurable difference, and lexicographic composition saves a second call per tied component.

Compared with built-ins: integer < is one instruction; str and tuple < already call the runtime (TextCompare through plenty_collection, and OrderCompare through the narrow plenty_order_compare). A user __cmp__ over integer fields is a direct call with field loads, and is expected to be no slower than tuple <; measure it (below).

Sort path

Two lowerings are possible for list.sort() on user elements:

  1. Runtime sort with a descriptor hook. Runtime type descriptors already carry one callback into generated code: the class __del__ adapter (Type::drop in plenty-runtime/src/aggregates.rs, emitted by src/codegen/metadata.rs). A compare slot of the same kind lets the runtime’s compare call __cmp__ for a record that has one. This makes sort, tuple operators with user components, and OrderCompare keys work with one runtime change, keeps the runtime’s row exchange, and adds one indirect call per comparison. It grows every descriptor from 64 to 72 bytes of static data, unless the slot replaces padding in a reorganized layout.
  2. Generated sort, specialized per element type as sort_by_key was before issue #331. It calls the hook directly, but the probe shows such a sort is 1.7 times slower than the runtime tuple sort because each step reaches elements through runtime calls, and it adds about 3 KB per element type.

Recommend option 1 for sort, sorted, tuple components, and keys, and direct generated calls for operators in ordinary and generic code.

Compile time

Detection is one method-name lookup per class or enum and one signature check, done once per type; operator typing gains one branch in is_orderable. Each instantiation of a generic body using < on a new type is already a separate specialization, so ordered user types add no new instantiation mechanism. The probe programs checked in about 2 ms each, too small to separate the cost.

Measurements required before shipping

  • Extend examples/collection_bench.py with sorts of list[(i64, i64)], list[Point] by __cmp__ (both lowerings above), list[Point] by sort_by_key, and a str-field class, at 100,000 and 1,000,000 elements, recording binary size per variant.
  • A Heap[Task] push/pop benchmark against HeapBy with the same order.
  • compile_bench --workload check and --workload aot (see profiling) before and after, with a generated workload of many ordered classes compared and sorted, to confirm detection and specialization cost stays in the noise.
  • Object-size check of the descriptor change across the test suite’s executables.

Naming

std.atomic already exports Ordering for memory ordering. It is not in the prelude, but from std.atomic import Ordering would collide with a prelude Ordering. Use Order for comparisons: short, distinct, and read naturally as Order.Less. The hook name __cmp__ follows Python 2 and matches the three-way shape; __order__ is the alternative if the Python 2 association is unwanted.

Order provides reverse(self) -> Order for descending orders and may provide then(self, next: Order) -> Order for short compositions. then evaluates its argument eagerly, so early returns stay the idiom for expensive fields.

Rejected

  • Six rich-comparison dunders: allows inconsistent operators and doubles calls in lexicographic composition.
  • PartialOrd/Ord split: two concepts for learners; Plenty already chose one total order for sorting.
  • Automatic structural ordering: makes field order semantic and gives order to types where it is meaningless.
  • Keeping structural == beside __cmp__: two definitions of equality that can disagree, breaking sorting and hashing invariants.
  • Fallible comparisons (-> Result[Order, E]): every operator and sort would become fallible; a failing comparison has no sound place in <.
  • A runtime callback for every comparison in generic code: the static call is cheaper and already available.

Open questions

  • Should __cmp__ also be forbidden from allocating, beyond rule 7?
  • Should built-in orderable types satisfy a protocol that requires __cmp__? That needs methods on i64 and tuples visible to protocol matching.
  • Should Order and compare be prelude names, or imported from a small module? The hook’s signature mentions Order, which argues for the prelude.
  • Does then earn its place, or should composition use early returns only?
  • Should nullary enums get a declared-order shortcut, or keep writing __cmp__ over a rank method?
  • Coordination with the hashing proposal: one equality hook, and the required agreement between hash and __cmp__.

Staged implementation

  1. Hook and operators. Recognize __cmp__ beside __init__/__del__ (src/frontend/classes.rs, src/frontend/enums.rs), validate its signature, and reject direct calls. Add Order to the prelude (src/frontend/enums.rs, src/frontend.rs) and compare for built-in orderable types. Extend Ty::is_orderable (src/op.rs) and binary operator typing (src/frontend.rs, near “operator {op} does not accept”) so comparisons on such types lower to the hook call and a tag test; route ==/!= there too instead of structural equality (lower_equality in src/codegen.rs). Enforce rule 7 through src/effects.rs. Generic users (std.math, std.test, Heap[T]) then work unchanged.
  2. Runtime ordering. Add the descriptor compare slot (plenty-runtime/src/aggregates.rs, src/codegen/metadata.rs) with an adapter like the __del__ hook, so sort, sorted, OrderCompare keys, and tuple components with user types use it (src/frontend/sorting.rs, int_cmp in src/codegen.rs).
  3. Documentation and lessons. Update the class, enum, tuple, list, and std.collections reference pages, std.time (which can then define Duration ordering), the priority-queue lesson, and B37.

Each stage ships with diagnostics tests for invalid signatures, direct calls, effectful bodies, and inconsistent orders that stay memory safe, plus the measurements above.

Timsort for stable, adaptive sorting

Status: Not implemented. This proposes an algorithm and an API direction, not an approved migration. List sorting and the key-sorting tutorial define today’s contracts. No implementation or performance improvement ships with this proposal. Scheduling belongs to B37. Issue #331’s sorting performance fix shipped allocation-free introsort with a heapsort fallback; it did not implement stability or Timsort.

Why consider it

Stable sorting preserves the input order of equal keys. Records already ordered by time can then be sorted by category without losing their time order within each category. This should be easy to request without adding an original-position tie breaker or constructing a separate decorated list. Exploiting existing order also suits logs, grouped records, and incrementally updated data.

Today sort() is infallible and allocation-free. The runtime uses Rust’s unstable slice sort for rows without inline payloads and three-way introsort (with a heapsort fallback) for wider inline rows (plenty-runtime/src/entries.rs). sort_by_key uses the same introsort for every row width, calling a generated comparator (src/codegen/sorting.rs) that recomputes both keys per comparison. sorted explicitly copies, then invokes the same natural-order sort. Keys cannot return references or Result, and user-defined natural order is still a separate proposal.

Issue #331 identified runtime element access, row movement, and descriptor comparison costs; the runtime-driven key sort and per-sort tuple and relocation plans removed most of them. Changing algorithms alone does not remove such costs. Timsort is a candidate for stable sorting and locality improvements, not evidence that it will beat the current scalar path or fix that issue by itself.

Algorithm recommendation

Prototype Timsort: detect existing ordered runs, extend short runs with stable binary insertion, and merge adjacent runs using bounded temporary storage. Galloping searches can reduce comparisons when one run repeatedly wins. CPython’s versioned design notes describe these mechanisms and their memory tradeoff.

Specify the merge scheduler separately. Classic Timsort uses run-length stack invariants; modern CPython uses Powersort scheduling within its adaptive sort. Compare a corrected classic policy with Powersort before selecting Plenty’s implementation. Munro and Wild’s paper gives Powersort a nearly optimal merge-cost bound relative to the existing runs. That bound concerns merge work, not guaranteed wall-clock optimality. If selected, describe Plenty’s implementation as Timsort-style run handling with Powersort scheduling, rather than claiming it uses the classic Timsort policy.

The required contract is stable ordering, O(n log n) worst-case comparisons, O(n) comparisons for a single ordered run, and documented workspace bounds. Constants and insertion/galloping thresholds are implementation details. A simple first version should reverse only strictly descending runs: reversing equal elements would destroy stability. For descending output, reverse the comparison direction, not the final ascending array. Forward merges take from the left on equality; backward merges take from the right.

Surface and compatibility

The recommended eventual teaching surface is items.sort()? and items.sort_by_key(key)?: stable order with explicit allocation failure. Keep infallible, allocation-free operations under explicit sort_unstable and sort_unstable_by_key names. These are proposed spellings. Stability is a useful default, but it does not make an allocating algorithm optimal for every purpose.

Changing existing return types is a source compatibility change. Algorithm adoption does not authorize it. A compatible first release could introduce sort_stable and sort_stable_by_key, each returning Result[(), AllocError], while retaining the existing methods unchanged. A later naming decision must include migration diagnostics, reference/tutorial updates, and call-site changes. sorted already returns AllocError, so a stable implementation can preserve its signature and source-preservation guarantee, while documenting extra memory.

Other choices have different costs:

ChoiceConsequence
Keep stable methods permanently explicitPreserves compatibility and makes scratch allocation visible; ordinary key sorting still loses ties unless the caller chooses stability.
Try scratch, fall back to allocation-free unstable sorting on allocation failureCan preserve an infallible return type, but introduces hidden allocation and cannot promise stability. This does not meet the proposed stable contract.
Fall back to a stable in-place mergeCan preserve stability and avoid returning allocation failure, but simple rotation-based merges can raise worst-case work to O(n log² n); sophisticated bounded-space algorithms add substantial complexity. Attempting allocation still changes today’s allocation-free promise.
Caller-owned reusable workspaceMakes repeated allocation avoidable; useful later, but capacity/type/lifetime management should not be required for the ordinary call.

Prefer a fallible stable operation over either fallback for the first implementation. Preserve an explicit allocation-free route, and decide the default names separately after measurements and compatibility review.

Storage, failure, and callbacks

For a straightforward row-merging implementation, reserve at most floor(n/2) whole rows before any mutation. Count bytes using the runtime row stride, including inline payloads, not Python’s pointer-sized estimates. Check all size arithmetic. Use a run stack whose capacity is justified for the chosen policy and supported length range. Allocation or size failure returns AllocError with the original list unchanged. Empty/singleton lists require no workspace; a read-only single-run check can avoid reserving for already ordered input. Pre-reservation deliberately trades potentially unused memory for a simple failure guarantee; adaptive buffer growth would need a weaker guarantee or rollback and is not the initial recommendation.

Moving a row transfers ownership; it must not clone, drop, or retain each element on every merge step. Relocate inline addresses before any comparison observes the row, including rows held in scratch storage. Every original value must have exactly one live owner and remain available for destruction exactly once. Existing swap-based safety does not automatically establish merge safety. A narrow audited runtime operation is preferable to exposing unchecked moves to Plenty programs. Keep safe orchestration in Plenty where it is efficient; measure the boundary costs identified in issue #331.

Initially retain current key rules: shared &T input, reusable shared-call callback, direct orderable result, consistent keys, and no concurrent access to the exclusively borrowed list. Recompute keys; do not promise a call count or evaluation order. Preserve signed-zero, NaN, string, and tuple total ordering. Key callbacks remain responsible for explicit allocations inside themselves. Caching keys once is a separate API decision with additional owned storage and observable callback semantics; fallible keys need their own cleanup/error design.

Inconsistent keys must never cause memory unsafety, ownership loss, or an unbounded loop; sortedness is only guaranteed for a consistent order. Internal index/stack checks must not rely on comparator consistency. Plenty fatal traps do not unwind, so this proposal promises neither rollback nor recovery after a callback trap. Ordinary allocation failure occurs before rearrangement.

Evidence required for adoption

The verification work by de Gouw and de Boer found errors in widely used Timsort implementations. Review stack invariants, capacity proofs, overflow checks, and merge termination explicitly; copying a familiar implementation is insufficient. Test adversarial run lengths as well as random permutations. Include equal-key identity tags in both directions, insertion and galloping boundaries, empty/singleton inputs, NaNs/signed zeros, wide inline records, affine owners/destructor counts, and injected allocation failures that verify unchanged input. Check callback loan rejection and inconsistent-callback memory safety. Differential tests must compare both sortedness and stable identity order, not just values.

Benchmark scalar, string, tuple, and keyed-record inputs: random, sorted, reverse, duplicate-heavy, almost sorted, and uneven runs. Record comparison and key-call counts, moved bytes, peak scratch, allocations, and elapsed time; retain the current sort as the baseline. Recreate issue #331’s workloads and use the collection benchmarks without treating old measurements as new evidence. Compare classic Timsort, Powersort scheduling, a simple stable mergesort, and the existing allocation-free implementation. A different unstable runtime sort could independently address #331; record-layout work in issue #330 is separate. Publish regressions as well as wins before choosing defaults. Any implementation PR must update contracts and executable lessons together; this proposal alone changes neither.

User-defined dictionary keys and set members

Status: Structural-key policy implemented. Named enums whose payloads are supported key types now work as dictionary keys and set members, including nested enums. Classes whose fields are supported keys, including move-only classes and enums containing them, and the KeyType constraint are also implemented. Anonymous sums with supported key members are implemented. Custom-cleanup keys are admitted for borrowing and ownership transfer; copying operations reject them. Named enum and tuple sets with supported finite domains through 256 values now use bitsets, including boolean and nested enum/tuple payloads. The collection reference owns the current contract. The design and historical analysis below describe the broader proposal; enum/class admission and the explicit generic constraint have shipped. Generic bodies remain checked per concrete instance: unconstrained parameters are not rejected solely for appearing in a collection signature. The original eager-rejection analysis below is historical, not current behavior.

Problem

Before structural key admission, a program that wanted dict[Point, V], set[Color] or dict[UserId, V] had to flatten keys into tuples, losing their nominal types. Eligible classes and enums now work directly. Generic code can state its key requirement explicitly:

def count[K: KeyType](xs: &list[K]) -> Result[dict[K, i64], AllocError]:
    ...

The bound rejects unsupported arguments at instantiation. Unconstrained generic functions continue to work when their concrete keys are valid; requiring a bound on every collection signature is not part of the implemented contract.

The suggested direction is a __hash__ dunder and a protocol requiring it as the key bound. This page assesses that idea against the current implementation, then recommends a narrower design.

How dictionaries and sets work today

All dictionary and set operations enter one runtime function, plenty_collection(op, args, descriptor, site) (plenty-runtime/src/aggregates.rs:1423), with an opcode and the collection’s static type descriptor. Generated code never hashes or probes.

AspectCurrent behavior
DescriptorType (aggregates.rs:36), 64 bytes, asserted layout: kind byte, affine, reflexive, inline payload size, key/value descriptors, name, variants (fields), and a drop function pointer at byte 56. Emitted by src/codegen/metadata.rs.
Hashaggregates/hashing.rs feeds the complete descriptor-guided key into SipHash-2-4: scalar words, length-framed strings, and kind B fields framed by variant tag and field count. Classes now have a field-count-framed structural branch as well.
SeedingOne secret 128-bit OS key per process, initialized on the first hash and shared by workers. Entropy failure traps; no fixed-seed production mode.
TableVec<usize> of entry indices plus one, linear probing, power-of-two size, load at most one half (try_reserve, aggregates.rs:546). No hash is cached; rehashing and removal (aggregates.rs:699, backward-shift repair) recompute the home bucket of every key they visit.
EntriesDense, type-sized rows (plenty-runtime/src/entries.rs). Inline payloads such as multi-field tuples live in the row. Iteration walks rows, not the bucket table, so dictionary order is insertion order regardless of hash values.
Equalityequal(a, b, ty) (aggregates.rs:742), structural, also used by == on classes, enums and collections (src/codegen.rs:2481 lowers it to collection op 8). Floats compare by IEEE rules.
Ownershiptry_insert retains copyable operands and adopts affine ones. Lookups take a value or reference. Borrowed iteration yields shared &K; mutable items() lends &mut V with shared &K (iterables). No operation lends &mut K.
AdmissionTy::hashable (src/collection.rs:189): integers, bool, str, byte strings, and tuples of these. Checked where collection types resolve (src/frontend.rs:885) and where literals infer (src/frontend/collections.rs:891, :3138).
CallbacksThe runtime already calls generated code: a class’s __del__ adapter through the descriptor’s drop slot, and executor/thread adapters with a three-pointer signature (src/codegen/executors.rs). Sorting has no user comparator; compare (aggregates.rs:808) is descriptor-interpreted like hash.

The runtime relies on two properties of every legal key, recorded in try_reserve: hashing never allocates and never calls user code. Table rebuilding computes hashes alone, in the middle of a reservation, with no error channel.

Issue #329 addressed two hash-quality defects independently of this proposal:

  • The former FNV-1a path had no finalizer. Multiplication carries bits only upward, so the low k bits of a string’s hash depend only on the low k bits of each byte. A 16-bucket table (the minimum) puts "a", "q", "A", "Q", "1" and "!" in one bucket. Tables below 256 buckets are affected.
  • It had no seed: colliding inputs could be precomputed. std.toml indexes parsed keys in dict[(i64, str), i64] (stdlib/std/toml.plenty:245), so untrusted TOML can force quadratic parsing. The dictionary reference already says adversarial collisions can make lookup linear.

The replacement keys the entire structured input, rather than applying a secret finalizer to an unkeyed prehash. It covers TOML’s index and duplicate-key set, and JSON’s duplicate-member set. HTTP and CSV currently use no dictionary/set tables in their implementations.

Historical cost of a composite key before keyed hashing

A probe on 2026-10-19 (release build of this branch, Linux x86-64, one pinned core; the machine had load average about 18, so treat these as indicative) ran 20 million membership tests, half hits, minimum of seven runs. These results predate issue #329’s keyed hashing and do not describe its current cost:

EntriesEmpty loopdict[i64, i64]dict[(i64, i64), i64]
1,0240.03 s0.39 s (about 18 ns/test)0.92 s (about 45 ns/test)
65,5360.04 s0.70 s (about 33 ns/test)1.53 s (about 75 ns/test)

A two-field tuple key costs about 2.4 times an integer key, including building the probe tuple. Any user type admitted through the same descriptor path should cost about the same as the tuple with the same fields. This must be confirmed by the benchmark in Measurements.

Precedents

Python. __hash__ returns an integer and must agree with __eq__: a == b implies hash(a) == hash(b). Nothing enforces it. Defining __eq__ without __hash__ makes a class unhashable; mutable built-ins are unhashable by convention. A key mutated after insertion silently becomes unreachable. Dataclasses derive both from fields (frozen=True, eq=True).

Rust. Hash and Eq are separate traits, usually derived together. A type does not return a hash; it feeds its fields to a Hasher (fn hash<H: Hasher>(&self, state: &mut H)). The collection chooses the hasher, so seeding (SipHash with a random key in std), algorithm changes and mixing quality stay out of user code. Consistency with Eq is a documented logic requirement, not checked; violations cause wrong results but no memory unsafety. Interior mutability makes key mutation possible (Cell in a key), and is documented as a logic error.

ShapeDistribution and seedingUser can break a == b ⇒ hash(a) == hash(b)
__hash__(self) -> u64User mixes bits; no seed; poor combinations (x ^ y) are easyYes, by hashing anything not determined by equality
__hash__(self, h: &mut Hasher)Collection owns mixing and seedOnly by feeding non-structural data or by custom equality
Derived from fieldsRuntime owns everythingNo

Assessing the __hash__ protocol idea

The idea needs several things Plenty lacks, and most of them work against the project’s goals.

  1. A protocol cannot express the bound. Protocols match ordinary methods (protocols). Integers, bool, and str have no methods, so dict[i64, V] would not satisfy a method protocol unless the compiler special-cased it. Every user key type would also need a hand-written method even when the derived answer is the only correct one. A bound that admits built-ins and derived types is a built-in constraint like IntType, not a protocol.
  2. Custom hash without custom equality is nearly useless. With structural, non-overridable equality, any pure function of the fields is a consistent hash. The only gain from a custom one is speed (hash an id field and skip the rest), at the price of a callback per hash. The cases people actually write __hash__ for (case-insensitive names, ignoring a cache field) change equality.
  3. Custom equality is a larger change than hashing. == would stop being structural for that type, equal in the runtime would need a callback, and collection equality, in, set algebra and dictionary comparison would all run user code. The classes reference promises equality “neither invokes user-defined magic methods”. Allowing __hash__ and __eq__ separately also imports Python’s unenforced consistency rule, which is exactly the kind of soft rule this language avoids.
  4. Callbacks break runtime invariants. A user hash would run inside try_reserve’s rebuild, inside removal’s cluster repair (once per displaced entry), and during set algebra. It would need a no-allocation, no-trap-on-error contract, an effect restriction, and a new descriptor slot. See Where hashing runs.

The idea is right that keys need a named, generic bound. It does not need a dunder.

Where hashing runs

A. Descriptor-interpreted (today)B. Runtime calls user hookC. Generated code passes the hash inD. Monomorphized table
MechanismExtend hash/equal to classesDescriptor gains hash (and equality) function pointers, called like dropEach call site computes the probe hash and passes it to plenty_collectionDict/set code generated per key type, in Plenty or codegen
Per lookupSame path as tuplesPlus one indirect call and an adapter per hash and per probe equalitySaves descriptor walk for the probe key onlyInlined hash and equality, no dispatch
Rehash, removalRecompute by interpretationOne callback per visited entryNeeds cached per-entry hashes, since the runtime cannot hash stored keysInlined
Memory per entryUnchangedUnchangedPlus 8 bytes per row or per bucketUnchanged or less
Code size, compile timeOne descriptor per key type, already emittedOne adapter function per key typeHash code at every call site or one helper per typeWhole table per instantiation; large
Runtime invariantsKept: no allocation, no user codeBroken; needs effect and allocation rulesKept for hashing; equality still interpretedMust re-implement reservation, failure, destruction-resume and removal semantics in generated code
Consistency with ==By construction: one equalOnly if equality is also the hook’sBy construction if derivedBy construction if derived

Measured context: in-cache integer lookups cost about 18 ns, almost all of it call, dispatch, and probing rather than hashing an integer. A callback adds an indirect call, an adapter that unpacks the slot, and possibly a Hasher object; expect a few nanoseconds per hash and per equality probe, on top of the tuple-equivalent cost. That estimate needs the measurement below before B is considered.

D gives the best per-operation speed and the worst compile time, code size, and correctness surface. It would duplicate aggregates.rs’s allocation-failure and destruction-order guarantees per type. It is the right tool only if the dispatcher itself, not hashing, dominates. Issue #18 gave list reads narrow runtime helpers for that reason; the same approach could apply to hashed lookups for all key types at once.

Recommendation

Derived structural keys, a built-in KeyType constraint, and no user hash or equality hooks. The rule has no exceptions and cannot be violated:

A key type is an integer, bool, str, byte string, or a tuple, class, or enum whose every field or payload is a key type. Recursive types are not key types. Dictionary keys, set members, and KeyType arguments must be key types. Hashing and equality are structural, so a == b always implies equal hashes.

class Point:
    x: i64
    y: i64

enum Shape:
    Dot(Point)
    Segment(Point, Point)

def count[K: KeyType](xs: &list[K]) -> Result[dict[K, i64], AllocError]:
    ...

seen: set[Shape] = set[Shape]()?          # accepted
weights: dict[Point, f64] = {}?           # accepted: values may be floats
bad: set[Sample] = set[Sample]()?         # rejected if Sample has an f64 field

Rationale

  • Consistency is impossible to break. One structural equal serves == and lookup; one structural hash reads the same fields. There is no __hash__ to forget, no __eq__ to disagree with it.
  • No new runtime invariant. Hashing still never allocates or calls user code, so try_reserve, removal repair, and worker-thread eligibility need no new rules. Effects are Compute only by construction (src/effects.rs).
  • Small change. The runtime already hashes kind B structurally; it needs a class branch. The frontend needs one predicate, one constraint, and better diagnostics. Existing keys keep identical code paths.
  • Custom identity stays explicit. A case-insensitive key is a key built from the normalized text (Name(text.lower()?)), not a hidden equality. The allocation is visible at construction, not repeated per probe.

Key immutability

Keys cannot change while stored, by existing rules plus the key-type rule:

  • Insertion moves an affine key in or copies a copyable one.
  • Every read of a stored key is a shared &K; no API lends &mut K, including mutable items() views and replace(d[k], v), which targets values.
  • Key types contain no interior mutability. Excluded by the rule, without a separate list: floats (NaN != NaN, and 0.0 == -0.0 with different bits), references, Box and recursive data, collections, closures and callables, generators, atomics (std.atomic), std.sync.Mutex, channels, CancellationToken, tasks, files, sockets and other handles. A class holding any of these is not a key type.
  • Strings share immutable storage, so retaining a str key is safe.

Affine keys

Every class not declared @copy is affine (moves and copies), so class keys are the first affine keys. Operations that today retain keys into new storage need the same treatment affine values already have:

OperationAffine key rule
insert, add, updateAlready adopt affine operands
get, [], in, pop, discardAccept &K; an owned temporary is consumed
Borrowed iteration, items()Already yield &K
keys()Shared KeyView[K], iterating &K, without copying
keys_snapshot(), into_keys()Transfer affine keys from an owned temporary; use copy(d)?.keys_snapshot() to preserve the source
union, intersection, difference, symmetric_difference, key-view to_set()Copy members into independent result storage; reject keys containing custom cleanup; sources remain unchanged on success or failure
Set and dict comprehensionsMove keys in as for insert

Copyable enums (payloads of copyable key types) avoid this table entirely, which motivates the staging below.

Generic bound

KeyType is a built-in constraint beside IntType and FloatType (src/frontend/generics.rs). It accepts exactly the current collection key predicate, on functions, methods, classes and enums. An unsuitable argument reports TYPE does not satisfy KeyType, even if the generic body does not use hashing. Unconstrained generic bodies remain checked per concrete instance; dict[K, V] and set[K] do not independently require an explicit bound.

Diagnostics now name the first unsupported component in declaration order, including class field names, enum variants and payload positions, and tuple element positions. The path ends with the rejected type or a specific reason such as an unsupported anonymous-sum member; see key diagnostics.

DoS resistance and seeding

This runtime concern was implemented independently of user key admission in issue #329. SipHash-2-4 consumes all structural bytes with one OS-generated key per process. Adding a finalizer alone would improve low-bit distribution but would not prevent precomputed full-hash collisions. The key is initialized lazily and atomically; interrupted entropy reads retry, short reads accumulate, and other failures trap rather than weaken the key. Constructing empty collections does not require entropy. See the collection contract.

Dictionary iteration stays deterministic through its entry order, so tests need no public seed override. Set order remains unspecified. Keyed hashing does not remove the table’s theoretical linear-probe worst case.

The pinned siphasher 1.0.4 backend uses the original SipHash-2-4 schedule: two compression rounds and four finalization rounds. SipHash-1-3 uses fewer rounds to reduce hashing cost; this change chooses the original schedule as a conservative security margin. The collection benchmarks quantify the cost: short-key lookup is more expensive, especially for integers. Hashing remains a runtime policy, so callers need no new API or manual mixing.

Superseded 2026-11-05 (issue #747): the runtime now uses SipHash-1-3, the schedule of Rust’s HashMap, CPython and Swift, and hashes top-level str, bytes, integer and bool keys directly without the descriptor walk. Nested keys keep the framed encoding. See the collection contract.

Costs

CostRecommended design
Lookup, user keySame descriptor path as a tuple with the same fields; about 45 ns versus 18 ns for i64 in the probe above
Existing keysSipHash-2-4 adds keyed mixing cost; see the collection benchmark for measured deltas
Per fieldOne recursive structural hash/equality walk. Record::slot recomputes each offset from the start, so hashing or comparing an n-field record is O(n²) descriptor reads; walking offsets incrementally fixes it for tuples and classes alike
Memory per entryUnchanged: 8-byte bucket at load at most one half, plus a row sized from the key’s inline payload, as for tuples
Code size, compile timeOne static descriptor per key type, which collection types already emit; no per-type functions
RehashEvery resize recomputes all hashes by interpretation; composite keys pay more than integers

Caching hashes is not part of the recommendation. It would cut rehash and removal cost and give a cheap pre-equality filter, for 8 bytes per bucket (doubling the table) or per row. Measure before adding it.

Measurements

Before implementation and before any follow-up optimization:

  1. Extend examples/collection_bench.py (method in examples/collection-bench.md) with insert, hit, miss and remove workloads at 1,024 and 65,536 entries for dict[i64, i64], dict[(i64, i64), i64], dict[Point, i64] (two i64 fields), a copyable enum key, and dict[str, i64] with short keys. Pin to physical cores as in profiling.
  2. Rerun against keyed hashing; report per-operation deltas for existing key types.
  3. To evaluate design B at all: prototype a descriptor hook that returns the structural hash through an adapter and compare against A on the same workloads.
  4. Compile time: compile_bench -- 1000 10 --workload aot before and after, per profiling; expected neutral.

Rejected alternatives

  • __hash__(self) -> u64. User mixing, no seeding, and consistency is unenforced. Adds a callback for no correctness gain under structural equality.
  • __hash__(self, h: &mut Hasher). Fixes distribution and seeding, but the only benefit without custom equality is skipping fields, paid for with a callback per hash and new runtime invariants.
  • __eq__ plus __hash__. Makes == non-structural, puts user code inside every collection algorithm, and reintroduces a consistency rule the compiler cannot check.
  • A method protocol as the bound. Cannot admit built-ins without a special case and forces boilerplate on every key type.
  • Monomorphized tables (D). Compile time, code size, and duplicated failure semantics, against fast compilation. Revisit only if dispatch, not hashing, dominates measured cost, and then prefer narrow runtime helpers.
  • Hashing floats by bits. Breaks a == b ⇒ hash(a) == hash(b) for 0.0/-0.0 and makes NaN keys unreachable. A user who needs float keys can store a canonical integer encoding explicitly.

Possible later extension: key projection

If custom identity proves necessary, the consistent form is one method that defines both equality and hashing by returning a key-type value:

class Name:
    text: str

    def __key__(self) -> str:
        ...  # normalized text

==, hash, in and lookup would all compare __key__() results, so they cannot disagree. It still needs a runtime callback (design B), a no-allocation rule on the projection (normalizing text usually allocates, which defeats the motivating example), and a change to the structural-equality promise. Explicit key construction covers the same cases today; this is not recommended now.

Ordering and sorted maps

Out of scope. User-defined ordering faces the same choice for < and sort: compare (aggregates.rs:808) is descriptor- interpreted like hash, and the same derived, structural rule keeps ordering consistent with equality. A sorted map would need both bounds; if ordering is derived structurally, every key type with no float fields is also totally ordered, and the two constraints could share one admission walk.

That proposal recommends a __cmp__ hook that also defines ==. A type that defines __cmp__ is therefore not a key type: its equality is user code, so a structural hash would not agree with it. Such values are keyed by a projection of the fields their __cmp__ compares.

Open questions

  1. Name: KeyType, Hashable, or Key. KeyType matches IntType.
  2. Resolved: classes with __del__, including nested cleanup, are keys when their fields are keys. Hashability does not imply duplicability. Non-mutating set algebra and key-view to_set() require explicit-copy eligibility, while insertion, borrowed views, relationships, in-place updates and owned extraction work without copying. Equal incoming keys are destroyed; existing key owners remain in the collection. No user-defined clone hook is added.
  3. Private fields: structural hashing reads them, as equality already does. Confirm that is acceptable for public key types across modules.
  4. Resolved: anonymous sums are key types when every member is.
  5. Whether to cache hashes, after measurement.

Implementation sketch

  1. Copyable enums. Extend Ty::hashable (src/collection.rs:189) to user and standard enums whose payloads are key types, rejecting recursion through type_facts (src/type_facts.rs). The runtime path already works for kind B. Update the three diagnostics (src/frontend.rs:885, src/frontend/collections.rs:891, :3138) to name the offending field. Tests in tests/test_collections.rs, test_dictionary_lookup.rs and test_set_operations.rs; reference updates to collections.md, enums.md, sets.md (which still lists only integers, booleans, and strings) and tutorial/04-collections/dictionaries.md.
  2. KeyType constraint. Add it to the built-in constraint table (src/frontend/generics.rs:19, diagnostic at :1001); let dict[K, V] resolve in generic signatures when K: KeyType. Update generics.
  3. Classes and affine enums. Add a class branch to write_key (plenty-runtime/src/aggregates/hashing.rs) over Record fields, plus a unit test beside aggregates/hash_tests.rs that equal classes in different storage hash equally. Audit each operation in Affine keys in src/frontend/collections.rs and the runtime’s retain/release paths (try_insert, Collection::at, set algebra). Use the runtime’s running-offset record iterator; field-offset traversal is already linear per pass (issue #330). Hash quality and per-process seeding are already implemented independently (issue #329); this sketch does not change their policy.

Derived serialization for classes and enums

Status: Not implemented. Nothing shipped; this page is the design record for issue #471. std.json and std.toml remain authoritative: neither derives codecs, and a typed codec is written by hand over JSON events or a TOML tree. No backlog entry exists yet; B51 lists derived typed codecs as a candidate under B38. The design questions are decided; see Decisions.

Problem

Reading a configuration file into a class means writing a codec by hand:

pub class Server:
    pub host: str
    pub port: u16
    pub tags: list[str]
    pub timeout: Option[i64]

A JSON reader for Server drives json.Decoder.next(), matches every Event, accepts members in any order, rejects duplicates and unknown members, reports which members are missing, checks that the port fits u16, and loops over the tag array. A TOML reader repeats the work over Toml.Table with find. Writers repeat it again in both directions. The code is long, easy to get subtly wrong (a forgotten duplicate check, a missing range check), and has to change with every field.

Rust solves this with serde derives; Python with runtime reflection (dataclasses, pydantic). Plenty has neither user macros nor runtime reflection, and its goals rule out hidden allocation and slow compilation. This page chooses how the per-type code is produced, how a type opts in, how each kind of type maps to formats, and how errors report where they happened.

What exists today

Probes in this section ran on local main at 7923a7c0 with a debug compiler.

FactConsequence for a design
A method may declare its own type parameters, including a bound that names another one: def encode[E, S: Sink[E]](self, out: &mut S) -> Result[(), E] compiles, both on a class and as a protocol requirementA format-neutral protocol can carry each format’s own error type without associated types
Protocols are structural; std.json.Encoder satisfied a user protocol with its existing null, integer, text, key, start_object and end_object methods, producing {"x":3,"label":"a"}The JSON encoder needs no adapter if the data model uses its method names
Unbounded generic bodies are checked per instance; std.fmt.display uses this to call a user method without a named boundEntry points can accept any type and report an unserializable one at the use site
A decode shape written by hand (an Option local per field, a str match on member names, construction after the loop) compiled and decoded members in either orderDerived decoding needs no new runtime machinery
Methods require a self receiver; a receiver-less def decode(n: i64) -> Point is rejectedDecoding, which has no value to call a method on, needs a type-directed hook
Protocols constrain only classes, enums and bytes; i64, str and list[T] have no methodsA protocol bound cannot admit built-in field types, as the hashing proposal also found
Generic code reaches another module’s methods only when the type and method are pub; automatic destruction ignores __del__ privacyCompiler-dispatched hooks can follow the __del__ rule instead
The runtime prints Point(x=3, y=4) by walking a static descriptor with field names (plenty-runtime/src/aggregates.rs)A descriptor-driven serializer is possible, at the cost discussed below
std.json has a resumable event decoder; std.toml has only a tree, because TOML tables can be extended anywhere in a fileThe data model must work over both an event stream and a tree
A closed set of declaration annotations exists with one member, @copy on classes; protocol method bodies agreed @final and @protocol.Name, not yet implementedAn opt-in can extend that closed set rather than introduce user decorators

Precedents

SystemWhere per-type code comes fromFormatsField options
Rust serdeProc-macro derive generates Serialize/Deserialize against a 29-type data model; formats implement Serializer/DeserializerAny, via one data model#[serde(rename, default, skip, flatten, tag)]
Swift CodableCompiler synthesizes encode(to:) and init(from:) when a type declares conformance; CodingKeys enum names fieldsAny, via Encoder/Decoder protocols with keyed containersRename by writing CodingKeys
Kotlin kotlinx.serializationCompiler plugin generates a serializer and a static descriptor per @Serializable classAny, via format-neutral encoders@SerialName, defaults from constructor
Zig std.jsonComptime field iteration (inline for (std.meta.fields(T))) in the format libraryEach format walks types itselfPer-type override functions
Go encoding/jsonRuntime reflection over struct metadataOne package per formatStruct tags (json:"name,omitempty")
OCaml ppx_deriving_yojson, C# System.Text.Json source generationGenerated code for one specific formatOne deriver per formatAttributes

Swift and Kotlin are the closest fit: the compiler generates ordinary code against a format-neutral protocol, and the user can write the same methods by hand. Serde does the same with a user macro. Zig and Go push the work into each format library.

Where the per-type code comes from

A. Compile-time field iterationB. Derived per-type code, format-neutral protocolC. Generated per-format codeD. Runtime descriptor walk
MechanismA library loop such as for field in fields[T]() unrolled at compile time, with type-shape matchingThe compiler writes __encode__/__decode__ for an opted-in type, calling a small std.serial protocol; each format implements the protocol onceThe compiler writes JSON code and TOML code per typeThe runtime walks the static type descriptor, as printing does
New languageCompile-time loops, field values typed per iteration, type-level match, and a partial-construction builder for decodingOne annotation, two hooks, two compiler-expanded std functionsOne annotation; format knowledge in the compilerNone visible; a runtime codec per format
Adding a formatWrite a type walkerImplement one protocol in PlentyChange the compilerChange the runtime (Rust)
Runtime costDirect callsDirect calls after monomorphization: the JSON instance calls json.Encoder methods, as C wouldDirect callsDescriptor interpretation per field, as == and printing do
Compile time, code sizeOne unrolled walk per type and formatOne body per type and format instance, linear in fieldsSame as BNone per type
Hand-written overridePer-type escape hatch neededWrite the hook; derived code for outer types calls itWrite per-format codeNeeds a descriptor callback slot, as hashing design B
VisibilityA library reads every type’s fields, including private onesCode is generated in the type’s own moduleSame as BRuntime reads everything
Hidden allocationDepends on the libraryNone in derived code (see costs)NoneRust codec allocates internally
ErrorsLibrary’s choiceOne path-carrying wrapperPer formatRuntime error type

A gives library authors the most power and would make @serial itself library code. It is also the largest language change: decoding needs a way to build a value field by field from compile-time-typed parts, and a library that iterates fields bypasses module privacy. It is a general metaprogramming feature, not a serialization feature; it should be justified on its own.

C has no runtime advantage over B. Generic instances are specialized per concrete type, so B’s JSON instance is direct calls on json.Encoder with no dispatch, which is the code C would write. C only moves format knowledge into the compiler.

D costs no compile time but conflicts with the “no runtime reflection” goal, cannot call a hand-written codec for a nested type without a new callback slot, and would put format code in Rust instead of checked Plenty.

Recommendation: B. The compiler knows one small data model, formats stay ordinary Plenty, and a derived method is the same code a user could write.

Recommendation

Opting in

A type opts in with one closed annotation on its declaration:

@serial
pub class Server:
    pub host: str
    pub port: u16
    pub tags: list[str]
    pub timeout: Option[i64]

@serial
pub enum Mode:
    Off
    Fixed(i64)
    Range(i64, i64)

@serial extends the closed annotation set agreed for protocol method bodies to class and enum declarations. The parser already reads class annotations for @copy; @serial adds a member and accepts annotations on enum. It derives each of two hooks the type does not already define:

def __encode__[E, S: Encoder[E]](self, out: &mut S) -> Result[(), SerialError[E]]
def __decode__[E, D: Decoder[E]](source: &mut D) -> Result[Self, SerialError[E]]

Self stands for the declaring type. __decode__ is receiver-less, like __init__ is a construction hook rather than a method; it is the only receiver-less hook and does not introduce static methods. A user may write either hook by hand, with or without @serial; derived code for an enclosing type calls whichever exists. The hooks are invoked only through std.serial.encode and std.serial.decode, as __init__ and __del__ are invoked only by construction and destruction, so there is one way to call them.

Without an opt-in, nothing changes: field names do not silently become a wire format, and renaming a field of an unrelated class never breaks a file format.

The data model

std.serial defines two protocols. The model has JSON’s shape, which TOML, YAML, CBOR and MessagePack also have, and uses std.json.Encoder’s method names so that the JSON encoder satisfies Encoder[JsonError] without an adapter.

pub protocol Encoder[E]:
    def null(self: &mut Encoder) -> Result[(), E]
    def boolean(self: &mut Encoder, value: bool) -> Result[(), E]
    def integer(self: &mut Encoder, value: i64) -> Result[(), E]
    def unsigned(self: &mut Encoder, value: u64) -> Result[(), E]
    def decimal(self: &mut Encoder, value: f64) -> Result[(), E]
    def text(self: &mut Encoder, value: &str) -> Result[(), E]
    def start_array(self: &mut Encoder) -> Result[(), E]
    def end_array(self: &mut Encoder) -> Result[(), E]
    def start_object(self: &mut Encoder) -> Result[(), E]
    def key(self: &mut Encoder, name: &str) -> Result[(), E]
    def end_object(self: &mut Encoder) -> Result[(), E]

pub protocol Decoder[E]:
    def null(self: &mut Decoder) -> Result[bool, E]       # consumes a null if next
    def boolean(self: &mut Decoder) -> Result[bool, E]
    def integer(self: &mut Decoder) -> Result[i64, E]
    def unsigned(self: &mut Decoder) -> Result[u64, E]
    def decimal(self: &mut Decoder) -> Result[f64, E]
    def text(self: &mut Decoder) -> Result[str, E]
    def start_array(self: &mut Decoder) -> Result[(), E]
    def next_element(self: &mut Decoder) -> Result[bool, E]  # False consumes the end
    def start_object(self: &mut Decoder) -> Result[(), E]
    def next_key(self: &mut Decoder) -> Result[Option[str], E]  # Nothing consumes the end
    def unknown_key(self: &mut Decoder, name: &str) -> Result[(), E]
    def reject(self: &Decoder, fault: Fault) -> E

The decoder is pull-based and typed: the derived code asks for the type it expects, so the format parses JSON number text directly into i64, u64 or f64, and TOML returns its stored Int or Float without a text round trip. reject builds the format’s located error from a model-level fault, as json.Decoder.reject does today. unknown_key applies the format’s policy for a member the type does not declare: skip its value, or return reject(Fault.UnknownField). Rejecting is the default; a per-call decode option skips unknown members instead. The policy is never a per-type setting.

FaultRaised by derived code when
UnexpectedKindThe next value is not the requested kind
MissingField(str)A required field is absent at the end of an object
DuplicateField(str)A field appears twice, whatever the format’s duplicate-key policy
UnknownField(str)Raised by the format’s unknown_key under the reject policy
UnknownVariant(str)An enum tag names no variant
OutOfRangeA number does not fit the field’s type
WrongLength(i64, i64)A tuple or variant payload array has the wrong length
DuplicateElementA set element or dictionary key repeats
UnrepresentableRaised by an encoder for a value its format cannot hold, such as null in TOML

Formats supply implementations in their own modules:

FormatEncoderDecoder
JSONjson.Encoder directlyAn adapter over json.Decoder, fed from a buffer or through read_event
JSON treeA builder producing JsonA cursor over &Json
TOMLA builder producing Toml, then the existing writer, which orders plain entries before sectionsA cursor over &Toml, after parse

How types map

Plenty typeModelJSONTOML
boolbooleantruetrue
i8..i64integer; decoding checks the rangenumberinteger
u8..u64unsigned; decoding checks the rangenumberinteger up to i64.MAX, else Unrepresentable
f32, f64decimal; an integer is accepted when decodingnumber; non-finite is the encoder’s existing NonFiniteNumberfloat, including inf and nan
strtextstringstring
Bytes, bytestext, standard base64 with paddingstringstring
list[T], set[T]arrayarrayarray, or array of tables
(A, B, ...)array of fixed lengtharrayarray
dict[str, V]object, insertion orderobjecttable
Box[T]T
Classobject, fields in declaration orderobjecttable
Option[T] fieldnull when Nothing; absent or null decodes as Nothingnullomitted
Option[T] elsewherenull or TnullUnrepresentable for Nothing
Nullary variantits name as text"Off""Off"
Variant with one payloadobject with one key{"Fixed": 5}{ Fixed = 5 }
Variant with several payloadsobject with one key, array value{"Range": [1, 9]}{ Range = [1, 9] }

Rejected at the use site with the path to the offending field: references (already excluded from fields), closures, generators, handles and synchronization types, anonymous sums (their members have no tag names), Result, Option[Option[T]] (its two Nothing levels would encode alike), dict with non-str keys (including integer keys), and any class or enum with neither @serial nor hand-written hooks:

error: `Config` cannot be decoded
  note: field `Config.server` has type `Server`, which has no `__decode__`
  help: add `@serial` to `Server`, or write `__decode__`

Recursive types through Box, lists or dictionaries are accepted, unlike automatic == and printing: the derived code recurses explicitly, and every format’s start_array and start_object enforce its max_depth in both directions, so a hostile document or a deep value built by hand stops at the limit instead of overflowing the stack.

Generic types derive per instance: Page[T] with items: list[T] is serializable exactly when T is, checked where it is used, with the usual instantiation note.

Field names and optional fields

Member names are the declared field names, unchanged. Encoding writes fields in declaration order; decoding accepts any order. Missing required fields, duplicates and (under the default policy) unknown members are errors. A member is optional when its field is Option[T], or when it corresponds to an __init__ parameter with a default (see Invariants and visibility). Plenty has no field defaults, so a class without __init__ has no other optional members.

A Nothing field is written as null in JSON, so every declared member appears in the document. TOML has no null, so its encoder omits the member instead. Both formats decode an absent member as Nothing, so a document written by one shape reads back in either.

Bytes and bytes are written as standard base64 text with padding. This suits JSON and TOML, which have no byte string; a future binary format can add a byte-string method to the model with a default that falls back to base64.

Enums

Enums use the external tag shown above: the variant name, alone for a nullary variant, otherwise as the only key of an object. This needs no reserved member name, works in every self-describing format, and lets a decoder read the tag before the payload, which an internally tagged form ({"type": "Fixed", ...}) cannot guarantee when members arrive in any order. Payloads are positional, so they are written as a value or an array, never as named members.

Entry points

std.serial provides two compiler-expanded generic functions:

def encode[T, E, S: Encoder[E]](value: &T, out: &mut S) -> Result[(), SerialError[E]]
def decode[T, E, D: Decoder[E]](source: &mut D) -> Result[T, SerialError[E]]

For a built-in type the compiler expands the mapping above; for a class or enum it calls the type’s hook. This is how a derived __encode__ handles a list[Server] field, and how a top-level array decodes without a wrapper class. Like copy and ==, they are defined for the types the rule admits rather than through a protocol bound, because no protocol can admit i64.

Formats add typed entry points named dump and load beside their existing ones. options carries the format’s limits and the unknown-member policy:

json.dump(&value, &options) -> Result[Bytes, SerialError[JsonError]]
json.load[Config](&source, &options) -> Result[Config, SerialError[JsonError]]
json.load_from[Config, R: Reader](&mut reader, &options)   # event-driven, bounded window
toml.dump(&value, &options) -> Result[str, SerialError[TomlError]]
toml.load[Config](&text, &options) -> Result[Config, SerialError[TomlError]]

json.load_from decodes straight from events, so memory is the result plus the decoder’s window. TOML parses to a tree first and writes from one, as its module already requires. The reader variant’s name follows dump/load; it is settled when the entry points are implemented.

Errors and field paths

pub class SerialError[E]:
    pub cause: E
    pub path: list[Step]
    pub complete: bool

pub enum Step:
    Field(str)
    Index(i64)
    Key(str)
    Variant(str)

The cause is the format’s own error, so JSON keeps its byte offset, line and column; TOML, decoding from a tree with no positions, relies on the path. A derived hook adds its step only while an error propagates out of it, so a successful encode or decode never touches the path. Field and variant names are string literals; a Key step moves the already decoded key. Adding a step is a fallible list operation: if memory is refused, complete becomes False and the cause is kept, so an out-of-memory condition never replaces the original error. SerialError displays as servers[2].port: number out of range at line 7, column 12.

Invariants and visibility

  • Construction. For a class without __init__, derived decoding uses the generated field constructor. An explicit __init__ may enforce invariants that field-wise construction would bypass, so decoding never bypasses it. When the __init__ parameters after self match the serialized fields exactly, by name and type, @serial derives __decode__ by decoding each member and calling __init__ with named arguments. A parameter with a default makes its member optional; when the member is absent, the call omits that argument and __init__ applies the default. An error from __init__ propagates with the class’s path. Otherwise, @serial derives only __encode__, and the class must define __decode__ by hand:

    error: `@serial` cannot derive `__decode__` for `Server`
      note: `Server.__init__` has no parameter for field `tags`
      help: match the `__init__` parameters to the fields, or write `__decode__`
    
  • Members. The serialized fields are all declared fields; names are the declared names, with no renaming.

  • Private fields. Included. The author opted in, the derived code lives in the type’s module, and structural printing and equality already include them.

  • Hook privacy. Hooks follow the __del__ rule: the compiler invokes them regardless of pub, so a module-private configuration class works with json.load. Without this, every serializable type would need to be public, since generic code in std.json reaches only public methods.

  • Destructors. A failed decode drops the fields already decoded, as a failed initializer does; the destructor of the incomplete value never runs.

Costs

OperationAllocatesTime
Derived __encode__Nothing; the format grows its own outputLinear in the value
Derived __decode__The value’s own strings, lists and dictionaries; field slots are stack localsLinear in the input; one str comparison chain per member name
Error propagationOne path step per enclosing hook, falliblyLinear in the depth
JSON eventsOne str per key and string event, as todayAs json.Decoder.next
TOMLThe tree, as parse and encode already doAs today
CompilationOne encode and one decode body per type and format instanceLinear in fields; measure before adopting

Member-name dispatch is a linear match over string literals. A type with many fields could use a length-then-bytes switch; measure first.

Rejected alternatives

  • Automatic serialization of every class, as printing works. Field renames would silently change file formats, and private data would be exported without a decision.
  • A named Serialize protocol bound. Cannot admit i64, str or list[T] without a special case, and cannot express receiver-less decoding.
  • Compile-time field iteration (A) as the first step. Larger than serialization needs; revisit only with independent uses. If it is ever added, @serial can become library code without changing the hooks.
  • Per-format generated code (C). No runtime gain over B after specialization; every new format would be a compiler change.
  • Runtime descriptor serialization (D). Runtime reflection in all but name, no path to hand-written nested codecs, and format logic outside checked Plenty.
  • A richer data model now (serde’s records, tuple structs and newtypes as separate kinds). It serves positional binary formats; JSON and TOML do not need it. It can be added as new protocol methods with defaults once protocol method bodies ship.
  • Internally or adjacently tagged enums as the default. Internal tags need a reserved member name and buffering when the tag arrives late.

Decisions

Each records the choice and the alternatives considered.

  1. Opt-in spelling: @serial. Rejected: separate @encode and @decode, and a serial clause in the class header. A split pair helps types that only ever write; @serial with a hand-written __decode__ covers that.
  2. Renaming members: declared names only, for now. Deferred: a closed field annotation such as @serial.name("userId"), which would be the first annotation on fields, and a per-call naming policy (snake to camel case).
  3. Unknown members: reject by default, with a per-call option to skip them. This matches the strictness of both parsers. Rejected: skipping by default, as serde and Swift do, which hides typos in configuration files.
  4. Nothing fields: null in JSON. Every declared member appears in a JSON document; TOML, which has no null, omits the member. Rejected: omitting in both formats for a uniform shape, which makes JSON output less explicit about the schema.
  5. Private fields: allowed and included. The author opted in, and printing and equality already include them. Rejected: excluding them, and rejecting @serial on a class with private fields.
  6. Hook privacy: the __del__ rule. Generated and hand-written hooks are invoked regardless of pub, so private types work. Rejected: requiring pub types and hooks, as std.fmt display methods do.
  7. Classes with __init__: decode through __init__ when its parameters exactly match the serialized fields; otherwise require a hand-written __decode__. Parameters with defaults make their members optional, and an absent member takes the default. Rejected: deriving encode only for every class with __init__, which forces hand-written decoders for the common case of a validating constructor over the fields; and field-wise construction that bypasses __init__.
  8. Entry points: dump and load in std.json and std.toml. Rejected: to_bytes/from_bytes and to_text/from_text, which differ between the two modules, and encode_typed/decode_typed, which collide with the Encoder/Decoder vocabulary.
  9. Bytes: base64 text. Rejected for now: rejecting Bytes fields. Binary formats can add a byte-string model method later.
  10. dict with integer keys: rejected for now. Deferred: writing keys as decimal text, as serde_json does.

Implementation sketch

Each stage is usable on its own.

  1. std.serial in Plenty. Encoder, Decoder, Fault, Step, SerialError, and the JSON decoder adapter. Verify that json.Encoder satisfies Encoder[JsonError]. Hand-written __encode__/__decode__ can be tested as ordinary methods before any compiler work.
  2. Hooks and entry points. Accept a receiver-less __decode__ and an __encode__ with the fixed signatures; expand std.serial.encode and decode per instance for built-ins and hooks; diagnostics naming the offending field path. Add json.dump, load and the reader variant.
  3. @serial. Parse the annotation on classes and enums and generate hook bodies as ordinary checked operations, so ownership, effects and diagnostics need no special path; decoding through a matching __init__ is an ordinary call with named arguments. --dump shows the generated bodies.
  4. TOML. Tree builder and tree cursor; toml.dump and load.
  5. Docs. Reference pages for std.serial, the json and toml additions, classes and enums; a tutorial lesson that replaces a hand-written codec.

Measurements

Before stage 3 is accepted:

  1. Compile time and object size for 200 classes of 10 fields, each encoded and decoded through JSON and TOML, against the same program with no derives (compile_bench method in profiling).
  2. Decode and encode throughput for the 8 MB document in the std.json costs, comparing derived code with json.parse to a tree and with a hand-written event codec. Derived code should match the hand-written codec.
  3. Member-name dispatch cost for 4, 16 and 64 fields.

Value generic parameters

Status: Partly implemented. Value parameters and Bounded[T, lo, hi] are implemented. The references own their current contracts; the array and general per-instance assertion sections below remain accepted design. B83/B96 track remaining scope. The original investigation used local main at 1b5b96b1 on 2026-10-27. Module-scope assertions from local PR #628 are implemented.

The implementation sequence was revised to serve bounded integers first. Const parameters can name a preceding T: IntType, so bounds retain its exact width and signedness. This extends the original concrete-integer-only parameter design.

Problem

Plenty cannot write a type whose size is a compile-time number. The language has inline aggregates (classes, tuples and enums store their fields inline since B35), but every inline aggregate has a fixed field list written out by hand. Code that wants “N values of T, stored inline” must either allocate a list or spell out N fields.

std.simd shows the cost. Its six types (i32x4, u32x4, f32x4, f64x2, mask32x4, mask64x2) take 614 lines of stdlib/std/simd.plenty. Each stores its lanes as enum payload fields, Lanes(i32, i32, i32, i32), so every method destructures both operands by name, and lane(index) is an if chain over the fields because inline storage cannot be indexed:

pub def lane(self, index: i64) -> Option[i32]:
    match self:
        case i32x4.Lanes(a0, a1, a2, a3):
            if index == 0:
                return Some(*a0)
            ...

The memory proposal names Array[T, N], BoundedList[T, N] and SmallList[T, N] as later types and defers them “until inline aggregates and basic constant parameters exist”. Inline aggregates now exist. This page designs the constant parameters and the array, spelled array[T, N] to match list and tuple.

Baseline at design acceptance

AspectCurrent behaviorSource
Parameters[T], [T: IntType], [T: FloatType], protocol and Callable bounds. Each is a type.type_parameters in src/frontend.rs
SubstitutionA name-to-TypeRef list; a body is substituted per instanceSubstitution, substitute in src/frontend/generics.rs
CheckingBodies are checked per used instance, not at declaration; signatures and bounds at declarationGenerics
InferenceAll arguments explicit, or all inferred structurally from inputs; output-only parameters need an explicit list; no number wideningInference
IdentityDifferent arguments are different nominal types, even for phantom parametersGeneric data
Limits16,384 instances per generic function or data type; 262,144 function and 65,536 data instances per compilation; dependency depth 256 plus the number of generic declarationsEmission, src/frontend/generics.rs, src/frontend/data.rs
Brackets in expressionsbase[...] that parses both ways is resolved by name: a local is indexed, a declaration takes type argumentsExpression::Ambiguous, src/frontend/modules.rs
Module constantsconst name: T = value, integer or float, literal or numeric type constant initializer; each read becomes an immediateModules, call-site substitution
Fixed small groupsTuples are inline; t[0] needs a literal indexTuples

Phantom type parameters already give nominal tags: Quantity[Meters] and Quantity[Seconds] are distinct types. Value parameters add two things phantom types cannot: storage sized by the value, and the value itself in the body without a runtime field.

Where value parameters would and would not help

CodeTodayWith value parameters
std.simd lane typesHand-written payload fields per shape; native lowering matches six names (src/codegen/portable_simd.rs)An internal Lanes[T, const N: i64] over array[T, N] could share bodies, with the public names kept as aliases. Lowering would match instance identities. The SIMD API keeps a closed public set; that decision stands.
Bounded containers (B13)Not expressible without allocationBoundedList[T, const N: i64] over inline storage reports capacity exhaustion without allocating
std.archive 512-byte tar blocks (block_size)Heap Bytesarray[u8, block_size] once reads can target a borrowed byte view (B11/B39)
std.random state() -> (u64, u64, u64, u64)TupleNo real gain: tuples cover small fixed groups in non-generic code
std.sketch HyperLogLog precisionRuntime field, validated, checked on merge, read from serialized dataWorse: from_bytes reads the precision from data, so it must stay a runtime value
std.crypto digest sizeRuntime Algorithm; finish_into writes into a caller buffer without allocatingNo gain: the algorithm is a runtime choice and the allocation-free path exists
clist chunksPhysical storage kept private so chunk size can change (chunks)Not wanted: a public chunk-size parameter would fix what the design keeps private

The array-oriented consumers above need indexable inline storage. B96 supplied a separate first consumer: bounds carried in an integer type. Value parameters and Bounded therefore ship before arrays; arrays retain their accepted scope.

Precedents

LanguageParameter typesArgument expressionsInferenceWhen checked
Rust (stable, min_const_generics, 1.51)Integers, bool, charLiteral, bare const parameter, or a { } block independent of generic parametersFrom [T; N] in argument typesGenerically, before monomorphization
Rust (generic_const_exprs, unstable)SameN + 1 and other expressions over parametersNot through expressionsNeeds where [(); N + 1]: evaluatability bounds
C++ non-type template parametersIntegers, enums, pointers; floats and structural classes since C++20Any constant expressionDeduced only from bare N; N + 1 is a non-deduced contextAt instantiation
D template value parametersAny type usable at compile timeAny CTFE-evaluable expressionFrom argument typesAt instantiation; if (n > 0) constraints
Zig comptime parametersAny type, including typeArbitrary compile-time evaluation, bounded by an eval-branch quotaExplicit arguments; types are valuesLazily, per instantiation
Mojo parametersAny compile-time typeArbitrary parameter expressionsFrom argumentsPer instantiation; SIMD[DType.float32, 4] is the stdlib vector

Lessons:

  • Rust’s stable subset has served for years; the hard part is expressions. With generic_const_exprs, N + 1 and 1 + N do not unify, well-formedness needs evaluatability bounds, and the feature has stayed incomplete. The Rust project has since pursued narrower argument forms. Plenty should not start there.
  • Before const generics, Rust implemented array traits only for lengths 0 to 32. That is the duplication std.simd shows today.
  • C++ and Zig show that arbitrary compile-time evaluation produces late, long diagnostics and needs evaluation budgets.
  • Plenty differs from Rust in one useful way: bodies are already checked per instance. Inside a body N is a known number, so arithmetic on it is ordinary arithmetic on a constant. Only type positions need restriction.

Options

OptionBenefitCost
A. Status quo: concrete types, hand-written or generatedNo language changeDuplication (std.simd); no bounded containers without allocation
B. Runtime capacity fieldsWorks today (BufReader, HyperLogLog)Heap storage; checks at run time; no inline arrays
C. Minimal value parameters: integer and bool values, bare arguments onlyCovers arrays, bounded containers and fixed lanes; no expression unification; finite instance setNo array[T, N + 1] in signatures
D. C++-style: expressions in non-inferred type positions, evaluated per instanceconcat and push can name result sizesErrors appear at the use, inside the callee’s signature; more checking machinery
E. Zig-style compile-time evaluationMost expressiveA compile-time interpreter, evaluation budgets, and generic code that is hard to read; against the small-language goal

Recommendation

Adopt option C, initially with Bounded[T, lo, hi], followed by the accepted array[T, N] consumer. Defer option D until code shows a need. Reject E. The decisions record what is settled.

Declaring

class Ring[T, const N: i64]:
    items: array[Option[T], N]
    head: i64
    count: i64

def filled[T, const N: i64](value: T) -> array[T, N]:
    array[T, N].filled(value)

const N: TypeName declares a value parameter. The keyword is already reserved and already means “compile-time value” at module scope. Without it, [N: i64] would read like a bound, and T: IntType (a type) beside N: i64 (a value) would differ only by what the right-hand name is. One marker keeps the rule mechanical.

Value parameter types: the eight integer types and bool; u8 and bool cost little once integers work. No floats (equality and NaN make identity awkward), no str, no user types. Value and type parameters may appear in any order. Functions, methods, classes and enums accept them, as they accept type parameters; protocols do not in the first version.

Arguments

A value argument is one of:

FormExampleNotes
Integer or bool literal, optionally negativearray[u8, 64]Typed by the parameter’s declared type, as a nongeneric parameter types its literal
Module constant, including an imported pub constarray[u8, block_size]Must have exactly the parameter’s type; no widening
A value parameter in scope, barearray[T, N]Forwarding only

Nothing else: no N + 1, no calls, no type constants such as u8.MAX (name a module constant instead). A derived size is named once as a module constant, which composes with the parallel work on constant initializers: whatever arithmetic a const initializer accepts is evaluated in the declaration, not in the brackets. Inside a generic, a derived size cannot be named; that is option D.

This rule also bounds expansion. A body can request only values that already appear in the program: literals, constants, and its own parameters. Value arguments cannot grow without end, so the existing type-driven limits remain the only expansion guard needed.

In bodies

N is an immutable value of its declared type. Each instance substitutes the number, so range(N), N * 2, if N > 0: and comparisons compile to immediates, as module constant reads do. A local cannot shadow a value parameter, matching the existing rule that a type parameter name cannot also name a value binding.

Since bodies are checked per instance, a body rejected for one value, such as array[T, N] with a negative N, is reported as an error of that instance, at the use that required it, through the existing instance diagnostics.

Type identity and checking

Two instances are the same type when their declarations match and their arguments are equal after resolving aliases and constants: with const width: i64 = 4, array[i64, width] is array[i64, 4]. Instances print with values: array[i64, 4], Flags[True]. A type alias can name a complete instance (type Block = array[u8, 512]), since aliases already name complete generic instances.

At declaration, the checker verifies kinds: a type where a type parameter is expected, a value of the right type where a value parameter is expected. Value constraints such as N > 0 are not part of the signature. A data type that needs one states it with a per-instance assertion; array rejects a negative length the same way, built in.

Per-instance assertions

The module-scope assert COND, "message" from local PR #628 is also accepted in class bodies and generic function bodies:

class Ring[T, const N: i64]:
    assert N > 0, "Ring needs a positive capacity"
    items: array[Option[T], N]
    head: i64
    count: i64

def best_window[const N: i64, const W: i64](xs: &array[i64, N]) -> i64:
    assert 0 < W and W <= N, "window must fit the array"
    ...
  • Position. Assertions come before any other item or statement in the body. There assert begins the static form, as at module scope; elsewhere in a function body it stays an ordinary name, as PR #628 leaves it.
  • Operands. The condition and message may use constants and the declaration’s value parameters only: no locals, fields, calls or type parameters. The checker rejects anything else at declaration, once, not per instance.
  • When it runs. Once per instance, after substitution, whenever the type or function is instantiated. That includes a type named only in a field, binding or signature with no method ever called. This matters because methods are checked only when used; an assertion attached to a method would never fire for an instance that only stores data.
  • Failure. Reported at the use that required the instance, with a note at the assertion, like the instance diagnostics added for issue #518:
main.plenty:9:11: cannot instantiate `Ring[i64, 0]`: static assertion failed: Ring needs a positive capacity
  ring.plenty:2:5: note: in instantiation `Ring[i64, 0]`: assert N > 0

In a nongeneric class or function the assertion is accepted and runs once, like a module assertion; it is mainly useful in generic code.

Inference

Value parameters are inferred by the existing structural match: array[T, N] against array[i64, 4] learns T = i64 and N = 4. A literal passed where the parameter type is an array also supplies N from its element count; see literals at generic parameters. Nothing else supplies evidence: not a list’s length, not an ordinary argument’s value. Output-only value parameters need the explicit list, as type parameters do: filled[i64, 4](0). Partial explicit lists stay unsupported (B16).

In expressions, filled[i64, 4](0) keeps today’s bracket rule: brackets after a local index it, brackets after a declaration are generic arguments. The type argument grammar gains literals, so xs[0] on a local still indexes.

The first consumer: array[T, N]

A built-in fixed-length inline sequence. This page fixes only what value parameters need; its complete API needs its own review.

PropertyContract
StorageN elements inline in the containing value, no header, no allocation
Copy and moveCopyable when T is copyable, otherwise moves, as tuples do
ConstructionComplete: filled(value) for copyable T, a literal with an expected array type, or a callback per index. Never partly initialized.
Accessa[i] with a runtime index, bounds-checked like list, trapping out of range; &mut a[i] under ordinary loan rules; len(a) is N
IterationBorrowed iteration yields references, as for lists
Equality, printingStructural, as for tuples
SizeElement size times N; large frames are bounded by the existing frame-size diagnostic (issue #441)

BoundedList[T, N] and a std.simd rewrite are library code over this type.

Array literals

A list literal [1, 2, 3] is typed array[T, N] when its expected type is an array instance, after resolving aliases. It then needs no ?:

lanes: array[i32, 4] = [1, 2, 3, 4]
values: list[i32] = [1, 2, 3, 4]?

This is sound because an array literal allocates nothing. Its storage is the destination itself: the local’s frame slot, the field, or the argument or return slot, as for a tuple. A list literal returns Result only because it allocates a heap buffer that can fail. Element expressions keep their own failures, so [f()?, 2] still propagates f’s error, as tuple components do. The element count is known when parsing, so a count other than N is a compile-time error, and T gives each element its expected type, as with add[u8](1, 2).

The expected type comes from the places that already supply one:

SourceExample
Annotated bindingxs: array[i64, 3] = [1, 2, 3]
Concrete parameter type, including constructor fields and explicit generic argument listschecksum([1, 2, 3]) for checksum(block: array[u8, 3]); sum_all[i64, 3]([1, 2, 3])
Generic parameter whose declared type is an arraysum_all([1, 2, 3]) for sum_all[T, const N: i64](xs: array[T, N]); see below
Declared return typereturn [0, 0] in a function returning array[i64, 2]

Without one, [...] stays a list. Comprehensions and empty [] are not array literals: a comprehension’s length is not static, and array[T, 0] is built with filled.

Literals at generic parameters

A generic parameter still supplies no expected type, since T and N are unknown until inference finishes. It does supply an expected shape. Parameter types are known from the signature before any argument is typed, so the checker can see that xs: array[T, N] is an array before it knows T or N:

  1. For each argument, the checker reads the parameter’s declared type after resolving aliases. If it is an array instance, a [...] literal there is an array literal. The shape is also visible through &, tuple components and the element type of an enclosing array, so [[1, 2], [3, 4]] for array[array[T, N], M] is two levels of array.
  2. The literal’s element count is evidence for the length argument: N = 3.
  3. Elements are typed as any generic argument is today, without an expected type: unsuffixed integers default to i64. Their common type is evidence for T. sum_all[u8, 3]([1, 2, 3]) or suffixed elements choose another type.
  4. Evidence from all arguments is merged and must agree, as now. Arguments still evaluate once, in source order.

The shape stops at type parameters and other nominal types. A parameter typed as bare T gets a list from [1, 2, 3], which needs ? as usual; so does Option[array[T, N]], whose argument is a constructor call, not a literal.

One call can mix shapes. For def pair[T, const N: i64](names: list[T], codes: array[T, N]), pair([1, 2]?, [3, 4, 5]) builds a list and a three-element array, and both give evidence for T. pair([1u8]?, [3, 4]) fails as add(1u8, 2) does: T is u8 from one argument and i64 from the other.

Instances, limits and compile cost

Each distinct value is a distinct instance with its own code. array[u8, 16] and array[u8, 32] duplicate every method they use. Value instances count against the same per-template (16,384) and per-compilation limits as type instances; no separate budget is needed, since arguments cannot grow.

Code size is the real cost. Libraries should keep length-independent work in functions that take a borrowed view, so only thin wrappers depend on N. That needs borrowed sub-range views (B11/B12); until then, the guidance belongs in library documentation rather than the compiler.

Interactions

AreaEffect
OwnershipValue arguments are copied numbers; no loans. array[T, N] follows tuple ownership.
--check-all-methodsUnchanged: it checks every method of each instance the program creates; a value-generic type with no instance stays unchecked
Compile cacheInstances are ordinary functions with N folded to immediates; content keys need no change (cache)
Runtime metadataarray needs a type descriptor carrying element descriptor and count, for drop, equality and printing
std.simdPublic shapes stay closed and named; any generic implementation is internal and must keep the native lowering’s matches
Module constantsConstants become usable as arguments; the constant work’s arithmetic and static assertions provide derived sizes and checks without bracket expressions
C exportsPublic instances already need public concrete arguments; a value argument is a number, so it adds no visibility requirement

Diagnostics

main.plenty:4:24: value arguments must be a literal, a module constant, or a value parameter; found `N + 1`
  note: name a derived size with a module constant: `const padded: i64 = ...`
main.plenty:6:11: `array` expects a type for `T`, found the value `4`
main.plenty:7:11: value argument for `N` must be `i64`, found `u8` constant `width`
main.plenty:9:5: cannot infer `N` for `filled`; supply every argument: `filled[i64, 4](0)`
main.plenty:12:9: `array[i64, -1]`: an array length cannot be negative
main.plenty:14:12: conflicting evidence for `N` in `zip_add`: 4 from argument 1 (`array[i64, 4]`), 3 from the array literal in argument 2
main.plenty:15:10: conflicting evidence for `T` in `pair`: `u8` from argument 1, `i64` from the array literal in argument 2
  note: give the literal's elements a type, such as `3u8`, or supply every argument: `pair[u8, 2](...)`
main.plenty:16:14: `?` needs a `Result`, found `array[i64, 3]`
  note: a literal passed as `array[T, N]` is an array, which allocates nothing and cannot fail
main.plenty:17:9: cannot infer `T` for `sum_all` from an empty literal; `[]` is not an array literal
  note: build an empty array with `filled`, or supply every argument
main.plenty:18:9: `sum_all` expects `array[T, N]` for `xs`, found `list[i64]`
  note: only a literal becomes an array here; a list does not convert

Rejected alternatives

AlternativeReason
[N: i64] without constConfuses bounds with value types; learners need one visible marker
Expressions in brackets (Rust generic_const_exprs)Unification of expressions, evaluatability bounds, and unbounded instance growth
Zig-style compile-time functionsA second evaluation model; against small-language and fast-compile goals
Float, str or user-type valuesIdentity and printing questions for no current use; str can be reconsidered after string constants land
Inferring N from runtime values or list lengthsHidden specialization per call site and no static guarantee
Lane-count-generic public std.simdAlready rejected by the SIMD API
Value parameters without a consumerNo motivating use; Bounded now supplies the first consumer

Decisions

  1. Spelling: [T, const N: i64]. Rejected: [T, N: i64], which reads like a bound.

  2. Value types: the eight integer types and bool. Rejected: i64 only, and floats, str and user types.

  3. Name and literal: lowercase array[T, N], matching list and tuple. A literal with an expected array type builds one inline without ?; see array literals.

  4. Value parameters first ship with Bounded. This supersedes the original decision to require array in the same change. Bounded integers provide a concrete use for values in type identity without indexable storage.

  5. Derived sizes (option D): deferred. Add expressions such as array[T, N + 1] in signatures only with evidence from BoundedList or std.simd code.

  6. Per-instance checks: assert in class and generic function bodies. The module-scope assert from PR #628 runs once per instance, from constants and value parameters, whenever the type or function is instantiated, with failures reported at the instantiation site; see per-instance assertions. Rejected: each type validating its own arguments, which only built-in types such as array can do at compile time; a user type would have to check in its constructor and fail at run time.

  7. Array literals at generic parameters. A literal where the parameter’s declared type is an array builds an array and supplies N, so sum_all([1, 2, 3]) needs no argument list; see literals at generic parameters. Rejected: requiring sum_all[i64, 3](...), which makes the common call verbose for no gain in clarity.

Implementation sketch

  1. Parameters. Parse const N: T in parameter lists and literals in type argument lists. Represent a generic argument as a type or a value; a separate argument kind is cleaner than a Ty leaf that every type position must reject, but touches every place that stores instance arguments as Vec<Ty>. Extend Substitution with value entries, substitute N in bodies as a literal, extend instance names and structural inference. Tests: kinds, constants as arguments, identity across aliases and constants, inference, rejected expressions, phantom value tags.
  2. Per-instance assertions. Accept assert at the start of class and function bodies, check at declaration that operands are constants and value parameters, and evaluate after substitution whenever an instance is created, including data-only instances whose methods are never checked. Failures use the instance diagnostic chain. Tests: failing and passing instances, a type used only as a field, rejected operands, assert as an ordinary name later in a body.
  3. array[T, N]. Inline layout, descriptor, construction including literals with an expected type or at array-typed generic parameters, checked indexing, borrowed iteration, equality, printing, drop of affine elements. Reference page, tutorial lesson in the generics chapter, cheat sheet entry.
  4. Library use. BoundedList[T, N] under B13. Measure an internal std.simd rewrite against the current code: native lowering must still recognize the shapes, and object code must not regress.
  5. Deferred: derived sizes. Only if stage 4 code shows a need: expressions in non-inferred output positions, evaluated after substitution.

std.plot: a ggplot2-style plotting module

Status: Partially implemented. Design accepted. The core/SVG reference describes the implemented subset: tables and record accessors, layer-specific data and inherited mappings, nine geoms, grouped counts/binning and bar positions, continuous/discrete position scales with transforms and explicit breaks/labels, named manual palettes and RGB gradients, fixed/free wrapped and grid facets, supplied interval/ribbon geoms, basic themes/legends, wrapped text boxes with layout preflight, and streaming SVG. This does not complete milestone 1: the remaining scales, smoothing, measured layout and the full example gallery remain future work, tracked under B80 in the backlog. The decisions remain the target design. Code fences here show proposed API and are not compiled; use the reference and tutorial for current API.

Problem

Plenty programs can compute summaries of data in lists, records, and std.dataframe frames, but cannot draw them. Today a user writes CSV and opens R or Python to plot it.

The plotting library most data analysts know well is R’s ggplot2, built on Wickham’s layered grammar of graphics (2010). A plot is data, plus aesthetic mappings from variables to visual properties, plus layers (a geom, a stat, and a position adjustment), plus scales, a coordinate system, facets, and a theme. Each part has defaults, so a short specification gives a complete plot with axes and legends.

This proposal designs std.plot: the same grammar, with ggplot2’s names and defaults, expressed in ordinary Plenty. A ggplot2 user should be able to read a std.plot program without documentation and write one after reading the correspondence table.

Goals

  • The grammar, not only the drawing. Mappings, layers with default stats, trained scales with automatic legends, facets, coordinates, and themes.
  • ggplot2 names and defaults. ggplot, aes, geom_point, scale_colour_brewer, facet_wrap, theme_minimal, labs. Default theme, palette, bin count, units (points for text, millimetres for sizes) and layout match ggplot2 so values transfer and plots look familiar.
  • Plenty’s builtin data. Lists of records, dictionaries of columns, lists and tuples are first-class input. This is required, not an add-on.
  • std.dataframe later, with the same plot API.
  • Plenty conventions. Errors are Result values with precise causes, allocation is explicit and fallible, ownership is ordinary, and the output is byte-for-byte deterministic.
  • Written in Plenty, following the standard library policy, with one justified native dependency already in the toolchain.

Non-goals

  • Interactive or on-screen display. Output is files or std.io streams.
  • R’s non-standard evaluation: arbitrary expressions inside aes().
  • ggplot2’s extension system (ggproto) and its ecosystem (ggrepel, patchwork, gganimate).
  • Complex-script text shaping (Arabic, Indic scripts) in the first releases.
  • Pixel-identical output to ggplot2. Layout and defaults match; renderers, fonts and smoothing details differ.

Precedents

ggplot2 composes with +: ggplot(mpg, aes(displ, hwy)) + geom_point(). + dispatches on the right operand (ggplot_add()), so layers, scales, themes and even lists of components can be added. aes() captures unevaluated expressions (quosures) and evaluates them against the data frame at build time, so aes(colour = factor(cyl)) works without quoting. after_stat(), after_scale() and stage() select when a mapping is evaluated. Version 4.0.0 (CRAN, 2025-09-11) moved to S7 classes and added theme-level ink/paper colours and palette settings; the grammar is unchanged.

plotnine (0.15.8) ports ggplot2 to Python. Python has operator overloading but no lazy evaluation, so plotnine keeps + (ggplot.__add__) and replaces unevaluated expressions with strings: in aes("wt", "mpg", color="factor(cyl)") each string names a column or is evaluated as an expression against the dataframe, with columns taking precedence over variables. It also accepts arrays and scalars as mapping values. Theme elements use legend_position for legend.position. Rendering goes through matplotlib. plotnine shows that strings are an acceptable replacement for NSE for ggplot2 users; its expression evaluation is the part that relies on Python’s dynamic eval.

plotters (0.3.7) is a Rust drawing library for charts. It is not related to the grammar of graphics: there are no aesthetic mappings, scale training, stats, automatic legends from mappings, or facets. A program sizes label areas in pixels, builds a Cartesian coordinate range by hand, configures a mesh, and draws series (LineSeries, point series, histograms) with explicit styles. It sits at the level of ggplot2’s grid graphics layer, roughly the bottom fifth of the work. Its backends are SVG and a bitmap backend that pulls in image, and fonts through font-kit (system font discovery) or ab_glyph. Rendering explains why it is not recommended as a backend.

Language probes

Probes ran on local main at dbcdf39a with a debug compiler. Since those probes, arithmetic operator hooks have been implemented (issue #512). The historical rejection below no longer applies; the composition recommendation still accounts for fallible allocation.

FeatureResultConsequence
Operator + on a class with __add__Rejected: operator `+` does not accept VNo + composition without a language change
Method with owned receiver self: PRejected: receivers are & or &mutBuilder methods cannot consume the plot
Free function consuming and returning a plotWorksadd(add(p, a)?, b)? is possible but reads inside out
Method borrowing self and returning a new value, chained on temporariesWorks: base.plus(a)?.plus(b)?Composition uses this
Anonymous sums with defaults: x: str | AfterStat | Unmapped = Unmapped.UnsetWorks; "wt" and after_stat("count") injectaes(x="wt", y=after_stat("count")) needs no wrapper
size: f64 | Unset given size=2Rejected: expected Unset | f64, got i64Numeric settings accept f64 | i64
Named arguments and defaults on functions and methodsWorksgeom_point(size=2, colour="red")
Multi-line list display as an argumentWorksOne component per line
Generic accessor num("wt", def(c: &Car) -> f64: c.wt) infers TWorks, via BoxedCallableRecords enter through typed accessors
Car.wt as a valueRejected; the selector exists only for clist.columnNo typed column handles yet
copy() of components holding closuresNot supportedThe plot itself holds no closures
Shared[T] field with share()Works; handles have no copy()Plots share their table without copying it

Also relevant: references cannot be stored in classes, so a plot cannot borrow caller data; wildcard imports and re-exports do not exist; a function may have at most 255 parameters; statements nest to 64 levels. The derived serialization proposal (B77, accepted, not implemented) gives classes an Encoder hook that a plot table can consume. std.compress provides zlib compression and CRC-32, which is all PNG needs beyond pixels.

Correspondence with ggplot2

ggplot2plotninestd.plot
ggplot(df, aes(x, y))ggplot(df, aes("x", "y"))ggplot(table, aes("x", "y"))?
p + geom_point() + labs(...)sameggplot(table, mapping, [geom_point(), labs(...)]?)?
base + geom_line() (base unchanged)samebase.plus([geom_line()]?)?
aes(colour = cyl)aes(color="cyl")aes(colour="cyl") (color= also accepted)
aes(colour = factor(cyl))aes(color="factor(cyl)")aes(colour=factor("cyl"))
aes(y = after_stat(density))aes(y=after_stat("density"))aes(y=after_stat("density"))
aes(y = 2 * hp)aes(y="2*hp")Compute the column when building the table
geom_point(colour = "red") (setting)samegeom_point(colour="red")
geom_bar(position = "dodge")samegeom_bar(position=position_dodge())
geom_smooth(method = "lm")samegeom_smooth(method=Smooth.Lm)
geom_histogram(binwidth = 500)samegeom_histogram(binwidth=500)
scale_x_log10()samescale_x_log10()
scale_colour_brewer(palette = "Set1")samescale_colour_brewer(palette=Brewer.Set1)
scale_y_continuous(limits = c(0, 50))limits=(0, 50)scale_y_continuous(limits=(0, 50))
coord_flip(), coord_cartesian(xlim = c(1, 5))samecoord_flip(), coord_cartesian(xlim=(1, 5))
facet_wrap(~class, ncol = 3)facet_wrap("class", ncol=3)facet_wrap("class", ncol=3)
facet_grid(drv ~ cyl)facet_grid("drv ~ cyl")facet_grid(rows="drv", cols="cyl")
theme_minimal(base_size = 12)sametheme_minimal(base_size=12)
theme(legend.position = "bottom")theme(legend_position="bottom")theme(legend_position=Legend.Bottom)
element_text(face = "bold"), element_blank()sameelement_text(face=Face.Bold), element_blank()
labs(title =, x =, colour =)samelabs(title=, x=, colour=)
ggsave("p.png", width = 6, height = 4, dpi = 300)p.save(...)p.save("p.png", width=6, height=4, dpi=300)?
Message: `stat_bin()` using `bins = 30`sameA note in the render result
Error: object x not found (at print)at drawPlotError.UnknownColumn from ggplot(...)?

The examples page shows six classic plots side by side in ggplot2, plotnine and std.plot, and a complete program.

Design

Data: from Plenty values to a plot table

ggplot2 plots a data frame; anything else first goes through fortify(). std.plot keeps that split. Typing happens at the boundary, where Plenty knows the record type. The grammar then works on a Table of named columns, exactly as ggplot2 works on a data frame.

pub class Table              # immutable after construction; columns behind Shared
    pub def share(self) -> Table          # another owner of the same columns; no allocation
    pub def height(self) -> i64
    pub def names(self) -> list[str] ...  # (Result-returning in the real signature)

enum Column                   # what a table column holds
    Num(list[f64])            # continuous
    Int(list[i64])            # continuous unless wrapped in factor()
    Text(list[str])           # discrete
    Bool(list[bool])          # discrete
    MaybeNum(list[Option[f64]])   # missing values allowed
    MaybeText(list[Option[str]])

Ways in, in the order they become available:

  1. Columns. table([("wt", Column.Num(wts)), ("cyl", Column.Int(cyls))]?)? moves owned lists in, without copying. columns(&values) copies a dict[str, list[f64]]; the dictionary’s key order becomes column order. Unequal lengths and duplicate names fail with PlotError.ShapeMismatch and PlotError.DuplicateColumn.

  2. Records with accessors. rows(&cars, fields) evaluates one typed accessor per column and copies the results:

    cars_table = rows(&cars, [
        num("wt", def(c: &Car) -> f64: c.wt)?,
        num("mpg", def(c: &Car) -> f64: c.mpg)?,
        integer("cyl", def(c: &Car) -> i64: c.cyl)?,
        text("model", def(c: &Car) -> str: c.model)?,
    ]?)?
    

    num, integer, text and flag box their callback and infer T from it (probed). The accessor list lives only for the call; the table holds plain columns. Derived values go here too, replacing ggplot2’s aes(y = 2 * hp): num("kpl", def(c: &Car) -> f64: c.mpg * 0.425)?.

  3. Records by field name, after B77: records(&cars)? drives each row’s derived __encode__ into a column builder that implements std.serial.Encoder[PlotError]. Scalar fields become columns named after the fields, with no intermediate tree. Nested records, lists and enums with payloads are rejected with the field path. A fields=["wt", "mpg"] argument limits the copy. This is the closest Plenty gets to ggplot(mtcars, ...) and should become the recommended path.

  4. std.dataframe, through a bridge module that only dataframe users import, so std.plot never links the Polars engine. ggplot_frame(&frame, mapping, components) asks the core for the columns the specification uses (required_columns), checks them against frame.schema(), and extracts only those with values_f64, values_str_nullable and the other typed readers. Aesthetics may also be dataframe Expr values, evaluated by one select before extraction. That is the dataframe counterpart of ggplot2’s expressions in aes() and of plotnine’s string expressions. A later version can push stat_count and stat_bin into Polars group_by.

Lists and tuples enter through columns or table. A list[(f64, f64)] of points goes through rows with two accessors.

Aesthetic mappings

pub enum Unmapped:
    Unset

pub type Aes = str | Factor | AfterStat | Stage | Unmapped

pub def aes(x: Aes = Unmapped.Unset, y: Aes = Unmapped.Unset,
            colour: Aes = Unmapped.Unset, color: Aes = Unmapped.Unset,
            fill: Aes = ..., alpha: Aes = ..., size: Aes = ..., shape: Aes = ...,
            linetype: Aes = ..., linewidth: Aes = ..., group: Aes = ...,
            label: Aes = ..., weight: Aes = ..., xmin: Aes = ..., xmax: Aes = ...,
            ymin: Aes = ..., ymax: Aes = ..., xend: Aes = ..., yend: Aes = ...) -> Mapping
  • A str names a column. aes("wt", "mpg") works positionally because x and y come first, as in ggplot2.
  • factor("cyl") treats a numeric column as discrete. Levels are the sorted distinct values; factor("class", levels=[...]?) orders them explicitly.
  • after_stat("count") names a variable computed by the layer’s stat. Each stat documents its variables (count, density, ncount, ndensity, prop, width), as in ggplot2.
  • stage(start="x", after_stat="count") arrives with after_scale in a later milestone.
  • There is no expression language. Computed values belong in the data step, where Plenty type-checks them; stat-computed values are limited to the named variables above, which ggplot2 provides to cover the common expressions.

Mapping and setting stay distinct, as in ggplot2: aes(colour="cyl") maps a column; geom_point(colour="red") sets a constant. ggplot2’s classic mistake, aes(colour = "blue"), maps a constant and produces a legend labelled “blue”. In std.plot it fails with UnknownColumn("blue"), and the message suggests geom_point(colour="blue").

colour and color are both accepted, in aes, geoms and scales; giving both is PlotError.Conflict. ggplot2 and plotnine accept both spellings, and users type whichever they learned.

Checking and errors

Column names in strings cannot be checked at compile time. They are checked eagerly instead: ggplot(...) and plus(...) validate every mapping against the table, every required aesthetic (geom_point needs x and y), and every setting (colour names, palette sizes), before anything is rendered. The error arises at the statement that introduced the mistake. ggplot2 and plotnine report the same mistakes only when the plot is printed or drawn.

pub enum PlotError:
    UnknownColumn(ColumnFault)    # name, aesthetic, layer, available names, nearest match
    MissingAesthetic(AesFault)    # "geom_point requires y"
    WrongKind(AesFault)           # discrete column for a continuous-only scale, etc.
    ShapeMismatch(ShapeFault)
    DuplicateColumn(str)
    Conflict(str)                 # colour and color both given; two coords
    InvalidSetting(SettingFault)  # unknown colour name, negative size, bins = 0
    Font(FontError)
    Io(IoError)
    Allocation(AllocError)

Faults name the layer by position and geom (layer 2 (geom_smooth)), the aesthetic, and the column, and print through std.report like other std errors:

error: layer 1 (geom_point): aesthetic `colour` maps unknown column "cly"
  available columns: wt, mpg, cyl, gear, model
  did you mean "cyl"?

Statically typed handles would move these checks to compile time. Plenty’s clist already has a compile-time selector, Car.wt, accepted only by column. Generalising it into a typed Field[Car, f64] value would allow aes(x=Car.wt, colour=factor(Car.cyl)), checked by the compiler. That is a language change; decision 6 defers it after records ships, rather than designing it now.

Composition

Plenty has operator hooks and explicit consuming method receivers. For this API, + would be a poor fit: every step can allocate, and Plenty makes that visible, so a chain would read ((ggplot(t, m)? + geom_point())? + labs(title="x"))?, as str already requires ("a" + b)?.

std.plot therefore composes in two ways that exist today:

p = ggplot(cars, aes("wt", "mpg", colour=factor("cyl")), [
    geom_point(size=2),
    geom_smooth(method=Smooth.Lm),
    labs(title="Fuel economy", colour="Cylinders"),
    theme_minimal(),
]?)?

with_line = p.plus([geom_line()]?)?        # p is unchanged
  • ggplot(data, mapping, components) takes the components as a list, in order, one per line. It reads top to bottom like a + chain. Later components override earlier ones by the same rules as ggplot2: a second x scale replaces the first, a complete theme replaces the theme, and theme(...) modifies it.
  • p.plus(components) borrows p and returns a new plot that shares p’s table and copies its component list (probed, including chaining on temporaries: base.plus(a)?.plus(b)?). This is ggplot2’s base <- ggplot(...); base + geom_point(); base + geom_line() idiom, and the way to add components conditionally: if smooth: p = p.plus(...)?.

Every component constructor returns one enum, Component, so a single list holds layers, scales, coords, facets, labels and themes. This mirrors adding a list() of components in ggplot2. There is no p.add(...) mutator: a plot is immutable once built, so validation happens once, and sharing a plot or its table between threads needs no locks.

Ownership and allocation

  • ggplot takes its Table by value. A table holds its columns in a Shared owner; table.share() gives another plot the same columns without copying. Tables are immutable, so sharing has value semantics.
  • Components are plain data: enums, numbers, strings and lists. They hold no closures, so a component list can be copied with copy and a plot can be rebuilt or shared freely.
  • Allocation happens in four visible places, each fallible: building the table (copies columns, except table which moves them), building the plot (the component list), rendering (per-layer computed data, scale breaks, text, and for PNG a pixel buffer of width × height × 4 bytes), and writing. Rendering to an SVG stream writes directly to a std.io writer, so the document is never held in memory. Nothing allocates behind a non-Result call.
  • Rendering borrows the plot; one plot can be saved in several formats.

The build pipeline

Rendering follows ggplot2’s ggplot_build() then ggplot_gtable():

  1. Resolve mappings per layer: inherit the plot mapping unless the layer sets inherit_aes=False; read columns.
  2. Facet: assign each row to a panel.
  3. Transform scales (log10, sqrt, reverse) before statistics, as ggplot2 does.
  4. Group by the interaction of discrete mappings and group.
  5. Compute stats per panel and group (identity, count, bin, smooth, boxplot, density); resolve after_stat mappings.
  6. Adjust positions: identity, stack, fill, dodge, jitter.
  7. Train and map scales: ranges and levels, breaks, labels, palettes; merge legends that share a title and levels.
  8. Coordinates: Cartesian limits (zoom without dropping data), flip, fixed aspect ratio; polar later.
  9. Lay out: measure text, size axes, strips, legends and titles, place panels.
  10. Draw primitives to a canvas: rectangles, polylines, polygons, circles, text with anchor and rotation, clip regions, groups.

Each stage is a Plenty function over columns of f64 and level indices. Stages 1–9 are independent of the output format.

Layers, stats and positions

MilestoneGeoms (default stat)StatsPositions
1point, line, path, col, bar (count), histogram (bin), smooth (smooth, Lm only), hline, vline, abline, segment, textidentity, count, bin, smoothidentity, stack, fill, dodge
2area, ribbon, boxplot, violin, density, freqpoly, errorbar, pointrange, linerange, tile, raster, rect, label, step, jitterboxplot, density, ydensity, summary, smooth with Loessjitter (seeded), dodge2
3contour, density_2d, bin_2d, hex, polygon, quantilecontour, bin_2d, ecdf, qqnudge

geom_smooth draws ggplot2’s 95% confidence ribbon (se=True). Linear regression standard errors need the Student t quantile, implemented in Plenty from the regularised incomplete beta function. Loess follows R’s stats::loess defaults (span 0.75, degree 2, tricube weights). ggplot2 picks loess below 1,000 rows and mgcv GAMs above; GAMs are not planned (see decision 5). std.stat already provides means, variances, quantiles and normal, uniform and exponential distributions (Normal has cdf and inverse_cdf); std.plot should use it and add the Student t distribution there rather than in the plot module.

Jitter uses a seeded std.random.Rng (default seed fixed, settable with position_jitter(seed=...)), so output stays deterministic.

Scales and guides

MilestoneScales
1scale_{x,y}_continuous, _discrete, _log10, _sqrt, _reverse; scale_{colour,fill}_hue (default discrete), _gradient (default continuous), _gradient2, _viridis_d, _viridis_c, _manual, _grey; scale_size, scale_alpha, scale_shape; xlim, ylim, lims
2_brewer, _distiller, _gradientn, _identity, scale_linetype, date and datetime scales over std.time and dataframe dates, guides(), guide_legend, guide_colourbar options
3Binned scales, sec_axis, custom transforms

Arguments follow ggplot2: name, breaks, labels, limits, expand, na_value, palette. Breaks use Wilkinson’s extended algorithm (as R’s scales::extended_breaks does). Labels for continuous scales format numbers with the same rules as scales::label_number defaults; labels= accepts a list of strings. A function-valued labels= would put a closure in the plot; instead a small LabelFormat enum covers the common cases (Percent, Comma, Dollar, Scientific, Number(accuracy)).

Colours are names (R’s 657 colour names, grey0–grey100) or hex "#RRGGBB" and "#RRGGBBAA". The default discrete palette is ggplot2’s HCL hue palette (hue 15–375, chroma 100, luminance 65); continuous is the #132B43–#56B1F7 gradient. ColorBrewer and viridis tables ship as generated Plenty data. Legends are generated from every non-position scale; scales with the same title and levels merge into one legend, as in ggplot2.

Coordinates and facets

  • Milestone 1: coord_cartesian(xlim=, ylim=, expand=), coord_flip(), coord_fixed(ratio=); facet_wrap(facets, nrow=, ncol=, scales=) and facet_grid(rows=, cols=, scales=, space=) with Scales.Fixed, Free, FreeX, FreeY. Facet variables are column names or lists of names.
  • Milestone 2: labeller choices (LabelBoth, LabelValue), margins.
  • Milestone 3: coord_polar, coord_radial, coord_trans.

Formula strings ("drv ~ cyl") are not accepted; rows= and cols= are the single spelling, matching ggplot2’s vars() interface. A string containing ~ fails with a hint.

Themes and labels

The default is theme_grey (grey panel, white grid), the look ggplot2 users recognise. theme_bw, theme_minimal, theme_classic, theme_light, theme_dark, theme_void and theme_linedraw take base_size, base_family, ink and paper.

theme(...) takes ggplot2’s element names with dots replaced by underscores, as plotnine does: legend_position, axis_text_x, panel_grid_minor, plot_title. Each parameter defaults to “inherit”, so a theme call changes only what it names. Element inheritance (axis_text_x inherits axis_text, which inherits text) follows ggplot2’s element tree. ggplot2 has about 130 theme elements; the full set fits within the 255-parameter limit, and milestone 1 ships the ~40 most used. Element constructors are element_text(size=, face=, colour=, angle=, hjust=, vjust=, margin=), element_line, element_rect and element_blank(). Enumerated settings are enums (Legend.Bottom, Face.Bold) rather than strings, so typos fail to compile.

labs(title=, subtitle=, caption=, tag=, x=, y=, colour=, fill=, ...), xlab(), ylab() and ggtitle() set labels. Without them, axis and legend titles are the mapped column names, as in ggplot2.

Notes instead of messages

ggplot2 prints messages and warnings: the bin count chosen by stat_bin, the smoothing method chosen by geom_smooth, rows removed for missing values. Plenty has no warning channel, and a library must not write to standard error on its own. Rendering returns a Rendered value whose notes list carries the same information; print(rendered.notes)? shows it. save is the convenience that writes the file and returns the notes. na_rm=True on a layer silences the missing-value note, as in ggplot2.

Rendering and output

Formats

  1. SVG first (milestone 1). It is text, so output is easy to test, diff and view, and needs no rasteriser. p.save("out.svg", ...) or p.write_svg(&mut writer, ...) for any std.io writer.
  2. PNG (milestone 2): an anti-aliased scanline rasteriser in Plenty (signed-area coverage accumulation, the technique used by font-rs and stb_truetype), then PNG chunks assembled in Plenty with std.compress.compress(Format.Zlib, ...) for IDAT and std.compress.crc32 for chunk checksums.
  3. PDF later, if requested; it shares the SVG canvas model.

save chooses the format from the extension, with width, height, units (Units.In, Cm, Mm, Px; inches by default) and dpi (default 300), as ggsave does.

Text and fonts

Layout needs text measurement even for SVG: axis label widths decide panel size, and legend widths decide plot width. ggplot2 asks the graphics device. std.plot uses bundled font data:

  • Metrics (advance widths and kerning pairs) for a default sans-serif font are generated into the module, so measurement needs no file system access and gives the same layout on every machine. The recommended font is Liberation Sans (SIL Open Font License), which is metric-compatible with Arial and Helvetica. An SVG names font-family="Liberation Sans, Arial, Helvetica, sans-serif", so viewers without the bundled font substitute one with the same widths and the measured layout still holds.
  • Outlines for PNG and for SVG with text=Text.Paths (exact rendering without viewer fonts) come from a subset of the same font, embedded as a b"..." byte literal and parsed in Plenty (cmap, hmtx, kern, glyf quadratic outlines). The subset covers Latin-1, Latin Extended-A, Greek, and plotting symbols (minus sign, multiplication sign, degree, micro, en dash). Emission removes it from programs that never draw outlines.
  • font_file(path) loads another TrueType font at run time; base_family then names it.
  • Missing glyphs render as the font’s .notdef and add a note.
  • Shaping is left-to-right with kerning only. Complex scripts are out of scope; if they are needed later, shaping is the one piece where wrapping an established engine is justified, subject to the allocation rule below.

Deterministic output

Given the same plot and toolchain, output bytes are identical:

  • coordinates are written with fixed precision (two decimals in SVG user units), elements in layer order, clip and gradient identifiers numbered sequentially;
  • no timestamps, random identifiers or host font lookup;
  • jitter uses a fixed default seed;
  • the rasteriser is Plenty floating-point code with no target-dependent fast paths, and PNG compression uses a fixed level of the pinned zlib-rs.

Tests compare SVG snapshots, like ggplot2’s vdiffr (which also renders through an SVG device with bundled Liberation fonts), and compare decoded PNG pixels rather than compressed bytes.

Write in Plenty or wrap

PartChoiceReason
Grammar: mappings, stats, scales, guides, facets, layoutPlentyNo crate implements it; plotters does not. This is most of the work in any case.
Statistics (bins, lm, loess, density, quantiles)PlentyFormulas over lists; builds on std.stat
Colour (HCL conversion, palettes)PlentyFormulas and tables
SVG writerPlentyText generation, a few hundred lines
Font parsing and text measurementPlentyTable parsing over bytes; ab_glyph and font-kit allocate infallibly, and font-kit adds host font discovery
RasteriserPlentyShort, numeric code the compiler should handle well; tiny-skia allocates infallibly. Measure before milestone 2 closes
zlib compression and CRC-32Existing native std.compress (zlib-rs)Already approved, reports allocation failure
Complex-script shapingNot planned; the only candidate to wrapA large established engine

plotters is not recommended as a backend. It would replace only the SVG writer and rasteriser, the smallest parts of the work. Its layout model (label areas sized in pixels by the caller) does not match ggplot2’s, which sizes from measured text, so std.plot would compute layout anyway and pass primitives across a foreign boundary. It allocates through Rust’s global allocator, which aborts on failure; the native-wrapper rule rejects that for new wrappers. Its bitmap backend brings image and font-kit, with system font lookup that breaks deterministic output. Its role here is a reference for primitive drawing and a comparison for rasteriser tests.

If the Plenty rasteriser cannot reach acceptable speed (target: a 2,400 × 1,800 PNG with 100,000 points in under 0.5 s on the reference laptop), the response is codegen work or std.simd, per the Plenty-first preference, before any wrapper.

Costs

  • Source size. Milestone 1 is likely 10–15k lines of Plenty, comparable to std.markdown plus std.regex. Importing a large std module costs front-end time; measure the compile time of a one-plot program at each milestone and keep it within a budget agreed before milestone 1 (proposed: under 150 ms over hello world with a warm cache).
  • Executable size. Emission drops unused std functions, so a program that only writes SVG carries neither the rasteriser nor the font outlines. Add a size check like the existing executable-size budgets.
  • Runtime. Each stage is linear in rows except sorting for lines, quantiles and loess (O(n²) per fit before interpolation). Rendering a million-point scatter to SVG produces a file of tens of megabytes, as in ggplot2; PNG is the format for large data.

Phased plan

  1. Core and SVG. Table, table, columns, rows; aes, factor, after_stat; milestone-1 geoms, stats, positions, scales, coords, facets and themes; layout with bundled metrics; SVG output; PlotError; notes; reference page, tutorial lesson, SVG snapshot tests. Exit: the six examples render and read correctly next to ggplot2’s output.
  2. Breadth and PNG. Milestone-2 geoms and scales, loess, jitter, dates, glyph outlines, rasteriser and PNG, text=Text.Paths.
  3. Records by name. records() over B77’s Encoder once B77 lands.
  4. Dataframes. The bridge module with Expr aesthetics and required-column extraction; later, stat push-down into Polars.
  5. Milestone-3 geoms and coords, as demand shows.

Stages 3 and 4 depend on other work and can move earlier if it lands first.

Decisions

The review accepted these choices. Each records the alternative considered.

  1. ggplot2 names verbatim, not Plenty-style names such as plot.point(). Recognition is the goal, and plotnine shows the names work outside R. Users import with from std.plot import ggplot, aes, geom_point or import std.plot as gg; wildcard imports are not proposed.
  2. Both colour and color are accepted; a call that gives both is PlotError.Conflict. This is two spellings for one thing, against the preference for single rules, but ggplot2 users write both and the cost is one parameter per colour aesthetic.
  3. Composition uses the component list and plus. No operator hooks are added for this: the ? after every step would make + worse than the list.
  4. ggplot2’s implicit defaults stay (bins=30, the smoothing method), each reported in the notes as ggplot2’s messages do. The plot shows the effect immediately, and requiring them would break the “write it as in ggplot2” goal.
  5. geom_smooth uses loess at every size, where ggplot2 switches to an mgcv GAM above 1,000 rows. Large inputs are evaluated on an interpolation grid as R’s loess(surface = "interpolate") does, and the note says so. Until loess lands in milestone 2, the default is Smooth.Lm, also noted.
  6. No typed column handles (aes(x=Car.wt)) for now. Strings checked at construction match ggplot2’s behaviour. Revisit after B77 lands: the per-column record accessor (num("wt", def(c: &Car) -> f64: c.wt)?) is verbose, and if records() does not remove most of that boilerplate, or mistyped column names prove common, typed handles become worth a language change.
  7. Default font: Liberation Sans, for Arial/Helvetica metric compatibility and the OFL licence. DejaVu Sans has wider coverage but different metrics from common viewer fonts.
  8. PNG uses a Plenty rasteriser, not a tiny-skia wrapper, with the speed target in Write in Plenty or wrap as the acceptance test.
  9. Missing values: rows missing any aesthetic a layer uses are dropped and counted in the notes, as ggplot2 does; na_rm=True silences the note.
  10. std.plot belongs in std, beside std.dataframe, not in a first-party package. The scope aims at data tools without third-party packages, and emission removes what a program does not use.
  11. The dataframe bridge is a separate submodule under std.plot’s namespace (as std.http has submodules), imported only by dataframe users, so neither module forces the other’s dependencies.

Risks

  • Scope. ggplot2 has about 50 geoms, 30 stats and over 100 scales. The milestones cut this to the plots most people make; the risk is users arriving with the missing tenth.
  • Fidelity expectations. Users will compare against ggplot2 output. Differences in fonts, break placement or smoothing will be reported as bugs. Matching defaults and testing against R reference values (breaks, lm standard errors, loess fits, histogram bins) limits this.
  • Text. Without shaping, labels in complex scripts render incorrectly. SVG viewers without metric-compatible fonts can misplace text slightly; text=Text.Paths is the exact fallback.
  • Runtime checking of names. Mapping errors appear when the program runs, not when it compiles. Eager validation at construction and suggestions limit the cost.
  • Compile time and binary size of a large std module; the budgets above make regressions visible.
  • Numerical accuracy of statistics written from scratch. Test against R.
  • Dependencies on unimplemented work: records() waits for B77; the dataframe bridge depends on the dataframe module’s typed readers staying stable.

References

std.plot examples beside ggplot2 and plotnine

Status: Not implemented. These examples illustrate the std.plot proposal. The Plenty fences show proposed API and are not compiled. Each example assumes a table with the columns of the R dataset named (mtcars, diamonds, mpg) and these imports:

from std.plot import ggplot, aes, factor, after_stat, labs, theme, theme_minimal
from std.plot import geom_point, geom_smooth, geom_histogram, geom_bar
from std.plot import facet_wrap, facet_grid, position_dodge, scale_colour_brewer
from std.plot import Smooth, Brewer, Legend

Scatter plot with a colour mapping

ggplot(mtcars, aes(wt, mpg, colour = factor(cyl))) +
  geom_point(size = 2)
(ggplot(mtcars, aes("wt", "mpg", color="factor(cyl)"))
 + geom_point(size=2))
p = ggplot(mtcars, aes("wt", "mpg", colour=factor("cyl")), [
    geom_point(size=2),
]?)?
p.save("cars.svg", width=6, height=4)?

factor("cyl") makes the integer column discrete, so the legend shows three levels instead of a gradient. size=2 is accepted because numeric settings take i64 as well as f64.

Points with a fitted line

ggplot(mtcars, aes(wt, mpg)) +
  geom_point() +
  geom_smooth(method = "lm")
(ggplot(mtcars, aes("wt", "mpg"))
 + geom_point()
 + geom_smooth(method="lm"))
p = ggplot(mtcars, aes("wt", "mpg"), [
    geom_point(),
    geom_smooth(method=Smooth.Lm),
]?)?

The line carries a 95% confidence band by default (se=True), as in ggplot2.

Histogram

ggplot(diamonds, aes(price)) +
  geom_histogram(binwidth = 500)
ggplot(diamonds, aes("price")) + geom_histogram(binwidth=500)
p = ggplot(diamonds, aes("price"), [
    geom_histogram(binwidth=500),
]?)?

Without binwidth or bins, the stat uses 30 bins and records a note, the counterpart of ggplot2’s `stat_bin()` using `bins = 30` message. Density instead of counts: aes("price", after_stat("density")).

Bars with fill and dodge

ggplot(diamonds, aes(cut, fill = clarity)) +
  geom_bar(position = "dodge")
(ggplot(diamonds, aes("cut", fill="clarity"))
 + geom_bar(position="dodge"))
p = ggplot(diamonds, aes("cut", fill="clarity"), [
    geom_bar(position=position_dodge()),
]?)?

geom_bar counts rows per cut and clarity, so y is not mapped. position_dodge(width=0.8) takes the same arguments as in ggplot2.

Facets

ggplot(mpg, aes(displ, hwy)) +
  geom_point() +
  facet_wrap(~class, ncol = 3)

ggplot(mpg, aes(displ, hwy)) +
  geom_point() +
  facet_grid(drv ~ cyl)
ggplot(mpg, aes("displ", "hwy")) + geom_point() + facet_wrap("class", ncol=3)

ggplot(mpg, aes("displ", "hwy")) + geom_point() + facet_grid("drv ~ cyl")
base = ggplot(mpg, aes("displ", "hwy"), [geom_point(),]?)?
wrapped = base.plus([facet_wrap("class", ncol=3)]?)?
grid = base.plus([facet_grid(rows="drv", cols="cyl")]?)?

base stays usable after each plus; both plots share its table.

Labels, scales and a theme

ggplot(mtcars, aes(wt, mpg, colour = factor(gear))) +
  geom_point(size = 3) +
  scale_colour_brewer(palette = "Set1") +
  labs(title = "Fuel economy by weight",
       x = "Weight (1000 lb)", y = "Miles per gallon", colour = "Gears") +
  theme_minimal(base_size = 12) +
  theme(legend.position = "bottom")
(ggplot(mtcars, aes("wt", "mpg", color="factor(gear)"))
 + geom_point(size=3)
 + scale_color_brewer(type="qual", palette="Set1")
 + labs(title="Fuel economy by weight",
        x="Weight (1000 lb)", y="Miles per gallon", color="Gears")
 + theme_minimal(base_size=12)
 + theme(legend_position="bottom"))
p = ggplot(mtcars, aes("wt", "mpg", colour=factor("gear")), [
    geom_point(size=3),
    scale_colour_brewer(palette=Brewer.Set1),
    labs(title="Fuel economy by weight",
         x="Weight (1000 lb)", y="Miles per gallon", colour="Gears"),
    theme_minimal(base_size=12),
    theme(legend_position=Legend.Bottom),
]?)?

Brewer.Set1 and Legend.Bottom are enum values, so a misspelt palette or position fails to compile.

A complete program from records

from std.plot import ggplot, aes, factor, geom_point, geom_smooth, labs
from std.plot import rows, num, integer, text, Smooth, PlotError

class Car:
    model: str
    wt: f64
    mpg: f64
    cyl: i64

def main() -> Result[(), PlotError | AllocError | IoError]:
    cars = [
        Car("Mazda RX4", 2.620, 21.0, 6),
        Car("Datsun 710", 2.320, 22.8, 4),
        Car("Hornet Sportabout", 3.440, 18.7, 8),
    ]?

    table = rows(&cars, [
        text("model", def(c: &Car) -> str: c.model)?,
        num("wt", def(c: &Car) -> f64: c.wt)?,
        num("mpg", def(c: &Car) -> f64: c.mpg)?,
        integer("cyl", def(c: &Car) -> i64: c.cyl)?,
    ]?)?

    p = ggplot(table, aes("wt", "mpg", colour=factor("cyl")), [
        geom_point(size=2),
        geom_smooth(method=Smooth.Lm, colour="grey30"),
        labs(x="Weight (1000 lb)", y="Miles per gallon", colour="Cylinders"),
    ]?)?

    notes = p.save("cars.svg", width=6, height=4)?
    print(notes)?
    Ok(())

geom_smooth(colour="grey30") is a setting: one fixed colour, so the line does not split by the plot’s colour mapping. In ggplot2 the same setting also overrides the inherited mapping for that layer.

After derived serialization lands, the rows call can become records(&cars)? for a class marked @serial.

A mistake and its error

p = ggplot(table, aes("wt", "mpg", colour="cly"), [geom_point(),]?)?
error: plot mapping: aesthetic `colour` maps unknown column "cly"
  available columns: model, wt, mpg, cyl
  did you mean "cyl"?

The error comes from ggplot(...), before rendering. ggplot2 reports object 'cly' not found when the plot is printed.

Exact decimal arithmetic (std.decimal)

Status: Not implemented. Nothing shipped; no reference page exists. Researched against local main at 70e85758 on 2026-10-31. Dataframe decimal columns wait on this design (B76). No backlog entry exists yet. Revised twice on 2026-10-31 after review; every decision is now settled.

Recommendation

Add std.decimal with one type, Decimal: a fixed-size, exact decimal of up to 38 significant digits and 0–38 fractional digits, written in Plenty as an allocation-free copy class. It follows CPython’s decimal wherever that does not conflict with the rules below, since CPython’s semantics are the most widely used and tested:

  • The scale is kept, as in Python: Decimal.parse("1.50") displays as 1.50, and 1.5 * 2.00 is 3.000. Equality and ordering are numeric across scales (1.0 == 1.00), and equal values hash equal, as in Python.
  • +, - and * are exact. A result that does not fit traps as integer overflow does, and inside a checked: block it becomes Err(OverflowError[Decimal]), as for i64. There are no decimal-specific checked methods, contexts or signals.
  • Division takes a NonZero[Decimal] divisor, so division by zero cannot be written.
  • Nothing rounds unless the call names places and a rounding mode, including conversion from f64. Division therefore has no / operator; it is a.div(divisor, places, mode). There is no context, no precision setting, and no silent rounding.
  • There is no NaN, infinity, or negative zero.
  • The digit limit matches Arrow/Polars Decimal128 and SQL DECIMAL(38, s), so dataframe and database values convert without loss.

Do not wrap rust_decimal, libmpdec, or decNumber for the default type. Leave arbitrary precision to a separate, later design if a workload needs it.

The type needs language work first (staged plan below): a 64×64→128-bit integer multiply; @arithmetic methods and hooks, so library types join the integer trap and checked: rules; the __cmp__ hook from the ordering proposal, so == is numeric; and a wider NonZero bound. Dictionary keys and decimal format specs follow later.

What financial code needs

NeedWhyThis design
Exact representation of decimal fractions0.1 + 0.2 must equal 0.3; amounts must round-trip through textCoefficient × 10-scale, scale kept
Exact addition, subtraction and multiplicationTotals, quantities × prices, rates × amountsExact or overflow; never rounded
Explicit rounding with named modesTax, interest and currency rules specify both the place and the rule (banker’s vs commercial)quantize(places, mode); seven modes
Division to stated placesUnit prices, rates, averagesdiv(divisor, places, mode) with a NonZero[Decimal] divisor
Overflow detectionA wrong total is worse than a crashThe integer trap, or Err inside checked:; never wrap
No surprise from binary floats0.1 as f64 is not 0.1from_f64 requires places and a mode
Parsing and formatting without lossCSV, JSON, SQL, user inputparse keeps every digit or fails; display keeps the scale
Splitting an amount without losing unitsInstalments, pro-rata allocationallocate (stage 2)
Interchange with columnar and SQL decimalsPolars, Parquet, databasesSame 38-digit range; scale kept like a column’s

Not needed for the common case, and excluded from the default: transcendental functions (ln, exp, sqrt), NaN payloads, signalling NaNs, subnormals, precision contexts, and numbers beyond 38 digits.

Candidates

The user’s phrase “the Rust decimal crate” most likely means rust_decimal (168 M downloads, the de facto Rust choice). The crate actually named decimal is an older binding to IBM’s decNumber decimal128 (last release 2.1.0, 2021). Both are covered.

CandidateModelSize and allocationRounding and errorsLicence, form
CPython decimal / libmpdecGeneral Decimal Arithmetic (GDA) spec, arbitrary precision; context with precision (default 28), rounding (default half-even), traps and sticky flags; NaN, ±Inf, −0; trailing zeros significantHeap coefficient per value. Allocation is fallible: mpd_switch_to_dyn sets MPD_Malloc_error and returns NaN (mpalloc.c)Every operation rounds to context precision; traps raise, otherwise flags are set. Python keeps the context in a contextvars variable: hidden per-thread stateBSD-2-Clause C, about 18,600 lines. CPython removed its bundled copy in May 2026 (9eb3b146686) and now requires a system libmpdec ≥ 2.5
decNumber via decimal or decGDA; fixed decimal128 (34 digits, IEEE 754-2008) plus arbitrary decNumberdecimal128 is 16 bytes and allocation-free; decNumber may mallocContext and status flags, as libmpdecICU licence C (about 16,800 lines), Apache-2.0 bindings; decimal unmaintained since 2021
rust_decimal 1.4396-bit coefficient, scale 0–28, 16 bytes; at most 28–29 digitsNo allocation for arithmeticRounds silently: a product whose scale exceeds 28 is rounded half-even, and one too small becomes zero (src/ops/mul.rs); division rounds to 28 digits. Operators panic on overflow; checked_* return Option. Eq/Hash are numeric (normalized)MIT Rust, about 12,300 lines
fastnum 0.7D128: 128-bit coefficient plus a 64-bit control block (scale, sign, flags), 24 bytes; NaN, ±Inf, ±0No allocationGDA-like context and signals. Default traps (division by zero, invalid, overflow) panic only in debug builds (context/signal_traps.rs); release builds return special valuesMIT/Apache-2.0 Rust, about 25,700 lines; MSRV 1.94
bigdecimalnum-bigint coefficient + i64 scaleVec per value; Rust’s global allocator aborts on failureExplicit precision on divisionMIT/Apache-2.0 Rust
Polars Decimal(p, s) (polars-compute 0.55.2 decimal.rs)i128 unscaled values, precision ≤ 38, scale fixed per columnColumn storageMultiplication and division round half-even back to the column scale; overflow returns NoneMIT Rust; reference algorithms, not an API
Plenty-native Decimal (recommended)Two base-1019 u64 limbs + scale + sign, as libmpdec stores words; 38 digits, scale 0–38, scale keptFour inline fields; never allocatesExact or trap; rounding only on requestPlenty source in stdlib/std/decimal.plenty

Why not wrap

Project policy (AGENTS.md, engine criteria) requires a concrete reason to wrap, such as substantial complexity or specialist security work. A fixed 128-bit decimal has neither:

  • The core is integer arithmetic on two 64-bit limbs plus scaling by powers of ten. Polars implements the equivalent i128 operations, including correctly rounded 256-by-128-bit division, in one 1,171-line file.
  • Every wrapped candidate carries semantics the recommended rules reject: rust_decimal rounds silently and stops at 28 digits, below the 38 that Polars and SQL use; fastnum returns NaN in release builds; libmpdec and decNumber need a context on every call and C builds for four targets, including MSVC.
  • A native call per addition costs more than the addition. A Plenty implementation inlines through ordinary Cranelift code.
  • Plenty code links per module, so a program that does not import std.decimal pays nothing either way (per-module linking, checked by tests/test_executable_size.rs).

Wrapping libmpdec remains the right choice if users later need the full GDA operation set (ln, exp, sqrt, pow with fractional exponents) at arbitrary precision. Its allocation failure is already reported as a status flag, so a wrapper can map it to AllocError without patching.

Semantic choices

Fixed 38 digits, not arbitrary precision

Fixed size is what makes the type a copy class: no allocation, no Result on every operation, no ownership bookkeeping. 38 digits holds any amount below 1020 currency units with 18 fractional digits (enough for Ethereum’s 10-18), and is the limit of Arrow Decimal128, Polars, Parquet, SQL Server, BigQuery NUMERIC and Snowflake NUMBER. IEEE decimal128 has only 34 digits and a binary-incompatible encoding (BID/DPD); rust_decimal has 28.

Arbitrary precision moves allocation into every arithmetic expression, (a * b)? everywhere, and needs a rule for how large results may grow. If a workload needs it, design it separately; this proposal does not depend on one.

Representation

Store the coefficient in two u64 words in base 1019, as libmpdec does on 64-bit targets (MPD_RADIX is 1019, MPD_RDIGITS 19, in mpdecimal.h). Two words hold exactly 38 digits, so the digit limit is “high word below 1019”; aligning scales, rounding at a digit, and parsing and printing split at word boundaries with powers of ten that fit one word. libmpdec multiplies words with a double-word product (_mpd_mul_words in typearith.h: __uint128_t or x86-64 mulq); u64.widening_mul is the Plenty equivalent. Reducing a product by the constant 1019 is a multiply by a precomputed reciprocal, so the same primitive serves; the prototype should confirm that before choosing between base 1019 and binary words. This is an implementation choice, not part of the API.

Scale, equality and hashing

The scale is kept, with Python’s ideal exponents (Lib/_pydecimal.py):

OperationResult scaleExample
parseAs written"1.50" → 1.50
a + b, a - bmax(scale_a, scale_b)1.5 + 2.25 → 3.75; 1.50 + 1.50 → 3.00
a * bscale_a + scale_b1.5 * 2.00 → 3.000
quantize(places, mode), div(d, places, mode)places1.5.quantize(2, …) → 1.50
normalize()Smallest exact scale3.000 → 3

A result needing more than 38 digits, counting trailing zeros, or more than 38 fractional digits overflows. Python would round it to the context precision instead; here that is a trap, or Err in checked:.

Python compares numerically: Decimal("1.0") == Decimal("1.00") is True and hash(Decimal("1.0")) == hash(Decimal("1.00")) (__eq__ uses _cmp; __hash__ hashes coefficient × 10-exp modulo the hash prime). Decimal matches both. This needs three things from Plenty:

  1. Equality and ordering: __cmp__. Plenty’s == on classes is structural and cannot be overridden (operator boundaries), so with the scale kept, structural == would make 1.0 == 1.00 false. The ordering proposal (B37) already gives the fix: defining __cmp__(self, other: &T) -> Order derives all six comparisons, and replaces structural == and != for the type (its rule 3), including inside collections. Decimal.__cmp__ aligns scales and compares coefficients; it allocates nothing. Decimal cannot ship before this, because its == would be wrong. A separate __eq__ is not needed and the ordering proposal rejects one; Python’s own six methods all reduce to _cmp.
  2. Keys: a key projection. Under the hashing proposal (B74), keys hash structurally, so a type defining __cmp__ is not a key type. Its key projection extension is the consistent route: Decimal.__key__() returns the normalized parts (coefficient without trailing zeros, scale, sign), a structural key value, so equal decimals hash and look up equally. That proposal’s main objection, that projections usually allocate, does not apply: normalizing is integer division by ten on two words. Until it lands, Decimal is not a key; code can key on d.normalize().parts().
  3. Consistency. __key__ equality and __cmp__ equality must agree, which the compiler cannot check. Restrict both hooks to std at first, as for @arithmetic, and test the agreement with differential tests against CPython.

No context, no special values

A GDA context is hidden mutable state (Python stores it in a contextvars variable), and its default precision of 28 silently rounds results that do not fit. Instead, an operation that can be inexact takes its places and rounding mode as arguments, and the others are exact. With no context there are no sticky flags to check, and with errors as values there is no need for NaN or infinity. Negative zero carries no financial meaning: parse("-0.00") is 0.00, where Python keeps -0.00 (it still compares equal to zero).

Floats

Python’s Decimal(0.1) and Decimal.from_float(0.1) convert exactly, giving 0.1000000000000000055511151231257827021181583404541015625; mixing floats into the constructor and comparisons is allowed unless the FloatOperation signal is trapped. That exact expansion is the confusion this module exists to avoid, and at 55 digits it does not fit anyway.

Decimal has no implicit or exact float conversion:

  • Decimal.from_f64(value, places, mode) rounds the float’s exact binary value to places with mode: from_f64(0.1, 2, Rounding.HalfEven) is 0.10. NaN, infinity, or a result over 38 digits is Err(RangeError.OutOfRange).
  • Operators and comparisons never accept f64; mixing is a compile error, as mixing i64 and f64 is.
  • parse is the way to write a decimal constant: Decimal.parse("0.1").
  • to_f64() returns the nearest f64.

Overflow: the integer trap and checked:

+, -, * and unary - on Decimal follow the integer rules:

  • Outside a checked: block, an unrepresentable result is an arithmetic trap at the caller’s position, with the standard-library caller note. Actors and test harnesses see it as TrapKind.Arithmetic, like integer overflow.
  • Written inside a checked: block, the same expression makes the block Err(OverflowError[Decimal]). A block mixing integer and decimal arithmetic gets both errors in its anonymous sum, for example OverflowError[i64] | OverflowError[Decimal].

The same rule covers the methods that can overflow (quantize, div, mul_quantize, div_rem), as it covers pow on integers.

OverflowError[T]’s wrapped field is the exact result modulo 2N for integers. For Decimal it is the decimal analogue: the exact result truncated toward zero to 38 fractional digits, with its coefficient reduced modulo 1038. It is deterministic and, like the integer field, mainly diagnostic.

Gap: library types in the trap rules

Today the rules do not reach library code. A checked: block checks only builtin integer operators, pow, and operators that @newtype classes forward; an operator hook such as Complex.__add__ is an ordinary call and keeps trapping inside a block. Standard-library code also has no way to raise the arithmetic trap for a condition that is not an integer overflow, such as a 39-digit decimal result.

The proposed fix is one general mechanism, not a decimal one. An @arithmetic decorator on a method or operator hook whose body returns Result[T, OverflowError[T]] (optionally | ZeroDivisionError) changes how callers see it:

  • Callers see the method as returning T. Outside checked:, the compiler turns an Err into the arithmetic trap at the call site, so the caller note and TrapKind.Arithmetic work as for integers.
  • A call or operator written lexically in a checked: block adds the declared errors to the block’s error type and exits the block on Err, as integer operators do.
  • The lexical rule is unchanged: integer arithmetic inside the hook body is ordinary code, and the hook decides what overflow means for its type.
  • Users never call a separate checked form. checked: is the only way to receive the error.

Closing the gap needs checker work (validate the decorator’s signature, type the call as T, contribute errors to enclosing blocks, and restrict the decorator to std) and code generation (the Err branch to the existing trap and the block’s exit edge). Other library types could adopt it later; std.time.Duration, which has checked_* methods returning RangeError, is one candidate, out of scope here.

Rounding modes

Seven modes, CPython’s set without ROUND_05UP, named by direction rather than by Python’s ambiguous ROUND_UP and ROUND_DOWN:

PlentyPython / GDAExample at 0 places: 2.5, −2.5, 2.4
HalfEvenROUND_HALF_EVEN2, −2, 2
HalfAwayFromZeroROUND_HALF_UP3, −3, 2
HalfTowardZeroROUND_HALF_DOWN2, −2, 2
TowardZeroROUND_DOWN2, −2, 2
AwayFromZeroROUND_UP3, −3, 3
FloorROUND_FLOOR2, −3, 2
CeilingROUND_CEILING3, −2, 3

GDA’s ROUND_05UP is omitted. Every rounding call names its mode; there is no default.

Division: a non-zero divisor

Division by zero is ruled out by the divisor’s type, following the decision behind NonZero[T] (issue #584): check once where the value is created, not at every use. The divisor is NonZero[Decimal], the same builtin type the integers use, because widening NonZero to library types later lets users define non-zero forms of their own numeric types.

Widening the NonZero bound

NonZero[T: IntType] becomes NonZero[T: ZeroTest]. ZeroTest is a built-in constraint, like IntType and the proposed KeyType, because a protocol cannot admit integers (they have no methods). It is satisfied by:

  • every integer type, unchanged: literals convert at compile time, 0 is a compile error, and operators forward to the builtin integer operators;
  • a @copy class that defines def is_zero(self) -> bool, computation only (no I/O, clock, entropy or process effects, as __cmp__ requires).

For such a class T, NonZero[T] provides what it provides for integers, built from T’s methods:

NonZero[i64]NonZero[Decimal]
Literal converts at compile time; 0 is a compile errorNo decimal literals. Decimal.nonzero(units: NonZero[i64], scale) and Decimal.nonzero_int(value: NonZero[i64]) take integer units, so Decimal.nonzero(0, 2) is a compile error
NonZero.try_from(value) -> Result[NonZero[i64], RangeError]Same; OutOfRange when is_zero(), so 0.00 is rejected
.get() -> i64.get() -> Decimal
* and unary - keep the type; + and - return TSame, through Decimal’s hooks
Mixed NonZero[T] and T operands are a compile error suggesting .get()Same
Copy, ==, ordering, printing, dictionary keyCopy; == and ordering from Decimal.__cmp__; printing; keys when Decimal is one

Decimal must provide @copy, is_zero (the coefficient is zero, at any scale), and * and unary - hooks that never turn non-zero operands into zero. Both hold: products are exact or overflow, and a product needing more than 38 fractional digits overflows instead of rounding to zero. Inside checked:, NonZero[Decimal] products report OverflowError[Decimal], as NonZero[i64] products report OverflowError[i64].

That last requirement is a contract the compiler cannot check. A user type whose * can produce zero (a matrix, a modular integer) would make NonZero[T] lie. So ZeroTest for classes is std-only at first, like @arithmetic; opening it to user types needs a rule for that contract.

The compiler work is in src/frontend/nonzero.rs: the bound, try_from through is_zero, forwarding *, unary -, +, - and comparisons through the class’s hooks rather than the integer operators, and the compile-time 0 check staying integer-only.

There is no flow narrowing (if d != 0: does not change d’s type), as for integers. A runtime count converts once: Decimal.nonzero_int(NonZero.try_from(n)?).

Division API

a.div(divisor: &NonZero[Decimal], places: i64, mode: Rounding) -> Decimal
a.div_rem(divisor: &NonZero[Decimal], mode: Rounding) -> (Decimal, Decimal)
  • div returns the quotient correctly rounded to places fractional digits (0–38; others trap like Decimal.of) with mode. A quotient over 38 digits overflows under the checked: rules above; it has no other failure.
  • div_rem rounds the quotient to an integer with mode and returns the exact remainder a - q * divisor. TowardZero gives Python’s Decimal // and % (-7 // 2 is -3, remainder -1); Floor gives Plenty’s integer // and % (-4, remainder 1). The two disagree, so the mode is required rather than picking one.

Why still no /: a non-zero divisor removes the zero case but not rounding. 1 / 3 has no exact result, so / would need fixed places and a fixed mode. Python rounds to the context precision, Polars to the column scale and rust_decimal to 28 digits; each is the silent rounding this design excludes, and a trap on every inexact quotient would make / fail for most inputs.

Integers differ today: i64 // NonZero[i64] is a compile error (the operand must be i64), and integer // and % still trap on zero. Issue #584 left NonZero divisors for // and % as separate work; std.decimal starts with the rule that work would adopt.

Parsing and formatting

Decimal.parse(text) follows the numeric parsing contract: it allocates nothing, trims white space, and accepts an optional sign, digits, an optional fraction, and an optional exponent (1.5e3). The scale is the written one, adjusted by the exponent and never below 0: "1.50" has scale 2, "1.5e3" is 1500, "1E+2" is 100 (Python keeps exponent 2 and prints 1E+2). More than 38 digits or 38 fractional digits is ParseError.OutOfRange; it never rounds. inf, nan, grouping and locale forms are ParseError.Invalid.

display writes the plain form with the stored scale (-1234.50), never an exponent; Python switches to exponent form for small values (1E-7). To show a fixed number of places, quantize first: total.quantize(2, Rounding.HalfEven).

Today an f-string spec on a class applies to its text: probing with std.complex, f"{Complex(1.25, 2.0):.3}" prints 1.2. For a decimal, {total:.2} would therefore cut 64.92 to 64. Until format specs reach library types (stage 3), the reference must warn against specs on Decimal.

Naming against Python

PythonPlentyReason
quantize(Decimal("0.01"), rounding)quantize(2, mode)Same operation and name; places as a count, since scale is never negative
round(d, n)Not providedPython’s uses the context’s rounding, and round(d) returns int
normalize()normalize()Same, but exact: no rounding to a precision
compare, <, ==compare, <, == from __cmp__Same
int(d)to_i64(mode)int truncates silently
as_tuple()parts()Coefficient words, scale, sign
ROUND_HALF_UP etc.Rounding.HalfAwayFromZero etc.Python’s UP/DOWN names read as directions they are not

Fit with the language today

NeedTodayConsequence
128-bit coefficientNo u128/i128; u64 has wrapping and checked operations but no high productTwo u64 words; add u64.widening_mul (Cranelift umulhi, available on x86-64 and AArch64 in the pinned 0.137)
Arithmetic operatorsOperator hooks as in std.complex+ - * and unary -
checked: and trapsInteger operators, pow, @newtype forwarding; hooks keep trapping; std cannot raise the arithmetic trap@arithmetic methods and hooks, std only
Numeric == and <Structural == only; no ordering hooks (B37)__cmp__ from the ordering proposal; a prerequisite
Dictionary keysNot available for classes (B74)__key__ projection after B74
Non-zero divisorNonZero[T] for integer T onlyZeroTest bound, std classes only
Literals and constantsconst accepts only scalars, str, bytesDecimal.of(1999, 2) is allocation-free and infallible; parse for text
Format specsApplied to display textquantize then display; later a spec hook

Decimal needs widening_mul on its own account: every product of two coefficients, and the 256-by-128-bit division behind div and quantize, is built from 64×64→128-bit products.

API sketch

from std.decimal import Decimal, Rounding

pub enum Rounding:
    HalfEven
    HalfAwayFromZero
    HalfTowardZero
    TowardZero
    AwayFromZero
    Floor
    Ceiling

@copy
pub class Decimal:
    # private: _high: u64, _low: u64 (base 10**19), _scale: u8, _negative: bool

    # Construction: allocation-free.
    Decimal.ZERO, Decimal.ONE
    Decimal.of(units: i64, scale: i64) -> Decimal        # of(1999, 2) is 19.99; traps outside 0..38
    Decimal.from_int(value: i64) -> Decimal
    Decimal.nonzero(units: NonZero[i64], scale: i64) -> NonZero[Decimal]
    Decimal.nonzero_int(value: NonZero[i64]) -> NonZero[Decimal]
    Decimal.parse(text: str) -> Result[Decimal, ParseError]
    Decimal.from_f64(value: f64, places: i64, mode: Rounding) -> Result[Decimal, RangeError]
    Decimal.from_parts(high: u64, low: u64, scale: i64, negative: bool) -> Result[Decimal, RangeError]

    # Exact arithmetic. @arithmetic: traps on overflow; Err(OverflowError[Decimal]) inside checked:.
    a + b, a - b, a * b, -a
    a.abs() -> Decimal

    # Rounding is always explicit. Also @arithmetic.
    a.quantize(places: i64, mode: Rounding) -> Decimal
    a.div(divisor: &NonZero[Decimal], places: i64, mode: Rounding) -> Decimal
    a.mul_quantize(&b, places: i64, mode: Rounding) -> Decimal   # exact product, then quantize
    a.div_rem(divisor: &NonZero[Decimal], mode: Rounding) -> (Decimal, Decimal)

    # Comparison: __cmp__ gives == != < <= > >= numerically across scales.
    a.compare(&b) -> Order
    a.is_zero() -> bool               # ZeroTest
    a.normalize() -> Decimal          # strip trailing fractional zeros; exact

    # Inspection and conversion.
    a.sign() -> i64
    a.scale() -> i64                  # stored scale
    a.is_integer() -> bool
    a.parts() -> (u64, u64, i64, bool)
    a.to_i64(mode: Rounding) -> Result[i64, RangeError]
    a.to_f64() -> f64                 # nearest f64
    a.to_units(scale: i64) -> Result[(u64, u64, bool), RangeError]   # unscaled value at a fixed scale; fails if not exact

    # Text.
    a.display(out)                    # plain form with the stored scale

Conversions report RangeError.OutOfRange, as integer try_from does; there is no DecimalError. No arithmetic reports division by zero, since no operation accepts a zero divisor.

Example, in the proposed API:

from std.decimal import Decimal, Rounding
from std.fmt import FormatError

def unit_price(total: Decimal, count: i64) -> Result[Decimal, RangeError]:
    per = Decimal.nonzero_int(NonZero.try_from(count)?)
    Ok(total.div(&per, 2, Rounding.HalfEven))

def main() -> Result[(), IoError | ParseError | RangeError | FormatError]:
    price = Decimal.parse("19.99")?
    rate = Decimal.parse("0.0825")?
    subtotal = price * Decimal.from_int(3)                 # 59.97
    tax = (subtotal * rate).quantize(2, Rounding.HalfEven) # 4.947525 -> 4.95
    total = subtotal + tax
    print(f"{total}"?)?                                    # 64.92
    print(f"{unit_price(total, 7)?}"?)?                    # 9.27
    half = Decimal.parse("1.50")?
    print(f"{half}"?)?                                     # 1.50
    print(half == Decimal.of(15, 1))?                      # True
    Ok(())

# Errors as values instead of a trap, with the integer rule.
def grand_total(lines: &list[Decimal]) -> Result[Decimal, OverflowError[Decimal]]:
    checked:
        total = Decimal.ZERO
        for line in lines:
            total += line
        total

Interchange

  • Polars/Parquet (B76): enable Polars’ dtype-decimal feature in plenty-dataframe and measure the dataframe executable’s growth. A column Decimal(p, s) reads as Decimal values with scale s, since the scale is kept; writing to a column needs each value at scale s exactly (to_units(s) fails with RangeError otherwise), so the caller quantizes first. Column arithmetic stays in Polars, with its own half-even rounding to the column scale, and the reference must say so.
  • JSON: Event.Number already exposes number text (std.json); add a decimal reader and writer without going through f64.
  • CSV, SQLite: parse fields and bind as text. SQLite has no exact decimal storage class; binding through REAL would lose digits.

Licences, size and platforms

A Plenty implementation ships no third-party code. If it ports algorithms from Polars’ decimal.rs (MIT), credit Polars in the module header, the reference page’s acknowledgements and scripts/release/third_party_notices.py. Tests can use the GDA decTest vectors and CPython’s decimal as a differential oracle (configured to 38 digits with traps for Inexact), without shipping either. Code size reaches only programs that import the module. The implementation uses only integer operations, so it behaves identically on Linux x86-64 and ARM64, macOS ARM64 and Windows MSVC; C candidates would need per-target builds and MSVC configuration.

Staged plan

  1. Prerequisites.
    • u64.widening_mul(other) -> (u64, u64) (high, low), lowered to umulhi and imul. Check the generated code before timing.
    • @arithmetic methods and operator hooks, std only (gap), with tests on a small test-only class: trap position and caller note, TrapKind.Arithmetic, checked: error types, and the lexical rule.
    • __cmp__ from the ordering proposal, at least for std classes, so Decimal’s == is numeric.
    • The ZeroTest bound for NonZero, std classes only.
  2. Core module. Decimal and Rounding; parsing and display with the stored scale; exact + - *; quantize, div, mul_quantize, div_rem, normalize, conversions; allocate(parts) for splitting an amount into parts that sum exactly. Reference page, a tutorial lesson replacing f64 money arithmetic, differential tests against CPython (38 digits, traps for Inexact and Rounded), allocation tests proving no allocation.
  3. Language integration. Dictionary keys through __key__ once B74 covers classes; a format-spec hook so {total:.2f} quantizes (half-even, as Python’s format does) instead of truncating text.
  4. Interchange. Polars decimal columns (B76), JSON number text, CSV helpers.
  5. Only on evidence. A standalone arbitrary-precision decimal design, or a fallible libmpdec wrapper for the full GDA operation set; a decimal literal suffix (19.99d) if parse and of prove too noisy; @arithmetic and ZeroTest for user types.

Decisions

Decided in review (2026-10-31):

  1. Scale. Kept, as in Python. Equality and ordering are numeric across scales through __cmp__, and keys hash the normalized value through __key__.
  2. Digit limit. 38 digits.
  3. Rounding mode. Always explicit, including from_f64 and div_rem.
  4. Multiply primitive. u64.widening_mul, as libmpdec uses a double-word multiply on base-1019 words.
  5. Non-zero divisor. NonZero[Decimal], through a ZeroTest bound; std classes now, user types later.
  6. wrapped. Truncated to 38 fractional digits, coefficient modulo 1038.
  7. @arithmetic. std only for now.
  8. Overflow traps like integers and is Err(OverflowError[Decimal]) inside checked:; no checked_* methods, contexts or signals.
  9. Division is div(divisor, places, mode); no /.
  10. No coupling to other numeric proposals. Arbitrary precision is a separate later design.

Still open, none blocking stage 1:

  1. Rounding names. Direction names (recommended) or CPython’s ROUND_HALF_UP style.
  2. User types. The rule for opening @arithmetic, ZeroTest and __key__ to user code: each rests on a contract the compiler cannot check (non-zero-preserving *, __key__ agreeing with __cmp__).
  3. Word base. Base 1019 (recommended, as libmpdec) or binary words, settled by the prototype; not user-visible.

A numerics core for scientific work

Status: Not implemented. This page scopes a numerical library for Plenty and recommends what to build first. It ships no module or API. It replaces issues #492 (optimization) and #493 (evaluating numeris). Scheduling belongs to B38. Shipped neighbours are std.linalg, std.stat, std.math, std.complex, std.random, std.dataframe and std.plot. Companion pages sketch the API and record the crate evaluation.

Goal

A graduate student or researcher in a STEM field should be able to load data, fit a model, solve an equation, integrate a function or an ODE, and plot the result using only std. In Python this needs NumPy and SciPy. Plenty should cover the parts of them that most research code uses, and stop there. Most of SciPy is not needed to reach that goal.

Recommendation

  1. One family, std.numeric, in Plenty. New modules live under std.numeric: optimize, integrate, differentiate, interpolate, special and fft, plus array helpers in std.numeric itself. The algorithms are written in Plenty. Dense linear algebra inside them calls std.linalg, which keeps faer.
  2. Adopt neither basin nor numeris as a dependency. Use them, with SciPy, QUADPACK, MINPACK and Cephes, as algorithm and test-vector references. The crate evaluation gives the reasons: objective functions are Plenty callbacks, neither crate reports allocation failure, and both are months old with frequent breaking releases.
  3. Keep the existing modules where they are. std.linalg and std.stat gain what the new modules need (least squares, more distributions); they are not moved under std.numeric. A reference index page lists the whole numerics core.
  4. Exchange data as list[f64] and la.Matrix. These are what std.dataframe returns from values_f64, what std.plot takes in Column.Num, and what std.linalg already uses. No new array type.
  5. Deliver in three milestones (below), each useful on its own, and add further scope only for a stated use case.

How large is SciPy, and what does Plenty take?

SciPy is a collection of subpackages, each a library in its own right, many wrapping decades of Fortran and C. Matching it would be a multi-year project and is not the goal. The table decides each subpackage. Core is in the milestones below. Later needs a use case and its own scope before work starts. Out is not planned.

SciPy / NumPy areaDecisionPlenty scope
numpy.ndarray, broadcastingOut for nowlist[f64], la.Matrix, iterators and std.simd. An N-dimensional array needs indexing overloads, views and a broadcasting design; it is a separate proposal if ever.
NumPy helpersCorelinspace, logspace, cumsum, diff, dot, norm, accurate fsum, polynomial polyval/polyfit/roots
numpy.randomShippedstd.random; new distributions sample through std.stat
scipy.optimizeCore subsetScalar roots and minimization; Nelder–Mead, BFGS, L-BFGS-B; nonlinear least squares and curve_fit; root finding for systems; dense linear programming
LaterDifferential evolution, SLSQP or other constrained NLP, mixed-integer LP, basin hopping
scipy.integrateCore subsetAdaptive Gauss–Kronrod quad, including infinite ranges; trapezoid/Simpson over samples; solve_ivp with Dormand–Prince 5(4) and one stiff method
LaterEvents, dblquad/nquad, boundary-value problems
scipy.differentiateCoreCentral differences with Richardson extrapolation; gradient, Jacobian and Hessian by finite differences
scipy.interpolateCore subsetLinear, cubic spline (natural, clamped, not-a-knot), PCHIP
LaterSmoothing splines, 2-D regular grids, B-spline objects
OutScattered-data RBF and triangulation
scipy.specialCore subsetGamma, log-gamma, digamma, beta, regularized incomplete gamma and beta with inverses, erf/erfc/erfinv
LaterBessel functions, elliptic integrals, orthogonal-polynomial evaluation
OutHypergeometric and the long tail
scipy.fftCore subsetComplex FFT of any length (mixed radix plus Bluestein), rfft/irfft, fftfreq
Later2-D FFT, DCT
scipy.linalgShipped, extendstd.linalg plus lstsq and det, which least squares and fitting need
LaterMatrix functions (expm), condition estimates, pseudoinverse
scipy.statsShipped, extendstd.stat gains Student’s t, χ², F, gamma, beta, binomial and Poisson once special exists
LaterHypothesis tests, linear regression summaries, kernel density estimation
scipy.signalLaterConvolution, filtering, peak finding, Welch spectra (std.signal already names Unix signals)
scipy.spatialLaterk-d tree and distance functions
scipy.sparse, sparse.linalgOutA sparse format and iterative solvers need their own proposal
ndimage, io, odr, cluster, datasets, constantsOutstd.image covers images; physical constants may be a small later table

The core is a few dozen algorithms, each documented in the literature with a reference implementation and published test problems. That is the size this proposal commits to.

Why std.numeric rather than top-level modules

std.linalg and std.stat are top-level, so std.optimize would match them. Top-level names were rejected:

  • Several SciPy names are already taken or ambiguous at the top level: std.signal is Unix signal handling, and std.special says nothing about numerics.
  • One prefix makes the family easy to find in the signature index and keeps shared types (tolerances, result records) in one parent module.
  • std already has this shape: std.cache.lru and std.sketch.xor sit under parent modules.

Moving std.linalg and std.stat would break shipped programs for no gain; they stay, and the reference index links them.

Why Plenty and not a wrapped crate

The standard library policy prefers Plenty unless a native library is clearly better. Here Plenty is better for three reasons:

  • The user’s function is the inner loop. Root finding, minimization, quadrature and ODE solving call a Plenty function thousands of times. A callback bound by F: Callable[...] is specialized at compile time and can be inlined. A native solver would call back across the engine’s C boundary on every evaluation and would have to carry the callback’s error out again.
  • Errors propagate unchanged. A probe with the release compiler built from main on 2026-11-02 compiled bisect[E, F: Callable[[f64], Result[f64, E]]](...) -> Result[f64, RootError | E] with a closure returning math.MathError, and a finite-difference gradient taking Callable[[&list[f64]], Result[f64, E]] and writing into &mut list[f64]. Both ran correctly. A failing objective, such as log of a negative number, leaves the solver with its own error, not a sentinel NaN.
  • Allocation failure. Plenty code reports AllocError by construction. Neither candidate crate does (evaluation).

The dense kernels that justify native code (GEMM, LU, QR, SVD, eigenvalues) are already native in std.linalg. Solvers needing them (Newton steps, Levenberg–Marquardt, trust regions, interior-point LP) call std.linalg. An FFT written in Plenty may be slower than a SIMD library. If measurement shows that, the bigint pattern applies: an allocation-free native kernel writing into caller-allocated buffers, added only when it is measured to pay.

Shared conventions

The API sketch has signatures. The rules:

  • f64 only. As in std.linalg and std.stat. f32 and generic float code wait until there is demand and a numeric protocol to bound them.
  • Fallible callbacks. Objectives return Result[f64, E], and solvers return Result[T, NumericError | E]. One form, not paired fallible and infallible variants; an infallible objective writes Ok(...).
  • No allocation per evaluation. Vector objectives borrow &list[f64] and write gradients into &mut list[f64] and Jacobians into &mut la.Matrix supplied by the solver. Workspace is allocated once per solve.
  • Non-convergence is an error that keeps the result. Reaching an iteration or evaluation limit returns Err(NumericError.Unconverged(report)). The report holds the last iterate, function value and counts. SciPy returns success=False in an ordinary result, which is easy to ignore. In Plenty the caller has to handle it, and still gets the last iterate.
  • Options are keyword arguments with documented defaults: xtol, rtol, atol, max_iterations, max_evaluations. No options dictionaries and no method-name strings: each method is its own function, so a misspelt method fails to compile.
  • Explicit randomness. Stochastic methods take &mut random.Rng from std.random; there is no global seed.
  • Nonfinite values. Inputs are checked; a NaN or infinity from a callback is NumericError.NonFinite, reported at the evaluation that produced it.
  • Documented accuracy. Each function’s reference page states its method, tolerance semantics and measured accuracy over stated input ranges.

Working with linalg, dataframes and plots

The exchange types already connect the three shipped modules:

FromToToday
std.dataframenumericsframe.values_f64("x") returns list[f64]
numericsstd.dataframedf.column_f64("fit", &values)
numericsstd.plotColumn.Num(values) takes the list without copying
numericsstd.linalgla.vector(&values) copies; matrix.to_list() copies back

The last row is the weak one. An optimizer converting between a list and a Matrix every iteration would copy every time. Milestone 2 therefore adds consuming conversions to std.linalg, such as la.vector_from(values), which takes ownership of a list without copying, and matrix.into_list(). Solvers can then keep state in lists and call linalg without copying.

A complete fit, sketched in the API page: read a CSV into a dataframe, take two columns as lists, fit a model with curve_fit, evaluate it on a linspace grid, and plot data and fit with geom_point and geom_line. Every step uses shipped or core APIs; the tutorial lesson for milestone 2 is this program.

Milestones

Each milestone ships reference pages, a tutorial lesson and its tests.

  1. Scalar problems and special functions. The std.numeric helpers; special (gamma family, incomplete gamma/beta, erf family); scalar root finding (Brent, Chandrupatla, Newton, secant); bounded scalar minimization (Brent); quad; derivative. std.stat gains t, χ², F, gamma, beta, binomial and Poisson distributions, which need incomplete gamma and beta. This also covers B38’s special functions and accurate summation.
  2. Fitting and multivariate problems. Nelder–Mead, BFGS and L-BFGS-B; least_squares (Levenberg–Marquardt and trust-region reflective for bounds) and curve_fit; systems of equations (Powell hybrid); finite-difference gradients, Jacobians and Hessians; interpolate; la.lstsq, la.det and the consuming conversions.
  3. ODEs, spectra and linear programming. solve_ivp with Dormand–Prince 5(4) and dense output, plus one stiff method (Radau IIA or a Rosenbrock method; chosen by measurement on the Robertson and Van der Pol problems); fft; dense linprog (bounded revised simplex first; an interior-point method if needed for larger problems).

After milestone 3, the Later rows above are candidates. Each needs a use case and its own scope under B38.

Testing and acceptance

  • Reference values. Expected values come from SciPy and mpmath, generated offline by a checked-in script and stored as fixtures. Tests do not need Python.
  • Standard problems. Moré–Garbow–Hillstrom functions (Rosenbrock, Powell singular, Brown badly scaled) for optimization; NIST StRD nonlinear regression datasets with certified values for least squares; QUADPACK’s test integrals, including endpoint singularities, for quad; Hairer’s nonstiff and stiff problems for solve_ivp; Netlib LP problems small enough to keep in the repository for linprog.
  • Accuracy. Special functions report maximum relative error in ULPs over stated ranges. A change that increases it fails the test.
  • Allocation and errors. Allocation-denial tests cover every solver’s workspace. Each solver has a test where the callback fails, showing that its error reaches the caller.
  • Cost. Each solver reports evaluation counts on the standard problems. Counts are compared with SciPy’s. Wall-clock comparisons are optional. They follow the profiling guide and do not promise a speedup.

Risks and open questions

  • Scope creep. Requests will arrive for SciPy features not listed here. The Later and Out rows are the answer until a use case is written down.
  • Numerical maintenance. Special functions and stiff ODE solvers are hard to get right. Ports keep the source algorithm’s structure and credit, so upstream fixes can be followed. SciPy and Cephes are BSD-3 and QUADPACK is public domain, so ports are allowed with attribution.
  • Fixed-size problems. Small, fixed-dimension problems (3-vectors, quaternions, Kalman filters) would avoid heap allocation with value generics (B83). The list-based API does not preclude adding fixed-size variants later.
  • FFT speed. A Plenty FFT may need std.simd or a native kernel to be competitive. Measure first.
  • Sparse problems. Large finite-element and network problems need sparse matrices, which this proposal leaves out. Researchers with those problems will still need another library.

Numerics API sketch

Companion to the numerics core proposal. Every signature here is hypothetical and may change before implementation. Fences are text because nothing on this page compiles today. Shipped calls, such as scan_csv, values_f64 and gg.table, use their current signatures.

Shared types (std.numeric)

enum NumericError:
    Domain            # invalid argument: empty bracket, negative tolerance
    NoBracket         # f(a) and f(b) have the same sign
    NonFinite         # an input or callback value was NaN or infinite
    Unconverged(Report)
    Linalg(la.LinalgError)
    Allocation(AllocError)

class Report:
    x: list[f64]          # last iterate; one element for scalar problems
    fun: f64              # objective value or residual norm at x
    iterations: i64
    evaluations: i64

A successful solve returns a result type for its problem, such as Root, Minimum, Fit, Integral or Solution. Each has the fields of Report plus fields specific to the problem, such as the error estimate for Integral and the covariance for Fit.

Array helpers:

def linspace(start: f64, stop: f64, n: Positive[i64]) -> Result[list[f64], AllocError]
def logspace(start: f64, stop: f64, n: Positive[i64], base: f64 = 10.0) -> Result[list[f64], AllocError]
def cumsum(values: &list[f64]) -> Result[list[f64], AllocError]
def diff(values: &list[f64]) -> Result[list[f64], AllocError]
def dot(a: &list[f64], b: &list[f64]) -> Result[f64, NumericError]
def norm(values: &list[f64]) -> f64              # scaled, no spurious overflow
def fsum(values: &list[f64]) -> f64              # correctly rounded sum
def polyval(coefficients: &list[f64], x: f64) -> f64
def polyfit(x: &list[f64], y: &list[f64], degree: i64) -> Result[list[f64], NumericError]
def roots(coefficients: &list[f64]) -> Result[(list[f64], list[f64]), NumericError]   # real, imaginary parts

roots uses companion-matrix eigenvalues from la.eig, so it needs the linalg engine; polyfit uses la.lstsq.

Scalar problems (milestone 1)

import std.numeric.optimize as opt
import std.numeric.integrate as integ
import std.numeric.differentiate as d
import std.math as math
from std.math import MathError

def brentq[E, F: Callable[[f64], Result[f64, E]]](
    f: &F, a: f64, b: f64, xtol: f64 = 2e-12, rtol: f64 = 8.9e-16,
    max_iterations: i64 = 100,
) -> Result[opt.Root, NumericError | E]

def newton[E, F: Callable[[f64], Result[(f64, f64), E]]](
    f_and_derivative: &F, x0: f64, ...
) -> Result[opt.Root, NumericError | E]

def minimize_scalar[E, F: Callable[[f64], Result[f64, E]]](
    f: &F, a: f64, b: f64, xtol: f64 = 1.48e-8, max_iterations: i64 = 500,
) -> Result[opt.Minimum, NumericError | E]

def quad[E, F: Callable[[f64], Result[f64, E]]](
    f: &F, a: f64, b: f64, atol: f64 = 1.49e-8, rtol: f64 = 1.49e-8,
    max_subintervals: i64 = 50,
) -> Result[integ.Integral, NumericError | E]

def derivative[E, F: Callable[[f64], Result[f64, E]]](
    f: &F, x: f64, order: i64 = 1, ...
) -> Result[d.Derivative, NumericError | E]

quad accepts infinite limits (f64.inf, -f64.inf) and maps them to a finite interval, as QUADPACK’s QAGI does. Its result includes an error estimate and the number of subintervals used.

def main() -> Result[(), AllocError | IoError | NumericError | MathError]:
    # Root of cos(x) = x
    root = opt.brentq(&(def(x: f64) -> Result[f64, MathError]: Ok(math.cos(x)? - x)), 0.0, 1.0)?
    print(root.x[0])?
    # Gaussian integral over the real line
    g = integ.quad(&(def(x: f64) -> Result[f64, MathError]: math.exp(-x * x)), -f64.inf, f64.inf)?
    print(g.value, g.error)?
    Ok(())

Multivariate problems (milestone 2)

def minimize_bfgs[E, F: Callable[[&list[f64], &mut list[f64]], Result[f64, E]]](
    f_and_gradient: &F, x0: &list[f64], gtol: f64 = 1e-5, max_iterations: i64 = 0,
) -> Result[opt.Minimum, NumericError | E]

def minimize_lbfgsb[E, F: Callable[[&list[f64], &mut list[f64]], Result[f64, E]]](
    f_and_gradient: &F, x0: &list[f64], lower: &list[f64], upper: &list[f64], ...
) -> Result[opt.Minimum, NumericError | E]

def minimize_nelder_mead[E, F: Callable[[&list[f64]], Result[f64, E]]](
    f: &F, x0: &list[f64], ...
) -> Result[opt.Minimum, NumericError | E]

def least_squares[E, F: Callable[[&list[f64], &mut list[f64]], Result[(), E]]](
    residuals: &F, x0: &list[f64], m: i64, ...
) -> Result[opt.Fit, NumericError | E]

def curve_fit[E, F: Callable[[f64, &list[f64]], Result[f64, E]]](
    model: &F, x: &list[f64], y: &list[f64], p0: &list[f64], ...
) -> Result[opt.Fit, NumericError | E]

def root[E, F: Callable[[&list[f64], &mut list[f64]], Result[(), E]]](
    f: &F, x0: &list[f64], ...
) -> Result[opt.Root, NumericError | E]

The solver owns the gradient and residual buffers and passes them as &mut list[f64], so an evaluation allocates nothing. If no gradient is available, d.gradient(&f, &x, &mut g) gives a finite-difference one. Wrap that in a closure to pass it as f_and_gradient. Separate gradient-free variants of every solver are not provided.

Fit adds covariance: la.Matrix, computed from the Jacobian at the solution as curve_fit does, and residual_norm.

Interpolation constructs an object once and evaluates it repeatedly:

import std.numeric.interpolate as ip

spline = ip.cubic_spline(&x, &y, boundary=ip.Boundary.NotAKnot)?
value = spline.at(2.5)?
values = spline.at_all(&grid)?          # list[f64]
slope = spline.derivative(2.5, 1)?

ODEs, spectra and linear programming (milestone 3)

def solve_ivp[E, F: Callable[[f64, &list[f64], &mut list[f64]], Result[(), E]]](
    rhs: &F, t0: f64, t1: f64, y0: &list[f64],
    method: integ.Method = integ.Method.DormandPrince54,
    rtol: f64 = 1e-3, atol: f64 = 1e-6, t_eval: Option[list[f64]] = Nothing,
) -> Result[integ.Solution, NumericError | E]

Method is an enum, not a string. A stiff method also takes a Jacobian callback; without one, it uses a finite-difference Jacobian. Solution holds t: list[f64] and y: list[list[f64]], one list per component, so each component can go straight into a plot Column.Num.

import std.numeric.fft as fft

spectrum = fft.rfft(&samples)?            # list[Complex]
frequencies = fft.rfftfreq(len(samples), 1.0 / rate)?
back = fft.irfft(&spectrum, len(samples))?

FFT outputs are list[Complex] from std.complex, a @copy class, so a list stores the values inline.

solution = opt.linprog(&c, a_ub=&a, b_ub=&b, lower=&lo, upper=&hi)?

linprog minimizes cᵀx subject to A_ub x ≤ b_ub, A_eq x = b_eq and bounds. The constraint matrices are la.Matrix. An infeasible or unbounded problem is an error variant, not a status code.

An end-to-end fit

This program shows the milestone 2 workflow: load data, fit an exponential decay, and plot data and fit. scan_csv, values_f64, gg.table and Column.Num exist today. The std.numeric calls are proposed.

import std.dataframe as df
import std.math as math
import std.numeric as num
import std.numeric.optimize as opt
import std.plot as gg
from std.math import MathError
from std.plot import Column, PlotError

def decay(t: f64, p: &list[f64]) -> Result[f64, MathError]:
    Ok(p[0] * math.exp(-p[1] * t)? + p[2])

def main() -> Result[(), AllocError | IoError | df.DataFrameError | NumericError | MathError | PlotError]:
    frame = df.scan_csv("decay.csv")?.collect()?
    t = frame.values_f64("t")?
    y = frame.values_f64("signal")?
    fit = opt.curve_fit(&decay, &t, &y, &[1.0, 0.1, 0.0]?)?
    print(fit.x, fit.residual_norm)?

    grid = num.linspace(t[0], t[len(t) - 1], 200)?
    mut curve = list[f64].with_capacity(len(grid))?
    for s in grid.iter():
        curve.append(decay(s, &fit.x)?)?

    points = gg.table([("t", Column.Num(t)), ("signal", Column.Num(y))]?)?
    line = gg.table([("t", Column.Num(grid)), ("signal", Column.Num(curve))]?)?
    chart = gg.ggplot(points, gg.aes("t", "signal"), [
        gg.geom_point(),
        gg.geom_line(data=line, colour="firebrick"),
    ]?)?
    chart.save("decay.svg", width=6.0, height=4.0)?
    Ok(())

The data passes from the dataframe to the solver to the plot as list[f64]. The only copies are values_f64 out of Polars storage and the model’s own evaluation.

Numerics crate evaluation

Companion to the numerics core proposal. It answers issue #493, which asked whether numeris should back linear algebra, statistics or optimization, and the native-library half of issue #492, which proposed optimization through basin. Facts are from the crates’ docs.rs pages read on 2026-11-03. Neither crate was built or measured.

Decision

Neither crate is a dependency. std.linalg keeps faer. The numerics modules are written in Plenty and may use either crate as a reference for algorithms and tests, with attribution.

The candidates

basin 3.2.0numeris 0.6.0faer 0.24.4 (shipped)
ScopeOptimization only: first-order, derivative-free, least squares, global/stochastic, constrained; Brent root findingBroad and SciPy-like: fixed and dynamic matrices, linalg, ODE, optim, quadrature, interpolation, FFT, special functions, statistics, control, estimation, image processingDense linear algebra
LicenseMIT OR Apache-2.0; PRIMA-derived solvers and the L-BFGS-B port are BSD-3MITMIT
First release0.0.1 on 2026-04-300.1.0 on 2026-02-28Released since 2021
ChurnAbout 40 versions; majors 1.0 (June), 2.0 and 3.0 (19–20 October), 3.2 on 2026-11-02About 37 versions, pre-1.0, single maintainerPinned through a fork
Required dependenciesnum-traits, rand, rand_chacha, rand_distr, web-timenum-traitsBuilt only into the separate linalg engine
Linear algebra backendGeneric over Vec<f64>, nalgebra, ndarray or faerIts own kernels; optional nalgebra conversion—
Fallible allocationNot documentedNot documented; dynamic types use Vec behind allocLow-level APIs take caller scratch; Plenty’s fork makes GEMM workspaces fallible

Reasons

Callbacks. An optimizer or ODE solver spends its time calling the user’s function. With a wrapped crate, every evaluation would cross from Rust into Plenty through the engine’s C interface. The callback’s environment and its typed error would have to cross as well, and the error would have to be carried back out through the crate’s own return path, which has no slot for it. A Plenty solver specializes the callback at compile time and propagates its error with ?. The proposal records a probe of this on the current compiler.

Allocation failure. The wrapper rules require a wrapped crate to report allocation failure as AllocError. Neither crate documents fallible allocation, and both allocate through ordinary Vec, which aborts on failure. faer needed a fork to meet this rule for GEMM alone. Repeating that for a second, broader crate would mean maintaining a second fork.

Maturity. Both crates are under a year old. basin made three major releases in a month. numeris has one maintainer and is pre-1.0. A standard library promises stability that its dependencies need to share. Pinning would work around churn but not supply fixes.

Replacing faer. numeris reports faster small fixed-size decompositions than faer on Apple Silicon. That is its stack-allocated Matrix<T, M, N>, which needs const generics. Plenty’s matrices are dynamically sized, and std.linalg’s faer integration already passes allocation-denial, concurrency and non-AVX tests. There is no measured reason to switch.

What each crate is still useful for

  • basin: the choice and structure of solvers (L-BFGS-B, trust-region reflective least squares, the PRIMA derivative-free family) and its test problems. Its separation of problem, solver, termination and driver is a reasonable model for keeping Plenty’s solvers small.
  • numeris: compact reference implementations for adaptive Runge–Kutta pairs, RODAS4, Bluestein FFT, natural splines and the special functions in milestone 1, with tests that can be checked against SciPy and mpmath.

Ported code keeps its license notice. BSD-3 portions (PRIMA, L-BFGS-B, SciPy, Cephes) need attribution in the source and the release notices.

When to revisit

Reopen the choice for a specific algorithm, not a whole crate, if:

  • a Plenty implementation is measured to be too slow and the work can be done without callbacks into Plenty, as in an FFT kernel writing into a caller-allocated buffer; and
  • the native code allocates nothing or reports allocation failure, and adds its own runtime entry symbol under the wrapper rules.

Units of measure

Status: Partly implemented. This page proposes compile-time dimensional analysis for Plenty: quantities whose physical dimension is part of the type, checked by the compiler and erased before code generation. Milestone 1, the language, shipped through issue #701 (local PRs #885, #887–#891); see dimensions, quantities and where the implementation differs. Milestone 2’s std.units shipped, including wrap_list and strip_list. Milestone 3’s shared instance bodies shipped; plot axis labels remain under B97. Companion pages record the precedents and the spike evidence. Researched against local main at bd407f05 on 2026-11-03; reconciled with eb772202.

Goal

A program that adds a length to a time, or passes an acceleration where a velocity is expected, should fail to compile. A program that passes these checks should run exactly as fast as the same program written with f64, and allocate nothing more.

This example needs std.units (milestone 2). Quantity is built in and needs no import.

import std.units as u
from std.units import Length, Time, Velocity

def speed(d: Quantity[f64, Length], t: Quantity[f64, Time]) -> Quantity[f64, Velocity]:
    d / t

def main() -> Result[(), IoError]:
    d = 3.2 * u.km
    t = 41.0 * u.minute
    v = speed(d, t)
    print(v, v.to(u.km / u.hour))?      # 1.3008130081300813 m/s 4.682926829268292
    print(d + t)?                       # compile-time error: Length + Time
    Ok(())

Recommendation

  1. Dimensions are a compiler-known kind of type. A module declares base dimensions (dimension Length("m")) and names derived ones (dimension Velocity = Length / Time). The compiler normalizes every dimension expression to a product of base dimensions with integer exponents, so Length / Time * Time is Length.
  2. Quantity[T, D] is a built-in numeric wrapper, like NonZero[T]. It has T’s representation. The compiler defines its operators: +, - and comparisons need equal dimensions; * and / combine them.
  3. The type carries the dimension, not the unit. A value is stored in the coherent SI unit of its dimension (metres, seconds, kilograms). km, minute and pound are constants, and q.to(unit) reads a value back in any unit. This is the uom and GNAT model.
  4. Storage types compose. T may be any number type, including Positive[i64], NonNegative[i64] and NonZero[i64]. An operation’s storage result is whatever T’s own operator returns; its dimension comes from the dimension algebra. Units never drop out of a result.
  5. std.units declares SI in Plenty: the seven base dimensions, named derived dimensions, prefixed and common units, CODATA constants, and dimension-aware sqrt, hypot and friends.

The recommendation needs no value generics, no type-level integers and no generic operator hooks, which Plenty lacks; see what the language could do before.

What the language could do before

The spikes ran on the release compiler built from main on 2026-11-02, before issue #701. Their library workarounds still behave as shown; the built-in Quantity[T, D] now covers the rows marked No and Partly.

NeedTodayEvidence
Distinct types per dimensionPhantom type parameters: Q[f64, Length] and Q[f64, Time] are different typesSpike 1
Same-dimension +, -, <@newtype forwarding over a phantom-tagged class; mixing dimensions is rejectedSpikes 1, 2
Code generic over one dimensiondef total[T, D](xs: &list[Q[T, D]]) -> Q[T, D]Spike 7
Zero costOne-field classes are represented as their field; operator hooks on a one-f64 class compile to the same instructions as a.x + b.x (operator lowering)Reference
Length * Length -> Area, genericallyNo. Operator hooks cannot have type parameters, a class has one __mul__ signature, and there are no associated types or type-level arithmeticSpike 4
A domain inside a quantityPartly. @newtype forwards Positive + Positive as Positive, but Positive - Positive returns a bare i64: the unit is lostSpike 3
Positive[f64]No. Integer domains take only signed integersSpike 5

So a phantom-tag library can check additive code today, but multiplication and division, the core of dimensional analysis, need a language change. Hand-written classes per dimension pair (spike 6) work for a handful of quantities and do not scale: each class can declare only one __mul__.

Options

OptionBenefitCost
A. Phantom-tag library with @newtype (works today)No language changeAdditive checks only; products need a hand-written function per pair
B. std library over value generics: Quantity[T, const L: i8, const M: i8, const T: i8, ...]Reuses B83Needs B83, its deferred derived-size expressions (L + L2), generic operator hooks with value parameters, and a second __mul__ for scalars. Seven fixed base dimensions, no user dimensions. Types print as Quantity[f64, 1, 0, -1, 0, 0, 0, 0]; a generic function over one dimension takes seven parameters
C. Compiler-known dimensions and built-in Quantity (recommended)Readable types and diagnostics; user dimensions; one parameter per generic dimension; no new operator-hook machineryA new kind of type and dimension expressions in the checker
D. C plus F#-style inference through dimension expressions in parameter typesdef sqrt[D](x: Quantity[T, D * D]) -> Quantity[T, D] infers DUnification over abelian groups in the checker; later, if code needs it
E. Runtime units (Pint, astropy, Measurement in Swift)No type-system workRun-time checks and storage per value; errors appear only on executed paths
F. Units in the type (mp-units, Unitful): km and m are distinct typesExact integer storage in any unitConversion and promotion rules between units of one dimension; many more types

Rust is a useful warning about option B. uom encodes exponents with typenum type-level integers because stable const generics cannot compute L1 + L2 in a signature; that needs the unstable generic_const_exprs. The const-generic ports (tiny-uom, const_unit_poc) require nightly features and stopped in 2020. Option B in Plenty would hit the same wall at the derived sizes that B83 deliberately defers.

C++ took the library route too: Boost.Units with template metaprogramming, then mp-units with C++20 class-type template parameters, now proposed for C++29 as P3045. Both are large libraries and rely on language features Plenty does not have and should not add for this. Ada GNAT and F# build dimensions into the compiler, as option C does. See the precedents.

Design

Declaring dimensions

dimension Length("m")
dimension Mass("kg")
dimension Time("s")
dimension Velocity = Length / Time
dimension Force = Mass * Length / Pow[Time, 2]
  • A base dimension has a name and the symbol of its coherent unit. Its identity is nominal, like a class: pkg.Length and std.units.Length differ.
  • A derived dimension is a transparent alias for a dimension expression, like a type alias. Velocity and Length / Time are the same type.
  • One is the built-in dimensionless dimension, the empty product.
  • dimension declarations are module-level and follow ordinary visibility (pub dimension Length("m")). dimension becomes a keyword; it is a declaration form, so this is the one source change existing programs could meet.

A dimension is a type with no values. It may appear only as a dimension argument, so Quantity[f64, Length] is valid and x: Length is not.

Dimension expressions

Wherever a dimension is expected, the compiler accepts:

FormMeaning
Length, Velocity, DA base dimension, alias or dimension parameter
A * B, A / BProduct and quotient, left-associative, with parentheses
Pow[A, n]Integer power; n is an integer literal, possibly negative
Root[A, n]nth root; every exponent of A must be divisible by n; n is nonzero, and a negative n takes the reciprocal
OneDimensionless

The normal form is a list of (base dimension, nonzero i8 exponent) pairs sorted by base identity. Two dimensions are equal exactly when their normal forms are equal. An exponent outside i8 is a compile-time error, including in an intermediate result that a later factor would cancel.

Exponents are integers. Root rejects a dimension it cannot divide exactly: the square root of Pow[Length, 3] is an error, not Length^(3/2). This excludes units such as V/√Hz from noise spectral density. F# added rational exponents in its fourth version for these cases; the normal form could store rationals later without changing any spelling.

Generic dimensions

D: Dimension declares a dimension parameter. Dimension is a built-in constraint, like IntType; it admits exactly the dimension types.

def mean[T: FloatType, D: Dimension](xs: &list[Quantity[T, D]]) -> Quantity[T, D]
def kinetic[M: Dimension, V: Dimension](m: Quantity[f64, M], v: Quantity[f64, V]) -> Quantity[f64, M * V * V]
def sqrt[T: FloatType, D: Dimension](x: Quantity[T, D]) -> Result[Quantity[T, Root[D, 2]], MathError]

One rule keeps inference structural, as it is today: in parameter types, a dimension argument is either concrete or a bare parameter. Return types, annotations and bodies may use any expression. Since Plenty checks bodies per instance, M * V * V and Root[D, 2] are evaluated after substitution, where every operand is known. This is the form B83 calls “expressions in non-inferred output positions”, applied to dimensions instead of integers.

A Root that does not divide is reported as an instance error at the call that required it, through the existing instance diagnostics:

main.plenty:8:9: cannot instantiate `units.sqrt[f64, Pow[Length, 3]]`: `Root[Pow[Length, 3], 2]` is not a whole dimension
  note: `Length` has exponent 3, which 2 does not divide

Option D, inferring D from x: Quantity[T, D * D], stays out until code needs it. Option C programs remain valid under it.

Quantities

Quantity[T, D] holds one T measured in the coherent unit of D. T is a number type: an integer, a float, or an integer domain type. D is a dimension.

OperationResult
Quantity[f64, Length](3.0)3 m. The constructor takes the coherent-unit value; prefer 3.0 * u.metre in application code.
q.get()The stored T, in coherent units
q.to(unit)(q / unit).get(), for float storage: unit is a quantity of the same dimension, such as u.km. Mismatched dimensions are a compile-time error.
a + b, a - bBoth Quantity[T, D]; result Quantity[R, D], where R is the type T + T or T - T returns
a * b, a / bQuantity[R, A * B] and Quantity[R, A / B]; / needs float storage
k * q, q * k, q / kA scalar T scales without changing the dimension
k / qQuantity[R, One / D]
a // b, a % bInteger storage: // divides dimensions, % keeps a’s and needs equal dimensions
-q, +qAs for T, dimension unchanged
<, <=, ==, …Equal dimensions; bool
+=, -=Equal dimensions. *= and /= take a scalar T only, since the dimension of a variable cannot change.
print(q), f"{q}"The value, then the coherent unit: 1.3 m/s, 9.81 m/s^2. A format spec applies to the number: f"{q:.2f}" is 1.30 m/s.
debugQuantity[f64, Length * Pow[Time, -1]](1.3), the normal form

Rules with no exceptions:

  • Scalars only scale. A plain T, including a literal, may multiply or divide a quantity, never be added to it or compared with it. q > 0.0 is an error; write q > 0.0 * u.metre. F#’s 0.0<_> wildcard is the exception this rule avoids.
  • Dimensionless is still a quantity. d / d is Quantity[f64, One]. .get() returns the f64. There is no implicit conversion.
  • Storage types never mix. Quantity[f32, D] + Quantity[f64, D] is an error, as f32 + f64 is.

Overflow, division by zero and checked: behave as they do for T. A quantity operation is T’s operation, so a checked: block reports OverflowError[T] or ZeroDivisionError.

Unit constants and conversions

A module constant may have a quantity type, with an initializer built from literals and other quantity constants:

pub const metre: Quantity[f64, Length] = Quantity[f64, Length](1.0)
pub const km: Quantity[f64, Length] = 1000.0 * metre
pub const hour: Quantity[f64, Time] = 3600.0 * second
pub const newton: Quantity[f64, Force] = kg * metre / (second * second)

Construction multiplies by the unit, and to divides by it. Both are one floating-point operation, folded at compile time when the operands are constants. Converting through the coherent unit can round: 0.1 * u.foot then .to(u.foot) need not return exactly 0.1. Programs that need exact values in a non-coherent unit should declare their own dimension or keep that unit’s integer count, as std.time.Duration keeps nanoseconds.

Unit constants are f64. f32 quantities convert with Quantity[f32, D].from(q), which narrows the stored value as f32(x) does; Quantity[f64, D].from(q) widens.

Domain storage

The user-facing question was whether Positive, NonNegative and NonZero can carry units. They can, by nesting the domain inside the quantity and taking the storage result from the domain’s own rules:

Expression, with p: Quantity[Positive[i64], Length]Result
p + p, p * pQuantity[Positive[i64], Length], Quantity[Positive[i64], Pow[Length, 2]]
p - p, -pQuantity[i64, Length]: the domain drops, the unit stays
n // p, with n: Quantity[i64, Area]Quantity[i64, Length], without the zero or MIN // -1 check
Quantity[Positive[i64], Length](0)Compile-time error, as for Positive[i64]
Quantity[Positive[i64], Length].try_from(k)Result[..., RangeError] for a computed k
Positive[Quantity[i64, Length]]Rejected: one nesting order, the domain inside

Today’s @newtype forwarding does the opposite for -: it returns the bare i64 (spike 3). The built-in rule rewraps every result.

Integer storage counts whole coherent units, so Quantity[i64, Length] holds whole metres. That suits counts such as pixels, bytes or items, declared as user dimensions, more than SI quantities.

Float domains do not exist. Mass, absolute temperature and the argument of a logarithm would benefit from a Positive[f64]. Its rules differ from the integer domain: it must exclude NaN, and * cannot preserve it because a product of two small positive values can underflow to zero. That belongs in a separate proposal. The storage rule above means a later float domain works inside quantities with no further units design.

std.units

Written in Plenty, as declarations and small functions:

PartContents
Base dimensionsLength (m), Mass (kg), Time (s), Current (A), Temperature (K), Amount (mol), Luminosity (cd)
Derived dimensionsArea, Volume, Velocity, Acceleration, Force, Pressure, Energy, Power, Charge, Voltage, Resistance, Frequency, Density, and the other named SI derived quantities
UnitsSI prefixes on the base and named derived units (km, ms, kN, MPa, kWh), plus minute, hour, day, litre, tonne, bar, atm, eV, and the international foot, inch, mile, pound and pound-force
ConstantsThe exact SI defining constants (c, h, e, k_B, N_A) and CODATA values such as G, typed
Functionssqrt, cbrt, abs, hypot, min, max, clamp, mean, sum; exp, log and trigonometry take Quantity[f64, One]
Temperaturefrom_celsius, to_celsius, from_fahrenheit, to_fahrenheit (affine; see out of scope)
Timefrom_duration(&Duration) -> Quantity[f64, Time] and a fallible to_duration
Listswrap_list[D](xs: list[T]) -> list[Quantity[T, D]] and strip_list, consuming the list without copying, since the element layouts are identical

Angles are dimensionless, as in SI: u.degree is Quantity[f64, One] holding π/180. A program that wants angles checked declares dimension Angle("rad") and converts at trigonometric calls.

Working with the numeric libraries

LibraryBoundary
std.dataframeunits.wrap_list[Length](frame.values_f64("x")?) gives typed values with no copy beyond values_f64
std.plotColumn.Num(units.strip_list(qs)), with a hand-written label for now. Labels from units are open design.
std.numeric (proposal)Solvers keep f64 callbacks. A callback strips and rewraps at its boundary. Quantity-generic solvers wait for evidence.
std.linalgMatrix stays f64. Elements of one matrix often have different dimensions, which a single D cannot express; out of scope

Plot axis labels

Open design, deferred from milestone 3 (B97). std cannot yet give a unit symbol as text, and a unit constant is only a value: u.km is Quantity[f64, Length] holding 1000.0, with no name. Candidate pieces:

  • symbol() on Quantity[T, D]. Returns the coherent-unit symbol that printing uses ("m/s") as a static string, without allocating; "" for a dimensionless quantity. This also serves hand-written labels: f"speed ({Quantity[f64, Velocity].symbol()})".
  • A quantity column in std.plot. For example plot.quantity("speed", &xs) stores the values through strip_list, with no copy, and records the dimension. With no explicit label, the axis title is speed (m/s). An explicit label wins.
  • A display unit that fits the data. Values of 2.5e6 m read better as 2500 km. The column would pick a unit whose scaled values are easy to read, for example so the largest magnitude lies in [1, 1000), and label the axis with that unit. Questions:
    • Where candidate units come from. Unit constants carry no names, so this needs a per-dimension table of (symbol, factor) pairs, from SI prefixes on base symbols (mm, m, km), from named units in std.units, or both. Named-unit display (kN, J) is related later work.
    • Prefixes on compound symbols: km/s or kN, and which factor of a product takes the prefix.
    • Scaling without a copy: divide at tick and label time rather than rewriting the column.
    • Overrides: the caller fixes the display unit and its symbol, for example plot.quantity_in("distance", &xs, u.km, "km"), since the constant has no name to show.
    • Shared axes: series on one axis must share one display unit.

Cost

  • Run time: none. Quantity[T, D] has T’s representation, operators lower to T’s instructions, and unit constants fold. Only print and formatting read the dimension, as a static symbol string.
  • Code size. Generic instances that differ only in dimension arguments have identical bodies unless they format a quantity. The implementation should emit one body for them. Rust and C++ rely on the linker’s identical code folding for this; Plenty’s per-instance checking makes it an explicit step after checking.
  • Compile time. A normal form is a short sorted list, interned with other types. Each dimension expression is normalized once per instance.

Implementation

Milestone 1 shipped as designed, with these differences. The reference pages are authoritative.

TopicThis designImplemented
Integer divisiona // b where T supports it/ is float-only and // integer-only for quantities; integer storage has no to, so use // and .get()
Root degreePositive nNonzero n; negative takes the reciprocal
Pow, RootReserved formsFallback names that a local declaration or import shadows
Exponentsi8 normal formLiteral n within i64; every intermediate exponent must fit i8
Debug outputQuantity[f64, Length / Time](1.3)Normal form: Quantity[f64, Length * Pow[Time, -1]](1.3); alias notes appear only in diagnostics
checked:OverflowError[T]Also ZeroDivisionError, as for the underlying arithmetic
ConstantsLiterals and quantity constantsAlso casts of extracted values, .get() and float .to; no user function calls
Zero-cost evidenceRelease instructions match f64Matched for f32 and f64 +, * and .to on x86-64; not measured on other targets
Instance sharingMilestone 1 if cheapMilestone 3, for every function: identical placed code is folded into one body. Instances that format a value or build a collection keep separate bodies, since their type descriptors name the dimension; functions used as values keep their own addresses
std.units listswrap_list[D](xs: list[T]) and strip_listf64 storage only, like the other std.units functions. A std-only compiler intrinsic, __list_retag, consumes the list and stores the target list type’s descriptor in its header; it accepts only a quantity and its own storage type as element types
std.units storageUnstatedUnits, constants and the std.math-based functions use f64; abs and clamp accept any storage
min, maxIn std.unitsNot repeated: the generic std.math.min and max already accept quantities
meanUnstatedReturns Option, Nothing for an empty list, as std.math.min_all does
Symbol orderSI order, kg*m/s^2Declaration order within a module, modules by name; std.units declares SI order

Out of scope

FeatureReason
Units in the type (option F)Conversion and promotion rules for one dimension; the dimension-only model covers the checks
Affine quantities (°C, °F, positions, points in time)from_celsius converts at the boundary. mp-units models these as quantity_point; add one only with use cases. std.time already separates Instant from Duration.
Quantity kinds: torque vs energy, Hz vs Bq, rad vs srSI gives them one dimension. A user dimension separates them where it matters.
Rational exponentsInteger exponents first; see dimension expressions
Logarithmic units (dB, neper)Not multiplicative; ordinary functions
Matrices and dataframes with per-column unitsNeeds a dimension per column or element, as UnitfulTensors.jl does; later
Dimension expressions in parameter types (option D)Inference by abelian unification; later if needed

Milestones

  1. Language. Delivered (issue #701). dimension declarations, the Dimension constraint, dimension expressions with Pow and Root, normal forms and identity, the parameter-type rule, built-in Quantity[T, D] for every number and integer domain storage type, operators, get, to, try_from, printing, quantity module constants and diagnostics. Reference page, tutorial lesson and cheat-sheet entry.
  2. std.units. Delivered. SI dimensions, units, constants and functions; temperature and Duration conversions; zero-copy list conversions.
  3. Measured follow-ups. Shared bodies for instances differing only in dimension (delivered); plot axis labels from units (deferred, design open); then any later item with a use case.

Testing and acceptance

  • Algebra: normalization is commutative and associative, cancels exactly, and Root rejects every non-divisible exponent. Property tests over random expressions compare normal forms against a reference evaluator.
  • Operators: every operator for every storage type, including each domain and checked: blocks, with expected storage and dimension results.
  • Rejections: each rule above has a diagnostic test, including additive scalars, mixed storage, Positive[Quantity[...]], dimensions in value positions and expressions in parameter types.
  • Zero cost: the operator-lowering probe extended to quantities: a + b, a * b and q.to(u.km) on Quantity[f64, D] must emit the same release instructions as the f64 program. Allocation-denial tests cover quantity arithmetic and wrap_list.
  • Library: std.units constants checked against the SI Brochure (9th edition) and CODATA 2022; conversions round-trip within one ulp where the factor is exact.

Risks and open questions

  • Diagnostics through aliases. Resolved: diagnostics show the normal form with a note naming the first visible matching alias in declaration or import order, using the importing module’s spelling.
  • Symbol spelling. Resolved: ASCII base symbols in declaration order within a module, with modules ordered by name. std.units declares SI order, so a newton prints kg*m/s^2. Debug spellings keep the normal form, sorted by qualified base name. Named-unit display (J) is later work.
  • The dimension keyword may conflict with existing identifiers outside the repository. Pow and Root are not keywords: a local declaration or import with either name keeps its ordinary meaning.
  • Learner cost. Units add a concept. The tutorial lesson should come after generics and teach the rules above without the algebra’s internals.

Units of measure: precedents

Research for the units of measure proposal, gathered on 2026-11-03. Each entry records the model, what it costs at run time, and what it means for Plenty. It contains no Plenty examples.

Summary

SystemWhere dimensions liveRun-time costUnit in type?Generic over dimension
Rust uomLibrary; typenum type-level integersNone; values normalized to base unitsNoYes, with trait bounds on typenum arithmetic
Rust const-generic portsLibrary; const exponentsNoneNoNeeds nightly generic_const_exprs
C++ Boost.UnitsLibrary; template metaprogrammingNoneUnit system in typeYes, verbose
C++ mp-units (P3045)Library; C++20 class-type template parametersNoneYesYes, with concepts
F# units of measureCompiler; abelian-group unificationNone; erasedUnits, not dimensionsYes, inferred
Ada (GNAT)Compiler aspects on subtypesNoneNoLimited
Julia Unitful.jlLibrary; type parameters, compiler specializesNone for homogeneous arraysYesYes, through dispatch
Python Pint, astropyRun timePer operation and per value——

Rust

uom 0.38. Quantities are generic over a dimension type whose seven exponents are typenum integers, the storage type and a unit system. Values are normalized to the system’s base units at construction, so Length::new::<kilometer>(3.2) stores 3200 m; arithmetic is then plain storage arithmetic. Results of * and / come from trait impls that add or subtract exponents with typenum operators. Errors name typenum types such as PInt<UInt<UTerm, B1>>. Plenty takes the normalization and the dimension-only type; it avoids the type-level integer encoding. Source: https://docs.rs/crate/uom/latest.

tiny-uom 0.1 (2020) and similar experiments replace typenum with const generics. Computing L1 + L2 in a signature needs the unstable generic_const_exprs (then const_evaluatable_checked), so they require nightly Rust and have not reached stable use. This is the problem B83 avoids by deferring derived sizes. Source: https://docs.rs/crate/tiny-uom/latest/source/README.md.

C++

Barton and Nackman (1994) showed dimensional analysis through template parameters. Boost.Units generalized it with template metaprogramming: zero cost, long compile times and diagnostics.

mp-units uses C++20 class types as template parameters, so a quantity is quantity<si::metre / si::second, double> with readable type names. It distinguishes quantity kinds (isq::height vs isq::width), separates points from differences (quantity_point, needed for °C), and keeps units in the type, converting on assignment where that is exact. P3045 proposes it for the C++ standard library; R7 is dated February 2026. The maintainer’s reports state that LEWG gave unanimous support at the Croydon meeting (March 2026) for the core, quantity kinds, the affine space and text output, and that the Brno meeting (June 2026) saw a new paper on non-negative absolute quantities. These are the maintainer’s reports, not committee minutes. Sources: https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2026/p3045r7, https://github.com/mpusz/mp-units, https://mpusz.github.io/mp-units/HEAD/blog/2026/03/28/report-from-the-croydon-2026-iso-c-committee-meeting, https://mpusz.github.io/mp-units/HEAD/blog/2026/06/13/report-from-the-brno-2026-iso-c-committee-meeting/.

The non-negative paper is the closest precedent for this proposal’s domain storage: mp-units is adding range constraints to quantities rather than wrapping quantities in a separate range type.

F#

Andrew Kennedy’s units of measure are part of the F# compiler. float<m/s> is a float with a unit; units form an abelian group, and inference solves equations over it, so let sqr x = x * x gets the type float<'u> -> float<'u ^ 2>. Units are erased after checking: a float<m> is a float at run time. F# 4.0 added rational exponents (cm Hz^(1/2) / W). F# writes literal zero as 0.0<_> to unify with any unit; this proposal rejects that exception and requires a typed zero.

F# puts units, not dimensions, in the type: float<km> and float<m> are different, converted by explicit multiplication. Option F in the proposal records why Plenty does not. Sources: https://learn.microsoft.com/en-us/archive/blogs/andrewkennedy/units-of-measure-in-f-part-three-generic-units, https://fsharp.github.io/fslang-spec/units-of-measure, https://www.infoq.com/news/2015/04/FSharp-4/.

Ada (GNAT)

GNAT adds Dimension_System and Dimension aspects. A type declares the base dimensions; subtypes declare exponents; System.Dim.Mks defines SI. The compiler checks dimensions statically and generates no extra code. It is the closest precedent for option C: a small compiler extension rather than a type-level library, with dimension-only types. Sources: https://gcc.gnu.org/onlinedocs/gnat_ugn/Performing-Dimensionality-Analysis-in-GNAT.html, https://www.adacore.com/uploads/blog/dc.pdf.

Julia

Unitful.jl encodes units and dimensions as type parameters, and Julia’s specializing compiler removes them for concrete types. Arrays of one quantity type are as fast as plain arrays. Users report long type names, slow arrays mixing units, and the need for UnitfulTensors.jl for matrices with per-element units. Plenty’s proposal leaves matrices with units out of scope for the same reason. Sources: https://discourse.julialang.org/t/physical-units-in-julia/76825, https://discourse.julialang.org/t/performance-of-unitful-arrays/52635, https://discourse.julialang.org/t/ann-unitfultensors-jl-efficient-arrays-with-units/114028.

Python

Pint and astropy.units attach a unit object to each value and check at run time. They are flexible and cost storage and time per operation; errors appear only on executed paths. They are option E in the proposal.

Motivating failure

The Mars Climate Orbiter was lost in 1999 after ground software produced impulse in pound-force seconds where the navigation software expected newton seconds. Both are impulse. A dimension-only type system would not have caught this, since the values had the same dimension; normalizing every value to a coherent unit at construction would have. This is why the proposal stores coherent-unit values and makes 3.0 * u.lbf * u.second the only way to write a pound-force second.

Units of measure: spike evidence

Spikes 1–8 ran on the release compiler built from main on 2026-11-02, before issue #701, and were rerun on eb772202 (after issue #701) with identical output except spike 5. They support the units of measure proposal. The built-in quantity section shows the same cases with Quantity[T, D]. Outputs are exact except that error paths are shortened to the file name.

Spike 1: Same-dimension arithmetic

Phantom tags give distinct types; @newtype forwards +, -, < and unary -.

# Phantom-tagged generic newtype: same-dimension arithmetic, distinct types.
class Length:
    pass
class Time:
    pass

@copy
@newtype
class Q[T, D]:
    value: T

def main() -> Result[(), IoError]:
    a = Q[f64, Length](3.0)
    b = Q[f64, Length](4.5)
    print(a + b, a - b, a < b, -a)?
    t = Q[f64, Time](2.0)
    print(t)?
    Ok(())

Output:

Q[f64, Length](value=7.5) Q[f64, Length](value=-1.5) True Q[f64, Length](value=-3.0)
Q[f64, Time](value=2.0)

Spike 2: Mixed dimensions are rejected

The forwarding rule already rejects Length + Time.

class Length:
    pass
class Time:
    pass
@copy
@newtype
class Q[T, D]:
    value: T
def main() -> Result[(), IoError]:
    a = Q[f64, Length](3.0)
    t = Q[f64, Time](2.0)
    print(a + t)?
    Ok(())

Output:

error: s2.plenty:12:11: `@newtype` class `Q[f64, Length]` forwards `+` only between two `Q[f64, Length]` values, not `Q[f64, Length]` and `Q[f64, Time]`; read the field, or declare an operator hook for these operands

Spike 3: Domain storage loses the unit on -

Positive - Positive returns i64, so d is a bare i64; the annotation n: i64 = d compiles.

# Domain storage through a newtype: does Positive survive + and -?
class Length:
    pass
@copy
@newtype
class Q[T, D]:
    value: T
def main() -> Result[(), IoError]:
    a = Q[Positive[i64], Length](3)
    b = Q[Positive[i64], Length](4)
    s = a + b
    d = a - b
    print(s, d)?
    n: i64 = d
    Ok(())

Output:

Q[Positive[i64], Length](value=7) -1

Spike 4: A generic operator hook is accepted when unused

The restriction is checked only at a use.

# Generic operator hook: can __mul__ take a different dimension?
class Q[D]:
    value: f64
    def __mul__[E](self, other: &Q[E]) -> f64:
        self.value * other.value
def main() -> () :
    ()

Output:

(compiles; no output)

Spike 4b: A generic operator hook is rejected when used

Length * Time cannot be written with one generic __mul__.

class L:
    pass
class T:
    pass
class Q[D]:
    value: f64
    def __mul__[E](self, other: &Q[E]) -> f64:
        self.value * other.value
def main() -> Result[(), IoError]:
    a = Q[L](2.0)
    b = Q[T](3.0)
    print(a * b)?
    Ok(())

Output:

error: s4b.plenty:7:5: operator hooks cannot have method type parameters, defaults, or yield

Spike 5: No float domain

Positive[f64] is rejected. On 2026-11-02 the error also suggested NonZero[f64], itself rejected; issue #700 removed that hint (local PR #883).

# Positive over floats?
def f(x: Positive[f64]) -> f64:
    x.get()
def main() -> ():
    ()

Output:

error: s5.plenty:2:10: `Positive` requires a signed integer type

Spike 5b: NonZero is integer-only

def f(x: NonZero[f64]) -> f64:
    x.get()
def main() -> Result[(), IoError]:
    print(f(2.0))?
    Ok(())

Output:

error: s5b.plenty:1:10: f64 does not satisfy IntType

Spike 6: Hand-written classes per dimension

Works, but each class has one __mul__ signature, so Metres * Metres and Metres * f64 cannot both use it; the scalar goes through __rmul__.

# One hook signature: scalar via __rmul__, quantity*quantity via __mul__.
@copy
class Metres:
    value: f64
    def __mul__(self, other: &Metres) -> SquareMetres:
        SquareMetres(self.value * other.value)
    def __rmul__(self, k: f64) -> Metres:
        Metres(k * self.value)
    def __truediv__(self, other: &Seconds) -> MetresPerSecond:
        MetresPerSecond(self.value / other.value)
@copy
class Seconds:
    value: f64
@copy
class SquareMetres:
    value: f64
@copy
class MetresPerSecond:
    value: f64
def main() -> Result[(), IoError]:
    m = Metres(3.0)
    print(m * m, 2.0 * m, m / Seconds(1.5))?
    Ok(())

Output:

SquareMetres(value=9.0) Metres(value=6.0) MetresPerSecond(value=2.0)

Spike 7: Code generic over one dimension

Additive unit-polymorphic code works today.

# Unit-polymorphic code over a phantom dimension, today.
class Length:
    pass
@copy
@newtype
class Q[T, D]:
    value: T
def total[T, D](xs: &list[Q[T, D]]) -> Q[T, D]:
    mut sum = xs[0]
    for i in range(1, len(xs)):
        sum = sum + xs[i]
    sum
def main() -> Result[(), AllocError | IoError]:
    xs = [Q[f64, Length](1.5), Q[f64, Length](2.0)]?
    print(total(&xs))?
    Ok(())

Output:

Q[f64, Length](value=3.5)

Spike 8: NonZero storage

* keeps NonZero; + returns a bare i64 and drops the unit.

# Positive survives + inside a newtype; NonZero divisor through a newtype.
class Count:
    pass
@copy
@newtype
class Q[T, D]:
    value: T
def main() -> Result[(), IoError]:
    a = Q[NonZero[i64], Count](3)
    b = Q[NonZero[i64], Count](-2)
    print(a * b, a + b)?
    Ok(())

Output:

Q[NonZero[i64], Count](value=-6) 1

Built-in quantity after issue #701

Run on eb772202. Spike 4b’s product and spike 3’s subtraction now work, and the proposal’s opening example, with local unit constants in place of std.units, prints the output the proposal gives.

dimension Length("m")
dimension Time("s")
dimension Velocity = Length / Time
const metre: Quantity[f64, Length] = Quantity[f64, Length](1.0)
const second: Quantity[f64, Time] = Quantity[f64, Time](1.0)
const km: Quantity[f64, Length] = 1000.0 * metre
const minute: Quantity[f64, Time] = 60.0 * second
const hour: Quantity[f64, Time] = 3600.0 * second

def speed(d: Quantity[f64, Length], t: Quantity[f64, Time]) -> Quantity[f64, Velocity]:
    d / t

def main() -> Result[(), IoError]:
    d = 3.2 * km
    t = 41.0 * minute
    v = speed(d, t)
    print(v, v.to(km / hour))?
    a = Quantity[f64, Length](2.0)
    b = Quantity[f64, Time](3.0)
    print(a * b)?
    p = Quantity[Positive[i64], Length](5)
    print(p - p, p * p)?
    Ok(())

Output:

1.3008130081300813 m/s 4.682926829268292
6.0 m*s
0 m 25 m^2

Mixed dimensions:

dimension Length("m")
dimension Time("s")
def main() -> Result[(), IoError]:
    d = Quantity[f64, Length](1.0)
    t = Quantity[f64, Time](1.0)
    print(d + t)?
    Ok(())
error: q2.plenty:6:11: quantity `+` requires equal dimensions, not `Length` and `Time`

p - p keeps the unit and drops the domain:

dimension Length("m")
def main() -> Result[(), IoError]:
    p = Quantity[Positive[i64], Length](5)
    q: Quantity[Positive[i64], Length] = p - p
    print(q)?
    Ok(())
error: q3.plenty:4:5: expected Quantity[Positive[i64], Length], got Quantity[i64, Length]

Six rules for a consistent core

Status: Accepted; partially implemented. The backlog tracks remaining scope. The evidence below records research against local main at b4580c2d on 2026-10-31, with probes on the debug compiler built that day. The evidence is the 2026-10-31 language consistency survey, recorded as issues #612–#637. Accepted and scheduled as backlog rows B88–B93 (staged plan); the review decisions settle five questions. A seventh rule from the same survey, a written convention page for the standard library, is separate documentation work and is only referenced here.

Summary

The survey found a coherent core with local exceptions. Each exception has its own explanation, and a learner has to memorise them one at a time. This proposal replaces about twenty such exceptions with six rules:

  1. One copy rule. Tuples, built-in types and function values copy when their parts copy. A declared class or enum copies only when it says @copy. The rule is stated in the same words in the reference, the cheat sheet and the agent skill.
  2. One formatting path. print(x) writes what f"{x}" writes, and str.repr(x) writes what f"{x!r}" writes. Both use a type’s display and debug methods, at the top level and inside aggregates.
  3. An owner converts to its view. Where a view type (&[T], &substr) is expected, a reference to the owner (&xs, &s, b"...") is accepted without [:]. Every byte owner and byte view is a byte source.
  4. Adapters belong to iterators. iter(source) accepts every for source and gives the same adapters and terminals. Only iterators (pipelines and generators) have adapter methods; collections, ranges, views and the free sources do not.
  5. An error type names only what can fail. Allocation-only failure is AllocError. A domain error type carries allocation as Allocation(AllocError). Invalid arguments follow one four-part policy.
  6. Every callable form is a value wherever a callback is accepted. Named functions, closures with or without captures, stored or literal, variant constructors and class constructors all pass to the same parameter, and a callback behaves the same however it is written.

Each rule is a change to behaviour or teaching, so each has its own backlog row and can ship independently. The rules do not depend on one another, except that rule 5 decides the error type that rule 2’s print reports.

How to read this page

Each rule has the same sections: what happens today, the proposed rule, alternatives, cost, what a learner no longer has to learn, and phasing. “Today” claims were checked against the reference pages linked in the text and, where marked probe, with plenty --check or a run on the 2026-10-31 debug build.

Earlier changes removed features rather than deprecating them (std.iter was removed in favour of iter(); see fluent iterator pipelines). This proposal follows that precedent: a breaking change ships with a diagnostic that states the new spelling, and the same change updates std, tests, the tutorial and the reference. Package language editions are not implemented and are not required.

Rule 1: one copy rule

Today

The bindings, moves and copies page says scalars, strings, ranges, named function values and “aggregates of copyable components” copy, and that classes are affine unless declared @copy. “Aggregates” includes tuples and enums, but the enums page never mentions copying. The agent-facing pages each give a different list (issue #636):

SourceSays copies
ReferenceScalars, str, ranges, function values, copyable aggregates, @copy classes
Cheat sheetScalars, str, copyable tuples, @copy classes; also says a @copy class “cannot take type parameters”, which classes contradicts (IntType/FloatType bounds are allowed)
plenty agent skillScalars, str, @copy classes

None of the agent-facing pages mentions enums, and the rule itself has an exception: a class must opt in, an enum copies implicitly.

That exception has a cost. Issue #535, which introduced @copy, made the annotation “part of the type’s public contract: removing it is a breaking change”. An enum’s copyability is the same contract, but it is implied by its payloads. Probe: enum E: A(i64) | B copies, so take(e); take(e) checks. Changing the payload to A(list[i64]) makes E affine, and the error appears at the second take(e) in every caller (use of moved binding e), not at the declaration that changed. @copy enum is rejected today (`@copy` applies only to class declarations).

Proposed rule

A value copies when its type is a scalar, str, bytes, a range, a function value, a tuple or built-in generic (Option, Result) whose parts copy, or a class or enum declared @copy. Every other value moves. @copy is checked at the declaration: every field or payload must copy, and the type has no __del__.

The rationale, stated with the rule: anonymous and built-in types copy by structure, because they have no author who could intend otherwise. A declared type copies only when its author says so, because copying is part of its public contract and must not change when a field changes.

The same sentence goes in the reference, the cheat sheet and the skill. The cheat sheet’s type-parameter claim is corrected. A test compares the three copies so they cannot drift again.

Alternatives

  • State today’s rule without changing it (classes opt in, enums by structure). This fixes the documentation but keeps the exception and the distant-caller breakage. Rejected as the end state; it is phase 1.
  • Make classes structural too, and remove @copy. One rule with no annotation, but a class that holds a token, an index or an invariant would silently copy, and adding a field would flip the contract of every class. This reverses issue #535. Rejected.
  • Allow @copy on enums but keep structural copying as the default. The annotation would only add a declaration-site check. Two ways to get the same behaviour, and the default still breaks distant callers. Rejected.

Cost and compatibility

  • Phase 1 is documentation and one test.
  • Phase 2 (@copy accepted on enums) is additive.
  • Phase 3 makes undeclared enums affine. std has 158 enum declarations; tests, examples and book pages have about 460. A migration script adds @copy to every enum whose payloads all copy, which preserves today’s behaviour exactly; reviewers then remove it where an enum should move. Generic enums whose copying depends on a type argument need a copyable bound, which is B84’s open work for generic @copy classes. Phase 3 waits for that bound (decided 2026-10-31), so B84 moves earlier: it now gates this rule as well as generic copy classes.
  • Copy-dependent behaviour elsewhere (dictionary-key rules in user type hashing, by-value receivers) keys off “copies”, so it follows the new rule without its own change.
  • The use-after-move diagnostic for an affine enum names the declaration and suggests @copy when its payloads copy.

What a learner no longer has to learn

Which kinds of declaration copy implicitly, and why an enum change broke a caller. The question “does this copy?” has a two-line answer.

Phasing

  1. One statement in all three places; fix the cheat sheet (issue #636).
  2. Accept and check @copy on enums.
  3. Undeclared enums move; migrate std, tests and the book.

Rule 2: one formatting path

Today

The formatting page says print and str.repr never call user formatting methods. The format strings page says a field without a conversion uses a public display method, !r uses debug, and also that “a field is written as print writes it”, which is false for such types (issue #615). Inside aggregates, neither path calls user methods. Probe, for a public class P whose display writes <3> and debug writes P#3:

ExpressionOutput
print(p)P(x=3)
str.repr(p)?P(x=3)
f"{p}"?<3>
f"{p!r}"?P#3
print(ps), f"{ps}"? for ps = [p][P(x=3)]

A non-pub class with display cannot appear in an f-string in its own module; the diagnostic names __plenty_fstring_… (issue #615). main’s error report originally used a third hook, std.report.Formattable.format, over a different Formatter type, and otherwise the str.repr spelling. Issue #634 subsequently unified the formatter, error, and reporting protocol in std.fmt; choosing display for entrypoint reports remains separate work. See entrypoint reporting.

Proposed rule

A value has two spellings. display is the default: print(x), f"{x}" and main’s error report write it. debug is the precise one: str.repr(x), f"{x!r}" and the elements of a formatted aggregate write it. A type without display uses its debug spelling; a type without debug uses its structure.

So print(p) writes <3>, print(ps) writes [P#3], as Python’s print([p]) uses each element’s repr. Strings stay raw at the top level of print and quoted inside aggregates, which is today’s behaviour and is the same rule (display of str is its text, debug is quoted).

A type’s own methods are found from its defining module whatever its visibility, so a private class formats in its module. Internal names never appear in diagnostics.

Alternatives

  • Make f-strings ignore display, as print does. Consistent, but removes the only user formatting hook from core syntax and breaks std types (std.path, std.complex, std.error and others) that define display. Rejected.
  • Keep the paths separate and fix the sentence in format-strings.md. Documents the difference instead of removing it; print(p) and f"{p}" still disagree. Rejected.
  • Use display inside aggregates too. Ambiguous for strings and for types whose display text contains separators; Python’s str/repr split exists for this reason. Rejected.

Cost and compatibility

  • print and str.repr start running user code. That code can fail with FormatError, so their result types follow rule 5: unchanged for values with built-in spellings, widened with FormatError when a user method can run. Programs that print such types and return Result[(), IoError] need the wider error; the diagnostic names the type whose method can fail.
  • Output changes for every printed value of a type with display or debug. Expected-output fences in the tutorial and reference that print std types with these methods change; the test suite finds each one.
  • main’s error report uses display. std.fmt.Formattable becomes a transitional fallback while the entrypoint hook migrates. Issue #634 delivered the shared formatter; the meaning of a method named format belongs to the std conventions page (rule 7). This rule fixes which hook core formatting calls.
  • Automatic formatting still rejects recursive types. Each user method an aggregate element calls gets its own std.fmt formatter with a fresh depth of 32 and no byte budget of its own; the methods do not share one depth count across the aggregate. The print or str.repr buffer grows with checked allocation as before.

What a learner no longer has to learn

Which of three output paths calls which hook, and why print and an f-string disagree. One rule: display for people, debug for precision.

Phasing

  1. Fix the false sentence and the private-class lookup; hide internal names (issue #615).
  2. Route print, str.repr and aggregate elements through display/debug.
  3. main’s report uses display; its shared formatter dependency is delivered by issue #634.

Rule 3: an owner converts to its view

Today

Update 2026-11-01: phases 1 and 3 shipped. Owners convert to views at view types, byte-reading std APIs take &[u8], and ByteSource is removed; naming it is a compile error pointing to &[u8]. The remaining &Bytes, start, stop inputs are listed under std conventions. The probes below record the state before phase 1.

Slice views give &[T] for a list, a Bytes buffer and a bytes literal, so “one &[u8] parameter reads any of them”. But “a list is not converted to a view implicitly; pass the whole list as &xs[:]”. Probes:

CallResult
total(&xs) for total(values: &[i64])Rejected: pass &xs[:]
n(b"lit") for n(t: &[u8])Rejected: pass &xs[:]
n(&s) or n("lit") for n(t: &substr)Rejected: pass &s[:]
s.startswith(w) for w: &substrRejected: expected str, got &substr
w.upper() for w: &substrRejected; the diagnostic lists the dozen methods a &substr has
n(&xs) for S: ByteSource and xs: list[u8]Rejected: protocols require classes, enums or bytes
n(&xs[:]) for the sameRejected, and the diagnostic names Result[list[u8], AllocError], the copying slice’s type

ByteSource has two implementations, Bytes and bytes; a &[u8] view and a list[u8] are not among them, although &[u8] already unifies all three owners. Byte APIs therefore take source: &S, start, stop triples that a view expresses directly.

Proposed rule

Where a view type is expected, a reference to an owner of that view is accepted: &xs for &[T], &mut xs for &mut [T], &buf or b"..." for &[u8], &s or a literal for &substr. The conversion is free: it creates the same view as [:], with no copy and no allocation. Every operation that reads text accepts &substr; every operation that reads bytes accepts &[u8].

&xs[:] keeps working. Expected-type positions are parameters, annotated locals and returns; a view still cannot be stored (B11).

Alternatives

  • Keep [:] explicit. The current rationale is that conversions should be visible. That matters for costs; this conversion has none, and the design goal is “no hidden allocation”, not “no hidden pointer arithmetic”. Rust performs the same coercion from &Vec<T> to &[T]. Rejected.
  • Accept views where owners are expected (a &substr argument for a str parameter). This would need an allocation or a second representation for str. Rejected; text-reading parameters change to &substr instead, which is B12’s planned scope.
  • Make &[u8] implement ByteSource and keep the protocol. Smaller change, but leaves two spellings for “bytes I read”. Accepted as the first phase only; std then moves to &[u8] parameters.

Cost and compatibility

  • Phase 1 is additive: code with [:] keeps compiling.
  • Phase 2 changes std signatures. Text APIs taking str to read change to &substr; callers passing str keep working through the conversion. Byte APIs taking &S, start, stop change to &[u8], so w.write(&buf, a, b) becomes w.write(&buf[a:b]). The std.io Writer protocol’s write signature changes, which breaks user writers; the std.io page and its examples change with it. Range-error results on those methods (BytesError.OutOfRange) disappear, because view bounds clamp.
  • ByteSource is removed once nothing in std takes it.
  • Per-type specialisation of byte generics goes away: one function body serves every byte owner. Codegen cost is a pointer and a length either way.
  • The misleading Result[list[u8], AllocError] diagnostic is fixed in phase 1.

What a learner no longer has to learn

When to write [:], which of &substr and str a text method wants, and the ByteSource protocol with its start/stop convention. One rule: pass the owner or a view of part of it.

Phasing

  1. Owner-to-view conversion at expected view types; &[u8] satisfies ByteSource; fix the diagnostic.
  2. Text-reading std APIs accept &substr (part of B12).
  3. Byte-reading std APIs take &[u8]; remove ByteSource.

Box already converts to &T, while Shared does not (survey section 2). &Shared[T] converts to &T as &Box[T] does (decided 2026-10-31). The content is immutable while shared, so the view is sound; the borrow keeps the handle, and so its count, alive for the view’s lifetime. This joins phase 1.

Rule 4: adapters belong to iterators

Today

Pipelines start with iter(source). Probe: iter(...) accepts every for source already: lists, sets, strings, bytes, ranges, reversed, enumerate, zip, keys(), values(), items(), generators and pipelines. Some sources also take adapters directly and others do not (issue #622):

SourceDirect .count() today
Generator, enumerate(&xs), zip(&xs, &ys), d.items()Accepted
range(3)Rejected: expected str, got range
xs (list)xs.map(...): unsupported method map; xs.count(): list count needs an element
reversed(&xs)Rejected: reversed is supported in loops only
d.keys()Rejected; suggests the allocating keys_snapshot()

The three dictionary views differ in kind (dictionaries, issue #624): keys() is a bindable KeyView with len and set algebra, values() cannot be bound or measured, and items() starts chains but cannot be bound. Terminals differ by item type: sum, min and max read copyable reference items through, while find and reduce reject them (issue #621). len(x) and in reject std containers such as Deque and Bytes that define .len() (issue #623).

Proposed rule

A collection, range or view is a source. An iterator is a single-use stream: a pipeline (Iter[T]) or a generator. iter(source) makes an iterator from any for source, and iter(it) on an iterator returns it. Adapters and terminals are methods of iterators only, and every adapter and terminal treats items the same way whatever the source.

This matches Python’s iterable/iterator split and Rust’s Vec/Iterator split. It is the only choice with no name clashes: a list already has count(e) and find, a string has count and find, a set has its own algebra, and none of them is consumed by iterating.

Under the rule:

  • enumerate(...), zip(...) and d.items() stay for sources and iter(...) arguments, but lose direct adapter methods; the iterator methods .enumerate() and .zip() already exist, so enumerate(&xs).count() becomes iter(&xs).enumerate().count().
  • keys(), values() and items() are the same kind of thing: bindable borrowed views with len, usable in for, in (keys) and iter(). KeyView is accepted wherever a borrowed set is read.
  • find, reduce and every other terminal copy copyable reference items as min and max do.
  • Diagnostics for an adapter on a source suggest iter(source).

Alternatives

  • Give every source the adapter methods directly. Clashes with existing collection methods (list.count, list.find, str.find), and makes it unclear whether a call consumes. Rejected.
  • Remove direct methods from generators too, so only Iter has them. A generator is single-use and supports next, exactly like a pipeline; denying it methods would be a distinction without a reason. Rejected.
  • Fix only the diagnostics. Issue #622’s minimum. Keeps two rules. Rejected as the end state; it is phase 1.

Cost and compatibility

  • Direct adapters on enumerate, zip and items() are rare: two uses in Plenty code and pages, about thirty in tests/*.rs (mostly test_pipelines.rs). Each becomes iter(...) or the iterator method.
  • Dictionary-view changes are additive.
  • find/reduce on copyable references is additive.
  • len method dispatch is implemented for classes and enums (issue #623). Membership on std containers still depends on B49’s protocol work; unsupported operations name their actual type.

What a learner no longer has to learn

Which sources take methods directly, three kinds of dictionary view, and which terminals read through references. One rule: iter() first, and everything that works on one iterator works on all.

Phasing

  1. Diagnostics suggest iter(source) (issue #622); find/reduce copy copyable items (issue #621).
  2. Uniform dictionary views (issue #624).
  3. Remove direct adapters from enumerate, zip and items().
  4. len method dispatch delivered (issue #623); in on std containers remains with B49.

Rule 5: an error type names only what can fail

Today

Allocation failure has several shapes (issue #632). The table and the probe describe the state before phasing items 2 and 3. Item 2 made f-strings without user formatting fail with AllocError; item 3 changed the FormatError, std.bytes and union rows and std.bytes.with_capacity(-1):

OperationAllocation failure reported as
[1, 2], a + b, copy(x)AllocError
An f-stringstd.fmt.FormatError.Bytes(BytesError.Allocation(...))
printIoError.Data(DataError.Allocation(...)) (I/O errors)
std.bytes.empty(), from_textBytesError.Allocation
clist.append(row)AppendError[T], which returns the row
sketch, linalg, mime, graphA union E | AllocError

Switching a + b to f"{a}{b}" changes the function’s error type, and FormatError needs an import although every f-string produces it. Probe: -> Result[str, FormatError] gives unknown type FormatError, and -> Result[str, AllocError] returning f"n={n}" is rejected. Tuple slices return Result although tuples never allocate (issue #625).

Invalid arguments follow four policies (issue #629):

ArgumentToday
range stepCompile error unless NonZero (ranges)
split(""), even a literalRun-time trap (text splitting)
repeat(-1)Empty result (text building)
list.with_capacity(-1)Err(CapacityOverflow) (construction)
std.bytes.with_capacity(-1)Err(OutOfRange)
.par(&pool, chunks=0)Trap with the unrelated integer cast out of range

Proposed rule

An operation’s error type names only failures it can produce. If allocation is its only failure, the type is AllocError. A domain error type that can also fail to allocate carries it as one variant, Allocation(AllocError). An operation that cannot fail returns no Result.

For f-strings: Result[str, AllocError] when every field has a built-in spelling and no spec is computed at run time; Result[str, AllocError | FormatError] when a field runs a user display/debug or a computed spec can be invalid. FormatError becomes a prelude name. print and str.repr follow the same rule (see rule 2). Tuple slices return the tuple.

Invalid arguments.

  1. A size passed to allocation that cannot be allocated, including a negative one, is AllocError.CapacityOverflow.
  2. Slice bounds and repeat counts clamp, as in Python. This is the operation’s defined result, not an invalid argument.
  3. Any other argument outside its documented domain traps, with a message naming the operation and the argument. A compile-time constant outside the domain is a compile error. Where a type can state the domain (NonZero for a range step), the parameter uses it.
  4. Err is for failures the caller cannot rule out by checking its arguments: allocation, I/O, and external data.

So split("") with a literal is a compile error and with a variable traps (as now); std.bytes.with_capacity(-1) is CapacityOverflow; chunks=0 traps naming chunks. repeat(-1) and slice clamping are unchanged.

Alternatives

  • Always a union, never a variant (IoError | AllocError everywhere). Uniform for matching, but print would need AllocError in about 390 main signatures that now return Result[(), IoError], and the survey’s agreed wording keeps domain errors for APIs that have them. Rejected. About 42 std error types already use an Allocation variant and four use a union; the four change.
  • Keep FormatError on every f-string and add it to the prelude. Fixes the import but not the signature change from + to an f-string, which is the more common edit. Rejected; the prelude addition is kept for the user-method case.
  • Make every invalid argument an Err. Puts a ? on calls whose failure is a bug the caller could rule out, which teaches ignoring it. Rejected.
  • Trap on negative repeat counts. Breaks the padding idiom " ".repeat(width - len(s)) that Python habits rely on. Rejected.

Cost and compatibility

  • f-string signatures: functions returning FormatError for f-strings that only use built-in spellings must change to AllocError (or keep a union that contains it). This is the largest edit; it is mechanical, and the diagnostic states the new type. Formatting through out.format(f"...") in display methods keeps FormatError.
  • FormatError.Bytes(BytesError.Allocation) becomes FormatError.Allocation(AllocError); code matching the nested shape changes.
  • std.bytes constructors return AllocError; their callers’ signatures narrow.
  • The four union-style std modules move to an Allocation variant. This is shared with the std conventions page (rule 7).
  • AppendError[T] remains: it returns the moved row, a separate convention (also issue #630 for spawn) that the std conventions page owns.
  • Tuple slices: t[0:2]? becomes an error with the slice-view wording (issue #625).

What a learner no longer has to learn

Five shapes of allocation failure, an import for f-strings, and four invalid-argument policies. One rule for each: the type names what can fail, and an argument bug traps.

Phasing

  1. FormatError in the prelude (issue #632); infallible tuple slices (issue #625); literal split("") rejected and the chunks message (issue #629).
  2. f-strings, print and str.repr report AllocError when no user method runs. Done: print and str.repr with B89 phase 2 (issue #615), f-strings with B92 phase 2. A :c spec counts as computed, since an out-of-range scalar is InvalidSpec.
  3. std error types: Allocation(AllocError) variants, std.bytes constructors, negative capacities. Done with B92 phase 3: FormatError.Allocation, std.bytes constructors return AllocError, and std.graph, std.sketch, std.linalg, std.shlex and std.mime (new MimeError) carry the variant; std.actor.stage returns StageError. Bytes methods that fail only to allocate, and sketch to_bytes, return AllocError. Out-of-domain arguments that depend on run-time state trap through the std-only __trap statement, with a message naming the operation and argument: Bytes ranges and indices (including read_file and read_stdin), and Rng.index, sample_indices and fill counts and ranges. Their results narrow: Bytes.get returns u8, slice Result[Bytes, AllocError], the scans plain values, and RandomError loses OutOfRange and Allocation. An empty random population stays EmptyRange, since data can produce it. Encoding offsets, UUID write_bytes, crypto finish_into, and CSV field positions now trap on caller bounds errors (issue #667). Encoding pack/unpack, UUID writes, and CSV positions return plain values. Stream read ranges and TLS session transfers follow the same trap rule (issue #666). Unreachable range variants and byte-error adapters are removed (issue #668). Statically checkable domains (positive capacities and limits) wait for a type that expresses them; BytesError.OutOfRange remains for std.actor.net connection limits.

Rule 6: every callable form is a value wherever a callback is accepted

Today

Function values and closures give four callable types, two constraints and three passing modes. Which forms pass depends on the callee (issues #617–#620). Probes:

FormPipeline filter/mapsort_by_keyf: Callable[...]f: &mut F
Named functionYesYesYesNo: named function declarations are immutable
Closure literalYesNo: sort_by_key requires a named function or reusable closureYes, without capturesNeeds a binding, as any borrow does
Stored closure, no capturesNo: pipeline callbacks require a named function or reusable closureYesYesYes, in a mut binding
Stored closure with capturesYesYesNo (code address only)Yes, in a mut binding
Variant constructor E.ANoNoYesNot probed
Some, Ok, ErrNot probedNot probedNo: Some is not a function valueNot probed
Class constructor PointNot probedNot probedNo: unknown binding PointNot probed

A closure with a mut capture passed as &F fails at instantiation with cannot borrow shared reference as mutable, without naming the capture (issue #619). Behaviour can differ too: inside checked:, callback overflow is Err for a closure literal and a trap for a named function (issue #614).

Proposed rule

Named functions, closures with or without captures (stored or literal), variant constructors and class constructors are callable values. Each passes wherever a callback with its signature is accepted, and it behaves the same whichever form it has.

Concretely:

  • Pipelines, sort_by_key and every other builtin callback parameter accept every form. Each currently accepts a different subset: pipelines reject a stored capture-free closure, sort_by_key rejects a literal.
  • &mut f is accepted for a named function, which has no state to mutate, so a &mut F parameter accepts every reusable callable. &F remains the default for read-only callbacks; its diagnostic for a mut capture names the capture and suggests &mut F.
  • A class constructor Point is a function value with the constructor’s positional signature. A fallible initializer gives a Result[Point, AllocError] result, as a call does.
  • Generic variants, including Some, Ok and Err, follow the rule generic functions already follow: an explicit instance (Option[i64].Some) or a concrete expected Callable specialises them. Probe: a generic ident[T] passed to f: Callable[[i64], i64] checks today, while Option[i64].Some is rejected.
  • Inside checked:, whether overflow is Err depends on where the code is written, not on the callback’s form (decided 2026-10-31, issue #614). Arithmetic whose text is in the block, including the bodies of closures defined in the block (literal or stored), reports Err. Any other function, a named function or a closure defined elsewhere, behaves as its signature says: an infallible signature keeps trapping, and a function that can fail returns Result, which the block handles with match or ? like any other call. Builtin reductions such as sum are calls, so their i64 result traps on overflow, as the pipelines page already says; to sum with checking, write fold with a closure literal in the block. Today a lowered closure literal reports Err while the numbers page says closures defined in a block keep trapping; both change to this rule.

Cost and compatibility

All of rule 6 is additive except two items. The checked: fix (issue #614) changes which programs trap: a closure defined in a block, stored or lowered, now reports Err, while sum inside a block, which reports Err today because of pipeline lowering, traps as documented. A closure defined in a block may not outlive it, since a later overflow would have no block to exit (decision 5). Constructor values make a class name usable as a value expression; how that interacts with local bindings of the same name needs a check against the name-resolution rules in names. Worker-thread effect certification originally rejected thin function values, including constructor values (local PR #681). Issue #661 adds conservative certification of every matching address in the closed program; see thin callable targets. This is an effect rule independent of callable forms.

What a learner no longer has to learn

Which callback positions accept which forms, why adding an unused capture fixes an error, and why &mut rejects a named function. One rule: anything you can call can be passed.

Phasing

  1. Pipelines and sort_by_key accept every closure form (issue #617); &mut named functions and the &F diagnostic (issue #619); lead the reference with the constraint form (issue #620).
  2. Variant constructors in pipelines; class constructors as values; generic variants with an expected type (issue #618).
  3. Inside checked:, code written in the block and closures defined in it report Err; other functions behave as their signatures say (issue #614). Delivered as lexical checked callbacks; the reference documents scope and callable-erasure limits.

What the six rules remove

RuleExceptions removedIssues
1Three copy lists; implicit enum copying#636
2print/f-string disagreement; private-class f-string failure; third report hook#615, part of #634
3[:] at view parameters; &substr rejected by text APIs; &[u8] not a byte sourceB12 scope
4Five sources without methods, three kinds of dictionary view, uneven terminals, len/in#621–#624
5Five allocation shapes; f-string import; four invalid-argument policies; fallible tuple slices#625, #629, #632
6Seven callable forms with different acceptance; form-dependent checked:#614, #617–#620

Not covered: the defects in survey section 1 that are bugs, not rules (issues #612, #613, #616), diagnostics for Python habits (issue #627), reference diagnostics (issue #628), and std conventions (rule 7; issues #630, #631, #633–#635). Survey items with existing backlog homes (B16 contextual literals, B36 builtin argument names, B11 borrowed returns) stay there.

Decisions

Recorded 2026-10-31 after review:

  1. Rule 1: phase 3 waits for B84’s copyable bound, and B84 is brought forward.
  2. Rule 3: &Shared[T] converts to &T, as Box does.
  3. Rule 6: Some, Ok and Err follow the generic-function rule; issue #556’s exclusion is lifted unless implementation shows the fallback is needed.
  4. Rule 6: checked: covers arithmetic written in the block and in closures defined in it, whatever their form. Every other function, including a named function passed as a callback and builtins such as sum, behaves as its signature says: it traps, or it returns a Result that the block handles or propagates with ?, which also leaves the block. A function’s possible failures are part of its signature, not of the context that calls it.
  5. Rule 6: a closure defined in a checked: block may not leave it. It leaves by assignment to a binding declared outside the block, as the block’s value or a return value, by being stored in a collection, field, Shared or channel, or by being passed by value to a function that keeps it (spawn, pool.submit). Passing it as &f or &mut f to a function called in the block is allowed. The checker can treat the closure as borrowing a block-scoped value, so existing borrow checking rejects every escape; the diagnostic names checked: and suggests defining the closure outside the block. Rejection can be relaxed later without breaking programs; trapping when called outside the block would reintroduce behaviour that depends on where a closure is called.

Staged plan and suggested backlog wording

Each rule’s phase 1 is small and independent; the issues listed are the work items. Later phases change behaviour and need the user’s agreement per row. The “Proposal” links below point into this page; in the backlog they become proposals/consistency-rules.md#....

IDAreaSuggested wording
B88Copy ruleOne copy rule in the reference, cheat sheet and skill, checked by a test; @copy on enums; undeclared enums affine after a migration that annotates every currently copyable enum. Generic enums wait for B84’s copyable bound, so B84 moves earlier. Proposal; issue #636.
B89Formatting pathprint/f"{x}"/main’s report use display; str.repr/!r/aggregate elements use debug; private types format in their module. Report hook migration uses the shared formatter delivered by issue #634. Proposal; issue #615.
B90Owner-to-view conversion&xs, &s, b"..." accepted at view types; &[u8] satisfies ByteSource; &Shared[T] converts to &T like Box; then byte-reading std APIs take &[u8] and ByteSource is removed. Text APIs accepting &substr stay in B12. Proposal.
B91Iteration sourcesAdapters only on iterators; iter() for every source; uniform dictionary views; uniform terminals over copyable references. Membership on std containers stays B49; len method dispatch is delivered (issue #623). Proposal; issues #621, #622, #624.
B92Error types and invalid argumentsAllocation-only failure is AllocError, including f-strings without user formatting; domain errors carry Allocation(AllocError); FormatError in the prelude; infallible tuple slices; the four-part invalid-argument policy. Proposal; issues #625, #629, #632.
B93Callable formsEvery callable form accepted at every callback parameter; &mut named functions; class constructors and generic variants, including Some/Ok/Err, as function values; inside checked:, closures defined in the block report Err and other functions follow their signatures. Callable names stay (issue #24). Proposal; issues #614, #617–#620.

Completed proposals

These proposals have shipped within their agreed scope, or completed the experiment or investigation identified in their entry. They preserve design rationale and alternatives; the reference and tutorial describe current behavior. Further extensions listed below do not make the completed scope pending again. The backlog remains the only work queue.

See the proposal index for partially implemented and unimplemented designs and the policy for moving completed proposals.

ProposalReferenceFurther work
Boxed protocol objectsProtocol objectsFurther protocol capabilities remain separate from the delivered erased-owner contract
Positive and NonNegative integer domain typesNumbers, std conventionsB96 Bounded; issue #686
Single-file scriptsScript runnerB17/B18: wider executable-cache coverage
Library-owned operations and backend hooks — investigation complete; no language changeProtocols, visibilityB29; future default/final methods and private-hook grants belong to Protocol method bodies
Fluent iterator pipelinesPipelinesB75 extensions; related B12, B22, B24
Owned response experimentField movesB72: real HTTP integration remains unexplored
Flat class layoutValue representations, memoryB19 performance extensions; related B35
Runtime allocation auditAllocation and trapsB13
References, ownership, and explicit copyingOwnershipB11
One immutable str with explicit lengthsTextB12, B27
Sum types and enumsEnums, matchingB19, B33
Generics, structural protocols, context managers, and error propagationGenerics, resource scopesB11, B14, B29
Native owned generatorsGeneratorsUnscheduled
Allocation-free generator frames and rangesGenerators, rangesB12
Callable surface reviewFunctionsB32
Stored closure environmentsConsuming and stored closuresB11, B32
Native concurrency contractConcurrencyB13, B24, B28
Bounded channel implementationChannelsDeferred B22 extensions
Named arguments and fresh defaultsSignatures and callsB36 refinements; unpacking remains separate
Human-readable formatting and error reportsstd.report, entry reportsB71 extensions; Failure remains a deliberate last resort
Explicit error composition, with experiment and production evidencestd.error, std.io.typed, std.http.typed, HTTP clientB71 additive extensions and deferred benchmark-template migration
Finite error sets as anonymous sumsAnonymous sums, propagationB14/B71: guards, residual-set inference and empty sums
Bundled HTTP acceptanceHTTP qualification, client, serverPreserve B66 gates; additional platforms and HTTP/2 or HTTP/3 are separate scope
Explicit applications, modules, and executable lessonsEntry point, modules, runnable documentationB17: re-exports and module caching; literate lesson sources not adopted
Plenty’s next language phase, a design reviewPer topic in its outcome tableB13, B23, and the extensions it lists
Python slice syntax, with its implementation planSlicesB39: borrowed views &xs[a:b] (issue #581); issue #583
Checked arithmetic blocksNumbersChecked shifts and casts unscheduled; class patterns landed with issue #558
Rust compiler boundaries and measured performance — steps 1–6 and legacy-syntax removalFrontend, cacheB17 compilation session; B19 module replay and frontend candidates

Positive and NonNegative integer domain types

Status: Implemented. The language types are recorded in B95 and the std migration in B95 std limits. The numbers reference defines the types and the std conventions the parameter rule. What shipped differs from the tables below as listed in As shipped; Bounded now has its own implementation under B96, and a compile-time check of fixed ceilings is issue #686.

Problem

A std parameter with a fixed domain, such as a byte limit that must not be negative or a buffer capacity that must be at least one, is checked inside the function and reported as an Err:

pub def parse(text: &str, max_bytes: i64) -> Result[Url, UrlError]   # InvalidLimit for max_bytes < 0
pub def buffered_reader[R](inner: R, capacity: i64) -> Result[BufReader[R], StreamError]   # OutOfRange for capacity <= 0

The failure is a caller bug that the caller could rule out, but every call pays for it: a ?, an error variant to match, and a check on each call. Almost every call passes a literal or a default, so the check can never fail at those sites. The consistency rules (rule 5.3) already say “where a type can state the domain, the parameter uses it”, and NonZero[i64] does this for range and slice steps. No type states “at least zero” or “at least one”.

The migration list covers the public std parameters and Limits fields whose lower bound is 0 or 1 and is checked at run time today.

Summary

QuestionDecision
New typesPositive[T] (value > 0) and NonNegative[T] (value >= 0)
TSigned integer types only: i8, i16, i32, i64
ConstructionInteger literal where one is expected (checked at compile time), or try_from
try_from errorRangeError.OutOfRange, as NonZero.try_from and T.try_from
Reading back.get() returns T; no implicit conversion to T
Arithmetic+ and * keep the type; - returns T; other operators are compile errors
Relation to NonZeroSeparate nominal types; NonZero.from(p) and NonNegative.from(p) widen a Positive
Copy, format, hashAs NonZero: copies, prints as its integer, valid dictionary key
len()Unchanged, returns i64; no implicit conversion
SizesStay i64 (integer types); only the wrapper is new

The types

Element type

Positive[T] and NonNegative[T] take a signed integer type. An unsigned type is a compile error that names the existing spelling, because the domain already has one:

Positive[u64]      # error: every nonzero `u64` is positive; use `NonZero[u64]`
NonNegative[u32]   # error: every `u32` is non-negative; use `u32`

One domain has one spelling. Std quantities are i64 by convention, so Positive[i64] and NonNegative[i64] are the forms std uses; the narrower signed widths exist for user data and cost nothing extra, since the implementation is generic over T.

Positive and NonNegative are builtin names and cannot be redefined, like NonZero.

Construction

The rules are those of NonZero, with a different domain check.

def per_page(items: i64, size: Positive[i64]) -> i64:
    items // size.get()

def main() -> Result[(), IoError | RangeError]:
    print(per_page(100, 20))?          # literal converts at compile time
    print(per_page(100, 0))?           # error: `0` is not a `Positive[i64]`
    configured = 25
    size = Positive.try_from(configured)?
    print(per_page(100, size))?
    limit: NonNegative[i64] = 0        # 0 is in the domain
    bad: NonNegative[i64] = -1         # error: `-1` is not a `NonNegative[i64]`
    Ok(())
  • An unsuffixed integer literal, possibly negated, converts where a Positive[T] or NonNegative[T] is expected: an argument, an annotated binding, a return value, a class constructor argument, a field assignment, a default parameter value, or inside Some(...). A literal outside the domain is a compile error. These are the positions where a NonZero literal converts today; a probe with the current compiler confirmed defaults, returns, constructor arguments, field assignment and Some(3) for an Option[NonZero[i64]] parameter.
  • Positive.try_from(value) returns Result[Positive[T], RangeError], with T taken from value; Positive[T].try_from(value) names T and requires a T. NonNegative.try_from is the same. Out of domain is Err(RangeError.OutOfRange).
  • The constructor is private: Positive(3) is a compile error that names the two conversions.

RangeError is the error every integer narrowing already uses (checked conversion), so one error type covers i32.try_from, NonZero.try_from, Positive.try_from and NonNegative.try_from, and a function that does several returns Result[_, RangeError] without a union. It has one variant, so an exhaustive match needs one arm.

Reading back and mixing with T

.get() returns the T. There is no implicit conversion to T: an operation that mixes Positive[i64] with i64 is a type error whose hint shows .get() or try_from, as for NonZero. Implicit unwrapping would let a domain value silently become an i64 and come back only through try_from, which hides where the check happens.

Arithmetic and comparison

Operands must have the same type. An integer literal beside a domain operand takes that operand’s type, so n + 1 is a NonNegative[i64] and size * 0 is a compile error for a Positive.

OperationPositive[T]NonNegative[T]Why
a + b, a += bPositive[T]NonNegative[T]Sum stays in the domain; overflow traps
a * b, a *= bPositive[T]NonNegative[T]Product stays in the domain; overflow traps
a - bTTCan leave the domain
unary +Positive[T]NonNegative[T]Identity
unary -TTLeaves the domain
//, %, bitwise, shiftsCompile error, suggests .get()SameAs NonZero; a domain divisor is a later addition (decision 4)
==, !=, <, <=, >, >=boolboolSame-type operands or a literal

Overflow in + and * traps, or is an Err in a checked: block, so a sum cannot wrap to a negative value. A comparison with a literal outside the domain, such as size > 0 for a Positive, is a compile error, as step == 0 is for NonZero: the literal is not a Positive. n >= 0 for a NonNegative compiles, since 0 is in the domain.

Relation to NonZero

A Positive[T] is nonzero, but it is not a NonZero[T]: Plenty has no subtyping, and each is a nominal @newtype class. Code that holds a Positive and needs a NonZero (a range step) or a NonNegative (a limit) widens it explicitly:

def chunks(total: i64, size: Positive[i64]) -> Result[(), IoError]:
    for start in range(0, total, NonZero.from(size)):
        print(start)?
    Ok(())

NonZero.from(p) and NonNegative.from(p) take a Positive[T], cannot fail, allocate nothing and compile to nothing. A from that cannot fail beside a try_from that can matches the pair the reader already sees in T.try_from. Widening is explicit (decision 3); see Alternatives.

There is no conversion from NonNegative to NonZero or Positive other than try_from(n.get()), since zero is in its domain.

Copy, format, hash and layout

As NonZero[T]:

  • the value has T’s representation and allocates nothing;
  • it copies, is a valid dictionary key and set element, and is usable as a field, collection element and @copy component;
  • print, f-strings and debug show the integer, and format specs apply as to T.

Both are declared in the builtin arithmetic-errors module beside NonZero (src/frontend/modules.rs), as private @newtype @copy classes with a get() method.

len() and sizes

len() returns i64 and stays so. Making it return NonNegative[i64] would break i < len(xs) with i: i64, range(len(xs)), and len(xs) - 1 used as an index, which are the most common uses of len(). A rule that converts a len() call where a NonNegative[i64] is expected, like a literal, is possible but is not needed now:

  • Allocation sizes are out of scope. list.with_capacity, Bytes.reserve, resize, LruCache(capacity) and std.actor.stage report a negative size as CapacityOverflow (rule 5.1, local PR #804), so with_capacity(len(x)) is unchanged.
  • No Positive parameter can take len(), which can be zero.
  • A search of stdlib/, the tutorial, the reference, examples/ and tests/ found two calls that pass a length expression to a migrated parameter, both inside std with arithmetic (text.plenty builder(self.max_bytes - used_bytes + ...) and regex.plenty self.scan(2 * len(self._names)), which is private). A len()-only rule would cover neither.

Where a length must become a limit, the call is explicit:

mut out = std.text.builder(NonNegative.try_from(2 * len(text))?)?

Inside std, public functions call .get() once and pass i64 to private helpers, so internal arithmetic is unchanged. A len() conversion rule is not part of this design (decision 5).

Migration list

A parameter migrates when its whole static domain is >= 0 or >= 1. The lists below are the public std and builtin parameters with that lower bound that return Err or trap today. P is Positive[i64] and N is NonNegative[i64]; each parameter is i64 today unless stated. Result types are unchanged except where noted, and follow PR #804 where it changes them.

Group A: the invalid-argument failure leaves the error type

Function or classParameters afterRemoved today
std.url.parse(text, max_bytes)max_bytes: NUrlError.InvalidLimit (its only use)
std.db.kv.Store.scan(..., max_rows, ...), scan_all(..., max_rows, ...)max_rows: NKvError.InvalidLimit (its only uses)
std.markdown.Limitsmax_input, max_depth, max_output: NMarkdownFault.InvalidLimits
std.http.body.FormLimitsmax_bytes, max_fields: NBodyError.InvalidLimit, with the two rows below
std.http.body.MultipartLimitsmax_parts, max_head, max_field, max_file, max_body: NSame
std.http.body.read_json(exchange, limits)via std.json.Limits.max_input: NSame
std.actor.net.connection(stream, limits)std.actor.net.Limits input, output: PBytesError.OutOfRange; the result can narrow to AllocError
builtin channel[T](capacity)capacity: P (from u64)ChannelError.InvalidCapacity
builtin ThreadPoolExecutor(max_workers, queue_capacity)both P (from u64)PoolError.InvalidSize
.par(&pool, chunks=N)chunks: PThe par chunks must be positive trap

The two builtins move from u64 to Positive[i64] rather than to NonZero[u64], which removes their listed deviation from the integer-type convention in the same break (decision 2, issue #665).

Group B: the call loses this failure; the error type keeps the variant for other causes

Function or classParameters afterVariant kept for
std.io.buffered_reader(inner, capacity), buffered_writer(inner, capacity)capacity: PStreamError.OutOfRange: stream ranges
std.io.typed.buffered_reader(inner, capacity)capacity: PSame
std.io.BufReader.read_until(delimiter, out, limit)limit: PSame
std.io.read_up_to(reader, out, limit)limit: NSame
std.process.Child.communicate(input, limit), output(limit)limit: NSame
std.csv.reader(inner, dialect, capacity)capacity: PCsvError.OutOfRange: field indexes
std.log.text_sink(writer, max_line)max_line: PLogError.InvalidSpec: specs
std.log.memory_sink(capacity)capacity: NSame
std.fmt.Formatter.floating(value, width), integer(value, width), unsigned(value, base, width)width: NFormatError.InvalidSpec: base, specs
std.text.builder(max_bytes), join(..., max_bytes)max_bytes: NTextError.InvalidLimit: wrapper geometry, below
std.text.hard_wrap(text, width, max_bytes)width: P, max_bytes: NSame
std.text.wrap, fill, shorten, text_wrapperwidth: P, max_bytes, max_input_bytes: NSame
std.text.dedent, indent, indent_withmax_bytes, max_input_bytes: NSame
std.text.TextWrapper fieldswidth: P, tabsize: N, max_lines: Option[P], max_bytes, max_input_bytes: NAn indent wider than width, a placeholder too long
std.fs.copy_from(..., max_bytes, ...), copy_file(..., max_bytes, ...)max_bytes: NFsError.InvalidInput: names, ceilings
std.fs.glob(parent, pattern, max_entries, max_matches, max_steps)all three NSame: pattern length
std.system.System.block_devices, disk_iomax_devices: NSystemError.InvalidInput: max_bytes, pid
std.system.System.process_ids(max_entries)max_entries: NSame
std.archive.zip_reader(inner, max_entries)max_entries: NArchiveError.InvalidInput: names, modes
std.compress.inflater(format, limit), decompress(format, data, limit)limit: NCompressError.OutOfRange: crc32 seed
std.http.encode_chunk_size(length)length: NStreamError.OutOfRange in HttpError
std.http.typed.fixed_body(inner, length)length: NFramingError.OutOfRange: read ranges
std.http.Limits, decoder and clientmax_headers, max_body: NHttpError.InvalidLimit: max_line, max_head >= 2
std.http.server limitsmax_line, max_head, max_headers: P, max_body: NOther Config fields with other bounds
std.http.websocket.Configmax_frame, max_message: PWebSocketError.InvalidConfig: one other mapping
std.actor.http.server configmax_requests: P, limits.max_line, max_head, max_headers: P, max_body: NServerError.InvalidConfig: capacity, heap, buffer minimums
std.actor.net.listener(..., capacity, heap, ...)capacity, heap: PNetError: transport failures
std.actor.net.Connection.reserve(wanted)wanted: NNetError: closed connection
std.actor.tls.over(net, session, input)input: PTlsError.Net: transport failures
std.image.Limitsmax_width, max_height, max_bytes: NImageError.OutOfRange: dimensions
std.stat.quantiles(values, n)n: PStatError.Domain: float parameters
std.math.factorial(n)n: NMathError.Domain; Overflow above 20 stays

Group C: the lower bound is typed; a fixed ceiling remains

Function or classParameters afterCeiling
std.fmt.formatter, stderr_formatter, string_formattermax_bytes: N, max_depth: Nmax_depth <= 64; also scratch.len() >= 32
std.json.Limits, std.toml.Limitsmax_depth, max_input, max_output: Nmax_depth ceiling
std.xml.Limitsmax_token: P, others Nmax_depth <= 1024
std.regex.Limitsall seven Nmax_nesting ceiling
std.template.Limitsmax_context_nodes, max_depth: P, others Nmax_depth <= 64
std.unicode.nfd, nfc, nfkd, nfkcmax_scalars: Ni64.MAX // 4
std.sketch.quantiles(capacity), quantiles_from_bytes(data, max_capacity)P1048576
std.sketch.frequent_items(capacity), frequent_items_from_bytes(data, max_capacity)P65536
std.sketch.xor.build, from_bytesmax_keys: Nkey_limit
std.sketch.cuckoo.cuckoo, from_bytesmax_buckets: Pbucket_limit; initial_buckets must be a power of two
std.fs.TempDir.cleanup_tree, std.fs.remove_treemax_entries, max_depth: Pmax_depth <= 64
std.fs.temporary_dir(parent, prefix, attempts)attempts: P1024
std.tempfile.directory(..., max_entries, max_depth)max_entries, max_depth: Pmax_depth <= 64
std.jwt decoding max_sizePmax_token_size
std.crypto.hkdf_sha256(..., size)size: N8160
std.image.new, from_pixelswidth, height: P2147483647
std.actor.Context.spawn, spawn_isolated, Stage.startlimit: Pmax_heap_limit
std.system.System.process(pid, max_bytes) and the max_bytes of load_average, uptime, cpu_count, memorymax_bytes: P!= i64.MAX; pid != 0

A ceiling keeps an Err variant, so the migration removes the ? from these calls only if the ceiling stops being an Err. Rule 5.3 says an argument outside its documented domain traps and a constant outside it is a compile error; under that rule these ceilings trap, as the run-time domains in PR #804 do, and the variants leave the error types. This design adopts that rule (decision 1): a ceiling traps, and a literal above a constant ceiling is a compile error.

Not migrated

ParameterReason
with_capacity, Bytes.reserve/resize, Heap.reserve, graph reserve, LruCache(capacity), std.actor.stage counts, std.secrets.token_bytes/token_hex/token_urlsafeAllocation sizes: CapacityOverflow (rule 5.1, PR #804)
str.repeat(count), slice boundsClamp (rule 5.2)
std.ready.selector(capacity)Clamps to 1..most_events today; a deviation to fix separately
regex replace limitA negative value means unlimited; a sentinel, not a domain (Option would state it)
Bytes range methods, std.actor.net.Connection.consume/commit/forward, Context.restart, std.image.Image.crop, std.random.Rng.index/sample_indices, csv/sqlite/watch indexesThe upper bound depends on the receiver or another argument; one trap covers both ends (PR #804)
std.http max_line, max_head (>= 2), std.sketch.hyperloglog precision (4..18), encode_png level (0..9), actor server capacity, heap_limit, buffer minimumsLower bound other than 0 or 1; needs Bounded
std.compress.crc32 seed (0..u32 max), archive modes (0..4095)The parameter should be a narrower type (u32, u16); separate fix
std.regex.find_all/split max_matches/max_pieces, std.http.cookie.cookies limit, std.db max_lengthNot checked today; their negative behaviour needs its own review first

As shipped

Differences from the migration list, recorded after the migration (2026-11-02):

  • Shared HTTP limits. std.http.Limits serves the client, decoder and both servers, and allows max_headers = 0. Its max_headers and max_body are NonNegative[i64]; max_line and max_head (>= 2) stay i64 with their InvalidLimit check, so the server rows’ P typing waits for Bounded (B96).
  • Removed beyond the tables. std.actor.StartError.Limit, SystemError.InvalidInput, CsvError.OutOfRange (field indexes already trapped after PR #804) and BytesError.OutOfRange (its last cause was the actor networking limits).
  • Narrowed results. std.actor.net.connection returns AllocError, so Listener.accept adds AllocError to NetError. listener keeps Result[Listener, NetError].
  • Also migrated. std.actor.tls server and client input sizes (they forward to over), std.system block_devices/disk_io max_bytes, std.compress.decompressor(limit), std.fmt.to_string/to_debug and std.report.to_string limits.
  • Ceilings that trap. As decision 1, plus std.actor.net.listener heap above max_heap_limit. quantiles_from_bytes and frequent_items_from_bytes had no ceiling and gained none. The compile-time rejection of a literal or module constant above a ceiling is implemented with std-only declaration metadata (issue #686); dynamic inputs retain the runtime guards.
  • Constants. A module constant could not have a domain type, so std passed constant limits through private getters such as def max_url() -> NonNegative[i64]:. Issue #685 later added domain constants (const max_url: NonNegative[i64] = 8192) and replaced the getters.
  • Internal lengths. Two std callers of encode_chunk_size convert a len() result with NonNegative.try_from(n).unwrap(), which cannot fail (decision 5).

Example

Before:

def lines(path: &Path) -> Result[list[str], AppError]:
    file = open_file(path)?
    mut reader = std.io.buffered_reader(file, 65536).map_error(AppError.Stream)?
    mut out = std.text.builder(1048576).map_error(AppError.Text)?
    ...

After, with literals: the two map_error adapters for impossible failures go, and the builder’s remaining failure is allocation.

def lines(path: &Path) -> Result[list[str], AppError]:
    file = open_file(path)?
    mut reader = std.io.buffered_reader(file, 65536)
    mut out = std.text.builder(1048576)?
    ...

buffered_reader can still fail to allocate its buffer, so it keeps a Result; whether that result narrows to AllocError follows rule 5. With a configured value, the check moves to where the value is read:

def configured(settings: &Settings) -> Result[Positive[i64], AppError]:
    Positive.try_from(settings.buffer_size).map_error(AppError.Config)

Alternatives

  • Bounded[T, lo, hi]. States every domain in the tables, including ceilings and >= 2. Initially deferred in favor of two fixed domains; bounded integers now implement this as a separate type using B83 value parameters. The arithmetic rules would depend on the bounds: Bounded[i64, 1, 64] + 1 has no simple result type. Bounded therefore uses explicit .get() for arithmetic; the two fixed domains keep their existing operations.
  • Unsigned parameters. max_bytes: u64 rules out negatives without a new type, but breaks the i64 convention for sizes, forces casts at every len() and index, and u64 cannot state >= 1. Rejected.
  • Plain std classes with try_from only. No compiler change, but no literal conversion, so every call writes Positive.try_from(64)?, which is the ? this proposal removes. Rejected.
  • Implicit widening to NonZero and NonNegative. Saves NonZero.from at range steps. It adds a third way for a domain value to appear where one is expected (literal, try_from, widening), and is subtyping in all but name. Rejected (decision 3).
  • Implicit unwrapping to T. size + offset with offset: i64 would work, but the domain value disappears silently and comes back only through try_from. Rejected, as for NonZero.
  • Keep run-time checks, document them. Leaves the ? and the variant on every call. Rejected by rule 5.

Cost and compatibility

  • Compiler. NonZero is 247 lines in src/frontend/nonzero.rs, plus hooks in operators, f-strings, metadata and diagnostics (116 mentions in 22 files under src/). Generalizing it to a domain enum (NonZero, Positive, NonNegative) with a per-domain literal check, operator table and diagnostic is the expected shape, not three copies. The signed-only bound is a new check: IntType admits all eight integer types. Codegen needs no change, since the representation is T.
  • Breaking changes. Every migrated parameter rejects a plain i64 variable; literals and defaults keep compiling. A call passing a variable adds Positive.try_from(x)? or NonNegative.try_from(x)?, and the diagnostic shows that form. Code that matches a removed variant (UrlError.InvalidLimit, KvError.InvalidLimit, MarkdownFault.InvalidLimits, BodyError.InvalidLimit, ChannelError.InvalidCapacity, PoolError.InvalidSize) gets an unknown variant error. Code that constructs or mutates a Limits with a variable converts it; code that reads a field calls .get().
  • Std internals. Public functions call .get() and pass i64 to private helpers, so most of the change is at signatures and in valid functions, which shrink or disappear.
  • Docs. The numbers reference gains the two types next to NonZero; each module page in the tables updates its signature, error and limit rows; conventions.md “Invalid arguments” and “Limits” state that a fixed lower bound is a type; the cheat sheet, tutorial lessons that pass computed limits, the plenty skill and signatures.md follow.
  • Performance. No change to representation or allocation; each migrated call drops one comparison and branch.

Phasing

  1. Language: Positive[T] and NonNegative[T], literal conversion, try_from, .get(), operators, NonZero.from/NonNegative.from, formatting and hashing, with the numbers reference and cheat sheet.
  2. Group A, which removes variants and needs the largest doc change, with the builtins channel, ThreadPoolExecutor and par(chunks).
  3. Group B, module by module; each module’s page and tests change together.
  4. Group C, whose ceilings change from Err to traps in the same step, so each module changes once.
  5. Delivered in issue #664: // and % with a plain integer dividend and a NonZero or Positive divisor, without the zero check. See nonzero arithmetic.

PR #804 merges first; this work changes the same std error types.

Decisions

Decided 2026-11-01, on review of local PR #812.

  1. Group C ceilings trap under rule 5.3. A literal above a constant ceiling is a compile error, and the ceiling’s Err variant leaves the error type.
  2. channel and ThreadPoolExecutor take Positive[i64], not NonZero[u64], which removes their u64 deviation from the integer-type convention. Tracked as issue #665, in phase 2.
  3. Widening is explicit: NonZero.from(p) and NonNegative.from(p). There is no implicit widening.
  4. Division by a domain divisor is delivered (issue #664). T // NonZero[T], T % NonZero[T], T // Positive[T] and T % Positive[T] return T without a zero check, including augmented assignment. A domain dividend still requires .get(); Positive also rules out the signed MIN // -1 overflow case.
  5. len() does not convert. It stays i64, and a length becomes a limit through NonNegative.try_from. This can be reopened if call sites show friction.
  6. Unsigned forms are rejected. Positive[u64] is a compile error that points to NonZero[u64], and NonNegative[u32] one that points to u32.

Single-file scripts with embedded manifests

Status: Implemented. Delivered under B17: plenty script, optional embedded manifests, and reusable script builds. The script reference describes current contracts and platform limits; this page records the design rationale. Ordinary single-file compilation and package resolution already exist. This proposal preserves native compilation and explicit main. Examples involving external dependencies are illustrative rather than standalone lessons.

Motivation and precedent

The separate language editions proposal considers an optional package edition that could also appear in this header. Neither proposal depends on implementing the other.

A small utility should be one file that can be copied, inspected, marked executable, and run. Requiring a directory, manifest and wrapper just to use a dependency discourages that workflow. The same file should remain ordinary Plenty, with static checking and the normal ownership and error rules.

The motivating example is the local ~/recipes/wtnew.rs: a worktree utility with a cargo +nightly -Zscript shebang and a ---cargo TOML dependency header. Its Git operations are application logic, not build steps; Plenty does not need a special command-execution language for this workflow.

Rust’s experimental RFC 3424 motivates optional manifests, reuse of package workflows, independence from surrounding workspaces, and diagnostics in the original file. Its examples use doc-comment fences. The Cargo Book’s experimental script documentation instead shows ---cargo frontmatter, as does the local example. These are design precedents, not a settled syntax specification for Plenty.

The user-facing shape

The explicit command is plenty script. A Unix executable script starts with:

#!/usr/bin/env -S plenty script

def main() -> Result[(), IoError]:
    print("hello from a script")?
    Ok(())
chmod +x hello.plenty
./hello.plenty
plenty script hello.plenty

No manifest is needed for the embedded standard library. The shebang is optional when invoking the command explicitly. env -S passes the subcommand; where it is unavailable, invoke plenty script directly. The compiler and supported native linker environment must still be installed.

For package metadata and dependencies, add a header after the optional shebang:

#!/usr/bin/env -S plenty script
---plenty
manifest-version = 1

[package]
name = "worktree_tool"
version = "0.1.0"
description = "A small repository helper"

[dependencies]
util = { path = "../util", version = "^0.1.0" }
---

import util

def main() -> Result[(), IoError]:
    print("helper loaded")?
    Ok(())

Here ../util is an existing Plenty library package with its own plenty.toml, not a bundled module. Git dependencies use the existing git plus tag or rev forms. This introduces neither a registry nor version-only lookup.

Invocation and arguments

plenty script [OPTIONS] [--] FILE [ARGS...]
plenty script --offline tool.plenty --help
plenty script --check tool.plenty
plenty script --compile -o ./tool tool.plenty

Runner options precede FILE. Everything after it goes unchanged to the program, including --help, --release, and a literal --. The optional -- before FILE permits filenames beginning with -. Thus ./tool.plenty --help asks the program for help; plenty script --help asks the runner. Preserve native argument bytes, matching args_bytes.

Default compile-and-run, --check, and --compile -o OUT share resolution and diagnostics. Check and compile modes reject program arguments and never run the application. --release, --offline, --locked, and --verbose retain their existing meanings where applicable. The default profile is debug. Build status goes to stderr; a fresh cached run is quiet.

Inherit the caller’s working directory, environment and streams. Expose the invoked script path as argument zero rather than the cached executable; retained binaries use normal executable argument zero. Preserve existing exit and signal-status behavior. Do not change directories to resolve imports or interpret the shebang as shell code.

Manifest and source rules

Reuse the strict TOML subset and dependency schema. A script adapter supplies defaults without weakening ordinary plenty.toml validation. An absent header means an empty manifest; absent manifest-version means 1. [package] is optional: default version 0.0.0, and derive a lowercase name from the file stem, replacing invalid characters with _ and prefixing script_ when needed for a valid nonreserved identifier. Explicit names follow package validation. Support description, authors, license and minimum plenty-version; do not automatically install or switch compilers.

There is exactly one executable target: this file. Reject [workspace], [lib], [[bin]], unsupported tables and unknown keys. Do not infer neighboring targets or join an enclosing workspace. Keep plenty FILE and package commands unchanged; plentyc remains a compiler, without dependency fetching.

Recognize one ---plenty opener and one --- closer before any declarations, allowing blank lines before the header. Unterminated, duplicate or misplaced headers are errors. Preserve original filenames, byte spans and line numbers for both TOML and Plenty diagnostics, including CRLF files. Never silently ignore a malformed header. Imported modules cannot carry script manifests; package metadata belongs to the selected root.

Resolve the script’s physical path once. Its parent is the module root and base for relative dependencies; a symlink launcher refers to resources beside its target. Permit sibling-module imports, but do not discover dependencies from the caller’s directory or a nearby plenty.toml. Normal alias-conflict and cycle rules apply. Sibling modules and path dependencies make a script convenient locally but no longer self-contained when copied elsewhere.

Storage, locking, and reproducibility

Keep generated files out of the script directory, including when it is read-only. Use private per-user storage with a namespace derived from the resolved script path; equal basenames must not share locks or outputs. This needs explicit storage roots in the package layer, rather than a generated project that loses the original source paths.

Recommend a cached dependency lock, reused across source edits. Dependency changes reconcile it using normal resolution rules. --locked refuses to create or change it, before fetching. --offline uses only cached sources. With no dependencies, an empty graph needs no lockfile and works with both flags on its first run.

A cached lock is not a portable reproducibility guarantee: deleting it or copying/moving the script can resolve a moved tag differently. Recommend an explicit --lockfile PATH for a durable, shareable lock. The option path is relative to the caller, while paths recorded inside the lock are relative to the script directory. The same locked/offline rules apply. Full Git revisions alone do not pin mutable tags in transitive dependencies; path dependencies remain live trees. Durable locks must not be removed by cache cleanup.

Executable reuse validates script/import contents, dependency graph, compiler/runtime identity, target, profile and linker inputs. Object/function caches help but do not themselves remove repeated linking. Plain plenty FILE already has executable reuse on supported Linux toolchains; embedded-dependency scripts must extend that contract. Concurrent launches need per-entry locking and atomic publication: never execute partial or failed builds, or replace a running executable in place. Include script outputs in cache inspection and cleanup.

Boundaries and alternatives

This is native compile-and-run convenience, not an interpreter, REPL, implicit top-level execution, new error policy or sandbox. Dependencies carry the same trust implications as ordinary package builds. No expression mode, stdin source, executable header hooks, library publication, or automatic header rewriting is included. Running a script never edits its source.

A #-commented TOML block avoids new source framing, but older compilers could silently ignore it. Raw frontmatter makes metadata explicit and older compilers reject it; the cost is a shared header reader and editor support. Implicitly enhancing plenty FILE saves a word but mixes the existing compiler shortcut with dependency resolution and persistent artifacts. Always placing locks beside scripts improves discoverability but breaks read-only directories.

To grow into a package, move the body to src/main.plenty and the header to plenty.toml, filling required fields and adjusting relative dependency paths. No language rewrite is needed. Existing module and entry-point rationale is recorded in Explicit applications and modules.

Original implementation assessment and acceptance

This is a moderate toolchain change, not a borrow-checker or code-generation project. A shebang-only prototype is small: at daf964d1, a real #!/usr/bin/env plenty program compiled and printed through plenty FILE; adding --help after its filename was rejected. The full proposal needs manifest adaptation, argument forwarding, isolated resolution/storage, source mapping and executable reuse, rather than just a CLI wrapper.

Likely boundaries are src/main.rs, src/compiler_cli.rs, src/project.rs, src/project/{manifest,toml,git,fingerprint}.rs, and the frontend source loader/LSP. Reuse package resolution behind explicit source/storage inputs. An unresolved header in plentyc should direct users to plenty script, not fetch dependencies.

Acceptance requires real executable-shebang tests, argument fidelity (spaces, empty and non-UTF-8 arguments), status propagation, original-source diagnostics, stdlib/path/Git dependencies, offline/locked failures, read-only directories, symlink launchers, surrounding-workspace independence, concurrent launches, and invalidation after source/dependency/compiler changes. Verify that warm runs skip compilation and linking, and failed builds never run an old binary.

The implementation settles storage under the existing Plenty cache, uses the standalone header delimiters shown here, and includes durable --lockfile support. A single-file test command needs its own entry-point/discovery design: existing package tests reject a test file mixing main and test_ functions. It is not implicitly solved here. Remaining toolchain/platform scope stays in B17/B18.

Library-owned operations and backend hooks

Status: Investigation complete. Issue #145 recommends an API design using existing composition, with capability parameters where needed; new protocol visibility is deferred. No new language feature or stdlib conversion is approved. The current contracts remain protocols, visibility, and classes. Any future access extension belongs to B29.

The later protocol-body proposal records agreed default/final methods and member-specific private-hook grants. That design is not implemented; the wrappers and capabilities here remain the existing-language solution. Its short programs contrast the proposed programming model with this assessment’s composition approach.

Introduction

A library sometimes needs to let other packages customize one step of an operation while keeping the surrounding algorithm under the library’s control. That algorithm may validate inputs, check state, acquire resources, and record success or failure. The customizable step is an implementation hook: it does the backend-specific work, but is not intended to be an independent application entry point.

For example, a library author provides Transport.send(message), which checks the message-size limit and connection state, asks a backend to send the message, and updates accounting after success. A backend author supplies the TCP-specific send_impl(message) hook, or an in-memory version for tests. An application author selects a backend and calls transport.send(message). These may be three different people working in three different packages.

Two mistakes should be prevented through that library interface. Application code should not accidentally call send_impl directly and skip validation or accounting. A backend’s customization should not replace the surrounding send algorithm and omit those checks either. Supplying the hook and calling the hook are therefore different permissions. A similar situation arises when Store.save(record) must validate a record and manage a transaction around a backend-specific write: application code should use save, and each backend should not have to reproduce the transaction algorithm.

C++ addresses this with the non-virtual interface (NVI) pattern: a public non-virtual method owns the algorithm and calls a private virtual hook. A backend class can override that hook, while application code using the base interface cannot call it directly. The public algorithm is not overridden through that interface. This is about separating customization from access, rather than sealing an interface against external implementations.

Rust traits do not directly offer that combination: individual trait methods cannot have independent private/public visibility, and a default method body can be replaced by an implementation. Rust can express the intent using an owning wrapper with a private backend field; a capability argument can further restrict calls to an otherwise public hook. The precedents explain and test these workarounds. The question for Plenty is whether existing composition provides clear enough guarantees, or whether protocols need a new way to permit external hook implementations without exposing ordinary calls to those hooks.

The aim is to prevent accidental bypass through the library abstraction, not to prove that a backend performs its work honestly. A backend can still expose its own separate API, and validation, partial failure and transaction recovery need explicit contracts regardless of method visibility.

Decision and roles

Use an ordinary owning wrapper for the public algorithm. Keep its backend field private. Use a public backend protocol when external implementations are required. If normal application code must also be unable to invoke a hook on a separately obtained backend, require a library-issued capability argument. Defer new protocol access or implementation syntax: the tested examples do not establish a missing guarantee that warrants that complexity.

This is the intent of C++’s non-virtual interface (NVI) pattern, not sealing. Precedents and probes explain the distinction and the Rust workarounds.

RoleTransport responsibilityPermitted customization
Library authorOwns Transport[B].send, validation, connection state and successful-send accountingChanges the algorithm and hook contract
Backend authorSupplies Memory.send_impl or TCP-specific work, possibly in another packageImplements the hook; does not replace Transport[B].send
Application authorConstructs the selected backend/wrapper and calls transport.send(message)Selects/configures a backend; cannot access the wrapper’s private field

One person may fill all three roles; their modules still have different access. The guarantees concern use through this abstraction. A backend author can expose another API on their concrete type or intentionally violate the hook contract. Neither NVI nor a token proves honest transmission, storage, or accounting.

Existing-language baseline

The complete three-module example executes with today’s compiler. The library author moves an externally defined public backend into Transport[B], keeps _backend, _limit, _open and _sent private, and implements one public inherent send method. A private bounded helper checks the backend protocol; class declarations do not yet support protocol bounds themselves.

send checks the open state and UTF-8 byte limit before invoking the backend. Only Ok increments the successful-message count. Err leaves the count unchanged and closes the wrapper: the backend may already have emitted bytes, and a message boundary is no longer assumed trustworthy. The toy error does not report a resumable offset. A production transport must either specify an all-or-nothing enqueue contract or preserve partial progress and recovery state, as std.io does. Hidden hooks do not resolve this choice.

There is no inheritance or protocol default body here. send_impl is absent from Transport[B], and the application cannot reach _backend. A backend method also named send cannot replace the wrapper’s method. No fixed or final annotation is needed. Public functions accepting a structural Sendable instead of the nominal wrapper would allow different algorithms; choose that contract only when such substitution is intended.

The remaining difference from private virtual hooks is precise: an application can construct a separate Memory and call its public send_impl. It cannot use that value to access the stored backend owner. Separate handles can still alias the same socket, database or other external resource; field privacy is not proof of exclusive access to that resource. The wrapper’s contract must account for permitted sharing. A library that only promises wrapper-local invariants can stop here.

Capability variant

The example also shows a public Permit type with a private constructor, passed as &Permit to the public hook. The library creates it only after validation. An external backend author can name Permit and implement the protocol; the application author cannot manufacture it through either constructor spelling. This separates the ability to supply the operation from possession of the authority needed to invoke it. The hook remains publicly nameable and callable by code that already has a permit; it is not a private method.

A borrowed argument avoids transferring ownership of the permit. Current Plenty disallows stored reference fields, and the hook’s result contains no returned borrow. However, ordinary classes support explicit copy, including their private fields. The tested variant adds a no-op private __del__ so copy(permit) is rejected under today’s custom-destruction rule. That is workaround boilerplate, not a proposed long-term spelling for noncopyable types. The compiler’s borrow rules also forbid escaping a captured borrowed permit; there is no permission to store a reference merely because it is in a closure.

A permit gates possession during the operation. It does not bind authority to one receiver or message, enforce exactly one nested call, or prevent deliberate delegation to another backend while the borrow is live. The library must not publish a permit factory/accessor, accept an application callback with a permit, or return a permit in an error if it wants this boundary. Token-bearing hooks also become part of the public implementor contract even when omitted from application documentation.

For transaction-sensitive hooks, use a transaction-specific authority rather than treating this generic send permit as proof of an active transaction. Represent the connection/transaction identity and allowed operations explicitly; a dummy token proves only possession.

Storage and transaction ownership

For Store.save(record), the library author owns validation and the transaction algorithm, a SQLite backend author supplies storage-specific operations, and the application author calls save. An in-memory backend follows the same contract. A plausible algorithm is:

validate record
acquire transaction (failure: no hook, no commit)
call storage hook with transaction context
on hook success: explicitly commit
on hook failure: explicitly roll back and retain both errors if rollback fails
on commit failure: follow the documented retry/rollback policy

This is an algorithm sketch, not new Plenty syntax or a working generic SQLite adapter. Beginning, committing and rolling back are themselves backend-specific; a single hidden write_impl does not make them backend-independent. A concrete transaction owner or backend protocol must supply those steps too.

The real SQLite transaction remains open after a busy commit so it can be retried, and rolls back when dropped unfinished. Choose explicitly whether a library wrapper retains that owner for retry or rolls back before returning an error. Never report commit success merely because the write hook succeeded. A borrowed transaction context can restrict writing to the transaction lifetime, but must also carry meaningful backend authority.

Resource scopes supply cleanup on ordinary exits; __exit__ returns unit and has no success/failure argument. It cannot replace explicit fallible commit/rollback handling or infer that a body succeeded. Traps do not unwind it. Coarse distinct transaction/state owners may enforce sequencing better than method hiding; see typestate. Ownership transfers, retryable failure and extraction of affine fields still need an API design.

What the stdlib actually needs

Current sourceAlgorithm and extension boundaryAssessment
stdlib/std/io.plenty: Reader, Writer, read_exact, write_allPublic one-attempt operations; free helpers add retry, bounds/progress validation and partial-failure policyPartial operations are useful application APIs. Do not make every stream operation a hidden hook.
stdlib/std/http.plenty: BodyReader.read_body, private stepWrapper checks ranges, decodes framing and poisons its state on failure; external streams supply Reader.readComposition already protects this algorithm. step is a library-owned shared helper, not an external hook.
stdlib/std/io.plenty: BufReader.innerPublic underlying owner intentionally permits closing/replacement, with a warning about bypassing buffered bytesChoosing public escape access is an API decision. An access feature cannot repair a deliberately exposed field.
stdlib/std/db/sqlite.plenty: TransactionConcrete transaction owner with commit/rollback lifecycleStrong motivation for explicit transaction authority and failure semantics, not evidence for private protocol hooks.

These sources were inspected at 5346f85. No stdlib conversion follows from this investigation.

Alternative: explicit protocol access grants

If later evidence requires lexically hidden external hooks, the narrowest candidate would keep the ordinary wrapper and add an explicit mapping from a backend’s private method to a public protocol requirement. Only the backend type’s defining module could issue that grant, for a named protocol identity and concrete arguments; the library could not grant itself access by declaring a matching signature. This is a semantic candidate, not proposed syntax.

A coherent version must confine invocation to the protocol’s owning module, not every generic function declaring the bound. Otherwise an application could write its own bounded trampoline and regain access. The access is to the listed method only, never other private members or the whole backend module; aliases, imports and specialization must not broaden it. Ordinary accessible methods would keep structural matching. A private requirement would need the explicit grant, and the same method call in an unauthorized module must fail even if conformance is known. Hidden type names in public signatures remain prohibited; do not silently admit private concrete backend types as a side effect.

This creates two conformance/access paths and provenance that specialization currently erases. It also couples a backend to a library protocol identity and requires coherent duplicate/missing/signature-mismatched grant diagnostics. Inference must check authority before using a private signature as evidence; it must never search private methods or guess an implementation. A useful error must identify the requiring call, protocol, candidate method and absent grant, without simply suggesting pub where hidden access was intended.

No inheritance, runtime protocol object, default method body or override system is needed. Their addition would increase scope without fixing access. Generic specialization can preserve today’s static dispatch, borrowing and allocation costs, but new access provenance would affect checking, caching and diagnostics. That cost is not justified by the current examples.

Tradeoffs and remaining evidence

The plain wrapper adds one owner and a small bounded delegation helper, keeps normal structural inference and requires no runtime allocation. A permit adds one public type and hook parameter, private construction and the current copy barrier; it makes capability errors visible at constructor/arity checks. Both retain ordinary exclusive backend borrows and Result-based cleanup.

Protocol grants could remove token plumbing and keep hook names off application call surfaces, but introduce a new access relation and implementor obligations. Neither approach proves semantic hook correctness. Choose a token when possession or transaction context is useful in its own right; use plain composition when the wrapper owns the only backend application code needs.

The decision against a new feature can be revisited with an external-backend API whose real invariant fails under the baseline, or measured token/delegation friction that exceeds the cost of grants. Identity-scoped transaction capabilities, noncopyable declarations, diagnostic ergonomics and compile-time measurements remain unanswered design questions, not scheduled implementation work.

Transport wrapper and capability examples

Status: Executable design example; not a stdlib API. These experiments use existing Plenty, supporting the assessment. They do not add stdlib types or change the language.

Baseline: three authors, three modules

Save the following files beside each other and run plentyc main.plenty. The library author owns library.plenty; the backend author owns backend.plenty; the application author owns main.plenty. Memory stands in for an external backend with success and failure outcomes. A TCP backend would need the partial-progress/error contract described in the assessment.

library.plenty:

pub enum TransportError:
    Closed
    TooLarge
    Backend

pub protocol Backend:
    def send_impl(self: &mut Backend, message: &str) -> Result[(), TransportError]:
        pass

pub class Transport[B]:
    _backend: B
    _limit: i64
    _open: bool
    _sent: i64

    pub def send(self: &mut Transport[B], message: &str) -> Result[(), TransportError]:
        if not self._open:
            return Err(TransportError.Closed)
        if message.byte_len() > self._limit:
            return Err(TransportError.TooLarge)
        match deliver(&mut self._backend, message):
            case Ok(_):
                pass
            case Err(cause):
                self._open = False
                return Err(cause)
        self._sent += 1
        Ok(())

    pub def sent(self) -> i64:
        self._sent

    pub def close(self: &mut Transport[B]) -> ():
        self._open = False

def deliver[B: Backend](backend: &mut B, message: &str) -> Result[(), TransportError]:
    backend.send_impl(message)

pub def transport[B: Backend](backend: B, limit: i64) -> Transport[B]:
    Transport(backend, limit, True, 0)

backend.plenty:

from library import TransportError

pub class Memory:
    pub def send_impl(self: &mut Memory, message: &str) -> Result[(), TransportError]:
        if message == "fail":
            return Err(TransportError.Backend)
        Ok(())

main.plenty:

from library import transport
from backend import Memory

def main() -> Result[(), Failure]:
    mut t = transport(Memory(), 4)
    print(str.repr(t.send("ok"))?)?
    print(str.repr(t.send("large"))?)?
    print(str.repr(t.send("fail"))?)?
    print(t.sent())?
    t.close()
    print(str.repr(t.send("ok"))?)?
    Ok(())

The calls produce success, TooLarge, Backend, count 1, and Closed. The application deliberately uses str.repr to show Result wrappers. The rejected oversized message never reaches the hook; a backend error poisons the transport and never increments the count. close is also an explicit operation.

transport[B: Backend] validates every requirement when constructed. The private deliver[B: Backend] helper supplies the bound for delegation from an otherwise unconstrained generic class method. This is existing Plenty; class Transport[B: Backend] is not available today.

An application cannot call t.send_impl (no such method) or t._backend (private field). It can construct a separate Memory and call its public send_impl directly. That separate value is not the owner moved into t, but separate handles could still alias the same external resource. Private wrapper fields do not establish exclusive access to the underlying socket or database.

Capability variant: authorize calls as well as implementations

Keep all three files above. In library.plenty, add this type before the protocol:

pub class Permit:
    _issued: bool

    def __del__(self: &mut Permit) -> ():
        pass

The generated field constructor is private because _issued is private. The destructor is deliberately empty: custom destruction disables structural copy. It is an awkward copy barrier, not cleanup logic or a proposed language idiom. The flag’s value is irrelevant; nominal type identity and private construction carry the authority.

Replace the protocol and private helper on the same page with:

pub protocol Backend:
    def send_impl(self: &mut Backend, message: &str, permit: &Permit) -> Result[(), TransportError]:
        pass

def deliver[B: Backend](backend: &mut B, message: &str) -> Result[(), TransportError]:
    permit = Permit(True)
    backend.send_impl(message, &permit)

In backend.plenty, import Permit alongside TransportError and add permit: &Permit after message: &str in Memory.send_impl. Its body can stay unchanged. The application file stays unchanged, and produces the same outcomes. All companion code for these replacements is the baseline on this page.

The external backend author can name the public type without constructing it. Ordinary application calls without a permit fail arity checking; both Permit(True) and Permit.new(True) fail private-constructor checking. Attempting saved = copy(permit).unwrap() in the backend fails with this resource cannot be copied. Removing the destructor makes that explicit copy legal, despite the private flag. This matters if the backend should be unable to retain an issued capability accidentally.

A hook can still invoke another hook with the borrowed permit during its call. The example does not prove exactly-once invocation, receiver/message identity, transaction sequencing, or correct external effects. Application code that is explicitly handed a permit also gains call authority. The library must choose which functions receive it.

Probe results

The compiler was freshly built from 5346f85 with cargo build --locked --offline --bin plentyc in the issue worktree, using a separate target directory. Both variants compiled and ran natively. These were focused manual probes, not additions to the tutorial’s executed-fence suite. Proposal fences intentionally use text.

Change to the exampleObserved result
Call t.send_impl("ok")Rejected: unknown method on Transport
Access t._backendRejected: private member
Make Memory.send_impl privateRejected at construction requiring the protocol; a matching name does not grant access
Make Memory privateRejected importing the class; a public method does not export its class
Call a separately constructed baseline backend directlyAccepted
Omit capability argument on a separate backendRejected: hook expects two arguments
Construct capability through either constructor spellingRejected: private Permit.__new__
Explicitly copy borrowed capabilityRejected with custom destruction; accepted without it

The source also confirms lexical access in src/frontend/protocols.rs (check calls check_member_at) and in protocol inference in src/frontend/generics.rs. Existing tests/test_modules.rs covers private classes/methods, transitive generic calls and definition-module access. The generic caller does not lend its access to the library’s helper.

C++ and Rust precedents

Status: investigation evidence. Supports the library-owned operations assessment, not a change to Plenty’s protocol semantics. Sources and probes checked 2026-10-17.

C++: override and call access are separate

The library author can write:

class Transport {
public:
    void send() { validate(); send_impl(); record_success(); }
    virtual ~Transport() = default;
private:
    virtual void send_impl() = 0;
    void validate();
    void record_success();
};

The backend author overrides send_impl privately in a derived class. An application author calls send through Transport&. The wrapper statically selects the library’s algorithm, which dynamically invokes the backend hook. Access does not participate in deciding whether one virtual function overrides another; call accessibility is checked separately using the static expression type. C++ working draft: virtual functions, access to virtual functions.

A derived class can declare another send that hides the base name, or expose another public operation. Calling through the derived type can therefore select another API; NVI does not prevent that. The guarantee concerns calls through the base abstraction. In the focused probe, counters showed that Transport&.send used the wrapper and the private override, while Memory.send selected the derived hiding method. Direct calls to the private hook through either type were rejected. The probe compiled and ran with GCC 16.2.1, C++20 and -Wall -Wextra -Werror.

The issue’s Standard C++ FAQ link returned HTTP 403 during this investigation; the accessible working draft and compiled probe supplied the evidence instead. Exceptions/partial external effects still need a library-specific contract; this minimal example omits them.

Rust: composition supplies the fixed operation

The Rust Reference on traits specifies replaceable defaults and rejects visibility annotations on individual trait items. A public trait with default send and required send_impl therefore does not express either hidden hook access or a fixed algorithm. This is a constraint on that spelling, not on Rust’s ability to express the intent.

A library author can instead write this public surface, with private fields:

#![allow(unused)]
fn main() {
pub struct Permit { _private: () }
pub trait Backend {
    fn send_impl(&mut self, message: &str, permit: &Permit) -> Result<(), ()>;
}
pub struct Transport<B> { backend: B, sent: usize }

impl<B: Backend> Transport<B> {
    pub fn send(&mut self, message: &str) -> Result<(), ()> {
        if message.len() > 4 { return Err(()); }
        self.backend.send_impl(message, &Permit { _private: () })?;
        self.sent += 1;
        Ok(())
    }
}
}

A public constructor implemented in the library moves B into the wrapper. An external backend author implements Backend and names Permit, without creating it. An application author obtains Transport<B> and calls its inherent send. This excerpt isolates the mechanism; the complete Plenty example also specifies connection state and poisoning after backend failure.

Without the token, a separate backend value’s trait hook is callable. With it, the hook still has public visibility, but callers need a value they cannot construct. Permit exposes no constructor, Default, Copy or Clone. The borrowed permit cannot be retained as an owned token by safe code; storing its reference requires a compatible lifetime that a callback-local permit does not provide. This does not restrict external trait implementations. Rust’s privacy rules make public trait items accessible but preserve private struct fields.

This adds a wrapper, inherent implementation/bound, delegation and one public token type/parameter. Separate crates need the extension trait and its argument types exposed as public contracts. A private implementation trait is simpler when all backends belong to the library, but removes external extensibility. Documentation hiding alone supplies no call restriction.

Sealing does not hide callable hooks

The Rust API Guidelines describe sealing to restrict downstream implementations. That is a different permission from calling a hook. In a separate-crate probe, the library put pub trait Hook in a private module, made public Api inherit it, and implemented both for Concrete. An application still compiled this generic call:

#![allow(unused)]
fn main() {
fn bypass<T: Api>(value: &T) -> i32 { value.hook() }
}

The application did not need to name the hidden supertrait. Bounds provide access to supertrait associated items, as the Rust Reference explains. Thus neither a private module path nor sealing is proof of hidden call access.

Rust probe outcomes

Separate library, backend and application crates compiled with rustc 1.99.0, edition 2021 and -D warnings; the application verified one successful send, size rejection, backend error and a count of one. No unsafe code was used.

Focused probeResult
External backend implements token-bearing traitAccepted
Application constructs private-field tokenE0451
Application calls hook without tokenE0061
Backend tries to clone &Permit into owned PermitE0308: reference cloned; token does not implement Clone
Visibility annotation on trait methodE0449
Implementor replaces a default sendAccepted; executed replacement returned the alternative value
Application’s generic bound calls private-module supertrait hookAccepted and ran

These small probes validate the language mechanisms, not transport reliability or database transaction semantics. They support using composition as the baseline and capability possession as a separate design choice.

Fluent iterator pipelines

Status: Implemented. All four agreed stages shipped. Stages 1 and 2 cover the serial core, pair callbacks, dictionary collection, generator fluent methods, explicit owned .generator() conversion, chain, flat_map, closure-literal splicing and comprehensions lowered through the plan. std.iter was removed rather than deprecated. Stage 3 covers the sum, min and max terminals, resumable adapters, next(p), and lending through &mut Iter[T], including pipelines of reference items. Bounded native substitution of scalar leaf callbacks also covers named functions and immutable scalar captures; it addresses the original performance gap (issue #399). Stage 4 covers .par(&pool) with the terminals below, plus sum, min, max and chunks=N. The pipeline reference lists the supported adapters, terminals, ownership rules and current limits. Current iteration is defined by iteration and generators, closures and ordered parallel mapping. Related backlog entries are B12 (iterator/view protocols, borrowed adapters), B49 (collections; its std.iter part was removed), B22 (broader parallel mapping) and B24 (automatic parallelization). B75 tracks remaining extensions, not the delivered stages. This page preserves the original design; examples may differ from the current reference.

Summary

Add iter(source), which starts a pipeline: a compiler-known, lazily evaluated chain of adapter methods (map, filter, take, …) ending in a loop, a comprehension, or a terminal method (fold, count, collect, …).

total = iter(&payments)
    .filter(def(p: &Payment) -> bool: p.name == "bob")
    .map(def(p: &Payment) -> i64: p.cents)
    .fold(0, def(acc: i64, cents: i64) -> i64: acc + cents)

A pipeline is not a generator and not a library object. The compiler lowers the whole chain at its consumer into one loop, the same loop a hand-written for or comprehension produces, with each callback called directly. Serial pipeline machinery allocates only when a terminal builds a collection, returning Result. A .par(&pool) step moves the rest of the pipeline onto an explicit thread pool, reusing the existing ordered map and fixed-tree reduce contracts.

The surface follows excitertools: one wrapper, chained methods, terminal “sinks”. The cost model follows Rust’s Iterator, but fusion is done by the compiler rather than by monomorphized adapter structs and an optimizer’s inliner, which Plenty does not have.

Starting point before B75

The original design started with iteration but no fluent interface or fused composition. There are no builtin top-level map or filter functions; the reusable adapters are the importable std.iter module.

FacilityFormCost
for over list, set, dict, str, bytes, rangeStatementDirect range source is a counted loop; collections use the loop plan in op.rs
enumerate, zip, reversedLoop/comprehension sources onlyFused into the loop plan; no iterator object. Rejected elsewhere
any, allBuiltin boolean reductionsShort-circuit, no allocation
dictionary.items()Loop intrinsic, two bindingsNo item tuples
Comprehensions[e for x in s if c]One fused loop; output growth returns Result[_, AllocError]
Generators, generator expressionsGenerator[T], inline frameNo heap allocation; each advance is a runtime call (below)
std.itermap, filter, take, fold, try_foldGeneric generator functions; one frame per adapter
std.math.sum_iter, prod_iterBorrowed-iterable foldsPlain loop
pool.map, pool.map_result, pool.reduce_treeExplicit parallel operationsOwned list or range input; eager, ordered

The original std.iter usage was nested calls, generator sources only:

# current
from std.iter import map, filter, fold
total = fold(map(filter(numbers(10), even), square), 0, add)

Gaps

  1. No method chaining. numbers(3).map(sq) reports “unsupported method map”. Built-in types have no user-extensible methods, and protocols deliberately add no methods on import.
  2. Generator sources only. A list must be moved into a generator expression, (n for n in values). There is no adapter over a borrowed collection, since a frame cannot hold a reference (B12/B49 wait on B11 stored references).
  3. No borrowed captures. Callbacks must own their captures; def [&mut seen] is rejected with “pass a borrowing closure by reference”, because the frame that stores the callback can outlive the loan.
  4. Per-item overhead. Every generator advance, including each adapter step, goes through collection_call(24) into the runtime dispatcher, then plenty_generator_resume (reentrancy guard), then an indirect call to the frame’s resume function, which dispatches its state through a linear compare chain and keeps every local in frame memory. Nothing is inlined: the compiler has no function inliner, and Cranelift sees each resume function separately.
  5. No storage. Frames cannot enter class fields, tuples or collections (“generators cannot be stored in generic data”), so a Rust-style adapter struct holding its source cannot be written in Plenty today.
  6. Missing operations. No chain, skip, take_while, flat_map, count, find, collect, or parallel chaining.

Measured cost

Release compiler at 6153b46f, x86-64 Linux, summing (n * n) % 1000 over the even numbers below 100,000,000:

ProgramTimevs. loop
for n in range(N): if n % 2 == 0: total += ...0.31 s1x
Named generator with the filter inside its body, consumed by for0.62 s2x
Named generator yielding every n, filtered in the loop0.91 s3x
fold(map(filter(numbers(N), even), sq), 0, add) with std.iter2.74 s9x
for v in ((n * n) % 1000 for n in range(N) if n % 2 == 0)5.15 s17x

The generator expression is the slowest because its outer range is stored in the frame and then iterated through the runtime, like any stored range (issue #371). A stored range alone, r = range(N) then for n in r, takes 4.71 s. Issue #371 has since been fixed: a stored range now runs as fast as a direct range, and the generator expression drops to about 0.86 s, the cost of one frame resume per element. These numbers set the target: a pipeline should be indistinguishable from the first row.

Design

Pipelines

iter(source) accepts every current for source and follows the same ownership rules:

SourceItem
iter(values) (owned list, set, dict)Owned element / key; consumes the collection
iter(&values)&T; borrows for the pipeline’s lifetime
iter(&mut values)&mut T (lists and dict values only, as today)
iter(range(...))Integer of the range’s type, counted loop
iter(text), iter(&buffer)str scalar, u8
iter(generator)Yielded value; consumes the frame
dictionary.items()Key/value reference pair

enumerate(...), zip(...) and reversed(...) become pipeline starts with the same meaning as iter(...).enumerate() and so on. They keep their current loop behavior, and gain chaining: enumerate(&rows).filter(...). This mirrors excitertools, whose module-level range, zip and friends also return Iter.

A pipeline’s type is Iter[T] for item type T, but like a generator it keeps a concrete identity (its source, adapters and closures), so two pipelines with equal T do not unify. Iter[T] describes the item contract only.

Where a pipeline may appear

One hard rule: an unfinished pipeline never leaves the scope that created it. It may be:

  • a for source or comprehension source;
  • the receiver of a terminal method;
  • bound to a local and used later; it is affine, and consuming it ends it;
  • lent to a function as &mut Iter[T] (next section); the caller keeps ownership and can continue or collect it afterwards;
  • turned into an owned generator with explicit .generator() when items must cross a function boundary, subject to the ownership restrictions below. That conversion is explicit and costs what generators cost today.

It may not be returned, passed by value, stored, captured, or held across yield. Because a pipeline never outlives its scope, it can hold borrowed sources and closures with borrowed captures. This removes gaps 2 and 3 without waiting for stored references (B11).

Generator conversion is an explicit lowering boundary, not permission to store an Iter value. The compiler consumes the plan and transfers its owned source, callback environments and progress into a generator frame. Conversion is allowed only when all of that state meets the existing generator capture, storage and yield rules. Borrowed sources, borrowed callback captures and sub-pipelines lending another pipeline are rejected, even if their yielded items are owned. No loan may cross yield, and no implicit copying makes a borrowed pipeline eligible. The resulting generator may leave the scope; the unfinished pipeline itself may not. Generators themselves expose the same fluent methods: source.map(f) builds a pipeline equivalent to iter(source).map(f), with no frame per adapter. .generator() creates one frame around the complete fused plan. This spelling replaces the originally proposed generator-expression bridge. There is no implicit conversion from an expected Generator[T] type.

Lending a pipeline to a function

Implemented in stage 3; the reference states the final rules and limits. This section keeps the design rationale.

A function can take a pipeline by exclusive reference, consume some of it, and return. The caller then continues from where the callee stopped:

def skip_header(lines: &mut Iter[str]) -> Result[(), CsvError]:
    match next(lines):
        case Some(header):
            check_header(&header)
        case Nothing:
            Err(CsvError.Empty)

def main() -> Result[(), CsvError | AllocError | IoError]:
    mut rows = iter(read_lines(path)?).filter(def(l: &str) -> bool: l != "")
    skip_header(&mut rows)?
    records: list[Record] = rows.map(parse_row).try_collect()?
    ...

Generators already work this way today: &mut Generator[i64] parameters specialize to the caller’s actual frame, and the caller can keep consuming after the call (verified with the current compiler). A pipeline reuses that model.

What the callee may do through &mut Iter[T]:

  • next(p) -> Option[T] takes one item;
  • for x in p: consumes items until the loop ends; after break, the rest stays in the pipeline;
  • a terminal (p.count(), or p.collect() with an expected type) consumes everything that remains;
  • an adapter (p.take(3), p.filter(...)) makes a sub-pipeline that borrows p and must be consumed inside the callee. Its own closures may borrow the callee’s locals, because it ends in the callee.

What it may not do: move, store, or return the pipeline, or add steps that persist after it returns. A reference parameter cannot be stored (B11 is not implemented), so the rule needs no new check: the pipeline stays owned by its creating scope by construction. Loans held by the pipeline’s own closures stay live through the call, as loans held by any lent value do.

Type. The parameter type is &mut Iter[T]. Like &mut Generator[T], it names the item contract and specializes to the concrete pipeline at each call; it is not a generic constraint. &Iter[T] (nothing can be advanced through a shared reference) and by-value Iter[T] parameters and returns are rejected. Iterable[T] was considered and rejected as the name: in Python and Rust “iterable” means something you can start iterating, such as a list, repeatedly. A pipeline is single-pass state, an iterator, and Iter matches both iter() and excitertools’ Iter class. Iterable[T] stays free for a possible later constraint meaning “any source iter() accepts”.

Passing by value and returning it (the alternative) would let a helper add steps: def evens(p: Iter[i64]) -> Iter[i64]: p.filter(even). It is feasible, since generator factories already infer a concrete returned identity, but it needs a return-derivation rule (a function may only return a pipeline derived from a pipeline parameter, never one it created), a rule for loans in the steps it adds, and it weakens the hard rule to “may leave if it comes back”. It is not part of this design; reusable steps are written as functions that return closures, e.g. .filter(longer_than(3)).

Complexity, stated honestly.

  • Resumable state. A pipeline consumed in one place keeps its progress in registers inside one fused loop. A lent pipeline can be consumed in several places, so its progress lives in slots: the source cursor, counters, chain’s phase, flat_map’s inner source, and zip’s pending left item. Each fused loop loads that state on entry and stores it on exit. The compiler sees every lend in the creating function, so pipelines that are never lent skip the store-back. The extra cost is per loop entry and exit, not per item; next(p) expands one step inline, still with no runtime dispatch.
  • Monomorphization. The callee is instantiated once per distinct pipeline type passed to it, as with generators today. Pipeline types include closure identities, so two call sites with different closure literals produce two instances. That is code size and compile time, bounded by the existing specialization limits, and must be measured in compile_bench.
  • Diagnostics. Errors inside an instance name the pipeline by its creation site, as closures are named today.
  • Every adapter must be resumable. Each new adapter specifies its slot state; this is a cost on every future addition, and the reason lending is staged after the serial core.

Implementation decisions. Lending follows option A: the lender passes its state and the borrower rewrites the plan’s operations to use it, writing state back only for lent pipelines, with next(p) inline. Where a choice affected borrow checking, the conservative option was taken:

  • State is passed per slot, not as one record. Each owned state slot is a hidden &mut argument after the declared ones; each reference held as state is a hidden by-value argument. This is equivalent to one record pointer without a new aggregate layout, and it reuses the existing frame specialization, call verification and borrow tracking for reference arguments. The borrower copies owned state into locals at entry and writes it back on every exit, so the lender never observes a half-updated slot.
  • Reference state is read-only. A reference the pipeline holds (a borrowed source, a closure literal’s borrowed capture) is passed by value and may not be reassigned in the borrower; the compiler reports any write. The callee cannot extend or shorten the loan behind it.
  • Caller loans stay with the caller. The plan’s loans are not reconstructed in the borrower, which has no names for them. The lender keeps each loan live across the call and uses it after the call returns, so a conflicting mutation before the pipeline ends is the same borrow error it is without lending. In the borrower, p itself is borrowed exclusively by every loop, next, terminal and sub-pipeline over it, so overlapping uses of p there are ordinary borrow conflicts.
  • Reference items borrow for the call only. A pipeline whose items are references, such as iter(&names).filter_value("ada"), can be lent. In the borrower each reference item has a loan rooted at one of the plan’s hidden parameters, which the borrower only reads and cannot name. An item is therefore usable anywhere in the call, including by local adapters, closures and further lends, but cannot leave it: returning an item, or a reference derived from one, is rejected with “an item of a lent pipeline cannot be returned”. The other routes out already reject references: Option and tuple payloads, class fields, generators, and escaping borrowing closures. Rooting items at the declared &mut Iter parameter instead would let a borrower return an item tied only to that parameter, which can outlive the pipeline’s loan of its source once the lender consumes the pipeline.
  • Limits. A lent pipeline may have at most 64 state slots, which keeps the borrower within the 256-slot frame limit. A pipeline is rejected when its state holds a reference inside another value (a bound closure with borrowed captures), or when it holds a reference into its own owned state: either would let the borrower’s copy alias the lender’s slot.

One borrower instance is compiled per lent plan, shared by recursive and repeated lends. In a probe lending three different pipelines to a one-line skip_one and a recursive drain, the compiler emitted six borrower instances of 360 to 910 bytes each. The compile_bench workload has no pipelines; its AOT median was unchanged (223.7 ms against 225.9 ms and 223.0 ms against 223.9 ms on main, alternating runs).

Adapters (core set)

Each adapter is lazy and consumes the pipeline it is called on.

AdapterCallbackItem out
.map(f)(Item) -> UU
.filter(keep)(&Elem) -> boolItem
.enumerate(start=0)none(i64, Item)
.zip(other)none; other is any pipeline source(Item, Other)
.take(n)noneItem, at most n
.skip(n)noneItem after the first n
.take_while(keep)(&Elem) -> boolItem until the first false
.filter_value(v=True)none; v: Elem or, for borrowed items, &ElemItem equal to v
.chain(other)none; same item typeItem from self, then other
.flat_map(f)(Item) -> sourceItems of each returned source

Rules that keep callbacks predictable:

  • Predicates always receive a shared reference to the element. For an owned item T that is &T; for a borrowed item &T it is the same &T, never &&T; for &mut T it is &T. map receives the item itself.
  • Pair items are passed as two arguments. See callbacks over pairs.
  • zip stops at the shorter input and tests the left side first, as today. chain requires both sides to have exactly the same item type.
  • take_while drops the first rejected item; there is no peeking.
  • flat_map’s callback may return any owned iterable: a list, set, dictionary, str, bytes, range, or generator, with the items a for loop over it would yield. There are no per-type exceptions. Returning a nested Iter is rejected by the same no-return rule as any other pipeline. A callback may use an eligible explicit generator conversion, with the frame cost and ownership restrictions described above.

Later additions, each needing a use case: step_by, map_while, scan, inspect, chunks(n) and windows(n) over borrowed lists (as borrowed sub-range views, which depend on B11/B39), dedup, rev for reversible pipelines, zip of more than two sources, peekable, group_by, and cycle. excitertools’ larger surface (map_reduce, send, I/O sources such as Iter.open) stays out of the language; such sources belong in std modules returning generators.

Callbacks over pairs

Some pipelines produce two values per item:

for name, score in scores.items():          # name: &str, score: &i64
for i, row in enumerate(&rows):             # i: i64, row: &Row
for a, b in zip(&left, &right):             # a: &T, b: &T

A loop binds the two values to two names, which works today. A callback needs a parameter list, and the question is what it looks like. The obvious answer is one tuple parameter, def(pair: (&str, &i64)) -> bool, but Plenty cannot form that type: tuples cannot hold references (“references cannot be stored in tuples”), because a tuple is a stored value and stored references are B11. That is also why borrowed zip/enumerate loops today require two unpacked bindings.

Options:

  1. Two parameters. def(name: &str, score: &i64) -> bool. Works with the current type system; matches the loop’s two bindings.
  2. One tuple parameter. Needs tuples of references (B11), and every use then indexes pair[0], pair[1].
  3. A destructuring parameter. def((name, score): (&str, &i64)) -> bool. Reads like the loop, but adds parameter-pattern syntax to the language and still names a reference tuple type.
  4. Two parameters for borrowed pairs, a tuple for owned pairs. What the first draft proposed; it makes the callback shape depend on whether the source was borrowed.

Recommendation: option 1, for every pair item, owned or borrowed. The hard rule is: a callback over a pair item takes two parameters. No exception for owned pairs, and no new syntax or types:

top: list[str] = scores.items()
    .filter(def(name: &str, score: &i64) -> bool: *score >= 90)
    .map(def(name: &str, score: &i64) -> str: copy(name))
    .collect()?

flagged: list[i64] = enumerate(&rows)
    .filter(def(i: &i64, row: &Row) -> bool: row.bad)
    .map(def(i: i64, row: &Row) -> i64: i)
    .collect()?

Only the top level is spread. In zip(enumerate(a), b) over owned a, the inner (i64, T) is an owned tuple passed as one argument. An inner pair that holds references cannot be formed and is rejected until B11; loops cannot unpack nested pairs today either. A map that returns an owned tuple produces a pair item, so the next callback takes two parameters. collect() into a dict[K, V] target consumes owned pairs directly. If B11 later adds tuples of references, option 1 stays valid and nothing has to change.

The accumulator is separate from the item spreading rule: pair fold and try_fold callbacks take (accumulator, left, right), even when the accumulator is a tuple. Pair reduce similarly takes (accumulator_pair, left, right). Predicates borrow each component; owned components therefore become &T and borrowed &T / &mut T components become &T for the predicate call.

Closure parameter types

Callbacks keep the current rule: parameter and result types are always written. This proposal does not add inference.

The cost is verbosity: def(n: i64) -> i64: n * n where Rust writes |n| n * n. The benefit is that each step of a chain states its item type where it is read. In a long chain, the item type changes at every map, and without annotations a reader must trace the chain from the source to know what n is at step five. Explicit types also make errors local: a wrong callback is reported at that callback, not as a mismatch several steps later. This matches the project’s preference for mandatory rules, and the rule is already taught. Relaxing it later (inferring parameter types from the adapter when the item type is known from the same chain, under B16) remains an open question; it can be added without breaking code that writes the types.

The closure and inference design history also constrains this choice: captures remain explicit, their loans stay live while callbacks may use them, and whether lowering inlines a callback must not change which captures are legal. Parameter inference would not imply capture ownership inference.

Terminals

TerminalResultNotes
.fold(initial, f)ALeft fold; f: (A, Item) -> A
.reduce(f)Option[Item]Left fold without an identity; Nothing when empty
.count()i64Checked; traps past i64.MAX like enumerate
.any(keep), .all(keep)boolShort-circuit; predicate rule above
.find(keep)Option[Item]First match; drops the rest
.for_each(f)()f: (Item) -> (); prefer a for loop when the body has statements
.last()Option[Item]
.collect()Result[C, AllocError]C is the expected list[T], set[T] or dict[K, V]; see below
.try_fold(initial, f)Result[A, E]f: (A, Item) -> Result[A, E]; stops at the first callback Err
.try_for_each(f)Result[(), E]f: (Item) -> Result[(), E]; stops at the first callback Err
.try_collect()Result[C, AllocError | E]Over Result[T, E] items; same targets; first Err wins
.sum()ItemNumeric items; zero when empty
.min(), .max()Option[Item]Numeric or str items; NaN propagates

For serial pipeline machinery, collect and try_collect are the only terminals that allocate. The try_ prefix means stopping at the first error, but the error source differs: try_collect consumes Result items, whereas try_fold and try_for_each consume ordinary items and call a fallible callback. This preserves the contract of the former std.iter.try_fold; these callback terminals do not implicitly unwrap Result items.

Early-exit terminals drop the remaining state when consuming an owned pipeline. Through &mut Iter[T], they leave the unvisited remainder with the caller, just as a borrowed consuming loop does. An item already passed to a callback or tested by a predicate has been consumed, including the item causing an error.

Agreed for stage 3: .sum(), .min() and .max() are terminals. Each compiles only when the item type supports the operation: sum needs numeric items, min and max need ordered items. Implemented for numeric items and, for min/max, str; borrowed items are read as *item. Tuples are rejected for now: a tuple containing a float has no NaN rule yet, so they use fold or reduce.

  • sum() returns Item; an empty pipeline gives zero. Integer overflow is a runtime error, as for +.
  • min() and max() return Option[Item], Nothing when empty. Ties keep the first item.
  • NaN propagates. For sum this is ordinary float addition. For min and max it is an explicit rule, IEEE 754-2019 minimum/maximum: any NaN item makes the result Some(NaN), and -0.0 orders below 0.0. A plain < loop would instead drop or keep a NaN depending on its position. std.math.min, max, min_all and max_all follow the same rule.
  • There is no parameter to ignore NaN. Filter first; the choice is then visible at the call. Separately named NaN-ignoring terminals can be added if filtering proves too common or slow.

NumPy, Julia, Java and IEEE 754-2019 also propagate by default; where both behaviors exist, most libraries use separate names rather than a flag. median is not planned: an exact median stores every item, and a streaming estimate is a different contract.

Choosing the collection

There is one collecting terminal. Its target comes from the expected type, as empty displays already do.

Hard rule: collect() and try_collect() build the collection named by their expected type. Without an expected collection type, they are a compile error. There is no default.

names: list[str] = iter(&people).map(def(p: &Person) -> str: copy(&p.name)).collect()?
tags: set[str] = iter(&posts).flat_map(post_tags).collect()?
index(iter(&people).map(def(p: &Person) -> (str, i64): (copy(&p.name), p.id)).collect()?)  # def index(by_name: dict[str, i64])
n = iter(&xs).collect()?.len()
# error: collect() needs an expected collection type; annotate the binding,
#        as in `name: list[i64] = ...collect()?`

Expected-type sites. The same sites that type an empty display today (verified with the current compiler):

  • an annotated binding, x: set[u64] = ....collect()?;
  • assignment to a binding or field whose type is known, xs = ....collect()?;
  • a non-generic parameter, take(....collect()?);
  • a function’s declared return type, through its Result;
  • a class field in a constructor call, Bag(items=....collect()?).

The expected type passes through ? and through the Result of the return type, exactly as name: list[i64] = []? does now. Sites that give no expected type: an unannotated binding, a generic parameter position (as type argument inference already states), a method receiver (....collect()?.len()), and later uses of a binding. The checker does not look ahead.

Precedent. The checker already types expressions from an expected type:

  • xs: list[u8] = [1, 2]? types the literals as u8; parameters do the same for numeric literals.
  • Empty displays take their element types from the expected Result type, and s = []? with no expected type is an error: “empty collection needs a type; annotate the binding, as in name: list[i64] = []?, or use a typed constructor”.
  • Atomic(value) takes T from the expected type, and expected types allow anonymous-sum injection and widening.

collect() extends this by letting the expected type choose the collection kind, not only its element type. Displays never needed that, because their syntax picks the kind. The checker change is to treat collect/try_collect as a contextually typed terminal in the same expected-type pass.

Target rules.

  • list[T], set[T] and dict[K, V] are accepted. Any other expected type, is an error naming the accepted kinds; aliases resolve first.
  • The element type must equal the item type exactly. Items are already typed by the chain (callback types are explicit), so nothing flows backwards into the pipeline and there is no conversion: x: list[u8] over i64 items is a mismatch, not a cast.
  • set[T] needs hashable items and keeps the first occurrence, as set[T].from does. dict[K, V] needs owned (K, V) pair items, and a later duplicate key overwrites, as dict displays do.
  • The Result wraps the target: Result[C, AllocError], and for try_collect over Result[T, E] items, Result[C, AllocError | E]. The expected type may be the bare C (with ?) or the whole Result.

Why an error, not a list default. Numeric literals default to i64, but the closer precedent is the empty display, which errors without context. A default would make xs = ...collect()? silently a list and change meaning when someone later adds an annotation elsewhere, and readers would need to know the default to read unannotated code. The error costs one annotation, which also documents the collection kind at the call site, where readers need it. It is the rule with no exception: every collect() has a visible or declared target type.

Alternatives considered.

  • Passing the container type, .collect(set), as excitertools does. Python has no static types, so the container must be a runtime argument there. In Plenty the element type is already known and the target is almost always declared nearby; a type-valued argument would be a new kind of argument (types are not values) and would duplicate the annotation. It also would not remove the need to check it against an expected type.
  • Separate terminals (collect_set(), collect_dict()). Explicit at the call, but three names for one operation, and an annotation would still be checked against them.
  • A type argument, collect[set[str]](). Turbofish; rejected.

list[T].from(pipeline) is not accepted; its diagnostic points to collect(), so there is one spelling.

Errors

Closure bodies never return from the enclosing function. A ? inside a callback returns from the callback, as it does today, so a fallible step produces Result items. Handle them with try_collect, which stops at the first Err and drops the rest of an owned pipeline, or with a for loop whose body uses ?:

parsed: list[i64] = iter(&lines).map(def(l: &str) -> Result[i64, ParseError]: parse_int(l)).try_collect()?

try_fold and try_for_each instead propagate their callback’s first error; handling a Result item inside that callback must be explicit. try_collect widens its item error with AllocError exactly as anonymous sums widen today. The callback terminals add no allocation error of their own. There is no implicit error collection and no hidden recovery.

Allocation

A serial pipeline allocates nothing of its own: no iterator object, no frame, no item buffer, no closure environment beyond the inline one. Only the collect family allocates, and it returns Result. Callback bodies and source expressions keep their own contracts. collect reserves capacity when the pipeline’s length is known exactly (a range or collection source with only map, enumerate, zip or take), and otherwise grows like a comprehension.

These guarantees follow the reference’s allocation boundaries: user callbacks, source expressions and destructors retain their own allocation behavior. Explicit generator conversion introduces an inline frame. Parallel execution is a separate boundary: job cells, output buffers and reduction scratch storage can allocate even for a non-collecting terminal, and failures must be reported through its parallel error type.

Ownership and cleanup

  • The pipeline owns its owned source, its closures and adapter state (counters). Dropping an unconsumed pipeline drops them in reverse creation order without running any callback or advancing any generator.
  • A borrowed source’s loan lasts until the pipeline is consumed or dropped, and protects the collection exactly like a for loop’s loan.
  • Closures may capture by value, mut, & or &mut; the captures’ loans last as long as the pipeline. Two adapters may not both hold &mut to the same binding, by the ordinary aliasing rule.
  • break, return and ? in a consuming loop drop the remaining source, pending items and closures when the loop owns the pipeline, as loops over owned generators do today. A consumer borrowing &mut Iter[T] instead releases its local state and loan, leaving the owner’s remaining pipeline available; early termination does not destroy the caller’s source or closures.
  • def once closures are rejected as adapter callbacks, since adapters call their callbacks repeatedly. A terminal that calls its callback at most once (none in the core set) could accept them later.

Efficiency model

Fusion at the consumer

A pipeline value is a compile-time plan: a source descriptor plus an ordered list of adapter steps, each with its inline closure environment and any counter slots. Nothing about the plan exists at run time except those slots.

When the plan reaches a consumer, the lowering in op.rs expands it into the same loop structure it already produces for for and comprehensions:

  • the source becomes its existing loop plan: a counted loop for ranges, an indexed loop for lists, a hash-table walk for dicts and sets, a resume call for generators;
  • filter and take_while become a conditional continue or break;
  • map becomes a call whose result replaces the item;
  • enumerate, take and skip become counter slots with checked arithmetic;
  • zip advances a second source plan in the same loop, as it does today;
  • chain becomes two loops sharing one outlined body, so the body is not duplicated;
  • flat_map becomes a nested loop;
  • the terminal or loop body is the innermost block.

This is how comprehensions already work, so the work is mainly generalizing an existing plan, not writing a new optimizer. Comprehensions become a spelling of a pipeline (see below).

Callback calls

A named function is called directly. A closure literal written inside the chain is known at the call, so its one-expression body is lowered in place, exactly like a comprehension’s element expression, with its captures resolved to the pipeline’s environment slots. A closure held in a local is called directly through its known producer with its environment address, with no indirect call. This removes all runtime dispatch from the hot loop; what remains is at most one direct call per callback per item, and none for closure literals.

A general function inliner is out of scope for this proposal. If measurement shows direct calls to small named functions matter, a size-bounded inliner at the operation level can follow as a separate B19 item.

Compile-time cost

Fusion adds no generic instances: an adapter is not a generic function, and a chain is not a nested type that must be instantiated and then optimized away. Lowering cost is linear in the number of steps at each consumer. Rust’s model creates one monomorphized type per adapter per chain and relies on LLVM to inline them away; Plenty would pay that as extra Cranelift functions and still not get the inlining. A pipeline that is not lent is lowered once, at its single consumer, because it is affine; a lent pipeline is lowered at each consuming site and in each callee instance. Apply the existing generator nesting limit (64 steps) to chain length.

Verification

Acceptance for the implementation, not optional extras:

  1. Operation equivalence. Snapshot tests compare the typed operations of a pipeline with the hand-written loop it replaces, for each adapter. Use the per-function dump being added in issue #360 (--dump ops/CLIF/asm).
  2. No runtime dispatch. A test inspects the CLIF of fused pipelines over range, list and borrowed-list sources and asserts there is no call to the collection dispatcher and no call_indirect inside the loop.
  3. No allocation. Allocation-checks tests (the allocation-checks feature) run every non-collecting terminal with allocation disabled.
  4. Benchmarks. Add the table above as a release benchmark. A pipeline over a range or list must be within 10% of the hand-written loop; report the generator-source rows separately, since their cost is the generator’s.
  5. Compile time. Add a pipeline-heavy workload to compile_bench and check that cold builds do not regress against the equivalent comprehension code.
  6. Semantics. Ordering of callback side effects, cleanup on early exit, loan errors for borrowed sources, partial consumption through &mut Iter[T] followed by collection in the caller, and every rejected placement (return, pass by value, store, capture, yield) with specific diagnostics. Cover owned generator conversion and rejection of borrowed conversion, flat_map returning Iter, and the distinct callback-error and item-error terminal contracts. For parallel stages, verify tree shape across worker counts and chunk boundaries, earliest-input error selection, and draining accepted work on early termination and infrastructure failure.

For loops, comprehensions and generator expressions

  • for x in pipeline: is the main consumer. for x in iter(values): is valid but redundant; prefer for x in values:. A lint can suggest the shorter form.
  • A list comprehension [f(x) for x in s if p(x)] lowers to the same plan as iter(s).filter(p).map(f).collect() into a list target. Comprehensions stay: they are the Python-shaped way to build a collection, and the tutorial teaches them first. The two forms must produce the same operations; that is a test, not a goal. Implemented for one for clause with any if clauses, for lists, sets and dictionaries; several for clauses keep the nested loop.
  • .generator() converts an eligible prepared pipeline into one frame. Ordinary generator expressions keep their existing syntax and owned-source rules; an unconverted pipeline is not a valid generator-expression source. Stored range sources already use counted iteration (issue #371).
  • There are no free any(xs)/all(xs) builtins, as there are no free sum, min or max. .any(keep)/.all(keep) take a predicate; .any()/.all() test bool or &bool items directly, as filter_value() does, and compile to the same loop the former builtins did. Over a generator expression the former builtins advanced a frame per item; iter(xs).any(keep) fuses the test into the source loop instead.

Parallel pipelines

.par(&pool) marks the boundary: steps before it run serially on the calling thread to produce input items; steps after it run on the pool’s workers. The pool is the existing ThreadPoolExecutor; there is no global pool, so thread creation stays explicit and fallible, as the parallelism proposal requires.

with ThreadPoolExecutor(8, 64)? as pool:
    lengths: list[i64] = iter(paths).par(&pool).map(file_size).collect()?
    longest = iter(range(1, n)).par(&pool).map(score).reduce(best)?

Stage 4 contract, extending current pool.map/pool.reduce_tree:

  • Sources. Owned lists and integer ranges, as pool.map accepts today. Borrowed lists follow once scoped pool jobs can borrow (B22). (Issue #455 added borrowed lists without user-facing scoped jobs: chunks read the rows in place because every terminal drains its jobs before returning.)
  • Allowed steps after par. map, filter and filter_value (over copyable items); enumerate must be applied before par. Order-dependent steps (take, skip, take_while, chain, zip, flat_map) are rejected after par with a diagnostic naming the step.
  • Callbacks. Statically named functions or reusable immutable closures that wholly own their captures and pass the existing worker eligibility and effect checks. mut, &mut and one-shot captures are rejected; shared borrows wait for scoped jobs. Generator sources are rejected, as frames are not worker-eligible.
  • Terminals. collect and try_collect into a list target (lowest-index error wins, as map_result), count, any/all, sum, min/max, and reduce. sum, min and max work per chunk, then combine chunk results in input order. Integer sum traps exactly when the serial sum would; min/max equal the serial results (first NaN wins, -0.0 < 0.0); a floating-point sum regroups additions by chunk. A parallel reduce works like sum: each chunk reduces its items left to right, then the chunk results are reduced in input order, skipping empty chunks. It is deterministic for a given input length and chunk count, but differs from the serial left fold for non-associative operations such as floating-point addition. pool.reduce_tree keeps one fixed grouping for code that needs it. There is no parallel fold: it would need an identity and associativity the type system cannot check. for loops and for_each are rejected after par, since their bodies would run concurrently.
  • Reduction grouping. A parallel reduce groups by chunk, as sum does, so no list of all items is collected; empty input yields Nothing. An earlier version of this proposal used the fixed-tree reduction contract of reduce_tree, so worker count and chunk boundaries could not change the grouping. That required collecting every item before reducing, which made reduce with a cheap function several times slower than serial. Code that needs the fixed tree calls pool.reduce_tree directly.
  • Early termination and effects. Follow the fallible mapping contract: try_collect selects the lowest original input index’s worker error, including across chunks. An earlier accepted input’s error takes precedence over a later submission failure. Accepted work is drained before returning, and partial outputs are dropped exactly once. For any/all, a decisive result may stop further submission, but already accepted chunks may still run callbacks; their effects are not undone. These terminals do not promise the serial callback count. An infrastructure failure already encountered is still reported, rather than hidden by a decisive boolean result.
  • Result type. Parallel terminals return Result[_, PoolMapError] or ParallelError[E], because submission can fail and the pool can be shut down.
  • Granularity. Each job runs the fused map/filter body over a contiguous chunk of input, not one item. The default number of chunks is the pool’s worker count (max_workers), read at run time, never the machine’s core count. .par(&pool, chunks=N) fixes the count; N is a positive i64. An input shorter than the count gets one chunk per item, with no empty chunks. Chunk boundaries depend only on input length and the count, never on scheduling, so the same program and pool size give the same results on any machine. Filtered chunks are concatenated in input order. Further tuning (chunk sizes, par_chunks, adaptive splitting) is deferred until there are benchmarks.

Compile-time checks the type system must enforce: worker eligibility of every callback and item type (existing closed-world check), no shared mutable borrows, no order-dependent step after par, and reduce only (not fold) for parallel combination. Associativity itself cannot be checked; the documentation states the grouping: per chunk, then chunk results in input order.

Automatic parallelization

Pipelines give B24 a natural unit: a fused loop body whose callbacks are pure by effect analysis, over a sized source. B24 remains deferred, and this proposal does not change that: a pipeline without .par is always serial. Explicit .par gives the evidence the parallelism proposal asked for: which workloads benefit and whether effect checks are sufficient.

Migration

Status: complete for stage 2. std.iter was removed outright instead of deprecated: it had no users outside the repository’s tests, and there is no deprecation-warning mechanism. Its tests now exercise fluent generator pipelines and std.error.map_error. The tutorial lesson moved to Transform a stream of values, after the borrowed-collection pipeline lesson. The items below record the original plan.

  • std.iter. Mark map, filter, take, fold and try_fold deprecated in a deliberate compatibility change after migration review. Fluent methods plus .generator() now cover their cross-function use cases; keep existing functions working until removal is separately approved. Internal uses are limited to tests/test_std_collections.rs and tests/test_trap_locations.rs.
  • Tutorial. Rewrite Transform a stream of values (then in the modules chapter) around pipelines (implemented, including explicit returning conversion). A later organization pass can move it out of the modules chapter, since iter needs no import; it belongs after comprehensions and closures in the iteration lessons. Review lessons that build lists by looping only to fold them, and the comprehension lessons for the equivalence note.
  • Reference. Add a pipeline page under iteration; update adapters (enumerate/zip become pipeline starts), the std.iter page, std.math (its list helpers stay; pipelines use the stage 3 terminals), parallel mapping, the cheat sheet and support.md.
  • Compatibility. No existing program changes meaning: iter is a new name (a user function named iter keeps priority, as with enumerate), and enumerate/zip/reversed loops lower as before.

Alternatives considered

Method syntax for std.iter (extension methods or uniform call syntax). Lets numbers(3).map(sq) call std.iter.map. Cheap to add, but keeps the 9x per-item cost and the generator-only, owned-capture limits, and an import that adds methods contradicts the protocol rule that imports never add methods.

The related protocol method bodies proposal distinguishes provided methods selected through a bound or protocol qualification from automatic method injection. Its agreed, unimplemented design preserves that import rule. Provided bodies could share adapter implementation logic, but do not by themselves add fluent methods to built-in generators or remove their frame and dispatch costs.

Rust’s model: generic adapter structs plus inlining. Requires storing generator frames and closures in generic class fields (rejected today), user-implementable iterator protocols with associated types (unsupported), and a function inliner in the Cranelift pipeline. It multiplies generic instances per chain and raises compile time, against a design goal, for performance the compiler would still have to recover.

Faster generators only. Inline the runtime dispatch, call resume directly when the producer is known, and use a jump table for state. Worth doing for named generators regardless (a separate B19 item), but a resume per adapter per item, with frame-backed locals, cannot match a fused loop, and it does not lift the borrowing limits.

Comprehensions only. Comprehensions already fuse, but cannot express take, take_while, chain, early-exit reductions, or par, and nested clauses become hard to read beyond two levels. The user request is precisely for the chaining form.

First-class storable pipelines from the start. Convenient, but storing a borrowed pipeline needs stored references (B11), and storing an owned one needs frame and closure storage in generic data. Keeping pipelines in their creating scope, with lending for helpers, ships the useful part now.

std.math helpers over lent pipelines. sum_iter(&mut pipeline, 0) and fold(0, math.max) were considered for stage 3 and deferred. They need module functions as values and generic function values without explicit type arguments, and they cannot reject unsuitable item types at the call. The terminals above replace them for pipelines.

Open questions

  1. User sources. A user iterator protocol (next(self: &mut S) -> Option[T]) could make classes pipeline sources once B12 settles; until then, a method returning a generator works.
  2. Closure parameter inference. Should callbacks later omit parameter types when the adapter knows the item type (B16)? This would shorten chains at the cost of the readability described in closure parameter types. Decide after stage 1 with real code to compare.
  3. Passing pipelines by value. Whether helpers that add steps (def evens(p: Iter[i64]) -> Iter[i64]) are worth weakening the never-leaves-its-scope rule. Not planned; revisit only with use cases that closure-returning helpers do not cover.

Staged plan and suggested backlog wording

  1. Stage 1 — serial core. iter, Iter[T], local-only placement, map/filter/enumerate/zip/take/skip/take_while, two-parameter pair callbacks, and terminals fold/reduce/count/any/all/find/for_each/collect/ try_fold/try_for_each/try_collect (target from the expected type), fused at the consumer, with the verification suite above. Issue #371, fixed, removed the stored-range slow path that would have misled the benchmarks. Small scalar leaf callback substitution and SSA scalar captures are also implemented; measurements cover the original workload. Generator fluent entry and the explicit owned .generator() bridge also landed independently of composition.
  2. Stage 2 — composition. chain, flat_map, closure-literal splicing, comprehensions lowered through the pipeline plan, tutorial rewrite and std.iter deprecation. chain and flat_map are implemented. Closure literals written in a chain are lowered in place for every adapter and callback terminal; literals with ?, match, several statements or a nested closure, and closures held in locals, keep the call. Single-for comprehensions lower through the plan and produce the same operations as the equivalent filter/map/collect chain; multi-for comprehensions and generator expressions keep their own lowering. Complete: the tutorial teaches pipelines, and std.iter was removed rather than deprecated (see Migration).
  3. Stage 3 — lending. &mut Iter[T] parameters specialized per pipeline type, next(p), resumable slot state for every adapter, store-back only for lent pipelines, and partial consumption followed by collection in the caller. Measure instance counts in compile_bench. The sum, min and max terminals described under Terminals are implemented. Every adapter is resumable, and next(p) works in the creating scope: a later loop or terminal continues after the items taken. In that scope the plan’s state already lives in the creator’s slots, so resuming needs no store-back; take_while and zip record that they have ended so a later consumer does not pull past the end. Lending is implemented: the callee receives the pipeline’s state through hidden parameters, copied in on entry and written back on every exit. Pipelines of reference items can be lent; their items cannot be returned. A pipeline holding a stored closure with borrowed captures cannot be lent; write the closure literal inside the adapter instead. The reference states the rules and limits.
  4. Stage 4 — parallel (implemented). .par(&pool) with chunked fused jobs (one chunk per pool worker by default, or chunks=N), the allowed-step rules, and collect/try_collect/reduce/count/any/all/sum/min/max. The reference lists the implemented contract. try_collect reports ParallelError[E | AllocError], since a chunk’s output list can fail to allocate on the worker; any/all do not stop early.

Suggested backlog row: B75 — Fluent iterator pipelines. “iter(source) pipelines with chained adapters and terminals, fused at the consumer into one loop with no adapter dispatch or serial machinery allocation outside collecting terminals. Stages: serial core, composition and std.iter deprecation, lending as &mut Iter[T], explicit .par(&pool). Depends on issue #371. Relates to B12, B22, B24, B49. Design: proposals/fluent-iterators.md.”

Issues found

  • Issue #371: generator expressions over range(...) and stored ranges iterate through the runtime, about 15x slower than a direct range loop. Fixed by local PR #436: they now compile to counted loops.

Owned response experiment and consuming receivers

Status: Experiment complete; consuming methods implemented. The source is in experiments/typestate/. The original model used consuming free functions against baseline 344dfdbd; it now exercises explicit owned receivers. This refreshes the typestate assessment. The HTTP server is unchanged. Individual field moves out of owned bindings were implemented later (issue #534), and the model now uses them. Real HTTP integration remains unexplored and is tracked in B72.

What the experiment establishes

Two public classes, Unanswered and Writer, hide their resource fields and constructors. Consuming methods implement these transitions:

Unanswered.begin(self: Unanswered, length) -> Result[Writer, StartError]
Writer.write(self: Writer, count)          -> Result[Writer, WriteError]
Writer.finish(self: Writer)                -> Result[(), FinishError]

The wire is a deterministic counter with an injected failure position, not a network transport. Negative lengths, oversized writes and incomplete finishes return the still-valid owner in an error payload for retry. A partial write is terminal: its error reports bytes accepted in that call and in total, and the connection drops. An error must never offer a retry owner after effects that make replaying the operation unsafe.

A freshly built debug compiler compiled and ran nine resource lifetimes. The runner checked exactly one destructor invocation per resource, including legal retries, cancellation, abandonment, an owner carried through ? in a retry error, and cleanup when a later function argument propagates an error. The partial-write case accepted two bytes, then one of the requested three; it reported (1, 3) and destroyed the connection before returning the error.

Twelve compile-fail probes cover writing/finishing before begin, repeated begin/finish, writing after finish, reusing the owner after a fallible write, forging either state, reading private storage, copying a resource-bearing state, consuming a borrowed writer, and using an owner as a whole after moving a field out of it. Compiler integration tests separately exercise owned receiver lowering, generics, protocols, generators and cleanup. These are reproducible assertions, not only a successful smoke check.

The cost is visible in the API: two public states and three error enums, new writer bindings after successful writes, and explicit matching to recover retry owners. begin moves the wire field out of the consumed Unanswered owner into a new Writer. Before field moves, Unanswered held an Option[Wire] and begin used replace(..., Nothing); neither version needs an allocation or a dummy connection. Constructor privacy maintains the invariant that a live public state contains a resource. No throughput, code-size, or allocation-count benchmark was performed; the cleanup trace is not a performance measurement.

Why this is not yet an HTTP server conversion

The actual std.http.server.Handler.handle receives &mut Exchange[T]. The server retains the exchange for request draining, reports and connection reuse. A handler cannot consume that borrowed owner. Fields such as request-body state, cancellation tokens and the connection also belong to the same exchange. An owned response API therefore needs a deliberate split of responsibilities, or a different handler contract, not just renamed methods.

The model’s recoverable incomplete finish is also deliberately different: today Exchange.end marks a short sized response broken, and a transport error during write leaves the response broken. The model does not prove that either real failure can be retried. Real integration must preserve partial-progress reporting, HEAD/chunked framing, cancellation and keep-alive behavior.

Keep ordinary writes borrowed if returning a writer for every write is too verbose; then terminal transport failure still needs a runtime broken-state check. Typestate is most promising at coarse begin/finish boundaries. Discarding an unfinished writer remains legal: deterministic resource cleanup is not a guarantee that a response completed. That stronger requirement belongs to linear obligations.

Consuming methods use existing ownership

Explicit self: Class reuses the owned-argument path: it transfers an entire value, rejects reuse and conflicting borrows, and transfers cleanup responsibility. value.finish() behaves like finish(value), with static dispatch and no new allocation required by the call itself. A compiler test compares optimized assembly for equivalent method and free-function calls.

src/frontend/classes.rs, src/frontend/enums.rs and src/frontend/protocols.rs admit owned receivers. Ordinary and generic method calls forward them as values; named/default-argument calls use the existing wrappers. Temporary borrowed views cannot be turned into owners by dispatch.

Bare self remains a shared borrow and self: &mut Class an exclusive borrow. Lifecycle methods and operator hooks keep their existing contracts. Owned parameters remain immutable bindings; a body can move self into a mutable local. Protocol matching distinguishes owned and borrowed requirements. An owned receiver cannot silently move through an affine reference, class field or indexed element, or return a reference to its dying storage. Copyable enums keep ordinary copy semantics rather than becoming affine solely because a method takes a value.

Tests cover enums and generic classes/methods, temporary and named receivers, protocol calls, named/default arguments, live loans, failure in a later argument, conditional moves and exactly-once destruction. Receiver evaluation precedes later arguments; mutable borrowing’s two-phase reservation does not allow reading the moved owner again. These are ordinary owned-function-call rules.

The first stdlib adoption is std.text.Builder.finish(). It consumes the builder on success or allocation failure; to_text() remains a borrowed snapshot for reuse and retry. std.text.join and wrap now finish their builders explicitly. Both finalization paths still copy output bytes; no zero-copy improvement is claimed. SQLite and HTTP lifecycle contracts are not converted.

Individual field moves

Implemented (issue #534). This section originally deferred field moves: they needed field-level availability and cleanup rules across branches, loops, reassignment and early returns, plus a rule for custom destructors that might inspect an extracted field. The implemented rules track each moved field like a binding, reject whole use of the owner until the field is assigned again, clean up only initialized fields, and reject moves out of a class with __del__. See field moves. Whole-object destructuring is still not included.

Flat class layout

Status: Stage 1 (one-field classes as newtypes), stage 2 (flat nested class records), stage 3 (fewer runtime walks) and stage 4 (flat tuple and variant payloads) implemented. Issue #547 asked why a class value is an address into its own storage, and whether classes could instead be laid out flat inside their owner. Values and memory own the shipped contract; this page records the findings and the design.

Why a class value holds an address

Every runtime slot is 128 bits: a payload word and a tag word. A class keeps its fields in inline storage that follows its slot in the owner (a local, a record field, a list element, a box), and the payload word is the address of that storage. The address exists because the payload word has to be something uniform:

  • Runtime operations (copy_payload, release, retain, hashing, equality, formatting) receive one u128 and a type descriptor and walk the record through the descriptor. They never receive the slot address.
  • References are (address, shift) pairs. &self.field is computed from the address of the record, which is read out of the class value word; a reference to a class value therefore projects fields with one load and an add, not by knowing where the value’s slot lives.
  • Sum payloads are packed into the same 128 bits as their tag; a variant holding a class holds its address word and leaves the tags free.
  • A zero payload word means “empty slot” for every type, which lets moved-out locals, fields, and partially initialized records release for free.

The cost is that every move of a class value is a copy of its inline bytes plus a fix of each nested class, range, closure, or inline-sum address (inline.rs::fix_record, the runtime’s Payload::relocate); issue #544 measured that as the remaining overhead of nested wrappers.

What depended on the address before stage 2

DependentHow it uses the word
lower_class (src/codegen/enums.rs)Field loads a slot at word + offset; FieldRef loads the word from the reference’s address and adds the offset
copy_inline_to, copy_record, fix_*, release_record (src/codegen/inline.rs)memcpy from the word, then rewrite nested words at base + offset + 16
runtime Type::record (plenty-runtime/src/aggregates.rs)value as u64 as *mut u128 is the record base for walks, drops, copies, hashing, equality, formatting
references (src/codegen/references.rs)write_reference copies inline payloads to address + 16 and stores the word at address
sum payloads (src/codegen/enums.rs, runtime Type::payload)a variant’s single class field is the class’s 128 bits; tags stack in the high word
generators and closuresclass captures and frame slots are ordinary slots with following inline bytes
Box[T], Shared[T], actorsthe box holds one slot plus inline bytes; copies between actors use copy_payload
C export (src/codegen/exports.rs, src/exports/handles.rs)a handle is a Box holding the instance; a borrowed &Class is the slot address inside the handle; an owned argument is copied out with copy_payload using the class descriptor
columnar lists (plenty-runtime/src/aggregates/clist.rs)the row class descriptor’s variants are the column schema

Stage 1: one-field classes are newtypes

Rule. A class with exactly one field and no __del__ is represented exactly as that field: same 128-bit slot, same inline bytes, same descriptor layout. Constructing it, reading or borrowing its field, and moving the field out are all identity operations in the generated code. No annotation; the rule is total.

Classes with __del__ are excluded because their record carries a state word (initialization completed, observer count) that has no home in the field’s own slot. Field-less classes already lower to a zero word and keep doing so.

Compiler

  • ClassType::newtype_field (src/record.rs) names the rule once; Ty::newtype_field and Ty::newtype_base expose it.
  • Layout queries delegate to the field: has_inline_storage, flat_record, inline_bytes, and the generator frame measurement.
  • lower_class: New is a zero word (an empty slot, which the field’s __init__ write fills); ArmDrop is a no-op; Field and FieldRef return their operand unchanged. The final unpack to the field’s native type is the only code emitted.
  • retain, release, and copy_inline_to unpack to the field type and recurse. fix_slot, fix_variant, slot_fixes, plain_variants and release_fields look through newtypes, so a record or sum holding a newtype copies and releases as if it held the field directly.
  • Columnar-list row access reads a one-field row as its scalar.
  • A newtype has no inline storage, so a function returning one built by a constructor call ends in a tail call. The constructor inlining from issue #548 now also lowers at that site: the caller’s locals are released first, as a real tail call would, and the shared exit then finds empty slots.

Runtime descriptor

A newtype’s descriptor carries the field’s layout (kind, affinity, inline bytes, links, variants, destructor hook) so every runtime walk treats the value as the field. The class name and a new wrapper link (byte 64; the descriptor grows from 64 to 72 bytes) record the source-level shape for formatting, which prints Wrap(x=1), and for the columnar-list schema.

FFI layout contract

C code never sees a class layout: a class crosses the boundary as an opaque handle (a Box holding one slot plus the descriptor’s inline bytes). The rule changes only what that slot holds for one-field classes, and every handle operation goes through the descriptor, so exported libraries, foreign pointers, and dynamic exports keep working without a header change. The contract FFI relies on is therefore:

  1. A handle points at a Box whose value slot is at offset 32.
  2. A borrowed class argument is the address of that slot.
  3. An owned class argument or result is copied in or out with the class descriptor’s copy_payload.

These three hold for every class, newtype or not.

Costs and limits

  • Type grows by 8 bytes, so every program’s read-only data changes; machine code for programs without one-field classes is unchanged.
  • A one-field class with __del__ keeps the record layout.
  • Formatting a newtype walks one extra descriptor link.

Measurements

Retired instructions (perf stat -e instructions:u) against main at c78365ad, which includes constructor inlining, compiled with --release:

ProgramBeforeAfter
Issue #544 reproduction, per cycle (file wrapped twice, 2000 iterations)486444
Issue #544 reproduction, 200 iterations, whole program340,245331,699
200 loopback connect/accept/close pairs through std.net2,194,6782,082,256
Nested two-field records copied through a loop (no one-field class)283,245283,175

The remaining per-cycle cost is caller-site bookkeeping around releases of slots the compiler could prove empty, not record copies. Programs without a one-field class produce identical machine code; only read-only data differs.

Stage 2: flat nested records

Goal. Store a class-typed field at a fixed offset inside the outermost owner without a word of its own, so a move is a byte copy with no address fixes.

Findings. After __init__, a nested class field’s word is redundant: it is always slot + 16 or zero (moved out). Dropping the word (“rule B”) saves 16 bytes and one load, test and store per nested class per move. It needs:

  • a presence marker for nested classes with __del__ (their state word can serve; FieldMove must clear it) and for partially moved records;
  • record_field, fix_*, release_record, and the runtime Record::slots walkers to compute nested addresses from the owner instead of loading them;
  • references to nested fields to stay (address, shift) pairs, which already works because FieldRef adds offsets to an address.

What it does not remove: the top-level word stays an address, because runtime operations and sum payloads still receive one u128. Flattening below the top level removes fixes but not copies; a move still copies the inline bytes.

First assessment. Stage 2 was first deferred: after stage 1 the remaining per-move cost of a nested record looked bounded by RECORD_FIX_LIMIT. Measuring it showed otherwise: in a loop that moves a three-level record through a function, the slot fixes were about 90 of the 146 instructions the callee compiled per call (6 fixes of 5 instructions per record copy, 3 copies), and the nested words added 96 of the 272 bytes copied per record.

Decision. Adopted. A field of a class record whose type is a class with a record is stored flat. The cases that keep a word followed by a record are:

  • top-level class values, whose word is the uniform 128-bit value runtime operations and sum payloads receive;
  • class payloads of enum variants and tuple components: only class records flatten their fields, so variant payload packing and tuple layout are unchanged (stage 4 later flattened tuples and multi-field variants);
  • field-less classes without __del__, which have no bytes to place.

Contract

  • Presence. An all-zero flat record is an empty field. Moving a field out or releasing it zeroes its bytes; a class with __del__ has a nonzero state word once constructed, and every other record with a record has at least one nonzero word, so empty and live never coincide.
  • References. A reference to a class value names the 16 bytes before its record. Readers use address + 16 without loading the word there; writers copy the record bytes, and writing an empty value zeroes them. Top-level values keep a real word at that address, flat fields do not, and no reader depends on which. Sort-key adapters, which receive runtime slots holding a row’s record address, convert to that form before calling compiled code.
  • Destructors. The runtime passes record - 16 to the __del__ hook, which must not read the word.
  • Newtype construction. A one-field class wrapping a class with a record is filled through a reference to the record, so New allocates the record’s storage as the wrapped class would.

Compiler and runtime

  • Ty::flat_field names the rule; Ty::field_bytes and ClassType::field_offset give a field’s width and offset.
  • lower_class projects flat fields by offset (class_field, referenced_record); read_typed_reference and write_record implement the reference contract.
  • fix_record, release_record and record_fixes (src/codegen/inline.rs) walk flat fields in place and emit no fix for them.
  • The runtime Field { at, flat } iterator (Record::slots) reads, writes and clears a field by kind. Copy, relocate, release, hashing, equality, ordering and formatting go through it. The resumable release walk clears a finished flat record, and Field::is_clear lets a resumed pass skip it. abandon_copy releases the copied prefix without allocating.

FFI

No change. A handle is still a Box holding one top-level slot, whose word addresses the record; borrowed and owned arguments use the same descriptor paths. Generated headers and .plentyi interfaces are byte-identical for a library exporting nested classes.

Measurements

Retired instructions (perf stat -e instructions:u, --release, mean of 5, pinned to 8 cores). Base is main at 5e6e98a4, before stage 2; main is 0ec41c06, which adds the empty-slot release removal (issue #549).

ProgramBaseMainStage 2
Three-level Point/Segment/Path workload5,033,105,2635,033,105,2673,700,365,927 (−26.5%)
Its move loop, per iteration1,8401,8401,302 (−29.2%)
Its list rows alone1,325,304,5631,325,304,5421,096,565,424 (−17.3%)
Nested two-field records copied through a loop282,667281,066280,637
Constructor fixture (wrapped file)331,192262,145262,143
Copies fixture, 5M wrapper cycles and 300k shelves5,612,022,5493,868,322,5853,570,485,254 (−7.7% vs main)
Constructor fixture, scaled2,225,242,405505,241,987505,241,982
300k shelves of nested records appended and destroyed3,387,022,8983,363,322,3953,065,484,601 (−8.9% vs main)
200 loopback connect/accept/close pairs through std.net1,864,6481,833,4131,832,631

Code size for the three-level workload fell from 130,321 to 121,521 bytes of .text. compile_bench (aot, 1000 × 10, pinned to 8 cores) retires 0.4% more instructions (12.28 G to 12.33 G for 3 repetitions); its median times (244–254 ms on main, 243–260 ms with stage 2) are within noise.

The first version of stage 2 made the shelves workload 7% slower than base. The runtime release walk restarts at an owner’s first field after each heap child, and with flat fields each restart re-entered nested records and scanned finished ones. Two runtime changes removed that cost:

  • The walk frees a string in place instead of queueing it and restarting; a string owns nothing and runs no user code, so the order is unobservable. A Shelf with three strings is now walked once instead of four times.
  • Record traversal decides a field’s kind once per field, and walks into a flat record build its Record directly instead of re-deriving it through Type::record.

Per shelf (callgrind, 20k shelves), the whole program fell from 9,716 instructions on base to 8,658; the release walk itself fell from 1,760 to 1,210, and relocation is level with base (685 and 663). Compiled moves no longer fix flat fields but still copy their bytes, so the per-move saving is the fixes, not the copy.

Stage 3: fewer runtime walks (issue #550)

Stage 2 left flat records copied by memmove and released and relocated by the runtime’s generic field walk. Profiles of the stage 2 workloads (callgrind, 20k rows of three-level Path records) put the release walk at 27% of the program, list appends’ copy_payload relocation at 16%, and the memmove calls for 176-byte record copies at 9%. Each change below targets one of those shares.

  • Inline copies. Copies and clears of 33 to 192 bytes are 16-byte vector loads and stores instead of memmove/memset calls. All loads precede the stores, so overlapping copies stay correct. Larger records, such as the 272-byte Shelf, still call memmove.
  • In-place moves. A local moved into a call argument, with only operand pushes between the move and the call, is passed at its own address: the callee copies arguments on entry, so the snapshot copy was redundant. Inlined constructors, generator producers and portable SIMD operations keep argument addresses and still receive a copy (inline::Moves).
  • Descriptor plans. A class descriptor with a record grows from 72 to 80 bytes; byte 72 links a Plan with the word offsets of the strings the record owns (through flat fields), whether they are all it owns, and whether its bytes contain addresses into themselves. The runtime frees such a record’s strings in one pass instead of walking its fields, skips relocation for a record without internal addresses, and skips per-row release for a list of records that own nothing. Records with __del__ at any depth, or any other managed field, keep the walk, so destructor order is unchanged.

The plan is data rather than a compiled hook. Descriptors are defined lazily while relocations are resolved, after the parallel function build; a compiled release function would need every eligible class declared before that build. For records owning only strings the data plan reaches the same result: no per-field dispatch, one load and release per string.

Measurements

Retired instructions (perf stat -e instructions:u, --release, mean of 5, pinned to 8 cores). Main is 240cdd71.

ProgramMainStage 3
Three-level Point/Segment/Path workload2,508,593,1681,312,945,806 (−47.7%)
2M moves of a Path through keep466,240,999376,240,717 (−19.3%)
300k Path rows appended and destroyed2,035,990,651932,943,557 (−54.2%)
300k shelves of nested records appended and destroyed2,380,500,5542,108,859,497 (−11.4%)
Copies fixture, scaled3,024,103,7042,756,512,591 (−8.8%)
Constructor fixture, scaled1,205,241,6331,205,241,629
Nested two-field records copied through a loop280,638277,878
Constructor fixture (wrapped file)262,142262,144
200 loopback connect/accept/close pairs through std.net2,045,0362,009,922 (−1.7%)

Wall-clock medians (15 runs): the three-level workload 254 to 140 ms, Path rows 187 to 116 ms, moves 68.7 to 27.5 ms, shelves 246 to 222 ms, copies 313 to 291 ms. Per Path row (callgrind, 20k rows) the program fell from 5,819 to 3,016 instructions: destroying the list, string frees included, from 1,611 to 110, and the append’s copy_payload from 920 to 204. Per shelf, destroying the list fell from 1,252 to 700 instructions, because its Label fields own only strings.

Inline copies cost code size: .text grew 0.2% to 1.7% across examples (classes 81,121 to 81,313 bytes, http_server_bench 556,705 to 566,257, semantic_search 1,768,878 to 1,773,166, actor_echo 252,641 to 256,193) and 1.3% to 3.2% for the record workloads. A 256-byte limit sped up none of the workloads and grew actor_echo by a further 2.5%; a 128-byte limit cost the moves workload 8%. compile_bench (aot, 1000 × 10) retires 0.18% more instructions (80.53 G to 80.68 G); its median times (332–340 ms on main, 331–336 ms with stage 3) are within noise.

Remaining

Shelves keep part of their walk cost: a Shelf holds ranges and sums, whose payload addresses depend on the active variant, so it is neither string-only nor free of internal addresses. Per shelf, relocation still costs about 930 instructions and the 272-byte record copy still calls memcpy. A plan entry per internal address, tested against the sum’s tag, would cover relocation.

Stage 4: flat tuple and variant payloads (issue #551)

Stages 2 and 3 flattened class fields of class records only. A tuple or enum variant with a class component still stored that component as a slot word followed by its record, so every move rewrote the word: copying a (Point, Point) through a function copied 96 bytes and then ran two address fixes, and a three-field variant carried one fix per class field.

Decision. A tuple component or a field of a variant with two or more fields is laid out as a class field is: a class with a record is stored flat at a fixed offset, with no word. Matching, unpacking, &mut t[0], field moves and constructors use the offsets class records use, and an all-zero flat record marks a moved-out component. A tuple or variant whose components are all flat records without internal addresses copies as plain bytes, so (Point, Point) is a 64-byte copy.

Kept: single-field variants. The payload of a one-field variant, such as Some(point), Ok(point) or Err(point), keeps the slot word followed by the record. Its 128-bit value is the variant’s payload, and sum tags stack in its high word, so the payload must be a full value as top-level values are; dropping the word would change the Option/Result contract that runtime operations, generic code and C headers use. Moving such a sum costs one address fix, the same as moving a bare class value. A workload of 2M Option[Segment] and Result[Segment, Problem] returns, matches and ? propagations retired the same instructions under callgrind before and after this stage (2,041,257,371 and 2,041,261,463), so flattening there has no measured gain to set against the contract change.

Runtime. Record::slots marks a variant or tuple field flat by the same flat_field rule as a class field, so release, retain, copy, relocate, hashing, equality, ordering and formatting walk tuples and variants without change. ranges::flat_record treats a multi-field tuple of flat records without internal addresses as plain bytes, as the compiler does.

Measurements

Retired instructions (perf stat -e instructions:u, --release, mean of 5, pinned to 8 cores). Main is d2078d82.

ProgramMainStage 4
2M swaps and moves of (Point, Point), 300k tuple rows1,758,757,5961,374,894,550 (−21.8%)
2M moves and matches of a Shape with Seg(Segment), Pair(Point, Point) and Tagged(Segment, i64), 300k rows1,820,986,4821,667,438,878 (−8.4%)
2M Option[Segment]/Result[Segment, Problem] returns and matches, 300k rows2,112,425,5882,114,518,618
Three-level Point/Segment/Path workload4,327,731,0124,325,030,979
2M moves of a Path through keep3,572,239,2833,572,239,260
300k Path rows appended and destroyed755,730,005753,030,049
Copies fixture, scaled2,756,512,6282,715,363,538 (−1.5%)
300k shelves of nested records appended and destroyed2,257,209,5732,218,460,498 (−1.7%)
Constructor fixture, scaled505,241,977505,241,981
200 loopback connect/accept/close pairs through std.net2,010,5592,010,513

The Option workload’s 0.1% difference is not in its code: its assembly is unchanged and callgrind counts match. Compiled per call, the tuple keep fell from 103 to 53 instructions and swap from 296 to 224; the variant keep from 218 to 155 and its consuming match from 687 to 542.

Runtime allocation audit

Status: Implemented. Reconciled with the compiler on 2026-10-10. Allocation and traps owns the contract; this page is the per-path audit behind it.

Remaining: public allocator selection, allocator lifetimes, and routing of container buffers (B13).

The memory and SIMD assessment retains the public allocator and buffer design alternatives. This audit records the allocation contract that shipped, not those future APIs.

Allocation is fallible by default in the current language. This audit covers generated programs, not compiler memory. Public allocating operations return Result; there are no try_ alternatives or prefix try displays. Postfix ? propagates failure, match handles it, and .unwrap() explicitly terminates on failure. Nonallocating operations keep their ordinary return types.

PathCurrent contract
Collections and reservationlist[T](), new, with_capacity, and reserve return checked results
Growth and duplicationappend, add, insert, extend, update, and copy use checked reservations and partial-owner guards
Class storageFields are stored inline; Class(...) and Class.new(...) allocate nothing and return Class. A fallible __init__ makes both return Result[Class, AllocError]
Enum storagePayloads are stored inline; variant constructors allocate nothing, for user enums and builtin sums alike
BoxesBox(value) returns Result[Box[T], AllocError]; it is the heap boundary for recursive types
Generator framesA call to a yielding function returns its concrete inline frame directly and allocates nothing
Rangesrange(...) returns a copyable inline value and allocates nothing
Iterator collectionList/set construction and from check output growth; iterator bodies handle their own operations
Literals/comprehensionsConstruction returns Result; failed growth stops evaluation and cleans partial contents
TuplesTuples are stored inline; a tuple display allocates nothing
StringsLiterals are immortal and strings of at most seven bytes are stored inline. + and builders return results. Indexing and iteration yield str and allocate nothing
Console/filesI/O results include allocation errors; partial reads/writes retain their documented effects
Formattingstr.repr and print use checked buffers. Printing an existing string needs no formatting allocation
EqualityStructural equality allocates nothing. It is rejected at compile time for recursive data (B33)
Indexed replacementList/dictionary replacement allocates nothing. Missing dictionary keys trap; adding keys uses insert
DestructionAn intrusive queue needs no allocated work list. User destructor bodies handle their own operation results
Allocator provenanceObjects and container buffers use the global allocator; the object allocator prefix was removed (issue #814) until allocator selection needs provenance

Class arguments evaluate left to right and move into the call. Storage cannot fail, because the instance is inline. Initializer failure drops initialized fields; the instance’s custom destructor becomes active only after successful initialization.

Collection display owners are allocated before entries or iterables are evaluated. The first failed growth skips later expressions and releases the initialized prefix. A ? inside an entry or filter returns from the enclosing function; the outer display is not an exception handler. Earlier side effects are not rolled back.

Result and Option wrappers, builtin error payloads, propagation, and native entrypoint status conversion need no allocation. A main that returns Err reports it on standard error and exits with status one. If formatting the report fails to allocate, it falls back to a fixed allocation-free message.

Failure sweeps exercise construction, growth, duplication, formatting buffers, final string storage, and stopped comprehensions. Native allocation accounting checks reclamation; Miri checks runtime layouts and pointer access. Destructors used in allocation-failure tests either avoid allocation or restore the test budget before explicitly allocating output.

Explicit unwrapping, invalid operations such as out-of-bounds indexing, and fatal runtime invariants can still terminate without unwinding. Public allocator selection, lifetimes, and routing of container buffers remain future work.

References, ownership, and explicit copying

Status: Implemented. Model A was accepted and shipped: mutable values move, borrows are checked, and copying is explicit. The ownership reference owns the contract. This superseded the original ownership proposal, which copied collections on update; that page was removed. Model B below is the rejected alternative.

Shipped beyond this text: returned references, field and element projections, reference rebinding, borrowed matching, user destructors (__del__), and Box[T] as a public owner type. An &T parameter borrows an owned argument implicitly. Allocating operations, including copy, return Result.

Remaining: stored references, relationships among several reference parameters, and disjoint-index loans (B11).

Motivation and current costs

Before this migration, assignment retained a collection handle. Mutation then copied the collection’s entries and retains nested values before changing the copy. The old append and indexed-update paths did this even with only one language-visible owner, making repeated appends quadratic. Updates now happen in place; explicit copy recursively duplicates mutable contents. Assignment, arguments, indexing, and enum payload projection all need review together; changing assignment alone is insufficient.

We want collection duplication to be explicit, while retaining fast compilation, typed function contracts, automatic reclamation, and one public str type.

Python assignment binds a name to an object; it does not copy integers or strings either. Immutability makes sharing those objects safe. Plenty can preserve that observable behavior while storing fixed-width scalars directly and sharing immutable string storage. String rebinding must remain distinct from mutation of the string’s bytes. An exclusive reference to a string binding could replace that binding without making the string object mutable.

Two coherent models

A. Transfer mutable values; borrow explicitly

  • Fixed-width scalars copy cheaply. Immutable str storage can be shared without copying its bytes or introducing another public string type.
  • Assignment and owned arguments transfer collections, generators, and aggregates that contain owned mutable values. Reading a transferred binding is an error.
  • &T lends read access, and &mut T lends exclusive write access. The owner survives a borrow but conflicting access is prohibited until its last use.
  • copy(value) explicitly produces an independent value. For nested mutable collections this recursively duplicates owned mutable contents; immutable strings may share storage. This differs from Python’s shallow copy.
  • Resource types such as generators have no general copy operation.

For example:

def append_three(values: &mut list[i64]) -> Result[(), AllocError]:
    values.append(3)?
    return Ok(())

def main() -> Result[(), AllocError]:
    mut original = [1, 2]?
    mut changed = copy(original)?
    changed.append(3)?                # original remains [1, 2]
    append_three(&mut original)?      # changes original without copying it
    mut destination = original        # transfers ownership
    # print(original)                 # error: moved binding
    return Ok(())

The copy builtin above takes a temporary shared borrow. Ordinary functions expose their borrowing or ownership contracts in signatures; the compiler does not infer cross-function contracts by examining implementations. mut permits writing through an owner or exclusive access path, not creating another unrestricted writer. References to whole bindings and references to their collection elements are different places and must be checked accordingly.

This is the accepted direction for static checking and explicit costs. Its main usability cost is that collection assignment does not preserve both names. It needs no lifetime syntax. Box[T] later became the one public owning pointer, for recursive data.

B. Shared mutable objects, like Python

  • Collection assignment and arguments retain handles to the same mutable object. Both names remain usable; mutations are visible through every alias.
  • copy(value) can naturally be shallow, like Python; nested mutable objects remain shared. Full independence then needs a separate deep-copy operation.
  • Binding rebinding permissions, permissions to mutate an object through a handle, and guarantees that an object cannot change must be distinguished. An immutable binding does not by itself make a shared object immutable.
  • Reference counting handles object lifetime, but does not establish exclusive access or prevent references into relocated buffers from dangling.

For public references into shared mutable collections, choose an explicit policy: conservatively reject conflicting accesses through all possible aliases, check borrows at runtime, or use stable handles that resolve accesses without exposing raw element addresses. Stable handles still require rules for removed/replaced elements and do not themselves provide exclusive access. A handle must not lower to a native exclusive reference without establishing exclusivity.

Runtime borrow guards are a practical implementation for unrestricted shared owners plus temporary direct references. Aliases may coexist; access that conflicts with a live guard fails at runtime. This adds checks and potential runtime errors, including in single-threaded code with nested function calls. It is not a general compile-time borrow-checking guarantee. A static alias analysis can remove some checks or reject more programs, but should not require whole-program analysis to understand a function’s contract.

This model is viable if preserving Python’s mutable aliasing is the priority. Future recursive structs would also require a policy for ownership cycles; reference counting alone would not reclaim them.

Migration requirements

  1. Define operations on source places in a typed control-flow graph before native lowering. Record reads, writes, moves, borrows, reborrows, and destruction.
  2. Separate lifetime ownership from permission to access. Internal retained temporaries and reference counts are not proof that a borrow is exclusive.
  3. Specify argument evaluation and temporary loans, branch joins, loops, and early exits. Use last-use liveness rather than ending every loan at block end.
  4. Define indexing and match projection without silently copying mutable payloads. Initially reject unsupported moves out of indexed storage; add explicit removal operations separately. Match may borrow a scrutinee or consume it, with field cleanup determined by that choice.
  5. Define iteration under mutation. Model A can borrow the collection during a loop; model B needs guarded iteration, invalidation detection, or documented mutation behavior. Do not preserve hidden snapshots by copying implicitly.
  6. Keep public references out of stored fields, returned values, and suspended generators in the first implementation. Add each only with an explicit lifetime contract; reject local references that escape or cross an unsupported yield. (Returned references have since shipped under a single-reference-parameter rule.)
  7. Test conflicting alias access, collection resizing with live element borrows, reference reassignment, reborrows, branch-dependent moves, early exits, and cleanup. A source check must run independently of Cranelift verification.

These requirements are met: the borrow checker runs on a source control-flow graph with last-use loans, before native lowering.

Sources

The selected model borrows specific ideas rather than adopting either language’s complete ownership model.

One immutable str with explicit lengths

Status: Implemented. One immutable str with byte and scalar lengths shipped. Text owns the source contract and value representation the layout.

Shipped differently: a string of at most seven bytes is stored inline in its value word, so a str is not always a pointer. + returns Result[str, AllocError]. Indexing and iteration allocate nothing. Counts are atomic. The runtime is Rust (plenty-runtime/src/strings.rs), so the C files and helper signatures below are historical. Line input is input() -> Result[Option[str], IoError]. Slicing, replace, strip, join, and str.repr were added later.

Shipped since this proposal: std.bytes provides owned binary buffers with explicit fallible allocation, and byte literals provide immutable bytes values. Remaining: broader C text/buffer representations (B27). Language-level slice syntax shipped with issue #579.

Language contract

There is one public string type, str. It is an immutable sequence of Unicode scalar values, stored as valid UTF-8. Assignment and passing arguments preserve independent value semantics while sharing immutable storage internally. mut permits replacing a binding; it does not make the string’s bytes mutable. No public owning-string/view-string distinction, implicit byte array coercion, nullable string, or string-specific lifetime parameters are needed.

Preserve the existing surface: literals, concatenation, equality, len, integer indexing, membership, iteration, typed arguments/results, and collection keys. len counts Unicode scalar values, not bytes or grapheme clusters. Indexing returns a one-scalar str, supports negative indices, and reports an out-of-range index at runtime. A combining mark is its own scalar. Neither equality nor hashing performs Unicode normalization. Surrogate code points are invalid.

Embedded U+0000 is ordinary content. Add the \0 literal escape and allow a literal NUL character inside a quoted string. Default storage has no trailing NUL byte, and no operation searches for a terminator. This requirement applies to static literals, empty strings, concatenation, indexing, and input alike. Additional escape forms and methods can be added independently; this change need not introduce slicing, formatting, case conversion, or a public bytes type.

Representation and ABI

Keep Ty::Str represented by one non-null machine-word pointer in Cranelift and collection slots. The pointer addresses an immutable allocation containing a common reference-counted object header, two unsigned 64-bit lengths, and inline UTF-8 bytes:

PlentyObject header       # {u64 refs, void (*destroy)(void *)}
u64 byte_len
u64 scalar_len
u8 data[byte_len]         # exact logical payload; no terminator

Use the ownership proposal’s common header verbatim: {u64 refs, void (*destroy)(void *)}. Offsets are byte_len=16, scalar_len=24, data=32. The callback lets the base runtime release objects without linking against optional collection or enum runtime code. Do not put a second reference count in the string header. Runtime C owns layout details; compiler literals must use the same documented offsets and 8-byte alignment. Use a shared compiler-side layout definition and C static assertions to catch drift. Current targets are 64-bit; reject unsupported targets instead of silently serializing a mismatched header.

Heap strings start with one owned reference. Static literal headers use the common immortal count sentinel (UINT64_MAX) and a null callback; retain/release must inspect the sentinel before attempting a write because literal data is read-only. Scalar count for literals is computed once by the compiler. The literal’s header and inline data can occupy one read-only object symbol, requiring no data-pointer relocation. The empty literal is a complete aligned header with both lengths zero.

The representation deliberately preserves PTR_TY, clif_type(Ty::Str), source function signature widths, local SSA variables, return values, and tail-call signature compatibility. A two-register (pointer, length) source ABI would force a much broader compiler rewrite without giving users an additional capability here. No monomorphization or Unicode library is needed.

Check byte-count, scalar-count, and allocation-size addition before allocating. Keep logical lengths at most i64::MAX, matching len and index types. Refcount overflow must trap; it must never wrap into zero or the immortal sentinel.

Runtime interface and ownership

Use ordinary C helper signatures taking const PlentyStr * and returning PlentyStr * where appropriate. All observing arguments below are borrowed for the duration of a call. Returned strings carry one owned reference. Generated code releases its owned input temporaries after an observing helper returns; helpers do not unexpectedly consume their inputs.

HelperBehavior
plenty_concat(a, b) -> strAllocate exact combined byte length, copy bytes, add cached scalar counts
plenty_str_eq(a, b) -> i8Compare byte lengths, then memcmp; pointer equality is a fast path
plenty_contains(haystack, needle) -> i8Bounded byte-substring search; empty needle is always present
plenty_println(s)fwrite(data, 1, byte_len, stdout) then newline
internal string length helperReturn cached scalar count
internal scalar index helperNormalize signed index without overflowing; scan UTF-8 boundaries; return a fresh one-scalar string
internal string hash helperHash exactly byte_len bytes, including NUL, using existing hash algorithm

Runtime helpers may return a retained existing string for empty concatenation or other safe optimizations later; callers must already treat results as owned. Strings have no child references, so releasing their last reference frees the allocation directly. Collection insertion retains stored string keys/values; projection retains the returned string; collection copying retains all copied children; collection destruction releases them. Enum string payloads follow the same contract. A string stored in a suspended generator is just another owned frame field, and yielded strings obtain an independent owned reference.

String equality and hashing must be changed together. Otherwise embedded NUL would corrupt dictionary/set key equivalence. Keep descriptor C strings as compiler-internal NUL-terminated metadata: those are not Plenty string values.

Strict validation rejects overlong encodings, truncated sequences, stray continuations, UTF-16 surrogates, and values above U+10FFFF. Compiler literals are already valid Rust UTF-8; concatenation and scalar extraction preserve the invariant. Validation is needed at external input boundaries only.

Iteration cost and cursor interface

Existing CollectionOp::IterGet(Ty::Str) uses a scalar index and rescans from the start. Simply replacing terminator checks with byte bounds fixes correctness but leaves full iteration quadratic. Do not conceal this cost in the design.

For linear string iteration, use a compiler-private byte cursor. The loop keeps an owned snapshot of the source, a byte offset, and the current one-scalar string. Its condition compares byte offset with byte length; scalar extraction copies the one-to-four bytes at that boundary. Its step adds the current scalar’s byte width. All three iteration consumers need this path: source for, comprehension clauses, and list/set construction from iterables. continue must execute the same cursor step before jumping to the condition. The byte cursor is an implementation detail; ordinary s[i] remains indexed by scalar number.

Suggested private operations are TextByteLen(str) -> i64 and TextAtByte(str, i64) -> str; they fit the existing fixed collection runtime ABI. Existing scalar Get(str) remains separate. Every internal byte access must enforce bounds and a UTF-8 boundary, even though well-formed generated loops satisfy both. An alternative unified iterator object is appropriate if the generator implementation already supplies one; it should own the string snapshot and byte cursor, with exactly the same semantics.

For the smallest coherent first patch, retain scalar-index iteration, document its cost, and land cursor iteration immediately afterward. This is a performance staging choice, never a reason to retain NUL-terminated storage.

Exact integration points

  1. runtime/plenty_runtime.c: define the common-header string layout, bounded helpers, strict input validation, and allocation/destruction integration. Remove strlen, strcmp, strstr, and fputs from all language-string operations. fputs remains fine for literal diagnostics.
  2. runtime/collections.c: replace string casts and scans in collection_hash, collection_value_equal, collection_print_value, collection_text_length, collection_text_at, and the descriptor-s dispatch. Delegate to shared string helpers, avoiding two inconsistent UTF-8 implementations. The existing runtime concatenation build places the base runtime before collections, so shared helpers need no separate link step.
  3. src/codegen.rs: change declare_str_data and declare_eof_empty_str to serialize complete aligned headers plus exact bytes in native target byte order. Update runtime comments and read-line fallback assumptions. Retain existing helper signatures at the CLIF level. String-pattern comparisons automatically use the new equality helper.
  4. src/frontend.rs: decode \0 and remove the in-quote literal NUL rejection. Do not loosen identifier or unquoted source token rules. Alias/type rules remain unchanged.
  5. src/frontend/collections.rs, src/collection.rs, and src/codegen/collections.rs: add the optional byte-cursor lowering described above. Ordinary scalar indexing signatures remain unchanged.
  6. The ARC integration pass of the original ownership proposal, since removed, inserts retain/drop at local loads, overwrites, temporaries, calls, returns, and control-flow exits. Do not ship partial retain/drop insertion that can prematurely free strings still reachable through a collection, enum, frame, or local binding.
  7. Update the language reference to describe the string representation; teach one immutable str, scalar indexing, and embedded NUL in the tutorial with runnable examples. Update existing NUL-rejection diagnostic tests.

Validation gates

  • Static, empty, concatenated, returned, mutually tail-called, and nested collection strings preserve their contents through native compilation.
  • len("a\0b") == 3; indexing 1 returns "\0"; raw printing preserves the NUL byte; substring search can find text after or spanning it.
  • "a\0b" != "a\0c"; dictionary/set keys distinguish those values and distinguish "a" from "a\0".
  • Empty-needle search, empty haystack, longer needle, final-byte matches, and equal-prefix unequal-length comparisons never read past payload bounds.
  • One-, two-, three-, and four-byte scalars; combining characters; negative indexing; both index boundaries; empty iteration; nested comprehension iteration; continue and break all work.
  • Input accepts valid UTF-8 including NUL and rejects each malformed encoding class, while preserving existing EOF and CRLF behavior.
  • ARC tests cover strings surviving source-binding replacement, collection projection, enum matching, yield/resume, and early return; repeated string concatenation under AddressSanitizer/LeakSanitizer catches ownership leaks and use-after-free. No writes to immortal read-only headers.
  • C runtime builds with warnings denied; formatting, Clippy, full tests, and executable tutorial examples pass. ASan/UBSan cover bounded operations using payloads deliberately lacking any accessible terminating byte.

Foreign interfaces

FFI must explicitly distinguish (pointer, byte_length) APIs from APIs requiring terminated C strings. Exporting to the latter creates a temporary terminated buffer and rejects embedded NUL, with lifetime tied to the call unless ownership is explicitly transferred. Importing C text validates UTF-8 and copies into the normal representation. FFI declarations and library loading must not dictate the default language string layout or add another ordinary string type.

Foreign text borrows shipped the exporting half as &str as utf8 and &str as c_string. Importing C text is B27.

Sum types and enums

Status: Implemented. Enums, matching, and sums and propagation own the contract.

Shipped differently: enums are inline values and construction allocates nothing; this page proposed heap tagged records. Bare Some, Nothing, Ok, and Err take their type from context, and ? exists. Unit payloads such as Result[(), E] are accepted. Recursive enums are accepted through Box[T] or a collection. An owned match moves an affine scrutinee, and borrowed matching exists. Generic enums and Variant.new(...) were added. Patterns nest, combine with |, and take no type arguments. Integer, str, and bool literals are patterns, and match accepts those types as scrutinees; see matching. The examples’ module-scope statements are no longer accepted.

Omitting a user-defined enum’s qualifier was declined: a bare name is always a binding and a qualified name is always a variant.

Shipped since this proposal: enum methods and protocol satisfaction, human-readable reports, and general anonymous sums, and bounded large dense integer/tag jump tables (issue #395; measurements).

Remaining: deep operations on recursive enums (B33). Guards, enums as dictionary keys or set elements, and match as an expression are unscheduled.

Contract

Add nominal concrete enums, compiler-known Option[T] and Result[T, E], and exhaustive match suites. Preserve one native value per Plenty value and existing independent-value semantics. This does not require structs, user generics, traits, dynamic dispatch, a new optimizer, or a borrow checker.

enum Reading:
    Missing
    Value(i64)
    Invalid(str)

def describe(reading: Reading) -> str:
    match reading:
        case Reading.Missing:
            "missing"
        case Reading.Value(value):
            if value < 0:
                "negative"
            else:
                "available"
        case Reading.Invalid(reason):
            reason

reading = Reading.Value(42)
print(describe(reading))

An enum declaration is module-scoped and contains one or more variants. Variants have either no payload or a fixed sequence of statically typed positional fields. Construction always names the enum and variant. Nullary variants are values (Reading.Missing); reject Reading.Missing() with a useful diagnostic. Payload variants require exactly the declared argument count and types. Arguments evaluate left to right. No implicit integer widening is introduced.

Reserve enum/type names in the same declaration namespace as aliases and built-in type names. Variant names need only be unique within their enum; they do not introduce globals. A binding must not silently change a qualified type constructor into a method call: resolving a type-qualified expression uses the type namespace and rejects ambiguous shadowing, consistently with existing type-constructor rules. Enums do not have methods in this slice.

Type identity and resolution

Two enum declarations with identical variants remain different types. A type alias is transparent: type Input = Reading names the same enum and may qualify its constructors and patterns. Forward references to aliases and nonrecursive enum types are accepted, following the existing module-wide alias behavior.

Collect declaration names before resolving any field types. Resolve a dependency graph containing aliases and enum fields, with explicit visiting/resolved states. Reject recursive enum declarations for the first slice, including recursion through collections. Give the dependency path in the error. This intentionally avoids accidentally promising recursive destruction, cyclic metadata, or mutable cyclic values; recursive sum types can follow with separately bounded implementation.

Ty::Enum(Rc<EnumDef>) is a small integration option for the current compiler. EnumDef contains a compilation-local nominal ID, display name, and resolved variants with field types. Its equality and hash use the ID, not structural field equality. Built-in instantiations are interned by constructor and concrete type arguments, so independently resolving Option[i64] produces the same identity. The resolver owns the intern table; code generation only consumes immutable definitions. A later global type-ID table can replace these references without changing source semantics.

Option and Result

Use compiler-known type constructors with ordinary enum behavior:

def find_positive(value: i64) -> Option[i64]:
    if value > 0:
        Option[i64].Some(value)
    else:
        Option[i64].Nothing

def checked_divide(left: i64, right: i64) -> Result[i64, str]:
    if right == 0:
        Result[i64, str].Err("division by zero")
    else:
        Result[i64, str].Ok(left // right)

The variants are Some(T) / Nothing and Ok(T) / Err(E). Nothing is a variant of one concrete option type, not a universal null value or a None type. Writing type Lookup = Option[i64] permits Lookup.Some(3) and Lookup.Nothing, reducing repetition without inference rules.

Require explicit type arguments or an alias initially. In particular, do not guess the absent side of Result from a constructor argument. Expected-type inference for abbreviated constructors is an ergonomic follow-up; it is not needed for a usable type-safe first implementation. No ?, implicit unwrapping, exception conversion, or truthiness is introduced.

The present frontend represents () as the absence of a stack result rather than a Ty. Keep that restriction explicit in the first implementation: payload fields and Option/Result arguments require stored value types, just as collection elements do today. Thus Result[(), E] initially diagnoses an unsupported unit payload. A concrete enum Outcome: Success; Failure(E) (written as indented variants) already expresses that result. Before claiming a complete standard Result facility, add unit as a first-class value type or erased field metadata; this must be a deliberate follow-up, not an accidental parser limitation.

Matching and control flow

Introduce Statement::Match { scrutinee, arms } and a frontend pattern AST. Patterns are qualified variants with bare immutable bindings or _ for each payload position, plus an optional whole-value wildcard _. The initial slice does not include guards, OR patterns, nested destructuring, literal payload patterns, or implicit capture patterns. These restrictions make finite coverage checking exact and linear in the number of variants and arms.

def unwrap_or(value: Option[i64], fallback: i64) -> i64:
    match value:
        case Option[i64].Some(number):
            number
        case Option[i64].Nothing:
            fallback

Evaluate the scrutinee exactly once. Each case gets a fresh local-name scope; bindings may shadow outer locals, but duplicate names within one pattern are an error. Restore the outer name scope after each arm and after the match. A binding copies an independent value; matching never partially moves its scrutinee in the first slice. A payload containing a collection therefore retains the existing independent-update behavior.

A variant arm covers that entire variant. A repeated variant is unreachable. A wildcard covers all remaining variants and must be last; diagnose a redundant wildcard after explicit complete coverage. Missing coverage lists missing qualified variants. Constructors and patterns from a different nominal enum are errors, even if their variant names and payload types happen to match.

Use the existing BlockResult::Exits rules: continuing arms must agree in result type; arms ending in return, break, or continue contribute no join value. If every arm exits, the match exits and subsequent statements are unreachable. A match in a function’s tail position yields its continuing arm’s final expression, exactly as if does today. Elsewhere the match result is discarded. Arbitrary inline x = match ... is deferred with the same limitation as multiline if; do not introduce a second expression-block grammar just for this feature.

Loop exits inside a case target the enclosing source loop. A continue in a for case runs the existing iterator step exactly once before jumping. Preserve tail-call marking for calls that are final expressions or explicit returns inside case arms.

The backend already has Op::Match, scalar patterns, an independent match checker, and Cranelift dispatch/join lowering. Reuse them rather than synthesizing nested source ifs. Scalar patterns may be exposed in the frontend at the same time: booleans require both values or _, whereas integers and strings require _, matching the existing checker. Do not expand scalar coverage algorithms as part of enum support.

Native representation and operation checking

Initially all enum values use a pointer-sized immutable tagged record. A concrete record carries a u64 tag followed by one u64 slot per active payload field. Small integers and booleans use the same pack/unpack rules as collection elements; strings, collections, and nested enums occupy pointer slots. Tags are assigned in declaration order and are private to the compiler/runtime ABI. Do not promise a C ABI, stable cross-version layout, unboxed layout, or niche optimization.

This representation fits current clif_type, function parameters/results, local variables, branch block parameters, collection storage, and tail calls. It avoids flattening aggregates throughout the operation stack or adding hidden result pointers. Allocation is a runtime cost accepted to keep the first implementation small and compilation fast. Shared immutable records mean assignment does not copy the payload or permit mutation through another binding.

Add a checked constructor operation carrying enum identity and variant index. It pops the declared fields in order and pushes the enum type. Extend match arms with payload destination slots, or equivalently a checked binding prologue whose field types are derived from the selected variant. Prefer arm-local binding metadata: it prevents an unchecked payload projection from being emitted outside a successful variant test. The independent operation checker verifies identity, tag bounds, arity, destination types, coverage, and each arm’s stack result.

A smaller integration alternative, preferred for the first implementation, stores the scrutinee in a hidden typed local, lowers a checked enum-tag operation to an integer, and reuses existing Op::Match with integer patterns. Each selected arm loads its payload from the hidden local with checked enum/variant/field metadata. Frontend coverage checking must run before this erasure; ordinary integer coverage does not know the finite enum domain. Emit an explicit invalid-tag trap fallback rather than treating an arbitrary integer tag as the final enum variant. Document the independent checker’s boundary: it verifies projection identity/index/type, while the structured frontend guarantees the projection follows its variant test.

Native dispatch loads the tag once, selects the arm, and only loads payload fields inside the selected case. A verified exhaustive match still traps on an invalid runtime tag to expose compiler/runtime bugs. Keep a compare chain for small or sparse sets. Sets of at least 256 distinct contiguous integer/tag values use a bounded jump table (issue #395). The B19 measurements support that conservative policy; smaller matches can regress with tables.

Allocation and enum type metadata should live behind a small runtime interface. An implementation can use generated read-only metadata or per-type compiler generated callbacks for equality, printing, and destruction. Do not bake display names into the nominal type identity or require source parsing at runtime. The shared runtime decision is a 16-byte managed header containing u64 refs and void (*destroy)(void *), followed by the enum tag and active payload slots. Immortal objects use UINT64_MAX references and a null callback. Generic retain/ release calls the object’s destructor at its last release; each enum destructor uses active-variant field metadata. This callback boundary also prevents scalar- only programs from acquiring link dependencies on every aggregate runtime module. Ownership/strings implementation owns this header contract; enum support reuses it. Until reclamation actually lands, explicitly document process-lifetime allocation as for current collections; tagging a record does not itself implement ownership.

Collections, strings, and ownership interfaces

Permit enums in list elements and dictionary values, including enum fields that contain collections. Continue to reject enums as set elements and dictionary keys initially, following the current closed hashable-type set. This avoids accidentally committing to public hash/eq traits or deriving hashability.

Current collection descriptors are recursive prefix strings and runtime operations perform generic equality and printing. Merely adding an enum pointer to Ty is not enough: list[Reading] must print and compare its active payload correctly. Extend descriptors with an enum identifier mapped to static variant metadata, or replace descriptors with shared static type metadata in one coordinated change. The enum descriptor must encode a concrete identity and field descriptor table; a bare opaque-pointer descriptor would silently give pointer equality, which is incorrect for independent values. Do not enable collection-containing enums until these runtime paths have tests. Equality requires the same nominal type, same tag, and pairwise equal active fields. No ordering operators are derived.

Printing uses qualified variant names, for example Reading.Missing and Reading.Value(42), with existing collection element/string formatting conventions. String payloads reference the single str runtime representation; enums do not create another string type. Their equality must compare the complete counted string, including embedded zero bytes, once the string runtime is updated.

All currently admitted enum fields have value semantics. Ownership refinement: an enum remains value-copyable only if all its fields are value-copyable; a future resource-owning or generator payload makes the enum move-only. Initially exclude those payload categories. Shared enum storage owns its fields, so final release destroys only the active variant. Match bindings retain/copy independently; no borrowed pattern bindings, references in enums, or partial moves are promised.

Generator next can return a concrete Option[T], and exhaustion is Nothing. The enum type/constructor ABI must therefore be usable from compiler-generated resume functions, without source-level generic functions. Generator state is not an enum language value and does not need to use this allocated record layout.

Implementation sequence and acceptance checks

  1. Add enum declaration parsing, name collection, type resolution and interning; diagnose duplicate names, bad type arguments and cycles. Keep this logic in a focused frontend/type module rather than growing the existing expression match.
  2. Add qualified constructors and typed operation checking; implement immutable record construction, printing, and equality in the runtime/native lowering. Verify enum arguments, results, aliases, and nominal type mismatches.
  3. Add frontend match suites, payload slots, exhaustive coverage and backend projection. Test all combinations of normal results and early exits.
  4. Instantiate Option/Result through the same enum metadata path. Verify nested options/results, aliases, explicit type argument errors and empty variant values.
  5. Connect collection descriptors and managed-object ownership hooks. Add nested enum/collection tests before advertising those combinations.
  6. Update the reference’s implemented/proposed boundary and the tutorial with runnable enum, Option and Result lessons. Its existing executable-guide test checks them.

Required regression cases include multiple fields with different integer widths; zero-field and string payload variants; constructors with side effects; repeated matches of the same value; zero bytes in string payloads; nested concrete enums; independent updates after extracting a collection; list equality/printing with distinct-but-equal enum allocations; wildcard coverage and arm-local shadowing; non-exhaustive/repeated/wrong-type variants; missing and excess payload arguments; wrong payload binding arity; all-exiting and partially exiting matches; break and continue in nested loops; direct and mutual tail calls inside case arms.

Run frontend diagnostics, native executable tests, the executable tutorial, the full existing suite, formatting and Clippy. Add a large concrete enum/alias input to catch accidental exponential resolution or repeated specialization; do not introduce a specialization optimizer or broad benchmark framework for this slice.

Generics, structural protocols, context managers, and error propagation

Status: Implemented. Proposed 2026-10-05; all four parts shipped. Generics, protocols, resource scopes, and sums and propagation own the contract.

Shipped differently: a protocol requirement ends in : with a pass body, and the protocol’s own name stands for the implementer; there is no Self. Generic bodies are checked for each concrete instance, not symbolically against their bounds. ? supports lossless anonymous-sum widening and may erase an error to Failure as a deliberate last resort. File is a builtin context manager. with &mut existing, returned references, and a Result-returning main shipped.

Remaining: combined bounds, protocol composition, and bounds on data (B29); stored references (B11). Erased dyn protocol values, fallible __exit__, yield inside with, and ? in generators are unscheduled.

The protocol-body proposal records the later agreed default/final method and private-hook design; none of it is implemented by this completed proposal. Error composition connects the shipped sum, mapping, and reporting extensions.

Starting point

At proposal time Plenty had concrete classes and inherent methods, Option[T] and Result[T, E], checked local/field borrowing, and deterministic cleanup. __del__ could not fail recoverably. Cleanup already ran on return, break, and continue. There were no user-defined generics, protocols, returned references, stored references, context managers, or ?. Built-in parameterized types were compiler-known special cases, not evidence of a general generic system.

The recommendations are:

  • Add ? directly for Result and Option; it needs no trait system.
  • Use explicit generic parameters with structural protocol constraints.
  • Resolve protocol methods to the concrete type’s inherent methods. Imports identify names; they never activate extension methods or implementations.
  • Implement concrete context managers independently of generics. A returned reference from __enter__ requires additional borrowing rules, however.
  • Begin with infallible scope exit and explicit fallible resource operations. Defer automatic fallible exit until error composition has a complete design.

Structural protocols with explicit generic parameters

Python protocols permit structural conformance without explicit inheritance. Go interfaces likewise describe sets of implementing types through methods. Both provide useful precedents; Plenty can adopt this idea without inheriting Python’s dynamic model or Go’s runtime interface values. Python typing specification, Go interface specification.

Proposed declarations:

protocol Writer:
    def write(self: &mut Self, data: &str) -> Result[(), WriteError]

def write_heading[W: Writer](writer: &mut W, heading: &str) -> Result[(), WriteError]:
    writer.write(heading)?
    return Ok(())

Self means the concrete implementing type and is a compiler-defined name inside protocols. WriteError must be declared or explicitly imported. protocol Writer: follows Plenty’s existing declaration shape. Protocol method declarations have signatures and no bodies; punctuation remains a parser decision.

A concrete class with an accessible matching write method satisfies Writer. No registration, inheritance, global implementation search, or implementation import is needed. A caller need not import Writer merely to call write_heading; the generic function’s signature already names its constraint. Calling a concrete object’s write method never depends on whether the caller imported this protocol. Across module boundaries, required concrete methods must be pub. Protocol conformance must not expose private methods through a public generic function.

Begin with required methods only. Defer protocol fields, default methods, inheritance/composition syntax, associated types, generic protocol methods, operator overloading, and recursive protocol constraints. A method-only protocol avoids exposing field representation and mutable-field variance questions. Python supports substantially more, including mutable attributes and generic protocols; its variance rules illustrate the additional complexity. Python generic protocols.

Type checking and ownership

The declaration’s constraints determine every operation permitted in a generic body. Do not accept arbitrary duck typing and discover missing methods only by expanding the first call site. Diagnose a missing constraint at the generic definition; diagnose a failed concrete conformance at the instantiation.

Initially require exact method parameter and return types after substituting Self and explicit type arguments. Include all ownership-relevant properties:

  • &Self, &mut Self, and an eventual owned receiver are distinct contracts.
  • T, &T, and &mut T parameters are distinct. Matching names alone is insufficient, and mutable access cannot substitute for shared access.
  • Returned references, when supported, include their source/escape contract.
  • Fallibility is represented by the exact Result return type.
  • Future thread-transfer and thread-sharing guarantees are separate checked capabilities; having appropriately named methods does not establish them.

An unknown T must be treated conservatively as an owned value. It can move, borrow, and drop; it cannot be silently duplicated because one instantiation happens to be an integer. An unconstrained generic identity function is useful:

def identity[T](value: T) -> T:
    return value

To duplicate a generic value, require an explicit supported copy capability or a user-provided operation. Allocation failure makes the exact copy API part of the fallible-allocation design. Do not decide it accidentally in the generic checker. In particular, a structural copy method is an explicit method call; it does not silently grant the compiler permission to copy values.

Invariant generic types and exact signature matching are a deliberately small first subset. More permissive variance or receiver adaptation can be evaluated later against actual examples. Protocol satisfaction checks shape and types, not behavioral laws: a method called write might still have incorrect behavior. Use nominal wrapper types when a domain requires explicit opt-in or distinction.

Compilation and representation

Type-check generic bodies against symbolic constraints, then instantiate only reachable concrete uses. Lower those instances to ordinary typed Plenty IR and Cranelift functions with direct calls. Cache by declaration identity plus concrete type arguments; keep the cache independent of import spelling. Report both the generic definition and instantiation chain in diagnostics.

Monomorphization is simple at runtime but can increase compilation time and binary size. Measure instance counts and time, reuse repeated instances, and diagnose recursively expanding instantiations. A documented implementation limit is preferable to compiler hangs. Ordinary recursion at the same type arguments does not need a new instance. Keep type-level evaluation, specialization, and overload search out of the first implementation.

Initially Writer is a constraint, not a storable runtime type. Write def f[T: Writer](x: &mut T) explicitly. Defer def f(x: &mut Writer) rather than letting it ambiguously mean either an implicit generic or dynamic dispatch. Likewise, a heterogeneous list[Writer] is not available in this first model. Generic functions can precede generic classes/enums; the latter require explicit layout, drop, and recursive-type rules.

An erased interface value could later use explicit syntax such as dyn Writer, with a data pointer and method table. It needs its own ownership/lifetime rules, representation, and restrictions on methods using Self and generics. It is not necessary to introduce those constraints for static generic calls. Rust’s dyn-compatibility rules are a useful checklist for this later feature, not a requirement to copy its surface design. Rust dyn compatibility.

? is a small language operation

For Result[T, E], value? evaluates value once, yields its Ok payload, or returns Err(e) from the enclosing function. That function must return Result[U, E] with the same error type after alias resolution. For Option[T], it yields the Some payload or returns Nothing from a function returning Option[U]. A unit success remains useful: write(data)? is an expression statement yielding ().

Use postfix precedence so open(path)?.read()? has a predictable parse. Preserve left-to-right evaluation and full-expression temporary cleanup. Move owned payloads once; do not copy the enclosing sum or its payload. On propagation, clean up exactly the scopes that an explicit return would cross, including partially evaluated expression temporaries and later context-manager exits.

Propagation must not allocate merely to rebuild Err or Nothing. This is an architectural requirement for recoverable allocation failure, not a property of today’s heap-backed enum representation. The implementation needs an allocation-free error carrier or an equivalent verified lowering; reusing an existing container is insufficient if its type metadata or ownership contract changes. Error mapping performed explicitly by user code may itself be fallible, but the primitive propagation path and OOM error construction must work without allocating. See memory, parallelism, and SIMD.

Rust supplies a familiar precedent but permits conversion on Result error propagation. Plenty should initially require exact error types and explicit conversion using match or a later named mapping operation. Do not implicitly convert Option into Result, infer a new union error type, or add an extensible Try protocol in this step. Rust try-propagation expression.

Reject ? in __del__, in unit-returning __init__, and in generators until generator completion/error semantics are designed. Fatal traps remain aborts; ? is ordinary typed control flow, not exception unwinding. A function entrypoint returning Result would require a separate startup/status-reporting convention.

Context managers without a trait dependency

Python’s with calls __enter__, binds its result, and arranges __exit__ on leaving the body. Python also supplies exception information and supports suppression. Plenty has no Python exception model to reproduce. Python context manager protocol, Python with statement.

The compiler can recognize concrete class methods directly, just as it already recognizes __del__. There is no generics prerequisite. A later structural protocol can describe this behavior for generic functions without changing concrete lookup.

Recommended first contract:

# Proposed signatures on a concrete class, not implemented today:
def __enter__(self: &mut File) -> &mut File
def __exit__(self: &mut File) -> ()

# Acquisition reports failure before the context has been entered:
with open_file(path)? as file:
    file.write(data)?
    file.flush()?

open_file, File, and its methods are illustrative APIs, not present library features. In this shape, fallible acquisition happens before with, entry is infallible, and exit performs infallible resource release. Durable writes and other operations whose failure matters remain explicit Result operations. Do not pretend that dropping a buffered file proves its data was persisted.

Keep Python’s meaning of as: it binds the result of __enter__, not the manager by an unrelated special alias rule. Owned entry results and unit entry are also possible. The common &mut self result is the important missing capability: the compiler must prove its origin is the hidden manager and prevent it escaping the scope. It must keep that manager alive, prevent moves/replacement while the loan exists, make the hidden owner inaccessible while the exclusive borrow is active, and end the loan before calling __exit__. This should reuse a general, narrow returned-reference rule rather than invent an unchecked exception for with. A smaller first slice can support entry methods returning owned values or () without returned-reference support.

Evaluate the manager expression once and move an owned result into a hidden mutable local. Supporting with &mut existing can follow after specifying reborrowing; do not implicitly consume and secretly restore an existing owner. For every successfully entered context:

  1. Run the body. Evaluate and preserve any pending return value before cleanup.
  2. Destroy body-local owned values in normal reverse binding order, and finish borrows of the manager. Any owned as binding is body-scoped too.
  3. Call __exit__ exactly once, then destroy the hidden manager normally.
  4. Continue the pending fallthrough, return, ?, break, or continue.

Nested contexts exit in reverse order. Failed acquisition does not enter the context. If entry later becomes fallible, failed entry must not call __exit__; normal manager destruction still cleans partially acquired state. __del__ provides fallback release and must recognize an already closed manager so that exit plus destruction does not release its resource twice. Fatal process aborts are outside this guarantee, consistently with ordinary destruction.

The compiler’s exit bookkeeping and release path must not require allocation. An infallible exit signature alone cannot prove arbitrary user cleanup avoids allocating: the eventual allocation-failure rules must also constrain cleanup implementations and ensure they can release resources under OOM. In particular, collecting exit errors into an allocating list would defeat recoverability.

Reject a returned/yielded borrow of the hidden manager. Initially reject yield inside with: retaining manager state and an exit obligation across suspension, and exiting abandoned generators, need explicit frame support. A later version can support suspension with owned managers once its loan and cleanup rules are verified. No async context managers are planned.

Fallible exit is a separate feature

If a context’s exit fails while its body is already returning an error, there are two errors. Silently overwriting either is a poor default; changing break or a successful return into a failure also needs an explicit enclosing type. Python exception suppression does not solve this typed-control-flow problem.

Start with __exit__ -> () and fallible explicit operations such as flush, commit, or finish inside the block. That gives an honest small feature now. For a later transactional context, a scoped expression could return an explicitly composed error enum with Body(E), Exit(F), and Both(E, F) cases. Its shape, allocation behavior, pending return values, nested exits, and handling of loop control must be specified before introducing fallible __exit__. Composing errors must itself work under allocation failure. A context must not implicitly swallow or transform a caller’s error without visible syntax.

An exit-reason enum distinguishing normal completion, return, and error may be useful later. For the first contract, transactional success is explicit through commit()?, and exit rolls back uncommitted state. This avoids assuming that leaving a block normally is sufficient evidence for a successful transaction.

Milestones and verification

  1. Add ? for the existing sums. Test nested expressions, unit success, owned payload movement, incompatible errors, and exactly-once cleanup on propagation.
  2. Settle module visibility and the representation of generic declarations. Add unconstrained generic functions, then required-method structural protocols with exact signatures. Measure compile time and emitted instance counts.
  3. Add the narrow returned-reference contract and its negative borrow tests. Add concrete with using the same exit machinery as existing control flow. Test nested contexts, acquisition failure, return/break/continue/?, escaping borrows, and repeated-close prevention.
  4. Add generic classes/enums and parameterized protocols when practical library examples require them. Keep erased interfaces and fallible context exits as explicit later decisions.

Protocol signature punctuation, returned-reference syntax, and escaping owned entry results are settled. Open decisions are an explicit conformance assertion for library authors and the eventual fallible-exit syntax. None requires an import-sensitive trait lookup system.

Native owned generators

Status: Implemented. Generators own the source contract and environments the frame and resume ABI.

Shipped differently: frames live inline in their owner’s storage and nothing is heap allocated; see allocation-free generator frames. Each producer has its own frame type, and Generator[T] names only the yield contract. yield moves its operand. Option and Result may own frames. There is no separate generator control-flow graph; continuation blocks are built during lowering. next(&mut g) was added. A generator body cannot use ?, take reference parameters, or yield inside with. The first example’s module-scope loop is no longer accepted.

Remaining (unscheduled): frames in collections, class fields, tuples, and user enums; yield from. Owned generator expressions are now implemented; see the reference.

Source contract

Use the compiler-known concrete type Generator[T] and statement-only yield:

def countdown(start: i64) -> Generator[i64]:
    mut remaining = start
    while remaining > 0:
        yield remaining
        remaining = remaining - 1

for number in countdown(3):
    print(number)

Calling a generator evaluates its arguments immediately, transfers those values into a new frame, and returns without executing its body. The first resume starts the body. Each resume runs until the next yield, bare return, or fallthrough. Each yield evaluates its operand exactly once and produces one owned value of the declared element type. There is no implicit yield from a tail expression.

A function containing yield is a generator function and must declare Generator[T]; ordinary functions may return that type by returning another generator. Determine whether a function contains a yield syntactically before checking its body, including yields in unreachable branches. No nested function syntax currently complicates this scan. An empty generator can contain if False: yield value (written as the usual indented suite). Do not turn every ordinary factory returning Generator[T] into a suspended function.

yield requires an expression of exactly T. A generator’s return accepts no expression and means completion; return value is an error even if value is itself a generator. Falling off the body also completes it. The initial stored type restriction excludes Generator[()], consistently with the enum proposal’s unit-payload restriction. Reject yielding at module scope, yielding in a function with a different declared result type, and nested generator element types.

Only value-copyable types may be yielded initially: scalar values, the one str type, current collections, and concrete enums whose fields are value-copyable. Yielding a managed local retains an independent value; it does not consume that local. A generator may own another generator as a parameter or local, but may not yield it. yield from, send, exception injection, generator expressions, closures, async/await, and public references are outside this slice.

Affine ownership and explicit advancement

Generator[T] is an affine owner of one advancing execution state. Assignment, argument passing, and return move it. Reading the old binding afterward is a compile-time error. mut controls replacing or explicitly advancing a binding; it does not make a generator copyable. Ordinary strings and collections retain their existing independent-value behavior.

mut numbers = countdown(2)
match next(numbers):
    case Option[i64].Some(number):
        print(number)
    case Option[i64].Nothing:
        print("finished")

next is a compiler-known operation on a named mutable Generator[T] binding. It temporarily borrows that owner exclusively and returns Option[T]. The borrow ends before subsequent source expressions execute and cannot escape into a value. This restricted operation needs no public reference syntax, lifetime parameters, user traits, or general borrow inference. Initially reject next(countdown(2)), next on an immutable binding, and non-generator operands with specific diagnostics. Users can name a mutable binding or use a for loop.

Exhaustion is stable: every later next returns Nothing, without rerunning the body or double-dropping locals. A generator that has completed remains an available owner until moved or dropped. Dropping an unstarted or suspended generator releases its captures and locals without executing any remaining user statements. No implicit user finalizer or exception-unwinding behavior is added.

Reject generators recursively inside list/dict/set elements, enum payloads, and Option/Result arguments in this slice. Those types currently copy projections, so admitting an affine field would require aggregate moves and consuming match semantics. Aliases must not bypass this restriction. No equality, ordering, hashing, collection indexing, or len is defined on a generator. Printing a generator should diagnose an unsupported printable type rather than expose its address or consume it.

Use the ownership proposal’s Available/Moved/MaybeMoved analysis on stable local IDs. Join only continuing paths. Every reachable loop backedge, including continue, must preserve availability of outer owners available on entry; move-then-reinitialize is valid, an unreinitialized move is rejected. Break paths join the zero-iteration path after the loop. This is deliberately conservative and should be described as such in diagnostics, not sold as a complete borrow checker.

Iteration consumers

for item in generator evaluates the iterable once and moves it into a hidden owner whose lifetime is the loop. Each condition check resumes once. A yield initializes the loop binding and enters the body; exhaustion exits. continue goes to that resume condition, and break destroys the hidden owner immediately. Returning from the consumer also destroys it. Consuming a named generator leaves the original binding moved even when it yields no values or the loop breaks on its first item. for does not require the source binding to be mutable because ownership is transferred to the compiler’s iterator binding.

Comprehensions and currently supported iterable constructors use the same adapter. For example, [n * n for n in countdown(4) if n > 1] remains eager as a list construction but obtains its input lazily. Preserve existing left-to-right clause evaluation and filter behavior. Creating an inner generator expression by calling a generator function happens each time that clause is entered; trying to repeatedly move the same outer generator binding across an outer loop’s backedge is rejected. Do not introduce Python’s tuple-producing dictionary iterators or a generic iterable protocol as a prerequisite.

Refactor the current collection iteration helper into a private iteration plan: setup operations, condition, item-binding prefix, step, and cleanup. Existing collections/ranges keep their snapshot and index step; strings can use the byte cursor proposed in strings.md; generators have an empty step because resume belongs in their condition. This removes the current assumption that every iterator has an integer index to increment. Every consumer must emit the same step for normal progress and continue and the same cleanup for exhaustion, break, return, and nested scope exits.

The simplest initial generator adapter calls the typed next helper into a hidden Option[T] local. Its condition tests Some; its body extracts an owned payload and drops the option owner before running user statements. Its exit cleans the final Nothing value and generator frame. This reuses checked enum operations and avoids inventing a special two-result source operation. A later optimization can use the native ready flag and output slot directly, without allocating one option record per iteration.

Native frame and callback ABI

Preserve one pointer-sized native value for Generator[T]. The frame begins with the shared managed header and stores a concrete resume callback:

typedef uint8_t (*PlentyResume)(void *frame, uint64_t *out);

typedef struct PlentyGenerator {
    PlentyObject object; /* { uint64_t refs; void (*destroy)(void *); } */
    PlentyResume resume;
    uint64_t state;
    uint64_t running;
    /* Concrete, aligned parameter and local slots follow. */
} PlentyGenerator;

uint8_t plenty_generator_resume(PlentyGenerator *frame, uint64_t *out);

The compiler emits constructor, resume, and destructor functions for each generator function, once per concrete source signature. Constructor and resume callbacks must use signatures/calling conventions compatible with the C runtime; the callback ABI is an internal implementation contract, not an FFI promise. The constructor allocates one concrete frame, zeroes its managed ownership slots, moves parameters into their slots, sets entry state, and returns an owned frame. It executes no body operations. Affine frames are never retained by source copy operations even though they share the common destruction header.

resume borrows the frame exclusively. Returning 1 writes one owned value to out; returning 0 does not initialize out. Pack integers and pointers with the same rules as collection slots. The caller must never read or release out on the completed path. A helper checks the completed state and a running flag, calls the concrete resume callback, and clears the flag. The guard diagnoses an invalid reentrant resume rather than corrupting the frame; public callbacks or concurrency are not introduced by this guard.

A generated typed wrapper converts this protocol into the existing concrete Option[T] constructors for source next and the initial iteration adapter. On a yielded path, construction borrows the owned output, retains a stored managed payload, and then releases the temporary output owner. On the completed path, it constructs Nothing. Keep this wrapper with enum-aware lowering so the base runtime does not acquire an unconditional dependency on enum allocation or metadata. The wrapper borrows, rather than consumes, the generator owner.

Assign dense continuation states in source control-flow order; reserve entry and completed states. Each yield writes the successor state, transfers the yielded owner to out, and returns 1. Completion releases live frame owners, clears their slots, records completed state, and returns 0. The generated destructor releases remaining non-null owner slots in reverse scope/declaration order and frees the allocation. The same destructor handles unstarted, suspended, and already completed frames without needing a distinct callback per suspension point. Internal null ownership slots are not nullable source values.

Typed control flow without a coroutine runtime

Do not suspend the C stack, keep Cranelift SSA values across native calls, use ucontext/threads, interpret the body, or collect all yields eagerly. Flatten the typed structured body into explicit basic blocks for the generator resume body. Keep the existing structured operation path for ordinary functions initially.

Introduce a small typed generator CFG with source spans, local IDs/types, ordinary operations, and terminators for jump, branch, match, yield, and complete. Flatten Op::Loop into header/body/exit, preserving the existing explicit step operations emitted before continue. Translate Break/Continue using an innermost-loop target stack, and translate Op::Match arms and joins directly. This also gives ownership checking and later borrow analysis a reusable place to record edge cleanup, without requiring a new optimizer or immediate migration of every non-generator function.

Require yield to leave an otherwise empty expression operand stack. As a statement, its argument is completely evaluated and consumed before suspension. The resume CFG can then split at yields without spilling partial expression evaluation or callee stack frames. Source calls remain ordinary native calls; only a call to a generator constructor returns a suspended frame.

For the first implementation, give all generator locals frame-backed slots, including hidden iterators, builders, parameters, and shadowed locals. This avoids liveness analysis, SSA reconstruction across resumes, and error-prone copy-in/copy-out ownership. A managed local load retains an expression owner; a move clears its slot; a store releases the old value and transfers the new owner. Scalars use the same frame storage without reference operations. Control-flow definite-initialization checking still rejects source reads from uninitialized locals. Managed slots start internally null for safe cleanup on paths where their declaration never executes.

Emit a state dispatch at resume entry to the entry or saved continuation block. Emit all frame loads/stores in that invocation normally; local expression SSA values need survive only until the current yield. Scope-exit cleanup happens on normal edges, breaks, continues, and returns before their destination. A yield is not scope exit: active owners remain in the frame. Locals dead across a yield may remain in fixed frame slots initially; frame-size/liveness optimization is a later improvement. This is finite per-generator storage, not accumulating one native stack frame per yielded item.

The independent checker must validate yield type, empty residual operand stack, frame slot type/ownership, valid continuation IDs, block-entry operand shapes, and initialized local use. It must reject affine duplication and a yield/complete terminator outside a generator body. It should validate all edges, including resume entry edges, rather than assuming structured source alone proves safety. Generated scope cleanup and definite-move facts must survive CFG conversion.

Calls from a generator before a yield are not tail calls: its frame must remain available for later resume. Ordinary functions forwarding a newly constructed generator can retain existing tail-call behavior. Do not disable ordinary direct or mutual tail calls merely because generators exist in the same module.

Implementation sequence

  1. Finish concrete Option/enums, common managed headers, complete value ARC, and deterministic scope cleanup. These are correctness dependencies for frames, not requirements for Rust-like references or a trait system.
  2. Add Generator[T], recursive placement validation, syntactic generator classification, typed yield/bare-return diagnostics, and affine place analysis.
  3. Add the typed resume CFG and independent checker. Test flattening of nested loops, match exits, and source locations before connecting native frames.
  4. Add frame layout, constructor/resume/destructor emission, runtime dispatch, and typed next wrapping. Keep frame locals in memory initially.
  5. Refactor the private iteration plan and wire generator for, comprehensions, and supported collection constructors through it. Preserve existing snapshot iteration, string behavior, and loop control tests.
  6. Add runnable tutorial lessons and mark only delivered behavior implemented in the reference. Record lazy effects, affine consumption, stable exhaustion, and the deliberately unsupported features explicitly.

Acceptance tests

  • Construction performs argument side effects once but no body side effects; first and subsequent resumes perform only effects before the corresponding yield. Drop of an unstarted or suspended frame never executes later prints.
  • Zero yields, conditional yields, multiple sequential yields, yields in nested loops/match arms, early completion, and repeated exhausted next all behave correctly. A long generator uses bounded native stack depth.
  • Iteration evaluates its source once; nested generators, filtered and nested comprehensions, break/continue, and consumer return resume the right frame and destroy only owners leaving scope.
  • Strings with embedded NUL, nested collections, and enum payloads survive yield and subsequent frame mutation/destruction. Updating a yielded collection does not change the frame’s retained independent value.
  • Diagnose use after assignment/call/return/iteration moves, one-branch moves, repeated moves across backedges, immutable next, unknown element types, generator-valued yields, forbidden aggregate placement through aliases, and valued returns from generator bodies. Accept move followed by reinitialization and exclude returning branches from joins.
  • Test frame destruction before first resume, at every suspension state, after full exhaustion, after break, and after consumer return. Include a frame owning another frame and shadowed managed locals on mutually exclusive paths.
  • Test dynamic allocation/drop counts plus ASan/UBSan where available; correct printed output alone does not catch leaks, duplicated ownership, invalid out slot reads, or double release after exhaustion. Static literals/metadata are excluded from dynamic counts.
  • Keep native compilation, existing non-generator tail calls, all executable tutorial examples, formatting, Clippy, and the full suite passing. Track compilation cost on a function with many yield points; generated code/frame metadata must grow linearly with that concrete body’s size.

Allocation-free generator frames and ranges

Status: Implemented (B25, B26). Generators and ranges own the contract, and environments the frame ABI.

Settled since: the “original design questions” below are answered. A generator parameter or annotation carries a hidden concrete producer type. A factory must resolve to one producer; no sum or boxed form joins different ones. Limits are 64 nesting levels, 256 consumer specializations, and 32-bit frame offsets.

Remaining: generators in collections or class fields (unscheduled); a public iterator protocol (B12).

Objective and boundary

Creating a generator should require no heap allocation for its execution frame. The compiler already emits a resumable native state machine; replacing that lowering is unnecessary. The change is how its state is represented, owned, passed, and returned.

This guarantee does not cover allocations performed while evaluating arguments or running the body. A generator may own a list or file, yield an allocated value, or execute fallible operations. Those operations retain their existing error contracts. Collecting a generator into a list still returns Result.

Proposed construction returns the generator value directly, without an AllocError wrapper. Body execution remains lazy. next returns Option[T], and exhaustion, consuming iteration, moves, and early cleanup retain their current semantics. This is a representation guarantee across supported uses, not an optional optimization limited to immediately consumed generators.

Concrete types and source annotations

The previous Generator[T] identified only the yielded type. A pointer to a heap frame let unrelated bodies, with different frame sizes, use that one type. Inline storage needs to preserve more information.

The recommended design gives each generator body and concrete specialization its own compiler-inferred type and finite layout. Users would still declare -> Generator[T] on a yielding function; this describes its yield contract, while the function identity determines the concrete returned frame. No user needs to spell a generated type name.

The concrete-type design is implemented. The alternative was keeping a uniform, size-erased owned type with fallible allocation; that cannot provide the same general inline-storage guarantee.

The original design questions were:

  • A factory returning Generator[T] should resolve to one concrete frame shape for each specialization. Returning different shapes on different branches requires a finite explicit sum or a future boxed form, with no hidden boxing.
  • Parameters written as Generator[T] could introduce a hidden concrete type parameter, specializing consumers by frame identity. Alternatively, require an explicit generic iterator constraint. Choose one rule consistently for values, references, aliases, and nested Option/Result annotations.
  • Option and Result containing a generator must preserve its concrete layout. The existing scalar-sized wrapper representation cannot simply retain a pointer to a callee’s expired stack frame.
  • Distinct generator bodies are distinct types even when they yield the same element type. Assignment, branch joins, and return diagnostics must explain this without exposing compiler-generated names as required source syntax.

Storage, moves, and destruction

Use caller-owned result storage for returned frames and a compiler-computed size/alignment for each concrete layout. Frames live inline in their owning locals, arguments, wrappers, or enclosing generator frames. The initial scope need not lift existing restrictions on generators in user collection/class storage.

Nested frame layouts form a dependency graph. Detect recursive inline layouts and explain the need for indirection; do not reserve a global maximum frame size or quietly allocate. Cache layout and specialization work to keep compilation predictable.

Moves transfer ownership exactly once. A moved frame cannot retain absolute pointers into its previous storage. Continue rejecting references across suspension; compute accesses relative to the active frame. Revisit tail-call eligibility whenever an argument points into storage that the tail call removes.

Keep initialized-state information sufficient to drop captures and suspended locals on exhaustion, break, return, propagation, or explicit drop. Destroy inline nested frames without freeing their storage as heap objects. Resuming after a move must use the new owner, and repeated exhaustion must stay stable.

How ranges fit

An allocation-free generator can implement range traversal, but a generator itself is single-use and has no length, indexing, or membership operations. Preserve these existing range capabilities with a small inline range[T] value holding its bounds and signed step. Each traversal creates a concrete generator from copied scalar fields, with no captured reference to the range.

Range construction and the existing indexed traversal are allocation-free without generator changes. Sharing a future generator implementation is an optional follow-up, not a prerequisite.

Using such a generator would require an iteration dispatch rule shared by for, comprehensions, and collection constructors. A public iterator protocol is a separate API decision; do not require one merely to remove the existing builtin allocations.

Typed range traversal must preserve unsigned bounds, descending signed steps, termination near integer limits, and once-only argument evaluation. A naive current += step generator is insufficient: even the final unused increment can overflow. Source-level implementation may need checked arithmetic helpers. Zero-step and length-limit policy are separate from allocation failure; today’s runtime rejects both. Do not silently change those rules with this refactor.

Evidence required

Disable heap allocation and exercise scalar-only generators through creation, resume, moves, factories, consumers, nested generators, and Option/Result. Verify cleanup exactly once before first resume, after suspension, on exhaustion, and on all early exits. Include compile-fail cases for incompatible frame types and recursive layouts, plus native memory checks for stack lifetime errors.

For ranges, cover repeated and nested traversal, indexing/membership/length, all integer widths, both step signs, maximum bounds, empty ranges, and unchanged invalid-input diagnostics. Check frame sizes and compilation cost as well as runtime allocation counts. Update the reference and runnable lessons only when the corresponding behavior is implemented.

Callable surface review

Status: Implemented (B31). Function values, closures, and consuming and stored closures own the contract.

Shipped since this review: explicit fallible shared-call erasure supports heterogeneous callback storage. Mutable/consuming erased modes and generator-containing stored environments remain B32; stored borrows remain B11.

Plenty needs callable values for callbacks, small local functions, and owned resource handoff. This review separates representation from ownership and avoids making every capability a prerequisite for writing an ordinary closure.

Decisions

SurfaceConcrete needDecision
def(...)A multiline function expressionKeep; the same typed function body as a named function
def [captures](../...)State without implicit sharing or allocationKeep explicit captures; infer the environment type locally
value, mut value, &value, &mut valueOwn or borrow state, with ordinary mutabilityTeach as the existing ownership and mutability rules composed together
Callable[[A], B] as a value typeStore different named functions in one registryKeep the code-address representation; it must not silently box captured state
F: Callable[[A], B]One reusable callback API for named and captured functionsPrefer this for generic consumers; infer F at calls
Closure[[A], B]Describe a factory’s concrete environment resultKeep for factories and specialized interfaces; remove it from the default consumer lesson
def once [captures]Transfer captured owners out of a callbackKeep the explicit consuming contract; do not infer it from body changes
OnceClosure[[A], B]Describe a factory returning a consuming environmentKeep as an advanced factory annotation
F: OnceCallable[[A], B]A task API that accepts all call modes and consumes its callbackKeep as an advanced constraint, never a stored value type

There are two representations: a code address and a concrete environment. There are two invocation contracts: borrowing an environment for reuse and consuming it. The public annotations describe these differences; they do not introduce four interchangeable forms of dynamically dispatched object.

Collapsing every value into Callable would require either a uniform erased environment with explicit storage, or a type spelling that sometimes denotes a code address and sometimes a producer-specific layout. Neither makes storage costs easier to predict. Keep the distinction visible at interfaces while letting ordinary local bindings infer it. OnceCallable could not be an ordinary method-only protocol today: function-call syntax and capture transfer remain compiler-checked operations. Ordinary protocols can now require a by-value self, but that alone does not describe callable invocation and captures.

Inferring once from moving a capture out would make a body edit change whether callers can invoke it again. An explicit word gives a local ownership contract, like the difference between an owned parameter and a reference. A resource handoff needs that distinction; an ordinary read-only callback does not.

What belongs in libraries

A registry of named callbacks needs only a collection of Callable values. A callback plus explicit state can already be a generic class with ordinary methods. Filtering, dispatching, and adapting these callbacks are library code; they need no new invocation syntax or builtin registry type. Concrete owned environments now use ordinary typed owner slots. Heterogeneous erasure still needs a separate representation and lifetime contract before library code can build an arbitrary closure registry.

The teaching path is named function values, then ordinary inferred closures, then a reusable generic callback API. Factory annotations, borrowed captures, and consuming callbacks are optional extensions for concrete use cases. The functions chapter marks that shorter path. Existing advanced lessons remain referenceable, so simplifying the entry path loses no examples.

This review does not claim that the current names must be permanent. It records why each retained distinction is necessary today, and replaces unnecessary consumer-specific annotations with the common generic interface.

Stored closure environments

Status: Implemented. This records B32’s storage decision. Consuming and stored closures owns the contract and environments the layout.

Shipped differently: classes, tuples, and enums are inline, so only a collection owner allocates. Environments that contain generators cannot enter heap storage.

Shared-call erasure implemented: explicit fallible BoxedCallable addresses heterogeneous shared-call storage (issue #25).

Remaining: mutable/consuming erased modes and generator-containing environments in heap owners (B32); escaping borrowed captures (B11).

A registry needs an owner for callback state, a call signature, and a cleanup contract. The existing generic class mechanism can express that owner when each instance has one known environment type. Closure bodies already have concrete identities and layouts. Preserve those identities when inferring a generic field or collection element, and place the environment directly in that owner’s slot. There is no separate environment allocation and no dynamic dispatch. Growing a containing collection retains its ordinary fallible contract.

Two calls to the same closure factory produce compatible environments. Different closure expressions remain different types, even with identical signatures. A finite family of environments can use an ordinary enum. A registry whose members must have arbitrary unrelated environments needs a separate erased representation; silently changing Callable would hide allocation and alter its thin-function ABI.

Owned boxing would give heterogeneous registries a fixed-size element but require fallible allocation, a call/drop table, and an explicit ownership-transfer contract on failed construction. Bounded inline erasure avoids that allocation but exposes a size/alignment budget, rejects oversized environments, and reserves that budget for every entry. Neither cost is justified for a homogeneous registry. Issue #25’s heterogeneous handler registry and branch-return factory supply the use case for explicit boxing. BoxedCallable leaves concrete storage allocation-free and gives heterogeneous shared-call owners one fallible allocation, a generated adapter, and the existing concrete drop descriptor. Failed construction drops the consumed callback. Broader invocation modes and bounded inline erasure remain separate contracts; erased captures and effects are conservatively ineligible for workers.

Stored environments must own their captures. Borrowed captures cannot escape into heap storage until stored-reference relationships are specified. Invocation follows the existing reusable or consuming contract: reusable mutable callbacks require an exclusive loan; one-shot callbacks must be extracted before invocation. Moving a container relocates inline environment addresses without cloning captured owners. Extraction and destruction must work with allocation disabled. Copy, equality, and hashing remain unavailable for environments, including through containing values.

The native implementation uses type-sized extraction storage, because the historical range-sized scratch area could not hold arbitrary environments. It reuses a removed row in the collection’s existing buffer instead of allocating during pop or remove. Nested environments must relocate after buffer growth, shifting, reversal, and transfer. Worker eligibility must inspect captures and callable bodies through stored environments, including their destructors; storage does not certify thread safety.

Native concurrency contract

Status: Implemented (B22). Scoped threads, bounded channels, executors, futures, parallel map and reduction, cancellation, and timed selection shipped. The concurrency reference owns the source contract and concurrency architecture the runtime. Later additions include atomics and mutexes and atomically counted Shared owners. These do not implement the linear execution obligations proposed separately.

Shipped differently: generator frames and executor or future handles cannot enter workers. Native files and network handles now cross threads under ordinary ownership and loan rules; the compiler certifies native effects without a stdlib exemption. Foreign calls and opaque foreign resources remain unsupported in workers. The audit below records the earlier restrictions. A failed fallible map returns the earliest failed input’s error as ParallelError[E].Worker(E), not an index. Start failure returns ThreadError.System(code), or a SpawnError[F] that carries the unstarted job. Executors, map, and reduction are compiler built-ins over a Rust runtime, not a Plenty library.

Remaining: transfer of custom-allocator values (B13); automatic parallelization (B24); callbacks on foreign threads (B28). B22’s deferred extensions are richer effects, rendezvous channels, streaming and broader mapping, and broader selection.

Runtime audit

ComponentCurrent evidenceConsequence
Type metadatasrc/codegen/metadata.rs emits immutable Type/Variant graphs; aggregates retain static pointersNo Rc<Type> conversion is needed; loaded code and descriptors must stay resident
Shared storageplenty-runtime/src/memory.rs now uses atomic counts, including immutable textCount handoff is tested concurrently; mutable payload access still needs compiler eligibility checks
DestructionThe final release reuses the dead count as a thread-local queue linkOnly the final releaser may enqueue; synchronize before repurposing the count
Inline framesGenerator cleanup uses the same final-release primitive; closures visit captured slotsInline frames finish synchronously rather than entering the heap destruction queue
AllocatorsAllocations retain an allocator identity; the built-in one uses the system allocatorCustom allocators need explicit transfer/deallocation permissions and lifetime guarantees
Files and foreign resourcesFile methods mutate state; foreign interfaces do not certify thread affinityExclude from initial cross-thread eligibility unless a contract proves transfer and drop are permitted
Runtime loadingLibrary leases release lookup ownership while code stays residentResidency prevents dangling code, but grants no concurrent-call or affinity permission
InstrumentationAllocation failure budgets are thread-local; live byte counters are atomicConcurrent tests must keep failure injection local and check leak baselines after joining

Use one atomic header contract initially, retaining the 16-byte ABI layout and immortal-literal sentinel. Retain requires an existing live owner; it does not publish payload writes. Release decrements with release ordering, and the final releaser acquires before destruction. Reusing the count as a queue link is safe only after that handoff, when no other owner remains. Keep queue operations thread-local and preserve synchronous nested drops from user destructors. The ordering follows the ownership-count argument in the Rustonomicon’s clone and drop chapters. Atomic counts do not make mutable payload access concurrent-safe.

Compiler facts, not user-defined marker methods

Compute transfer and immutable-sharing eligibility structurally. Scalars and immutable text are eligible once runtime sharing is safe. A moved collection requires eligible contents and allocator; a shared collection additionally requires that all reachable observations are safe under shared access. An exclusive borrow transfers only within a scope that joins before that borrow ends. Captures and suspended generator locals participate in the same check.

Do not infer a destructor’s thread safety merely from its fields. A destructor or named callback can call a foreign function with affinity requirements even when it captures nothing. Eligibility needs a call-effect summary, an explicit trusted foreign contract, or a conservative rejection. Recursive data analysis must compute a bounded fixed point over nominal declarations, not recurse indefinitely. Imports never change eligibility implicitly.

Ownership at task boundaries

A scoped task can borrow its parent’s data. Its scope joins every started task before locals are reclaimed, including on return, ?, and loop exits. Start failure must preserve the unstarted job. The initial API borrows owners and reusable closures, leaving the job with the caller on failure; a consuming submission API would need to return unstarted ownership with its error. Join transfers one result to one consumer. A task returning Result[T, E] keeps that application error distinct from failure to create a thread or allocate task storage.

Bounded channels own their queue storage and transfer values. A full queue may block under an explicit operation; a nonblocking send reports full/closed with the unsent value intact. Closing wakes waiters. Queued values drop exactly once when no receiver remains. Queue allocation is fallible; send into reserved capacity must not add an unreported allocation.

Familiar executor behavior

Use Python’s explicit executor construction, submit, result retrieval, ordered map, and context-managed shutdown as the interface model. Python also offers a buffer limit for mapped work and warns about workers waiting on work in the same exhausted pool. Python executor documentation.

Plenty should require bounded submission storage and report submission failure with ownership of the unqueued job. Exiting the executor joins running jobs and reclaims unconsumed results; shutdown may cancel pending jobs but cannot kill running ones. Worker code must not assume that waiting on another job in its own pool makes progress. Thread count and queue capacity are explicit settings. Result handles here are blocking task handles, not language async futures.

Parallel map returns results in input order; a deterministic fallible variant returns the error of the earliest failed input after earlier work finishes. Cancellation is cooperative. Dropping partial outputs and joining tasks cannot undo external effects. Parallel reduction needs an explicit reassociation policy, especially for floating point. Ordinary loops and comprehensions retain serial semantics.

Executor scheduling, adapters, and traversal belong in libraries where possible. The compiler provides checked ownership/effect facts; the runtime provides fallible thread creation, synchronization, and deterministic cleanup primitives.

Bounded channel implementation decision

Status: Implemented. Channels own the source contract and concurrency architecture the runtime.

Shipped differently: construction makes one allocation, and the endpoint pair is an inline tuple. Timed selection later added waiter nodes to the core.

Remaining: zero-capacity rendezvous, and send-side or larger selection (deferred B22 extensions).

Plenty channels use a fixed-capacity MPMC ring, explicit endpoint sharing, and fallible construction. Both blocking and nonblocking operations preserve unsent ownership. This decision builds on established contracts while preserving Plenty’s recoverable allocation policy.

Contracts borrowed from existing implementations

Rust’s bounded standard channel supplies backpressure and sender cloning. Its receive side is single-consumer. Crossbeam supports multiple competing receivers and distinguishes an empty/full queue from a disconnected peer. Buffered messages remain receivable after senders disconnect; an unsuccessful send returns the message. Plenty adopts these useful behaviors, without implying broadcast or fairness among waiting threads.

Crossbeam’s public bounded constructor returns handles directly. The inspected 0.5.17 implementation allocates its array and shared counter internally, without a recoverable allocation result. Wrapping it would not satisfy Plenty’s allocation contract. No Crossbeam source is copied into the runtime.

Synchronization choice

One mutex protects queue positions and disconnection state. Separate condition variables park blocked readers and writers. Predicate loops follow the standard condition-variable contract, including spurious wakeups. Normal transfers notify one peer; disconnection wakes all affected peers. User destructors never execute under this lock.

parking_lot offers compact synchronization and useful wakeup policies, including requeueing waiters during notification. Plenty’s currently supported Linux target already has inline, futex-backed std mutexes and condition variables. Source inspection and allocation-disabled native tests establish the needed allocation behavior on that target, so this implementation adds no dependency. This is a target property to revalidate when porting, not a cross-platform std API guarantee.

Ownership and costs

Construction makes one allocation, for the core and ring. The endpoint pair is an inline tuple. Both endpoint headers live inside the pinned core. Sharing increments the appropriate atomic count without allocating another endpoint object. Last-drop disconnects that family; a separate lifetime count keeps the core alive through queued-message destruction and both endpoint callbacks.

Fixed storage bounds queue memory. Each message is embedded using its native typed slot size; inline closure captures do not need boxing. Heap-owned messages transfer their ownership pointers. Receive copies inline payload bytes into caller storage before freeing a ring slot. These are representation moves, with one logical owner throughout.

Receiver ownership cycles are rejected by a finite compiler type-graph check: queued messages must not retain the receiver whose last-drop would drain them. Ordinary recursive data and sender-based reply paths remain valid. Endpoint context managers reuse lexical cleanup; placing the receiving guard after its producer task closes that handle before joining on early exit.

The mutex serializes transfers and provides a straightforward publication point. There is no lock-free or strict fairness claim. Positive capacities cover the initial contract; zero-capacity rendezvous is a distinct handshake rather than a disguised one-slot queue. Scheduling and subsequent extensions remain in the backlog.

Named arguments and fresh defaults

Status: implemented for named functions, methods and constructors. See the current call rules for supported defaults and deliberate restrictions. This is the narrow revision of B36, separating named arguments/defaults from the older unpacking proposal. It does not approve splat unpacking, variadics, /, or * signature markers. The discussion below records the design rationale; named calls and fresh expression defaults now use compiler-generated wrappers with ordinary checked ownership. Callable values and builtins remain positional. Protocol-constrained generic bodies currently require full positional method calls conservatively.

Construction should communicate structure

At baseline bef41b1, stdlib/std/json.plenty’s decoder (line 701) builds Decoder with 28 positional fields. Many have identical types. Names would help review, but 28 independently supplied values remain a poor abstraction.

Prefer an explicit Decoder.__init__ that accepts meaningful inputs, initializes internal state by field name, and establishes invariants. An instance cannot be created uninitialized and then escape while a caller fills its fields: current initialization rules require all fields on every successful initializer exit. This is desirable.

Use a configuration class for coherent caller choices; std.json.Limits already does this. Group cursor/location, token-decoding, and container state only where the grouping has a real invariant. Passing a new class containing the same 28 unexplained arguments merely relocates the problem.

A builder is useful when construction is incremental, validation spans several settings, or a sequence of required choices matters. Its build returns a valid object or a typed error. It is not necessary for every record. Current explicit initializers allow only unit or allocation failure, so factories remain useful for domain validation errors. Typestate can strengthen a builder when required stages justify extra types.

Other review sites: json.encoder, http.server.config, http.client.config, and http.body.multipart_limits. Names help callers distinguish timeouts, capacities, and limits; structure and constructors keep internal state private.

Named call arguments without splats

Example call syntax:

def connect(host: &str, port: u16, timeout_ns: u64 = 1000000u64) -> Result[Connection, NetError]:
    ...

connect("example", timeout_ns=5000000u64, port=443u16)
  • Positional arguments must precede every named argument. Names may then appear in any order. No positional argument may follow a named argument.
  • Bind names statically; diagnose unknown, repeated, missing, and multiply supplied parameters, including a positional value plus the same name.
  • Evaluate the callee/receiver once, then supplied argument expressions once in written order, regardless of parameter order. Only placement into parameter slots is reordered. Type-check each expression against its resolved parameter.
  • Preserve move/borrow checking and early-propagation cleanup of evaluated arguments. A method receiver reservation must span the same evaluation steps; reordering must not relax alias restrictions.
  • Defaults follow required parameters in a declaration. Naming a required parameter does not make it optional. Explicitly passing Nothing is a value, not an instruction to evaluate a default.
  • Cover named functions, statically resolved methods (excluding self as a supplied argument), and generated/explicit constructors. Constructor visibility and private-field restrictions remain unchanged.

Parameter names become source API commitments. A rename can break callers. Callable values currently have positional structural signatures without names; keep their calls positional. Protocol parameter names currently need not match implementations. Initially keep calls through protocol constraints positional too; do not infer a keyword contract from whichever concrete class specializes the body. Adding protocol-level names later would be an explicit compatibility decision. Builtins require a deliberate inventory of stable parameter names.

This needs argument-binding and lowering work, diagnostics, imported signature metadata, and ownership tests. No runtime argument dictionary is required, but calling it a trivial parser change would underestimate the work.

Defaults: two stages with one freshness rule

Start with the older proposal’s allocation-free literal defaults. Then consider fresh expressions, using the same omitted-argument semantics. Defaults are evaluated once per omitted argument per invocation, never stored as a shared mutable object. Python’s documented definition-time evaluation is explicitly not the intended model. Python default arguments

For fresh expressions, evaluate supplied arguments first in written order, then omitted defaults in parameter declaration order, before entering the body. Resolve default names in the function’s defining module. Initially prohibit references to parameters or self, avoiding reordered/dependent initialization. Reject dependency cycles rather than inventing a dependency scheduler. Callable values still require full positional arity; omission is not part of their type.

A default expression must produce the parameter’s value using ordinary typing. Construct a fresh owner for an omitted owned parameter. For an omitted borrowed parameter, a proposed extension materializes a fresh temporary owner and lends it to that parameter for the call. Use Plenty’s existing annotation placement:

def work(data: &mut list[i64] = []?,
         lookups: &mut dict[str, f64] = {}?) -> Result[Option[bool], AllocError]:
    ...

The caller supplying either argument must still supply a legal exclusive loan. An omitted argument gets separate storage for every call. Mutation of an omitted scratch buffer is discarded at call completion; callers needing that output must pass storage or receive an owned result. Empty-container expected types come from the parameter. References into these temporary owners must not escape in returned references or borrowed closures. Reject such calls when the return provenance could include an omitted temporary; never extend its lifetime implicitly or store it in an aggregate.

Allocation failure cannot disappear into an Option

In Plenty, [] and {} produce Result because allocation is fallible. A bare default =[] therefore does not currently type-check as a list, and an Option[bool] return has no place for AllocError. Freshness alone does not solve this mismatch.

Recommended rule: explicit ? inside a default propagates through the declared callee result contract, not the lexical caller’s return type. A call with an omitted fallible default still has exactly the declared return type. The example therefore returns Result[Option[bool], AllocError]. On default failure the body does not run, evaluated owned arguments/defaults are cleaned up, and that call produces Err. Supplying all arguments does not change its static return type. Error conversions must be explicit as elsewhere (or follow a separately accepted error-set rule); no hidden Failure conversion or trap is introduced.

Lower this as a compiler-generated default wrapper or equivalent call-site control flow with callee-local propagation. Simply pasting a ? expression into the caller would give incorrect semantics. Diagnostics should identify the declaration/default and the call that omitted it. Keep this stage separate from allocation-free defaults because its cleanup and effect semantics are substantially more involved.

An alternative is to accept an optional owned buffer and construct it inside a fallible function today. That is more ceremony but uses existing rules. Do not make omission alone change a function’s return type or silently abort on OOM.

Evaluation criteria

Use named arguments on HTTP limits and timeout configuration, and compare a structured JSON initializer against the current 28-value constructor. Test out-of-order keywords with observable side effects; positional/name collisions; imports, generics, methods, constructors and visibility; cleanup after argument or default failure; repeated mutable-default calls; explicit arguments skipping defaults; nonescaping default loans; and callable/protocol restrictions.

Field defaults are a separate construction feature, not implied by parameter defaults. Avoid combining this work with splats or a general builder framework. The reference and runnable tutorial describe the implemented behavior. The stdlib’s http.body.multipart_limits now uses named fields as a small adoption example; broad constructor redesign remains a separate refactor.

Human-readable formatting and error reports

The formatter families were subsequently unified for issue #634. Current reports use format[W](self, output: &mut std.fmt.Formatter[W]) and the shared std.fmt.FormatError; see the report reference. The concrete destination design below records the original implementation.

Status: Implemented. Enum methods, ordinary enum protocol satisfaction, the concrete std.report formatting core, dispatch over anonymous sums, entrypoint selection, and typed context formatting have shipped. Entry reports, formatting, and std.error define current behavior.

Floating-point report formatting and dynamic cause-chain interfaces are extensions, not prerequisites for this completed core. The anonymous-sum decision retains builtin Failure as a last resort after removing the cases that forced erasure. The composition overview explains how mapping, sums, reporting, and typed streams fit together; remaining work stays in B71 of the backlog.

Separate data, presentation, and reporting

An error stays ordinary typed data. Neither Result[T, E] nor ? requires a reporting protocol. Finite sums compose possible causes without deciding their human-readable wording or recovery policy.

Structural representation remains available for debugging and fallback reports. A general human-readable formatting protocol lets classes and enums supply messages. It is useful beyond errors; a nominal error marker is not required. Context[E] and domain wrappers retain operation, path, and cause information. Application reporting chooses destination, prefix, fallback, and exit status. Formatting a value does not itself print or terminate the process.

Entrypoint reports select Formattable when the concrete type satisfies it, otherwise use structural rendering with special explanations for some builtins. Context.describe() still uses structural cause text, while describe_with() accepts caller-provided wording. std.fmt.Formattable provides borrowed formatting into a concrete destination; entrypoint reports and eligible Context values use it. Classes and enums can also supply the existing std.fmt display/debug methods. Those methods use a generic writer and allocated scratch and remain useful for richer application formatting.

Borrowed formatting into a destination

The protocol borrows its receiver and writes text through a mutable formatter. An enum implementation matches its variants and formats their details. It does not consume the error or require constructing an owned string. A convenience API may collect the same output into a fallible allocated string.

The core formatter must support an allocation-free reporting path for fixed messages and scalar payloads. This is not a promise that every user-defined implementation allocates nothing: custom computation and string collection keep their ordinary fallibility. Formatting errors must be explicit, must not replace the original application error, and must terminate reporting rather than recurse into reporting the formatting error itself.

The original implemented contract was Formattable.format(self, output: &mut Formatter) -> Result[(), FormatError] in std.report. Formatter stores a private inline destination enum: stderr or an owned byte collector, plus byte/depth counters. The stderr variant creates no scratch buffer or heap owner. Text, signed/unsigned integers and booleans need no allocation; collection and finish explicitly return storage failures. Floating-point spelling and rich format specifications remain with std.fmt in this first core slice.

render borrows the value, checks its protocol and restores the depth counter on success or error. Each native stderr attempt reports its exact byte progress; interruption is retried, and other output errors make that formatter terminal. No boxed protocol value, generic writer parameter, prefix, newline, exit policy, or automatic fallback is hidden in this core. See its reference for byte limits, partial output, and collection guarantees.

Enum protocols and sum formatting

Ordinary protocol satisfaction now includes enums, with enum methods subject to the same visibility, receiver, ownership, and effect rules as class methods. Methods use ordinary indented def declarations alongside variants; shared and mutable receivers borrow the enum. No special error-only implementation mechanism is needed.

For the formatting protocol, A | B satisfies the protocol exactly when every normalized member satisfies it. Generated dispatch examines the tag and borrows the active payload; it adds no allocation or hidden existential container. Aliases and singleton normalization give the same formatting behavior.

This rule does not automatically forward arbitrary protocols through sums. Methods involving Self, mutation, or different result types require a separate general contract. The first version grants only the specified formatting behavior.

Reporting at the application boundary

The entrypoint policy chooses human-readable formatting when the concrete error type satisfies the protocol, otherwise structural representation. For a sum with an unformattable member, use structural reporting for the sum as a whole; do not introduce a hidden per-member relaxation of the protocol bound.

If the selected rendering fails or is unavailable, emit a fixed minimal report on a best-effort basis. The original error is destroyed once and the failing entrypoint still returns status one. Intermediate propagation remains silent. Output failures, including a closed stderr pipe, must not trigger another report. Preserve the existing entrypoint treatment of stderr failure.

Streaming can fail after emitting a prefix or part of a message. Normal output has a 16,384-byte budget and depth limit 64. A formatting failure appends the fixed suffix [report unavailable]\n outside that budget; a terminal transport failure stops output instead. Reports are not atomic, and the reporter never retries a user formatter after partial output. Ordinary traps retain their existing non-unwinding semantics; a best-effort reporter cannot catch arbitrary traps.

Context formatting presents the operation/path and then the typed cause. Explicit caller-provided descriptions remain useful. This avoids duplicating the cause in both a wrapper’s message and an automatically appended chain. A heterogeneous source() interface, dynamic error objects, and general cause-chain traversal are deferred; typed context composition does not depend on them.

Acceptance criteria

Check class and enum protocol satisfaction, missing-method diagnostics, borrowed payload preservation, sum dispatch for every member, singleton/alias behavior, and rejection of a formatting bound when any member lacks it. Verify that plain unformattable data remains legal in Err.

Exercise fixed messages under forced allocation failure, scalar formatting, string collection failure, nested contexts, custom formatter failure before and after output, unsupported structural rendering, closed stderr, exactly-once payload destruction, and identical compile-and-run/native executable reports.

The existing context description/report convenience APIs preserve their behavior. Conditional Context[E] formatting is a narrow compiler-supported rule for the trusted standard type, rather than a general conditional-method feature. The formatter method, result and destination API are defined in the std.report reference; the entrypoint reference defines reporting limits and fallback.

Explicit error composition

Status: Implemented through HTTP client adoption. Library mapping, general anonymous sums, human-readable reporting, typed streams and HTTP framing have shipped. The HTTP client’s head, write and body paths preserve concrete causes without internal stored-error retrieval. Installed CLI/data and JSON-service checks cover application errors, reports, recovery and cleanup. ? supports lossless error-set widening without user-defined conversions. Builtin Failure remains a deliberate last resort; issue #342 is closed. The legacy response Reader bridge remains available beside typed response reads and partial body-collection progress.

The agreed B71 acceptance batch is complete. Floating-point report formatting, dynamic error objects and other additive extensions remain in B71; the benchmark-template migration stays deferred during performance comparisons. The companion error sets page records the sum contract and the last-resort role of builtin Failure; formatting records the shipped enum-capable reporting design. Typestate addresses lifecycle invariants.

Reading this family

QuestionDesign and current contract
How do I deliberately translate an error?Mapping below; std.error
How do I retain several possible error types?Anonymous-sum design; current sums
How do I present an error to a person?Completed reporting design; std.report
How do stream adapters preserve a transport cause?Experiment and production evidence; std.io.typed, std.http.typed

Problem and recommendation

At the original investigation baseline, ? propagated an identical error type, except for deliberate erasure to Failure. Library adapters repeated success forwarding just to translate failures. A second, independent problem is that std.io.Reader fixes its error type to StreamError: HTTP adapters store richer causes in a separate field and return a generic error through the stream interface.

Explicit result mapping and an isolated parameterized-stream experiment established the baseline. General anonymous sums and human-readable formatting for enums and classes subsequently shipped. Issue #342 removed the export and retained-job cases that forced Failure; the builtin stays as a last resort. Production typed-stream adoption then removed the HTTP client’s internal error side channels. Keep named domain errors where they encode useful policy: neither mapping nor sums alone can recover causes discarded by a stream API.

Evidence in the stdlib

Locations below describe the inspected baseline, commit bef41b1; function names are the durable anchors when line numbers move.

LocationExisting burdenCandidate improvement
stdlib/std/http/body.plenty:79–140, allocated, http_allocated, lifted, http, json, storedRepeated Ok forwarding around error translationResult mapping; retain classifiers for deliberate allocation normalization
stdlib/std/http/client.plenty:333–387, alloc, bytes_of, framed, netted, parsedSeveral near-identical wrappersMap only the failure payload
stdlib/std/json.plenty:121–147, faulted, reserved, unplacedNested adaptation before parsing can proceedShared mapping operation with named classifiers
stdlib/std/http/client.plenty:599–640, Wire.noted, fault, writtenError stored in _failure, generic stream error returned, cause retrieved laterPreserve the transport error as part of the returned type
stdlib/std/http/client.plenty:690–707, Response.read, take_failureCaller must retrieve richer error separatelyTyped stream error path with optional compatibility adapter
stdlib/std/http.plenty:633–647, BodyReader.readFraming faults become Io(InvalidInput) plus side storageParameterized framing/stream error
stdlib/std/error.plenty:62–85, context, path_contextAlready preserves a typed cause and handles failed context allocationReuse this foundation; do not replace it with unconditional boxing

These are opportunities, not claims that all conversions can disappear. BytesError.InvalidUtf8 becoming BodyError.InvalidUtf8, for example, is intentional public API policy.

Explicit mapping

Possible future method spelling (not current executable Plenty):

body = std.json.encode(value, limits).map_err(body_json_error)?

def body_json_error(cause: JsonError) -> BodyError:
    BodyError.Json(cause)

The operation consumes Result[T, E] and a mapper constrained by F: OnceCallable[[E], G], returning Result[T, G]. The implemented std.error.map_error(result, mapper) provides these semantics as an ordinary library function. Method syntax remains an ergonomic choice, not required semantics. See the reference and the tutorial.

  • Evaluate the result and mapper expressions once, left to right.
  • Invoke the mapper exactly once on Err, never on Ok.
  • Move the original error into the mapper; forward success without copying.
  • Drop an unused mapper and its captures on success. Ownership and early-exit cleanup apply normally, including failure while evaluating the mapper.
  • Mapping itself allocates nothing. Callback construction and execution retain their ordinary effects; do not silently box captured callbacks.
  • A mapper returns G, not a secretly flattened Result[G, X]. Fallible context enrichment needs an explicit API and policy for preserving the cause.
  • ? still performs no user-defined conversion. The call site names the policy.

This removes repetitive success branches while leaving semantic conversions reviewable. A helper can still be worthwhile when it communicates a domain decision. Do not mechanically inline all classifiers or introduce complex closures solely to reduce line count.

Rust’s thiserror generates error implementations and selected conversions; it reduces declaration boilerplate while retaining a named error type. Plenty could later consider declared conversions, but implicit conversion at every ? would need ambiguity and coherence rules. Explicit mapping is the smaller initial contract. thiserror documentation

Does an error protocol solve it?

The idea is coherent, but has three distinct meanings:

MeaningWhat it providesFit today
E: Error generic constraintOperations available on one statically known EClasses and enums can satisfy protocols; a reporting bound is optional
Result[T, Error] holding arbitrary implementationsAn owned existential value with hidden concrete typeNot supported; protocols are not runtime value types
`Result[T, AB]`One of a finite, statically declared set

A generic bound does not allow one specialization to return unrelated concrete types on different branches. It still needs an enum, union, or erased container. Nor does a common reporting interface establish how to classify interruption, retryability, partial progress, or a framing failure.

An existential error container must specify storage, destruction, dispatch, thread transfer, reporting, and whether downcasting is supported. Arbitrarily large owned payloads cannot all fit a fixed-size inline object; boxing or an explicit capacity/fallback policy is needed. Allocation while reporting an allocation failure is a particular problem for Plenty. Borrowed errors cannot generally escape the failing function’s locals.

Rust trait objects make runtime dispatch explicit. anyhow offers an erased error with contextual reporting and downcasting, trading away an exhaustive signature of possible causes. That can suit application boundaries, but it is not equivalent to a finite typed sum. Rust trait objects, anyhow documentation

Do not make a reporting protocol mandatory for error payloads: Plenty currently allows ordinary typed data in Err. Reporting and recoverability are separate. The implemented formatting protocol provides presentation without an existential error container. Enums can satisfy protocols, and anonymous sums forward this formatting protocol when all members qualify. Builtin Failure stays as a last resort, for when no precise error can be written (issue #342); an erased error container and heterogeneous cause-chain interface remain deferred.

Removing the stream error side channel

Current parameterized protocols already substitute type arguments in result positions. A minimal Source[E] and generic reader using an enum error type also passed the local debug compiler smoke check. Prototype the following shape in an isolated library module, before changing std.io:

protocol Reader[E]:
    def read(self: &mut Reader[E], buffer: &mut Bytes,
             start: i64, stop: i64) -> Result[i64, E]:
        pass

enum ReadFailure[E]:
    Source(E)
    UnexpectedEof
    InvalidProgress
    OutOfRange
    Allocation(AllocError)

This is a proposed interface shape, not a claim that changing Reader alone is sufficient. Helpers generate their own errors, and retry logic needs a way to recognize interruption. Keep that policy explicit: use a classifier callback or a protocol parameter for a classifier class, or require a normalized stream envelope that distinguishes Interrupted from Source(E). Compare these options against concrete adapters before selecting one.

read_exact[E, R: Reader[E]] would return progress plus ReadFailure[E]; copy_all must preserve independent reader/writer error types and the failing side. HTTP framing would similarly need HttpError[E] (or a distinct generic framing error), separating transport causes from protocol failures. Existing StreamError is still useful for OS adapters and compatibility facades.

Avoid recursive type equations such as making ClientError contain HttpError[ClientError]. Define a transport-only error underneath framing and an exchange-level error above it. Separate EOF, malformed progress, cancellation, and protocol faults; arbitrary E cannot be interrogated as if it were StreamError. Parameter threading increases signatures and specialization; measure whether eliminating side storage pays for that complexity.

Retain old APIs during a migration, or make a deliberate coordinated breaking change. Remove _failure/take_failure only when every relevant adapter and caller receives the full cause directly. Mapping alone cannot achieve this.

Acceptance criteria

The evidence companion records the mapper comparisons, prototype tests and production adoption. The HTTP qualification record records the installed CLI/data and JSON-service acceptance. The criteria below preserve the checks that motivated those changes.

Compare before/after HTTP body adapters, a JSON classifier, and one complete HTTP-over-stream path. Count adapter functions and side-channel fields, not just lines. Verify exact cause preservation, once-only mapper invocation, success without mapping allocation, destruction of affine payloads/captures, partial progress, interruption, cancellation, and allocation failure. Check diagnostic quality for incompatible error types and protocol inference. Tutorial/reference changes belong with an implementation, not this proposal.

Prototype findings

The experiment record records mapper adoption, specialization limits, and the typed-stream prototype. It distinguishes the isolated examples from the public library APIs.

Additive production slice

The production slice provides typed I/O helpers and fixed-length/chunked/close-delimited HTTP body readers, validated-head integration and full client adoption. Internal transport side channels are removed; the legacy response Reader bridge remains for compatibility. The evidence page records the delivered scope and limits.

Error-composition experiments and typed streams

Status: Implemented through HTTP client adoption. This is the evidence companion to the composition overview, not a separate proposal or work queue. It accompanies the completed B71 acceptance batch. Mapping, typed streams and framing, and the HTTP client’s head, write and body paths have shipped. Current contracts belong to std.error, std.io.typed, and std.http.typed.

Prototype findings

The first implementation needs no compiler or runtime changes. map_error uses OnceCallable, a mutable local callback binding, and an ordinary match. The mutable binding accepts stateful reusable callbacks as well as named functions and consuming closures. A callback is owned even when unused, so success drops its captures. Error values need no reporting protocol.

The initial representative stdlib changes covered three result adapters: std/http/body.plenty removes json and maps directly in read_json and send_json; std/http/client.plenty uses mapping in framed; and std/json.plenty uses it in faulted. Three explicit success-forwarding branches disappear. Two convenience adapters remain and one non-generic bytes_fault classifier exposes the JSON policy. The number of named functions is therefore unchanged, while one adapter disappears completely. Other conversion helpers initially remained available for comparison.

A wider initial conversion of eleven adapters exceeded the compiler’s global 256 generic-function specialization cap in the offline HTTP JSON service test. Keeping a generic convenience adapter around map_error adds another instance for each success/callback type. The follow-up for issue #141 separates a 256-level dependency-depth guard from a 16,384-instance total resource budget; independent specializations no longer consume the depth guard. It restores the wider eleven-adapter adoption: body allocation, byte classification, HTTP and storage conversions; client allocation; and JSON allocation and location-free errors now also use mapping. Their conversion policies are unchanged. This addresses the observed scaling constraint without removing finite compiler limits or introducing allocation. See specialization limits.

Worker integration exposed a real ergonomic restriction: thin function values lose their target identity inside the generic helper, so thread effect checking rejects their indirect calls. The stdlib adopters use concrete consuming closures (def once (...)) instead; their bodies remain inspectable and allocate nothing. Named classifiers are called directly from these closures. This keeps worker checks intact but adds a wrapper at those sites. Mapping does not solve that existing callable-design limitation.

The isolated modules in examples/error_composition/ explore a normalized StepError[E] envelope: Interrupted requests retry; Source(E) preserves everything else, including cancellation and allocation failure. Reader[E] and Writer[E] feed read_exact, write_all, and copy_all; helper-generated failures have their own variants. Copy errors distinguish reader and writer types and count bytes delivered to the writer, including partial writes. Caller-provided scratch storage avoids allocation in the helpers.

FixedBody[E, R] demonstrates Content-Length body framing over a typed transport. Its failures distinguish Transport(E), truncation, invalid progress, invalid range, and reuse after failure. Interruption leaves it usable; terminal framing/transport failures mark it broken. The full body-copy path returns the original transport cause through FramingError[E] and CopyFailure[FramingError[E], W]. It contains zero error-storage fields and zero retrieval methods. At that prototype stage, production _failure fields and take_failure methods were retained: the experiment alone did not justify their removal.

This is one complete body-transfer path, not a complete HTTP client. It assumes a previously validated Content-Length and response semantics. It does not parse heads, implement chunked/close-delimited framing, perform TLS or network I/O, or manage connection reuse. Scripted sources exercise rich TLS-like errors and cancellation as typed data, not real handshakes or cancellation tokens. Those boundaries must be covered before a production migration.

Protocol-bound inference recovers the error types from concrete methods, even through the framing adapter: fixed_body(source, length) and copy_all(&mut body, &mut writer, &mut scratch) need no explicit type arguments. The generic definitions and nested error signatures are still long. A normalized envelope keeps retry policy simple but requires adapters to wrap their failures. A classifier callback would leave transport signatures alone, at the cost of threading retry policy through helpers; it remains an alternative, not implemented here. Arbitrary errors must not be retried by guessing their meaning.

Focused tests live in tests/test_error_composition.rs and tests/test_typed_stream_prototype.rs. They cover mapper ownership, allocation failure, nested results, evaluation order, progress, retry, EOF, cancellation, failure-side attribution, and typed transport preservation. These findings supported the additive production framing below; mapping remains independently usable.

Additive production slice

std.io.typed now makes the prototype’s typed protocols and progress helpers available without copying example modules. Its owned legacy reader/writer bridges preserve the returned StreamError and classify only explicit interruption as retryable. A consuming free function recovers each bridge’s stream for explicit flushing, closing or seeking. The private optional slot supports extraction under existing field-move rules; it allocates no storage.

std.http.typed initially provided fixed-length and close-delimited body readers. Close-delimited EOF is sticky, empty reads do not consume input, and its connection is never reusable. Both return FramingError[E] through the typed protocol without error-storage fields or retrieval methods. The production tests run the prototype’s cause/progress cases against both implementations and exercise real std.io.Cursor bridges through both framing modes. The standalone standard.plenty example is the public-module counterpart to main.plenty.

The subsequent chunked reader preserves typed transport errors and payload progress through size lines, data CRLF and bounded trailers. It retains parsing state across interruption and does not read into the next message. Framing and allocation failures use FramingError.Protocol(HttpError); the concrete transport cause stays in Transport(E). Its scripted regression tests inject interruptions and transport errors throughout metadata and payload, and check limits, malformed input and terminal-state behavior. The original isolated prototype remains a fixed-body experiment rather than a duplicate chunk parser.

Production now also includes parsed-head framing validation, bounded typed head readers and writers, typed buffering, and guarded recovery of completed persistent connections. Shared grammar/serialization helpers keep legacy and typed HTTP validation aligned. Head failures count consumed wire bytes; writes count delivered wire bytes; body-copy and collection counts refer to payload.

The HTTP client uses these paths for CONNECT, request heads, in-memory and streamed uploads, response heads and all body modes. Its existing TLS/network transports, deadlines, cancellation, redirect policy and pool remain in charge of connection lifetime. Internal Wire/Dialled error storage and retrieval are removed. Response.read/take_failure remains a compatibility bridge; typed_response carries the full ClientError through typed stream helpers, and read_all_progress retains bytes appended before failure. Existing send/fetch signatures still return causes without per-head/write counts; those are available from the lower-level typed operations. No builtin Failure, general-sum, or reporting change was needed for this migration.

On 2026-10-18, a local unoptimized compiler comparison of main.plenty against standard.plenty used five --compile runs each with PLENTY_CACHE=off. Median wall time was 0.196 s versus 0.216 s (ranges 0.163–0.209 s and 0.181–0.225 s); linked executable sizes were 3,286,648 and 3,290,816 bytes. Other validation builds were running concurrently, so these timings are a bounded smoke measurement, not a performance guarantee. The public fixed path adds roughly 4 KiB in this probe; it imports the legacy stream module to define bridges. Both copy signatures still carry two error types and two stream types, while fixed framing carries one error and one reader type. Call sites infer these parameters. No specialization-count instrumentation was added, and this small case does not establish scaling for a complete HTTP client.

Finite error sets as anonymous sums

Status: Implemented within the agreed initial contract. General anonymous sums have shipped, as described in the anonymous-sum reference. It adopts case cause: Type: at the case position, grouped types and inline normalization/injection/widening, including Result widening and borrowed matching. Human-readable reporting has also shipped. Builtin Failure stays as a last resort, for when no precise error can be written; issue #342 closed after removing the cases that forced erasure. The current propagation reference remains authoritative. Case guards, residual-set inference and empty sums are deferred extensions in B14/B71 of the backlog, not missing requirements of this completed initial contract. Human-readable presentation is specified separately in formatting and error reports. The composition overview separates this language feature from explicit mapping and typed stream APIs.

Motivation and research

The proposed signature Result[(), IoError | AllocError | SomeError] describes one error drawn from a finite set. It does not mean simultaneous errors, a dynamically typed value, or automatic recovery. This could remove wrapper-only enum declarations while retaining payloads and compile-time coverage checks.

Dillon McMahon’s article argues for composing precise error sets, widening them when combining operations, and narrowing them after recovery. It also separates recovery type information from operational context. Those are useful design questions, not proof that one implementation is best for Plenty. The Missing Piece in Rust Error Handling

The eros documentation describes typed and erased errors, widening and recovery. Crucially, version 0.8.1 documents boxed storage for ErrorUnion. Its ergonomics therefore do not demonstrate allocation-free inline error composition. Plenty should evaluate its own representation rather than copy that storage choice. eros documentation

The reported mixed Reddit reaction is not used as evidence here: no particular thread was supplied or independently assessed. The following objections are design analysis, not attributed community consensus.

Agreed initial contract

The following notation describes the implemented composition contract:

def load() -> Result[Config, IoError | ParseError]:
    source = read_source()?
    parse_source(source)

A | B is a general type expression, available in ordinary value positions as well as Result errors. Error composition supplies the first acceptance cases; the type system must not restrict sums to the second argument of Result.

  1. Treat A | B as a tagged, owned sum of whole nominal types, not a C union and not an untagged overlap of values. No boxing is implicit. Storage must accommodate the largest payload and its tag; existing recursive-data rules still apply. Large payloads can enlarge every enclosing Result.
  2. Normalize parentheses, order, duplicates, and resolved aliases: A | B and B | A have the same identity; A | A is A. Preserve distinct nominal wrappers. Publish no stable tag-number/FFI-layout guarantee by accident.
  3. Permit lossless injection of E into a destination containing E, and widening of a source set into a superset. Do not invoke arbitrary user conversions. Apply these rules in expected-type checking for bindings, assignments, arguments, Err, direct returns, and ?. Also permit Result[T, E] to widen to Result[T, F] when E is a member/subset of F: success moves unchanged and failure is retagged without allocation. This does not establish covariance for arbitrary generic containers or references, and does not change the success type.
  4. Reject propagation when any source alternative is absent. Diagnostics name the missing alternatives and source operation. The separately specified builtin Failure erasure at ? remains a deliberate last resort; it is not lossless sum widening. Issue #342 superseded the original removal plan.
  5. Provide exhaustive, unambiguous matching by member type using case cause: MemberType:. Existing enum constructor patterns cannot identify every class payload; existing pattern | keeps its separate meaning.
  6. Do not implicitly flatten named enum variants. HttpError | AllocError may still contain HttpError.Allocation; deduplicating that semantic overlap requires an explicit classifier. Otherwise API abstraction is broken.
  7. A first version can match the entire set without automatic residual-type inference. Recovery that subtracts a member type is a further feature, not free syntax sugar. An uninhabited empty set and conversion to infallible T also need a decision; do not use inhabited unit as an empty error type.

Function return types remain declared. Do not infer a public error set from the body or silently widen a declared binding after a later assignment. Expected types guide injection; inference must not guess between distinct generic substitutions that satisfy membership. Normalize again after concrete generic substitution, so a generic set can collapse to a single member.

Injection and widening evaluate the source once and preserve the active payload. Owned noncopyable payloads move; inactive alternatives are never initialized or dropped. Early exits and destruction follow the existing sum ownership rules. Copyability and thread-transfer eligibility require every member to qualify. Existing restrictions on sum payloads and recursive storage continue to apply; general sums do not independently introduce stored references.

Matching and remaining surface decisions

Matching must select a whole member type and bind its payload, including class payloads. The implemented spelling is case cause: IoError: at the case position. It does not extend existing pattern |. An enum member can then be matched by its constructors normally.

The initial contract requires exhaustive coverage of normalized members, or a wildcard. Case guards remain unsupported; a future guarded arm alone cannot establish coverage. Reject duplicate unguarded member arms, including members that coincide after generic substitution. Specify those diagnostics on concrete specializations rather than pretending generic alternatives always remain distinct.

Owned matching follows ordinary move rules; matching through & or &mut borrows the active payload with the corresponding permission and cannot let it outlive the owner. A wildcard does not infer a smaller residual sum in this version. Empty sums, automatic error-set subtraction, and automatic conversion to infallible values are deferred.

The implementation puts | below references and type applications, uses (T) for grouping and (T,) for singleton tuples, and allows typed-member binding when normalization leaves an ordinary type. Callable brackets delimit their argument/result types. Diagnostics cover these cases alongside generic inference ambiguity; ambiguous constructors require an explicitly typed intermediate.

Replacing Failure

Replace uses of builtin Failure with precise errors or finite sums, optionally named by ordinary type aliases. Keep domain enums where they express policy; map_error remains the explicit way to classify or wrap a cause. Do not replace erasure with an implicit catch-all member or mandatory reporting bound.

The original alternative was to remove the prelude builtin and its special checking/code generation entirely, then reject old erasure uses. That alternative was superseded by the issue #342 decision: retain Failure as a last resort and remove the cases that force it. C exports now accept payload-free user error enums; explicit retained-job errors and drop_job() preserve start/submission causes without mandatory erasure. Examples, templates and diagnostics use precise errors, except the benchmark template deliberately deferred during performance comparisons. Tests of Failure itself remain intentional.

The mutual-TLS lesson and installed CLI/data and JSON-service workflows are acceptance cases for precise errors and reporting; see the qualification record. Retaining erasure is a documented policy, not unfinished builtin removal.

What sets do not solve

An I/O failure while reading and the same I/O type while writing become the same set member. std.io.CopyError.Read versus .Write carries useful meaning that a bare union discards. Use named operation wrappers or structured context. Likewise, JSON offsets and partial byte counts must remain attached to causes.

A public set exposes its member types and changes when another alternative is added. Exhaustive callers then need updating. Named domain enums can provide a more stable abstraction boundary, though adding their variants also affects exhaustive matching. An erased error hides alternatives but sacrifices static coverage. Choose based on the caller’s recovery contract.

Unions do not automatically fix Reader returning StreamError: the stream interface must still become generic or adopt a different envelope. Nor do they replace allocation-aware context construction in std.error.Context[E].

Concrete evaluation targets

Try sets on internal composition near http/body.plenty’s read_json and send_json, and the HTTP client’s framed/netted boundaries. Compare with explicit mapping into the current named types. Preserve BodyError.status classification, allocation normalization, JSON locations, and retry policy.

Do not start by replacing CopyError, Partial, JsonError, or every public HTTP error. They carry semantics beyond membership. Require comparisons of signature readability, diagnostics, representation size, compile time, and client recovery code. Tests must cover aliases, duplicate types, permutation, nested sets, ambiguous generic members, missing-member diagnostics, payload destruction, exhaustive matching, and no implicit allocation.

Explicit mapping is the implemented baseline. Add ordinary non-error sum values to the acceptance cases, plus Result widening on both success and failure, generic normalization, borrowed/mutable matching, and entrypoint reporting of each active member. Builtin Failure and its erasure stay as a last resort, for when no precise error can be written (issue #342). Production typed-stream adoption subsequently shipped under the separate composition proposal: sums alone cannot restore causes already discarded by an adapter.

Bundled HTTP acceptance

Status: Implemented and qualified for x86-64 Linux GNU. The HTTP framing layer, client, server, TLS, and deployment helpers ship over checked native I/O effects and bounded workers. The server parks request heads and TLS handshakes, supports signal-driven draining, and passes nginx/Caddy reverse-proxy tests, which CI runs. The qualification record records every B66 gate passing, including coverage-guided fuzzing, retained corpora, resource-failure tests and installed offline application checks. Issue #103 is closed. The linked references own the production-readiness claim, plain and HTTPS baselines, operational limits and maintenance process. HTTP/2, HTTP/3 and additional platforms are outside this completed scope; the requirements below preserve the original design rationale.

The library scope supplies the broader context. Typed stream composition has also shipped through HTTP client adoption; its API acceptance is distinct from these server qualification gates. Platform qualification also remains distinct from HTTP behavior on the current supported host.

The bundled server should remain usable in production without requiring a third-party replacement for a demonstration server. Go’s combined HTTP client/server surface motivated this goal in issue #49; no Go compatibility or claim about its protective defaults is implied.

Execution and ownership

Define trusted native I/O effects and transferable/thread-safe handle ownership before scheduling socket/TLS calls in workers. Current worker restrictions do not permit arbitrary native calls or files. Preserve ordinary checks for std; never exempt the whole namespace. A measured bounded blocking model may be sufficient initially; polling is an alternative for many idle connections. Unbounded threads per connection are not an acceptable default. Async language syntax is not a prerequisite.

Handlers and shared state must obey ownership/thread-safety rules. Basic routing must work without heterogeneous erased closures. Shutdown deadlines must not reclaim borrowed storage still in use. Fatal traps currently do not unwind: expected handler failures use Result, while arbitrary fatal traps remain process-fatal. Per-request recovery cannot be promised by a catch-all mechanism.

Client and wire contracts

Use HTTP/1.1 as the initial conformance boundary. Bound header count/bytes, body/buffer sizes and parser work. Reject malformed/conflicting length or transfer encoding, truncated bodies and invalid headers with protocol errors, not unbounded allocation or accidental traps. Define reuse/framing policy and streamed request/response bodies with partial-progress I/O.

The reusable client needs pooling, DNS/connect/TLS/request deadlines, cancellation, redirect/proxy policy and bounded streaming. Certificate/hostname verification is enabled by default. Avoid automatic retries once request effects may have happened. Maintain backend/trust-root versions and updates; tests use local certificates/services.

Server and deployment contracts

Provide method/path routing, composable handlers, JSON/form helpers, cookies and bounded multipart parsing/storage. Bound connections and queued work with backpressure and documented overload behavior. Specify header/idle/handshake and body/write deadlines plus explicit controls for long-lived streams. Cancellation/failure releases handles and multipart temporary storage.

Support direct TLS and documented reverse-proxy deployment. Forwarded identity or addresses require an explicit trusted-proxy policy. If static-file helpers are adopted, enforce filesystem-root/symlink rules with filesystem-aware checks; string cleanup alone is insufficient.

Graceful shutdown stops accepting, drains or cooperatively cancels active work under a documented deadline policy, and releases resources after safe joining. Provide structured access/error events and metrics hooks with caller-selected sinks and no mandatory telemetry vendor.

Qualification gates

Production qualification is a separate issue from a core server implementation. Require parser fuzzing and retained regression corpora, protocol conformance/ interoperability, slow-client/cancellation/allocation-failure tests, sustained load and resource-leak tests, and repeatable throughput/latency/memory baselines. Publish workloads, hardware, versions, supported platforms and configured limits. Exercise overload and graceful shutdown while work is active.

Run the motivating CLI and JSON service integration examples with an installed, relocated toolchain offline, including verified TLS and proxy fixtures. Check unused-module/backend linking. Document expected failures, fatal boundaries, operational guidance and an ongoing security-fix/maintenance process before calling the server production-ready. HTTP/2 and HTTP/3 remain separate decisions.

Explicit applications, modules, and executable lessons

Status: Implemented. The baseline below is b690afe, when the tutorial was one file, TUTORIAL.md.

  • Entry point: implemented. A parameterless main returns (), i32, Result[(), E], or Result[i32, E]; see entry point and entry reports.
  • Modules: implemented. Absolute imports, aliases, pub visibility, and library checking; see modules and visibility. The source root flag is --module-root DIR.
  • Executable lessons: implemented as executable Markdown; literate sources not adopted. tests/test_tutorial.rs runs every plenty and plenty-error fence in the tutorial and design/examples/ pages, in SUMMARY.md order; see runnable documentation.

Literate lesson sources (B21): not adopted. Reviewed on 2026-10-24 against the tutorial at 161e1c63: 213 tutorial pages plus design/examples/, with 420 run-pass and 57 compile-fail examples. The Markdown harness already meets the goals the literate design below sets out:

  • Every executed fence is a complete program, compiled as written. No lesson hides setup, so every displayed example is copyable and runnable.
  • Expectations are reviewed, hand-written output and error fences. Nothing regenerates them, so a compiler regression fails instead of rewriting the book.
  • Each runnable example is checked as an AOT binary; compile-fail examples must produce the expected diagnostic without running.
  • Multi-file examples are first-class: plenty-file companion modules, sh plenty-build library builds, and project-file packages run with sh plenty-project. The harness rejects orphan expectations and companions on another page.
  • A failure names the page and line of the Markdown the author edits.

Literate sources would add a manifest, a renderer, a --check comparison of generated pages, and a migration of every page, and authors would edit one file while reviewers read another. The one clear gain, editor and language server diagnostics while drafting, does not require generating the book: a draft can be written in a scratch .plenty file and pasted into its fence. The question of comment blocks versus sidecar expectation files is therefore closed.

Shipped since this proposal: packages provide manifests, locked path/Git dependencies, workspaces, package commands, and per-target build reuse. Import cycles are rejected. Remaining (B17): re-exports and finer module caching/reuse. Registry resolution remains separate package scope.

Current behavior and the proposed entry point

At the research baseline (b690afe), frontend::compile collected top-level statements into __plenty_entry; their bindings were startup locals rather than globals. The implementation now calls the declared main instead. The native runtime still supplies the platform entry point separately.

Recommend requiring an entry-module function for every binary:

def main() -> ():
    print("Hello, Plenty!")

Start with zero parameters and return () or i32. Unit means successful exit; i32 supplies the process status, subject to the host’s exit-status conventions. main need not be pub, and imported functions named main are ordinary functions. A library has no entry-point requirement. Checking a library module must not produce a missing-main error; checking/building a binary must.

Once error propagation and allocation-free error reporting are ready, consider main() -> Result[(), E]. Do not silently add an implicit formatting/protocol requirement on arbitrary E now. Applications can initially use a small i32 main that explicitly matches a fallible application function’s result. (main now accepts both Result forms.)

Allow only declarations and imports at module scope initially. Defer module-level constants until their permitted constant expressions are specified. Reject executable statements with a diagnostic suggesting moving them into main. There should be one application rule for compile-and-run and binary output, not an implicit script fallback when main is absent. This is an intentional breaking change on the language-development branch, accompanied by migrated examples.

Python makes top-level execution meaningful both for scripts and imports, and documents the __name__ == '__main__' convention. Plenty can adopt familiar spelling without importing executable-module initialization semantics. See Python’s entry-module documentation.

Imports name source modules; they do not execute them

Proposed source layout, with src selected explicitly as the module root:

src/
    app/
        main.plenty
        geometry.plenty
        storage/
            files.plenty

Use absolute paths and familiar name binding:

import app.geometry
import app.storage.files as files
from app.geometry import Point
from app.geometry import distance as measure

import app.geometry makes app.geometry reachable. An alias binds exactly the alias; a from import binds the requested declaration. Importing a module does not inject its members into the caller’s unqualified scope. A file’s symbols must come from its declarations, lexical bindings, documented built-ins, or explicit imports. Importing a protocol must never silently add methods to unrelated types.

Directories initially serve only as namespaces, without executable __init__ files or implicit package initialization. Resolve a.b to <root>/a/b.plenty; reject ambiguous file/directory module collisions. No relative imports, wildcard imports, import hooks, conditional imports, or automatic transitive re-exports. Importing app.geometry must not grant access to a sibling module simply because some other file imported that sibling.

Python separates finding a module from binding its name and has extensive dynamic loading rules. We need its visible syntax, not that entire runtime system. See Python import statements and the import system.

Initially use one source root: an explicit build/CLI option, defaulting to the entry file’s directory for standalone programs. A nested entry file that imports app.geometry therefore needs the root above app, not an implicit search through parent directories. Resolve against the configured root, never ambient Python paths or arbitrary environment search paths. A project manifest can eventually make the root and dependency mapping durable; no package manager is necessary yet.

Load each reachable module once by canonical module identity; include paths in all diagnostics. Initially reject import cycles with the actual cycle path. Collect public declarations before checking bodies in dependency order. Later, declaration cycles may be supported independently of recursive value layouts.

Private by default, with explicit public API

Use pub for top-level functions, types, class fields, and methods:

pub class Point:
    pub x: f64
    pub y: f64

    pub def length_squared(self) -> f64:
        self.x * self.x + self.y * self.y

Recommend module privacy initially: private names and members are accessible throughout their defining module, including sibling helper functions, but not from other modules. Directory ancestry grants no extra access. This borrows Rust’s explicit pub vocabulary with a smaller privacy model; Rust itself has additional descendant and restricted-visibility rules. See Rust’s visibility reference.

Rules needed alongside parsing the keyword:

  • A public class does not automatically expose its fields or ordinary methods.
  • Public signatures cannot leak private nominal types; transparent aliases must not bypass this check. Resolve actual type identity before enforcing visibility.
  • A public enum exposes its variants as part of its exhaustively matchable API in the first version. Hiding variants requires a separate non-exhaustive design.
  • A generated field constructor is externally callable only when the class and all fields are public. Otherwise require a public __init__ or public factory to establish invariants. An explicit __init__ has its own visibility.
  • __del__ remains a compiler-invoked lifecycle hook, never a public callable destructor. Access restrictions must not prevent automatic cleanup.
  • Borrowing private fields, generated structural printing/equality, and generic protocol checks must respect encapsulation. Initially keep compiler-generated printing/equality behavior documented as structural; do not mistake privacy for data secrecy. Decide whether opaque public classes should later opt out.
  • Imports are private bindings. Defer pub import re-exports rather than letting ordinary imports accidentally widen a module’s API.
  • pub controls Plenty visibility, not C ABI exports. Foreign exports require their own explicit declaration and ABI validation.

The main implementation work is resolved symbol identity and access checking, not accepting a new keyword. Give every nominal type/function a ModuleId plus local identity instead of using an unqualified string as global identity. Local aliases preserve identity; two unrelated modules may each define Point.

Compiler changes that keep later features possible

Introduce a path-aware compilation session owning sources, module IDs, an import graph, public signature tables, and diagnostics. Keep source-string APIs for isolated checks/tests, with explicit virtual-module identity and an optional resolver; do not make them silently read unrelated filesystem modules.

Resolve imported type/member syntax before lowering runtime expressions. A qualified symbol path and a field access may look similar but are not the same operation. Likewise, importing a module is not loading a DLL; both may be needed to use a foreign library, and should remain separate concepts.

Initially lower the reachable source graph into one object using the existing pipeline. Stable internal symbol IDs should allow later per-module object caching and generic instantiation caches without promising separate compilation now. Public signatures include ownership, borrowing, and future fallibility contracts; callers must not inspect callee bodies to infer them. Module visibility checks must run before both ordinary and generic calls are emitted.

The tutorial is already executable, but its source can improve

At this snapshot, TUTORIAL.md contained 43 successful programs and 11 expected compile errors; the lessons are now the pages under book/src/tutorial/. tests/test_tutorial.rs extracts them directly. It invokes both compile-and-run and explicit AOT compilation; successful binaries must produce the documented output, and invalid programs must produce the expected diagnostic without effects. Prose is not semantically checked. The Markdown is currently the source of truth; the lack of standalone files is an authoring/reuse limitation.

The proposed literate-source approach is workable. Recommend one independently compilable .plenty file per example, with ordered lesson metadata and optional named display regions:

#| ## Choose the width of an integer
#|
#| Suffixes make the width explicit.
# docs:begin example
def main() -> ():
    count: u8 = 42u8
    print(count)
# docs:end example

The #| lines become Markdown prose; region markers remain inert comments to the compiler. Use a small manifest to order files and identify run-pass versus compile-fail examples, expected stdout/diagnostics, fixture stdin, and any module root. A chapter may comprise multiple example files. Prose-only sections can also be listed. Do not concatenate independent programs into one execution unit.

The first lesson shows the full required main. Later lessons may show named regions, with a link to the complete runnable source and an explicit statement that the displayed region is an excerpt. An example with hidden setup is no longer honestly described as independently copyable from the rendered snippet. Negative examples are separately labeled intentionally invalid source files.

Proposed build flow:

  1. Parse the manifest, prose, and regions; reject duplicate IDs, unmatched markers, missing files, invalid paths, and orphan expectations.
  2. Compile/run each complete source with isolated temporary outputs. Validate expected stdout, exit status, and stderr/diagnostics. Use bounded execution time and deterministic fixtures so an accidental loop cannot hang CI.
  3. Render validated actual outputs and code regions into the tutorial pages. Preserve source locations in failure reports and links to complete examples.
  4. A --check mode renders to memory and fails on differences from the checked-in tutorial. A generation command writes it deliberately. CI never updates it.

Keep reviewed expectations, not only whatever output happened to be generated. Otherwise a compiler regression would rewrite the book and appear successful. Snapshot acceptance must be an explicit author action. Preserve the existing two-command validation when moving it into a shared harness. Once migrated, edit prose/code in the source lessons and treat the tutorial pages as generated; do not maintain duplicate editable examples in both places.

TinyTemplate can handle ordering and presentation, but it does not execute or validate programs. Its variables, loops, and template calls are sufficient for precomputed lesson records; region markers should be parsed by the doc builder, not treated as executable template commands. A simple renderer may need fewer moving parts initially. If using TinyTemplate, disable HTML escaping for trusted Markdown/code fragments, and choose fences that safely contain backticks. See TinyTemplate’s overview and formatter configuration.

  1. Clarify the current tutorial entry behavior immediately without claiming a language change. This is documentation of implemented behavior.
  2. Implement required binary main and migrate executable examples together.
  3. Add modules, stable identities, and privacy in one coherent change; then expose standard-library operations through explicit imports.
  4. Move lessons to runnable literate files with output snapshots and generated Markdown. This can happen alongside step 2 if the migration is easier once.

Steps 1-3 are complete, except that builtins stay in a prelude that needs no import. Step 4 was not adopted; the status block gives the reasons. Declarations are private to their module, main accepts (), i32, and both Result forms, and the source root flag is --module-root.

Plenty’s next language phase

Status: Investigation complete. This design review consolidated the entry-point and module investigation and three research proposals against baseline b690afe. The review is complete: ten of its eleven recommendations shipped at least in part, several differently, and the one that did not, allocators, is B13. Every remaining extension in the table below has a backlog entry. Outcomes were checked against the reference at 161e1c63 on 2026-10-24. The text below the outcome table describes the baseline and its recommendations, not the current language.

Outcome

TopicOutcomeReference or backlog
Entry pointImplemented; main also returns Result[(), E] or Result[i32, E]Entry point
Modules and visibilityImplementedModules, visibility; B17
C interoperationImplemented: .plentyi interfaces, adapters, exports, runtime loadingInterop; B27, B28
Context managersImplemented, with with &mut existingResource scopes
GenericsImplemented; bodies are checked per instance, not against boundsGenerics; B29
AllocationImplemented: allocating operations return Result. Class, tuple, enum, generator, and range values are inline and construct without allocation; a fallible class initializer returns ResultAllocation and traps
AllocatorsNot implemented; objects only record an internal allocatorB13
Error propagationImplemented, including anonymous-sum widening; Failure erasure remains a deliberate last resortError composition, sums and propagation; B14/B71
ParallelismImplemented: scoped threads, bounded channels, ordered pool.map and fixed-tree reduction; automatic parallelization deferredConcurrency; B24
SIMDPartly: optimized builds vectorize one constant-comparison filter and count() on i32/u32 clist columns; portable std.simd remains a proposalAutomatic SIMD counts; B23
Runnable lessonsImplemented as executable Markdown; literate lesson sources not adoptedRunnable documentation, decision

Recommendations at the baseline

The proposed direction fits Plenty: explicit names, structural interfaces, native ownership and cleanup, and predictable ahead-of-time compilation. The biggest changes to make early are module identities, an allocation-failure representation that does not allocate, and a clear boundary between Plenty’s private runtime representation and the C ABI. Context managers need no general trait system. Threads will need more runtime work than their surface syntax suggests.

TopicRecommendationMain dependency or limitation
Binary entrypointRequire def main() -> () or -> i32 in the entry moduleReplace current top-level execution and migrate examples
Modules and visibilityAbsolute imports; private declarations by default; explicit pubStable declaration identities and path-aware diagnostics
C interoperationReusable trusted interface files, checked adapters, ordinary C ABIExplicit ownership, lifetime, layout, and error contracts
Context managersConcrete __enter__/__exit__ methods and deterministic with cleanupReturned-borrow rules for managers yielding references
GenericsExplicit type parameters constrained by structural protocol Name: declarationsStatic constraint checking and cached specialization
AllocationEvery allocation-capable operation exposes failure; allocation-free error constructionReplace hidden runtime allocations and current boxed error values
AllocatorsStartup default plus an allocator handle retained by each allocation ownerProvenance, allocator lifetime, and failure-atomic operations
Error propagationPostfix ?, initially with the same Result error type or within OptionCorrect ownership transfer and cleanup on early return
ParallelismScoped threads, bounded channels, then explicit parallel collection operationsThread-safe shared storage and compiler-checked transfer rules
SIMDExplicit vector operations and contiguous buffers, starting with a narrow target contractBackend support, scalar semantics, bounds, and data layout
TutorialRunnable literate lesson sources with checked outputs generate MarkdownA small extraction/validation tool; existing tests provide the baseline

Entrypoints, imports, and names

The suspicion about startup was correct at the research baseline. The compiler wrapped top-level statements in a generated entry function. A user function named main was just another function; it was not called automatically. Top-level bindings were startup locals, not globals visible to other functions. That behavior has now been replaced by explicit main, and the tutorial teaches the new contract.

A required binary main makes that contract clearer. Recommend a parameterless main returning unit or an i32 process status. It need not be pub. A library has no required entrypoint, and an imported function named main has no special meaning. Module top levels should contain declarations and imports, with globals and constant initialization designed separately. Fallible application code can initially be called from a small main that explicitly handles its result; a Result-returning entrypoint can follow once reporting and exit status are defined.

Use familiar absolute import package.module and from package.module import Name, including explicit aliases. Imported names and builtins account for everything a file can reference beyond its own declarations. Start with one explicit source root, no ambient path search, no executable package initializers, no wildcard imports, and no import cycles. These restrictions keep resolution and initialization easy to understand and compilation inexpensive.

pub should govern access to functions, types, fields, and methods. Public classes do not automatically expose all fields or methods. An automatically generated field constructor should be public only when the class and all its fields are public; otherwise construction needs an accessible initializer or factory. Compiler-invoked destruction must still work for private destructors.

For a simple initial rule, privacy can be module-based, including class members. Class-private access could be added deliberately if wanted. Privacy controls source access; today’s structural printing and equality mean it is not a promise that representation details are secret. Public signatures also need checking for accidental exposure of private types.

Detailed resolution, visibility, and migration rules are in Entrypoints, modules, and executable tutorials.

Structural protocols without import-sensitive behavior

Structural protocols are a good fit. Python’s Protocol supplies useful precedent for matching declared capabilities without requiring explicit inheritance; Go’s interfaces offer another precedent. Plenty can make that matching entirely static for generic calls. Neither requires importing an unrelated trait to make a method appear. See the Python protocol specification and Go interface types.

Recommend protocol Writer: with typed method requirements, and an explicitly generic function such as def save[W: Writer](writer: &mut W) -> Result[(), IoError]. This is proposed syntax. A class satisfies the constraint when its accessible methods match the required receiver, parameters, returns, borrowing, and failure behavior. Imports only provide names; they never activate methods. An adapter class can give an external type a different interface when necessary.

Start with protocols as generic constraints, not runtime values. Accepting a protocol-typed object through a vtable, or storing different implementations in one list, is a separate feature. This avoids silently introducing allocation or dynamic dispatch. Unknown generic values remain owned values; generic code cannot assume copying is free.

Check a generic body against its declared constraints, then specialize reachable concrete instantiations and cache them. Detect expanding recursive instantiation and measure generated-code growth. This costs some compile time, but remains a tractable first design without specialization, blanket implementations, or a large trait-resolution engine.

The protocols, generics, and context managers proposal contains the proposed matching and specialization rules.

Context managers and ?

Concrete context managers can precede generics. The compiler already knows a concrete class’s methods and can check a with operation directly. Returning a reference from __enter__ does require extending today’s borrowing rules: a hidden owner must remain alive, the yielded borrow cannot escape it, and the borrow must end before mutable exit/cleanup access.

The essential behavior is once-only cleanup on normal completion, return, break, continue, and propagated failure. After successful entry, body locals are cleaned up, __exit__ runs, then the hidden manager is dropped. An entry failure must not call __exit__ as though entry had succeeded. Defer yielding from inside a context until suspended-generator cleanup is specified.

Start with infallible __exit__ -> (), and make operations that can fail visibly explicit: acquisition before entry, and flush, commit, or finish before exit. The destructor remains a cleanup fallback. Fallible exit is possible later, but needs an answer for simultaneous body and exit errors; silently discarding one would be a poor default. An infallible signature also does not by itself prove that arbitrary user cleanup never allocates.

Postfix ? is worthwhile, especially with fallible allocation. Start with direct propagation: Result[T, E] into a function returning Result[U, E], and Option[T] into Option[U]. Avoid implicit error conversions or cross-family propagation. Evaluate the operand once, move its payload, and perform the same cleanup as an explicit early return. This can be implemented against today’s sums first, but it must not be described as OOM-safe until error propagation itself is allocation-free.

C libraries and trusted interfaces

One correction to the premise: Rust does support C shared libraries, exporting cdylib libraries, and runtime loading. Dynamic linking is possible; the hard part is establishing the lifetime, aliasing, ownership, and callback contracts that the C ABI does not express. Rust documents its library linkage forms, and libloading supports runtime loading.

Plenty can make that work more convenient with reusable interface files. Separate the raw native signature from a normal Plenty-facing API, and generate or write adapters between them. The interface is a trusted contract maintained once by a binding author. Callers can then use ordinary ownership, borrowing, and Result without repeating low-level boundary work at each call.

Those contracts need more than const: buffer lengths, nullability, retained aliases, borrowed lifetimes, ownership transfer on success and failure, the exact release function, error reporting, callback retention, and thread restrictions. Annotations cannot prove that arbitrary native code honors its promises; relaxing the rest of Plenty’s ownership rules would not solve that problem.

Cython’s declaration files and Zig’s explicit C ABI/layout facilities are useful models. C-header translation could eventually be a separate tool, with compiler flags and target information, but cannot infer all semantic contracts. It should not be a prerequisite for a small, useful initial FFI.

Keep Plenty’s str, class, collection, and enum representations private. Begin with C scalars, opaque handles, and explicit pointer/length or C-string adapters. Retain the matching allocator and release function for owned foreign resources. Opt-in C-layout records can follow. pub visibility must remain separate from a stable C export declaration.

Ordinary shared-library linking and runtime loading should be separate stages. Known signatures can be called indirectly after runtime loading with AOT code; this does not require reviving the JIT. Initially retain loaded libraries for the process lifetime, and defer retained or foreign-thread callbacks until their ownership and synchronization rules exist.

The FFI and dynamic libraries proposal documents the contract fields, current backend assumptions, and staged delivery plan.

Recoverable allocation is an architectural commitment

Rust already has fallible allocation-related operations such as Vec::try_reserve. Plenty’s opportunity is to apply a consistent recoverable policy across its own language and runtime operations.

Today’s runtime cannot make that promise. Result and Option are managed, heap-allocated sum values. Runtime type metadata, collection rebuilding, strings, generators, and some structural equality operations also allocate. Even string indexing and range construction need auditing. An allocation failure cannot reliably be reported by attempting another allocation to construct Err.

First establish allocation-free finite sum/error lowering and static type metadata. Keep that lowering contract internal and evolvable. Then audit every allocation-capable operation, including cleanup and propagation. A small AllocError must be representable without allocating or formatting a message.

Fallibility belongs to an operation’s static interface even when a particular call happens to fit existing capacity. Literals, comprehensions, concatenation, copying, class construction, and generator creation all need an explicit policy, not just append. Whether their syntax yields Result directly or uses an explicit allocation effect remains an open design choice; hidden automatic ? would undermine the desired visibility of failure.

Failed mutations should preserve existing logical contents, and failed builders must release partially constructed results. Decide how failed insertion handles an owned input: consuming and dropping it is simple; returning it with the error needs a deliberate recovery API. Test every allocation position with fault injection, including attempts to report and clean up after failure.

Allocator selection is feasible without exposing a different list type for every allocator. Recommend a default chosen at startup and an explicit constructor argument for custom allocation. Each owner retains allocator provenance and keeps the allocator alive until deallocation. Nested values retain their own allocation provenance; inserting them does not magically relocate them. Resettable arenas need additional lifetime constraints and should come later.

Fixed-capacity containers and inline-small containers are both useful but have different guarantees: the latter can still allocate after spilling. Neither needs to complicate the initial ordinary collection type. Budgeted allocators would make predictable service-level refusal practical, although operating-system overcommit means no language can guarantee recovery from every physical-memory exhaustion event.

The memory, parallelism, and SIMD proposal gives the allocation audit, failure guarantees, allocator design, and sources.

Parallelism and SIMD

Owned values make concurrency easier to reason about, but moving a collection does not prove all its storage is exclusive. Shared immutable strings and runtime metadata now use atomic counts and immutable descriptors respectively. Allocators, destructors, and foreign handles can also have thread affinity. These need compiler-recognized transfer/share rules and runtime changes before safe thread APIs.

Start with scoped threads that join on every exit, bounded channels with defined failure ownership, and explicit parallel map/collect/reduce operations using a configured pool. Define result order, error selection, cancellation, and cleanup before implementation. A thread-count setting should initially affect explicitly parallel work only.

Automatic parallelization is a later optimization requiring proof that observable behavior is preserved, including allocation failure, side effects, destruction, and floating-point reduction order. Structural method matching alone cannot prove those properties.

SIMD should have its own narrow, testable contract. Explicit small vector values and contiguous numeric buffers are a reasonable first step; current generic list storage is not automatically a contiguous native numeric array. Begin with operations the selected Cranelift targets can implement, define scalar-equivalent fallbacks and tail handling, and leave CPU dispatch and wider target-specific vectors for later. Do not promise that all backends support identical native instructions or that floating-point reassociation is invisible.

Runnable lessons and generated documentation

The proposed documentation workflow is workable. The tutorial already has direct executable coverage: 43 successful examples and 11 rejected examples. Its test checks output through both compile-and-run and explicitly compiled executables, and checks expected diagnostics for rejected examples. That test passed after the startup clarification in this change.

Standalone .plenty lessons would improve editing, discovery, and reuse. Give each lesson a complete runnable program, prose in ordinary comments, and inert markers selecting displayed code regions. An ordered manifest selects lessons; sidecars can hold expected output or diagnostics. Show the complete main in the first example and label later shortened snippets as excerpts with links to the full source.

The documentation tool should run and validate the whole program against reviewed expectations before generating Markdown. It must not silently turn changed output into a new expected result. CI should check that the committed document matches the generated version. Compile-fail lessons and future multi-file module examples need first-class support too.

TinyTemplate can render the final document, but extraction, execution, validation, and ordering are separate work. Its default HTML escaping needs disabling for trusted Markdown. A simple renderer may be sufficient; the important design is a single authored source and independently checked behavior. Details and TinyTemplate references are in the tutorial workflow proposal.

Work tracking

The original cross-feature implementation sequence has been retired. The backlog is the single maintained list of remaining work and proposed order; the implementation status records the result. The sections above preserve the design reasoning and open questions from this review, not a current task queue.

Checked arithmetic blocks

Status: Implemented (issue #557). The numbers reference defines current behavior; this page records the decisions and rationale.

Problem

Integer overflow and division by zero trap. That is the right default when they mean a bug, but code that takes sizes, counts or prices from input needs to report them instead. Before this change the only recoverable options were try_from after widening, or std.math helpers, which do not compose across an expression.

Rejected: a Result from every operator

Local PR #678 made every integer + - * // % return Result and changed existing code to .unwrap() or ? each one. It was abandoned unmerged:

  • ordinary math became noisy, and most call sites only unwrapped;
  • every operation carried a payload and a branch on the result, which was slower than the existing inline trap check;
  • the tutorial had to teach Result before the first addition.

Its error types, float div_checked, T.MIN % -1 == 0, the overflow code generation and constant-divisor branch elimination were kept.

Decision: an expression block

r = checked:
    a * b + c

checked: opens an indented suite whose last expression is the value. The block’s type is Result[T, E], where E is the anonymous sum of OverflowError[T] per operand width and ZeroDivisionError that its operators can produce. The first failure exits the block with Err.

Outside any block, arithmetic compiles exactly as before, so the cost is paid only where it is asked for.

Lexical scope

Only arithmetic written in the block’s suite is checked. Calls, closures (including closures defined in the block) and other functions keep their own behavior. A dynamic rule, where a callee’s arithmetic depends on its caller, would need either two compilations of every function or a run-time mode flag on every operation, and would make a function’s failure behavior depend on code it cannot see. Comprehensions and compound assignment are lowered inline, so they are checked when written in the block.

Control flow

?, return, break and continue keep their meaning: the block catches only arithmetic failures. A failure exits like ?: it releases the locals the block declared and the temporaries on the operand stack, then produces Err. There is no rollback. An outer variable assigned before the failure keeps the new value, and a compound assignment whose operation failed does not write. Rollback would need copies of every mutated value and would hide allocation.

Settled questions

QuestionDecisionReason
Where a block may appearBinding or assignment value, after return, or a function’s or case’s final expressionThe positions where a match block produces a value; no new expression grammar
checked as a keywordContextual: checked followed by : and a newlineExisting names such as checked: i64 = 0 keep working
NestingThe innermost block catchesEach block’s error type stays local and predictable
A block with no failing operationCompile-time errorResult with an empty error set would be misleading, and the block is likely misplaced
Every path exitsCompile-time errorThe block has no value
Discarded blockRejected as an unused ResultSame rule as any Result
Reference resultRejectedLocals the block owns are released on failure
x += 1, items[i] += 1CheckedLexically arithmetic; the earlier rejection of compound assignment is not needed
@newtype operators (issue #555)Forwarded integer operators are checked and report OverflowError of the field typeForwarding is inline lowering of the field’s operator; user-defined hooks are calls and keep trapping
Constant operandsEvaluated at run time inside a block; module constants still reject overflow at compile timeA block’s purpose is to produce the Err value
Constant divisorError type unchanged; impossible branches omittedTypes follow operators, not values, so editing a constant does not change a signature
Shifts and castsUnchangedOut of scope; try_from covers casts
Negative pow exponentStill trapsIt has no integer result to wrap
Declared wider errorKept when it accepts every possible failureLets a block be a function body returning a larger error set

Code generation

Op::CheckedBlock holds the body and the result type; each checked operator is Op::CheckedArith, carrying the operation, operand type and cleanup for the block’s locals. Code generation computes the wrapped result and an overflow flag with Cranelift’s *_overflow instructions, tests the divisor for zero and the signed MIN // -1 case, and branches to a cold block that builds the error, releases stack values above the block’s base, runs the cleanup and jumps to the block’s exit. The success path wraps the value in Ok. A flag known to be zero, such as the zero test for a nonzero constant divisor, emits no branch.

Integer pow in a block calls plenty_int_pow_checked, which returns the wrapped result and an overflow flag. Float div_checked is a one-operation block.

Programs without checked: produce the same objects as before; the comparison is recorded in the PR for issue #557.

Follow-up

  • Class-destructuring patterns for OverflowError landed with issue #558: case Err(OverflowError(wrapped)); see class patterns.
  • Checked shifts and casts, if a use case needs them.

Rust compiler boundaries and measured performance

Status: Completed. Steps 1–6 are implemented (issues #691–#696), and the legacy stack syntax is removed (issue #698). Replayable module lowering remains a design. No speedup is claimed. The frontend reference and cache reference describe current behavior. Remaining scope and priority belong to B17/B19 in the backlog.

Assessment and scope

The compiler’s strongest properties are an independent typed-operation checker, deterministic parallel emission and diagnostics, explicit ownership checks, cycle-safe nominal metadata, and profiling-backed allocation decisions. Preserve them; the aim is code that is easier to change, not a different architecture.

The main concern is coupling: parsing, generic expansion, local inference, ownership provenance, editor recording and operation construction meet in a large frontend with shared mutable tables, and diagnostics travel between phases as rendered text. Improvements should make those dependencies explicit and independently testable.

The source map was rechecked against local main 6477751e (0.8.7). Step 6’s profile is recorded in measured attribution; no correctness audit was performed, and historical numbers below retain their original scope. The runtime crate, optional dataframe/linalg engines and standard-library adapters were not audited. This is not an unsafe-code or security review. There is no proposed learner-visible language change.

Design map

QuestionDetailed design
How should lowering, syntax and ambient state be separated?Frontend and session boundaries
How should errors reach CLI, workers and editors?Structured diagnostics
Which hot representations merit experiments?Identity, type facts and dataflow
What must exist before caching lowered imports?Replayable module results
How do we decide whether a change is worthwhile?Validation and adoption

These pages describe design dependencies and alternatives, not a second schedule. The backlog owns priority, and the shared local issues register owns defects, claims and PR state. Existing closed investigations remain closed unless new evidence justifies a separately scoped follow-up.

Implementation order

Each step is one local PR that preserves behavior unless it says otherwise.

  1. Structured diagnostics. The editor renders errors to text and reparses that text for positions; the parallel checker flattens errors to strings. Replace both with one sendable diagnostic value. This fixes a representation, not an IR.
  2. Scoped ambient state. Install the editor recorder through a guard that restores the previous state on return and unwind, and make teardown skipping a per-call option instead of a process-global flag.
  3. Split src/op.rs by responsibility. It holds the IR definition, the historical stack-syntax compiler and the independent checker. Separate files give the checker an explicit boundary and keep the legacy compiler out of the IR module.
  4. Separate the syntax layer from lowering. Tokens, the lexer, the syntax tree and the core parser make up about a third of src/frontend.rs and do not use lowering state. Moving them into child modules makes that independence checked by visibility rather than convention.
  5. Group lowering state. Start with passive pipeline state on Lower, then borrow/provenance state if the first extraction shows a clear gain. The pipeline grouping did not show one, so borrow/provenance state stays on Lower.
  6. Attribution before optimization. Function-name and nominal-type lookups are candidates only. Profile them and record the result; change the representation only on measured evidence. The measured attribution found neither material, so no representation changed.

Steps 1 and 2 both edit src/frontend/ide.rs; steps 1, 4 and 5 edit src/frontend.rs. Land them in order rather than concurrently.

Replayable module lowering is a larger architectural opportunity, not a quick cache patch. Imported source dependencies, generic instances and inferred returned-reference facts must be reproducible before persistence is safe. It stays a design until steps 1–5 land and a std-heavy application shows the need.

Several unimplemented plans meet these boundaries:

  • B17 editor work (find references, document symbols, signature help and incremental module re-checking) consumes structured diagnostics and needs a recorder whose lifetime is explicit.
  • B19 module caching needs the replay contract in replayable module results.
  • B20’s GCC and Rust-source backends consume checked operations; a separate checker module and, later, function IDs in the IR affect what a second backend must reimplement.
  • Language editions would need per-module options to flow through lowering, which explicit lowering state makes easier.

Existing work not to redo

The performance delivery record documents:

  • issue #302: shared token/type children, callable signatures and FxHash tables;
  • issue #303: borrowed loan-graph operations and reusable liveness buffers;
  • issue #305: large-workload slowdown attributed to cache/TLB misses rather than increasing instruction work per unit;
  • issue #315: batched codegen and mimalloc addressing allocator contention;
  • issue #321: selective CLI teardown skipping; bump allocation was rejected;
  • issue #317: syntax-tree and separately compiled module experiments did not justify adopting those cache designs.

Avoid blanket clone removal, replacing Arc with Rc, a global arena rewrite, more threads without contention measurements, or replacing Cranelift without matched workload evidence. These are not prerequisites for the proposed boundaries. Compiler latency, generated-program performance and Rust build time are distinct outcomes; report them separately.

Frontend and session boundaries

Parent proposal. The syntax/IR split, pipeline state and scoped ambient state are implemented (issues #692–#695); the session and further lowering-state options remain design sketches.

Source map

The table records the layout these steps started from, at 6477751e.

File / symbolResponsibility and coupling
src/frontend.rs: Token, FileId, LexerTokens, file interning and streaming item tokenization, about 400 lines at the root of the frontend.
src/frontend.rs: Function, TypeRef, Stmt, ExprThe syntax tree, plus TypeRef::resolve, which turns written types into Ty against aliases and so belongs to resolution rather than syntax.
src/frontend.rs: ParserCore statement/expression parsing, about 1,650 lines. Feature modules (classes.rs, enums.rs, patterns.rs, collections.rs, fstrings.rs, protocols.rs, annotations.rs) add their own impl Parser blocks beside their impl Lower blocks.
src/frontend.rs: LowerHolds function names/slots, mutable function and generic tables, heap, immutable aliases/access, loans, captures, pipeline state, scopes and source provenance: about 50 fields.
lower, lower_function, lower_bodyCoordinate declaration expansion, registration, on-demand specialization, body lowering and finalization.
src/frontend/functions.rs: Functions, SymbolIndexed function table: signatures, declarations and returned-field facts. complete releases eligible syntax bodies.
src/frontend/generics.rs: EngineInstance preparation and pending/compiled work interact with ordinary lowering.
src/frontend/references.rs, provenance.rs, retention.rsBorrow origins and lifetime-related facts used during expression lowering.
src/frontend/pipelines.rs and its childrenPipeline plans, callbacks, generator adaptation and specialized lowering.
src/frontend/ide.rsAmbient per-thread recording and error recovery alter analysis behavior.
src/op.rsThe operation IR (about 1,350 lines), the historical stack-syntax compiler (about 600 lines, with src/lexer.rs) and the independent checker (about 1,250 lines).

At 6477751e, src/frontend.rs was about 7,650 lines. After the moves it is about 4,950, with frontend/lexer.rs, syntax.rs and parser.rs beside it; src/op.rs keeps the IR (about 1,500 lines), with op/check.rs. The legacy stack-syntax compiler, split into op/legacy.rs, was later removed with its --legacy flag (issue #698). File length is evidence of navigation cost, not a reason by itself to rewrite the frontend. A move is worthwhile where it turns an existing independence into one the compiler enforces; adding files without changing dependencies would not address the main concern.

Syntax and IR files

The core parser does not read Lower, the heap, the generic engine or module tables. Move tokens and the lexer, the syntax tree, and the core parser into child modules of the frontend (for example frontend/lexer.rs, frontend/syntax.rs and frontend/parser.rs). Keep the per-feature impl Parser blocks where they are: grouping a feature’s parsing and lowering together is deliberate. TypeRef::resolve and the builtin-type tables stay with resolution. Visibility then shows which syntax items lowering uses.

Split src/op.rs into the IR (op.rs), the checker (op/check.rs) and the legacy stack-syntax compiler (op/legacy.rs, with src/lexer.rs). The checker then depends on the IR’s public surface only, which is the contract a second backend would also consume. Removing the legacy syntax entirely (the CLI’s --legacy flag and the backend tests written in it) was a separate product decision, not part of this split; issue #698 made it.

These are pure moves: no behavior change, and the diff should review as one.

Lowering state

Separate state by lifetime and responsibility, not by matching every syntax variant with a trait:

  1. Compilation environment: aliases, access policy, source identity and shared function/type tables. Distinguish read-only inputs from mutable specialization/string services. Do not place every service behind a mutex just to make the context shareable.
  2. Function state: current function identity, parameters, local slots, bindings, return inference and scope stacks. Preserve the different meanings of module scope, unit return and a value return; Option<Type> currently encodes those distinctions.
  3. Borrow/provenance state: loans, reference-local associations, return origins, value origins and slot names. Expose operations that update related facts together rather than public mutable field collections.
  4. Pipeline state: plans, resumed/end flags, lent parameters, callback attribution and consumption state. Give this state explicit entry/exit operations before moving pipeline lowering methods.

The first extraction, frontend::pipelines::State, is passive pipeline state. Its nine fields are read and written almost only by src/frontend/pipelines/ (plus three sites in references.rs, generators.rs and the generator bridge), and every function starts with them empty. Group them in one struct owned by the pipelines module, with a Default initializer, while retaining methods on Lower. The struct’s documentation then says what the pipeline code owns, and Lower’s construction loses nine lines. Move operations only when their actual dependencies are understood. Borrow/provenance state is used throughout expression lowering and is more correctness-sensitive; extract it only if the pipeline step shows a clear reduction in coupling. Grouping pipeline state documented ownership but did not change who calls what, so borrow/provenance state was not extracted.

The goal is that a reviewer can identify which service a lowering operation may mutate. Avoid a giant context with unrestricted mutable access, cosmetic substructures whose callers still mutate every field, or a visitor/trait framework without multiple real implementations.

Keep independent op::check validation. A future validated-program wrapper could make backend entry requirements explicit, but must account for passes that transform operations after checking. A type name alone does not prove that transformed IR remains valid.

Ambient state and lifetime policy

Three mechanisms deserved separate treatment; the recommended smallest change below is implemented (issue #692). This list records the starting state:

  • src/frontend.rs (now frontend/lexer.rs): process-global FILES: RwLock<Files> maps paths to compact FileIds. It intentionally avoids atomic path clones per token. It retains each distinct path for the process lifetime.
  • src/frontend/ide.rs: thread-local ACTIVE and RECORDER; analyze enables recording, catches unwinds, disables recording and takes the result. A nested analyze on the same thread would overwrite the outer recorder and then switch recording off for the rest of the outer analysis.
  • src/lib.rs: public skip_teardown() sets a process-global atomic. src/compiler_cli.rs calls it for one-shot checks; the language server, plenty build, plenty test and library callers must not.

These are design constraints, not reproduced leaks or crashes.

Options

Smallest change (recommended): retain the file interner. Install the recorder through a guard that saves the previous ACTIVE/RECORDER and restores them on drop, so nesting and unwinding are both correct. Replace skip_teardown() with a CheckOptions field set by the CLI; a library caller then cannot change another caller’s policy. This changes a public function, so the PR must say so; the CLI is its only caller in this repository.

Broader change: a compilation session owns source identities and analysis services, with per-invocation options. Default library APIs create a normal, fully reclaimed session; the LSP can intentionally reuse one. Token IDs remain compact and every escaped analysis/artifact retains or translates its source identity.

Prefer the smallest change unless bounded source retention or session reuse has a measured consumer. A session is not automatically a cache, a scheduler, or a requirement to make lowering concurrent.

Required invariants

No file ID may resolve through the wrong session or outlive its source table. Do not recycle IDs while retained diagnostics, ASTs or operations refer to them. Keep cached artifact identities independent of process-local file numbers. Distinguish source-path interning from retaining source contents.

Teardown skipping must remain an explicit one-shot CLI optimization, never a default for embedding.

Test nested analysis and unwind restoration of the recorder. Measure retained paths/allocations separately from allocator RSS, which may remain high after correct destruction.

Structured compiler diagnostics

Parent proposal. The diagnostic contract here is proposed; existing CLI spelling remains the compatibility baseline.

Current conversion chain

src/frontend.rs defines Located with a location string, message and optional ownership Demand. Attributed marks a generic diagnostic already attributed to its proper instance, but holds only rendered text. Token::error constructs located errors. sendable preserves those two concrete types across worker boundaries and renders other boxed errors to strings.

src/op.rs::check_with converts function-check errors with to_string() to collect parallel results, then selects the first error in program order. Ownership checking builds richer messages from origins and demand information in src/ownership.rs, but returns them as text.

In src/frontend/ide.rs, recover stores deduplicated strings in Recorder.diagnostics. During finish, diagnostic searches the first line for a source:line:column: prefix and parses it back into a public IDE Diagnostic, which already exposes file, line, column and message. src/lsp.rs::check converts these into client diagnostics.

The problem is loss of structure between producers and consumers, not the absence of a public location structure. Human-readable text is functioning as an internal interchange format, and the reparse can misplace a message whose text happens to contain name:1:2: .

Introduce one internal diagnostic type, outside the IDE module, carrying:

  • the source path, line and column, each optional: an error with no source position (I/O, linking, a legacy-syntax error) has none;
  • the message text;
  • whether generic attribution has already happened (today’s Attributed);
  • the ownership Demand, where callers currently inspect it.

Do not record an end range: tokens record a start position, not an expression’s extent, and a fabricated span is worse than none. Related locations, diagnostic codes and fix suggestions wait for a consumer.

Display stays path:line:column: message, so CLI text and existing test snapshots are unchanged. Keep source positions in the compiler’s coordinate convention and convert to negotiated UTF-8/UTF-16/UTF-32 units only in the LSP adapter. Ty, and therefore Demand, is already Send + Sync, so the type can cross worker threads without conversion.

Error-type alternatives

Preferred incremental option: the concrete diagnostic boxed inside the existing Box<dyn Error> results, recovered by downcast at the boundaries that need structure (worker join, editor recording). External failures stay ordinary boxed errors without positions.

Alternative: a compiler-error enum distinguishing diagnostics, I/O and backend/link failures. It gives exhaustive handling but means migrating every Result signature in the compiler for no current consumer. Adopt only if callers need those distinctions.

Do not add a diagnostic framework dependency to remove these string conversions.

Migration

  1. Replace Located and Attributed with the new type; Token::error and demand_error construct it. Keep rendered messages identical.
  2. Make the worker boundary in check_with and sendable keep the value instead of flattening it. Preserve program-order error selection.
  3. Store structured diagnostics in the editor recorder and convert them to the public Diagnostic directly. An error that is not a compiler diagnostic has no position; never guess one from its text.

Deduplication needs a deliberate key: position and message. Two different source sites with the same wording must not collapse; the same error recovered twice must. Test both.

Acceptance

Use tests/test_lsp.rs, test_python_habit_diagnostics.rs, test_callable_diagnostics.rs, test_storage_diagnostics.rs, test_recursive_tail_errors.rs, and generic/borrowing regressions. Add focused tests for:

  • a message whose text contains a path:line:column: lookalike;
  • colons and spaces in paths;
  • identical wording at distinct sites and true duplicate recovery;
  • generic definition/use attribution and ownership demand inspection;
  • the same diagnostic at one and many workers.

The refactor succeeds when internal text reparsing is removed for compiler diagnostics while CLI text, attribution and diagnostic order remain unchanged. No speedup is claimed.

Identity, type facts and dataflow

Parent proposal. These are experiments to evaluate, not claims that every inspected lookup is a bottleneck.

Function identity across phase boundaries

src/frontend/functions.rs already has Symbol(u32), a name-to-symbol map and indexed entries for signatures/declarations. Returned-field facts are keyed by symbol. Do not propose implementing this table again.

The remaining boundary is visible in src/op.rs: DefineFn(String, ...), Call(String) and TailCall(String) retain function names. check_with collects Sigs = HashMap<&str, &FnSig>. This table borrows names and signatures; it does not copy every signature. src/codegen/symbols.rs also builds a name map when snapshotting declared Cranelift functions, while its function declarations are already indexed by Cranelift FuncId.

Options to compare

  1. Keep the current maps as control. Measure signature collection, call lookup and whole-program checking, rather than counting map sites.
  2. Add a checker-local indexed view, retaining name-bearing operations. This isolates table construction/layout experiments, but calls still require name resolution unless translated once.
  3. Carry a compiler-owned function ID through modern IR and use indexed signatures/call targets. Retain names for display, exports and external linkage.
  4. Compare interning all identifiers only if function IDs leave substantial measured name cost. This is a larger parser/AST/environment change, not a natural consequence of option 3.

Prefer a bounded checker experiment before changing the entire IR. For ID-based IR, use a neutral identity module rather than exposing a frontend-private Symbol as the backend contract.

IDs must identify the owning program/table. Never serialize a transient index as a stable cache identity without deterministic remapping. Preserve generic instance identity, function ordering, dump names, exported symbols and cache-on/off object identity. Avoid a refactor that saves lookups but adds a second full copy of every body.

Nominal type access and fact caching

src/nominal.rs::Types owns mutex-protected class/enum slot vectors and a name-keyed facts map. Nominal distinguishes local/stored definitions and strong/weak table links. get returns an Arc; try_get upgrades a slot and clones the finalized definition handle. Escaping stored fields promote weak edges so recursive declarations do not create ownership cycles.

src/type_facts.rs::Ty::facts already handles simple types directly, combines collection/box content facts, and caches complete class/enum facts. Nominal::define clears table facts as definitions become available. Incomplete recursive graphs must not acquire permanent final facts.

Explore these progressively:

  • Attribute atomic clone/upgrade traffic, facts-map hashing and locking with sampling/counters. A lock in source is not evidence of contention.
  • Compare indexed per-definition fact storage against the shared name map. Specify how finalizing any dependency invalidates prior results.
  • Compare a build-then-freeze definition table, with borrowed read-only access during checking/emission. First identify on-demand specialization boundaries: freezing too early prevents legitimate instance construction.
  • Consider broader type IDs only after these smaller experiments. Structural types, local builtin sums and escaped recursive definitions make a wholesale type arena substantially more complex than replacing an Arc<Ty>.

An epoch scheme could preserve invalidation with indexed caches; a finalized immutable graph might permit one-time facts. Neither design is safe merely because the field is called OnceLock. Prove that no definition consumed by the cached fact can change within that phase.

Retain cycle release, weak-edge promotion, stable display names and complete graph checks. Use src/nominal/clone_tests.rs, nominal unit tests, tests/test_recursive_data.rs, test_type_cycles.rs, test_generic_data_safety.rs and concurrent compiler checks.

Measured attribution (issue #696)

Sampled on 2026-11-02 at the head of the step 1–5 branches, with a release build using frame pointers, perf record -e cpu-clock on eight physical cores and the compiled-function cache off. Shares are of all samples, by self time.

FamilyGenerated checkStd-heavy check
collect_sigs (building the name-keyed map)1.5%0.7%
Lookups in that map0.1%<0.1%
Nominal handle clone, get and drop5.6%5.6%
Type facts, including cached nominal facts0.1%0.2%
Mutex/futex/Once0.2%0.1%
Ty clone and drop4.1%4.3%
mimalloc9.3%13.2%

The generated check is compile_bench 1000 20 --workload check --no-warmup (39,003 lines; median 138 ms per check with frame pointers). The std-heavy check is 40 plenty --check runs of a program importing std.http.server, std.http.client, std.db.sqlite, std.json, std.csv, std.toml, std.markdown, std.xml, std.url, std.regex, std.time and std.text (about 100 ms each).

Conclusions:

  • Function identity: the signature map costs at most about 1.7% of a check, almost all of it the walk that collects signatures rather than hashing or lookup. Function IDs in the IR (options 2–4) would not pay for their migration. Revisit only if a workload attributes a material share to collect_sigs or call lookup.
  • Nominal facts and locking: fact computation and locks are below 0.3%. Indexed fact storage, epochs and a frozen table are not justified by cost.
  • Nominal handles: clone/get/drop traffic is about 5.6%, spread across many callers: Ty::clone in lowering, generator measurement and validation. Together with Ty clone/drop and allocation it is a cost of the type representation, not of the table. A change here is the broader type-ID or clone-reduction work this page already defers, and it needs its own design and matched timings.

The memcmp, memmove and allocator shares are larger than any family above. memcmp is mostly under ownership::check (1.7% of the std-heavy run); it was not investigated further here.

Ownership dataflow: conditional follow-up only

Checker::loans in src/ownership.rs skips loan-free bodies, borrows operations in its CFG and reuses sorted Live buffers. Its liveness fixed point scans reachable nodes until no set changes. Issue #303 already removed operation cloning and per-round tree allocation; do not undo those improvements.

Only if loan-heavy workloads attribute significant time here, compare:

  • a predecessor worklist that revisits nodes affected by changed successors;
  • dense bitsets for functions with many dense loan IDs, against sorted sparse buffers for small/sparse sets;
  • buffer reuse across functions without retaining excessive peak storage.

Measure nodes, edges, loans, rounds, set sizes and allocations. Full scans can beat worklists on small functions; dense bitsets can waste memory. Any hybrid threshold must come from workload evidence.

A traversal helper may reduce duplicated nested-block enumeration (in_body, has_loans, codegen walkers), but passes differ in cleanup, nested-function and control-flow semantics. Share only a precisely specified traversal with tests for each operation family, not one universal visitor that silently changes the graph.

Replayable module-lowering results

Parent proposal. No lowered-module cache is implemented by this design. The module-cache investigation is the evidence baseline; this page describes the boundary it requires.

Evidence and limits

The recorded investigation at 00d32a13, on an AMD Ryzen 7 7840HS with seven repetitions and pinned physical cores, measured a synthetic ten-module application importing std.http.server:

Object-build modeMedian
Cache disabled45.50 ms
Warm function cache44.54 ms
Unchanged program cache3.99 ms
Ordinary body edit44.00 ms

These exclude native linking and are not current-main benchmark results. The body-edit profile recorded parsing 8.35 ms, resolution 3.32 ms and program lowering 19.28 ms; lower-function time is contained in lowering. Do not sum nested rows or promise the sum as achievable savings.

A scalar separate-compilation spike was slower than the normal compiler. There is no evidence that independently packaging current modules as native libraries solves frontend reuse. Existing whole-program and compiled-function caches remain the default.

Relevant implementation boundaries

  • src/frontend/modules.rs and its children load/resolve imports and produce Resolved; source identities and access maps are part of that environment.
  • src/frontend.rs::lower consumes Resolved but mutates shared string, function and generic state while producing one Program.
  • src/frontend/generics.rs and functions.rs own pending instances and body-derived facts.
  • src/frontend/references.rs infers returned-field projections that callers use for borrow checking.
  • src/cache.rs, src/cache/program.rs, src/cache/compiled.rs and src/cache/executable.rs cover existing cache layers; a module artifact must not silently reuse incompatible keys or change cache-off behavior.

Proposed replay contract

A result needs explicit inputs, outputs and identity relocation, not simply a serialized Vec<Op>.

PartRequired information
InputsCanonical module identity/text, import resolution, compiler/settings identity and imported semantic dependencies
DeclarationsSignatures, aliases, nominal layouts, visibility, protocols and generated methods
DefinitionsGeneric templates and instantiated bodies, including their dependency provenance
Derived interfaceReturned-reference projections and any inferred capture/effect facts consumed by other bodies
Lowered contentTyped operations, referenced strings/types/functions and native substitutions
AttributionSource identities, original declaration positions, instantiation origins and deterministic diagnostic order

Replay must remap local string/type/function/source identities into the new environment in a deterministic order. Reject unsupported combinations rather than quietly sharing mutable tables between saved and replayed compilations. Handle cyclic nominal graphs without recursive serialization loops.

First extract an in-memory result for a restricted, explicitly documented module class. Replay it against a fresh imported environment and compare both accepted and rejected programs with normal lowering. Keeping all original global tables alive and returning pointers into them does not demonstrate independent replay.

Conservative full imported-source digests are acceptable initially. Precise interface invalidation is a separate refinement: a body can change semantics that callers need without changing its written signature.

Critical invalidation probe

The existing experiments/module-cache/check_dependencies.py changes an imported borrow-returning function from lending pair.x to lending pair.y, keeping select(pair: &mut Pair) -> &mut i64 unchanged. An unchanged caller mutating pair.y must change from accepted to rejected.

Retain this probe, generic-body edits, imported layout/visibility changes, import-target replacement, missing/deleted files and native substitution changes. Compare diagnostic attribution as well as executable output.

Alternatives and adoption gate

A conservative lowered-import result is the primary experiment. Separately compare lowering only reachable standard-library functions, checking whether on-demand instantiation and diagnostics remain consistent. Streaming functions can reduce peak memory but does not by itself create a module cache; mutually dependent inference/effects must still complete before consumers need them.

Only add persistence after fresh-environment replay is correct and faster on a real std-heavy application. Account for hashing, validation, decoding, relocation and merge allocations. A large theoretical reusable frontend share does not establish net benefit. Corrupt/version-mismatched artifacts must miss safely; invalidation must not suppress errors. No remote cache follows without a separately agreed trust contract.

Validation and adoption

Parent proposal. These are acceptance criteria for possible implementations, not checks claimed to have run for this assessment.

Behavior-preserving changes

Each implementation should have a bounded responsibility, its own local PR and a clear comparison against its base. Do not combine diagnostic migration, function IDs, type storage and module persistence in one refactor.

For Rust changes, run formatting, strict workspace/all-target Clippy and the full workspace/all-feature suite following the development workflow. Targeted tests help development but do not replace the full handoff checks.

BoundaryExisting coverage to retain / extend
Diagnostics and editortests/test_lsp.rs, diagnostic-specific files, generic instantiation and borrowing diagnostics
Function IDs / IRsrc/op.rs unit tests, tests/test_generics.rs, test_recursive_tail_errors.rs, test_compile_cache.rs, test_compiler_parallelism.rs
Nominal factsNominal clone/cycle unit tests, test_recursive_data.rs, test_type_cycles.rs, test_generic_data_safety.rs
Ownership worklistssrc/ownership/graph_tests.rs, test_borrowing.rs, test_call_borrowing.rs, test_borrowed_match.rs
Module replaytest_modules.rs, test_compile_cache.rs, module-edit harness and returned-projection dependency probe
Session stateRepeated library checks, editor recovery, independent concurrent sessions and unwind/nesting policy
File moves (syntax, IR, checker)Full suite; no behavior change; review with git diff --color-moved so only visibility and use lines read as edits

Add regression tests for the actual new boundary rather than only testing through the CLI. Preserve error order across workers, object identity where the existing contract requires it, source/trap attribution, runtime cleanup, generic limits and invalid-program rejection.

Measurement matrix

Follow profiling and module-edit measurements. Use an isolated target directory/cache directory so profiling builds do not replace another worker’s compiler or contaminate warm-cache comparisons.

Measure at least:

  • small checks and larger generated programs across multiple sizes;
  • ordinary and generic-heavy checking;
  • std-heavy imports and a representative application;
  • cache-off object emission, warm-function emission, unchanged-program hits, body edits, interface edits and generic edits;
  • one worker/core and the intended multicore configuration.

Use release, feature-disabled compilers for final latency comparisons. Use phase profiling for attribution, profile-alloc for allocation traffic, and a sampling profiler for uninstrumented leaf costs. Inclusive rows cannot be added as independent phase costs. Integration tests use a different allocator path from compiler executables; do not interpret their wall time as the CLI’s allocation performance.

Record exact compiler commits, Rust/Cranelift versions, CPU/affinity, allocator, flags, source size, cache state, warm-up policy and every timed sample. Alternate baseline/candidate runs where possible. Report median, dispersion, user/system CPU, peak RSS, allocation bytes/counts, instructions and cache/TLB counters when available. Explain unavailable counters. Keep linking separate from object emission.

Decision rules

  • A structural refactor can be adopted without a speedup if it measurably simplifies ownership/dependencies and preserves behavior without a material performance or memory regression. Review the resulting API, not only file size or line count.
  • A performance change needs a repeatable end-to-end benefit beyond observed noise, not merely a smaller microbenchmark. Identify any small-program, multicore, memory, artifact-size or generated-code tradeoff explicitly.
  • A module replay design needs correct invalidation on rejected as well as accepted inputs, plus net savings after replay costs.
  • A session change needs defined lifetimes and ordinary destruction for embedding. Correct destruction does not require the allocator to return every freed page to the operating system immediately.
  • A failed experiment is useful evidence. Keep its method/results and reject the complexity rather than weakening checks or moving the benchmark target.

No fixed percentage is promised. Agree on workload-specific acceptance and acceptable regressions before implementing an invasive performance design.

Documentation and handoff

Keep reference pages describing shipped behavior until implementation lands. Then update the frontend/cache/library API references actually affected. Update learner material only for observable changes; do not manufacture tutorial changes for an internal representation refactor.

Run all three docs CI checks for book changes: source links, Python helper tests and the complete versioned-site builder with the pinned mdBook. In each PR, state what ran and what did not. Record delivered scope in completed-work records and narrow the backlog, without marking a design proposal as a shipped compiler optimization.

Python slice syntax

Status: Implemented (issue #579). Copying slices of lists, text, Bytes, bytes, tuples and clists shipped as one change rather than the planned sequence; the slices reference owns the contract. Shipped differently: tuple slices of affine components and slices of temporary clists are rejected; Python-style copy() of such components is not implemented. Element types containing Shared fields copy with their handles shared since issue #582. Remaining: borrowed views &xs[a:b] (issue #581, B39/B11), negative tuple indices (issue #583), and first-class slice(...) values. The original text below was researched against local main at f681a0d5 on 2026-10-30; the implementation and documentation plan is a companion page.

Problem

Slicing is among Python’s most used features, and Plenty’s Python-shaped syntax makes people and coding agents reach for it first. B39 left it out because buffer[a:b] would hide a copy. This proposal keeps Python’s syntax and results, and makes the copy visible the way list literals do: a slice returns Result[_, AllocError] and needs ?.

def main() -> Result[(), AllocError | IoError]:
    xs = [0, 1, 2, 3, 4]?
    print(xs[1:3]?, xs[-2:]?, xs[::-1]?)?      # [1, 2] [3, 4] [4, 3, 2, 1, 0]
    print("Aé🙂Z"[1:3]?)?                      # é🙂
    Ok(())

Decisions

These were settled with the user on the issue and in review of local PR #694:

  1. A slice that produces a new value is fallible: ys = xs[1:3]?, typed Result[_, AllocError].
  2. Elements are duplicated with copy() semantics: deep and fallible in the same AllocError. Shared[T] elements are duplicated with share(). Element types that copy() rejects are a compile error naming the type.
  3. Copying slices come first. Borrowed views will be spelled &xs[1:3]; the syntax reserves that form.
  4. Out-of-range bounds clamp exactly as in Python, including negative indices and negative steps.
  5. First-class slice(...) values are deferred.
  6. Receivers: list, str, Bytes, bytes, tuples and clist.
  7. Every slice returns a Result, tuples included, for consistency. The compiler may remove work it can prove unnecessary.
  8. A bytes slice returns an owned Bytes, with no import needed.
  9. copy() will duplicate nested Shared handles with share() (issue #582). Slices of elements that contain Shared fields depend on that change.
  10. Tuple indexing will accept negative literals, as in Python: t[-1] (issue #583).
  11. str slices always copy. & and &mut applied to a slice are rejected until borrowed views exist (issue #581).
  12. str.slice() stays until slice syntax lands; whether to deprecate it is decided then, by its remaining use.
  13. The step has a nonzero integer type, as range steps will (issue #584), so a zero step cannot be written or computed. See the step.

Plenty has no shared mutable aliasing, so a deep copy differs from Python’s shallow copy only where a Python program mutates an element through one list and reads it through the other. Plenty can express that only through Shared[Mutex[T]] elements, and slices share() those, which gives Python’s reference behavior.

Syntax

A subscript whose brackets contain a top-level : is a slice:

subscript = primary "[" expression "]"                       # index, unchanged
slice     = primary "[" [start] ":" [stop] [":" [step]] "]"

xs[:], xs[::], xs[a:], xs[:b], xs[::c] and xs[a:b:c] are all slices. A comma inside slice brackets (m[1:2, 3]) is rejected; there are no multi-dimensional subscripts. The parser builds a new Slice expression rather than reusing Index, so later passes cannot confuse the two.

A slice is a postfix operation with the same precedence as indexing, calls, member access and ?. These parse as written:

SourceReading
xs[1:3]?Copy, then propagate allocation failure
xs[1:3]?[0]Index the copy
len(xs[1:3]?)Length of the copy
s[1:]?.upper()?Method on the copied text
rows[0][1:]?Slice an element of a nested list
&xs[1:3]Reserved for a borrowed view; rejected for now

No conflict with generic brackets

NAME[...] is ambiguous in today’s parser: brackets after a name or member path first parse as type arguments, and module resolution later picks the generic or index reading. Issue #562 is a case where it picks wrongly: TABLE[n] on a module constant is read as a generic call. A type-argument list can never contain :, so type_arguments fails at the colon and the parser falls back to the subscript path. No Ambiguous node is built, and TABLE[n:] works even while TABLE[n] does not. Issue #562 remains a separate fix.

The only other expressions with a colon are anonymous functions, def(x: i64) -> i64: x + 1, whose parser consumes its own colons, and dict literals, which are inside braces. Neither can end a bound early.

Bounds and normalization

Bounds are i64 expressions, or references to i64, as indices are. Unsuffixed literals infer i64; other integer types need an explicit cast such as i64(n). The step is a nonzero i64, described in the step; an omitted step is 1. Omitted bounds depend on the sign of the step, as in Python. Positions count elements, Unicode scalars for str, bytes for Bytes and bytes, and rows for clist.

Normalization follows CPython’s PySlice_Unpack and PySlice_AdjustIndices. This algorithm is normative:

positions(length, start: Option[i64], stop: Option[i64], step: NonZero[i64]):
    # length >= 0. Arithmetic is exact: compute in i128, or first clamp a step
    # of i64.MIN to -i64.MAX as CPython does. Both give the same positions,
    # since a step of magnitude >= length selects at most one element.
    # step != 0 by its type; there is no zero case.
    if step > 0:
        low, high = 0, length                 # clamp targets
        first, last = 0, length               # omitted start, omitted stop
    else:
        low, high = -1, length - 1
        first, last = length - 1, -1          # -1 here means "before 0",
                                              # not the last element
    adjust(bound):
        if bound < 0:
            bound = bound + length
            if bound < 0:
                bound = low
        elif bound >= length:
            bound = high
        return bound
    s = adjust(start) if start is present else first
    e = adjust(stop) if stop is present else last
    if step > 0:
        count = (e - s - 1) // step + 1 if s < e else 0
    else:
        count = (s - e - 1) // (-step) + 1 if e < s else 0
    return s, s + step, s + 2 * step, ... (count positions)

The existing runtime helper slice_bounds in plenty-runtime/src/aggregates.rs is the step-1 case of this algorithm; the implementation generalizes it.

These are the issue’s examples with a = [0, 1, 2, 3, 4]. Each result matches CPython 3, checked with slice.indices while writing this page.

ExpressionMeaningNormalized (s, e, step)Result
a[1:3]start through stop - 1(1, 3, 1)[1, 2]
a[1:]start through the end(1, 5, 1)[1, 2, 3, 4]
a[:3]Beginning through stop - 1(0, 3, 1)[0, 1, 2]
a[:]A copy of the whole sequence(0, 5, 1)[0, 1, 2, 3, 4]
a[1:4:2]start, not past stop, by step(1, 4, 2)[1, 3]
a[::2]Every second item(0, 5, 2)[0, 2, 4]
a[-1]Last item: an index, not a slice—4
a[-2:]Last two items(3, 5, 1)[3, 4]
a[:-2]All except the last two(0, 3, 1)[0, 1, 2]
a[::-1]All items, reversed(4, -1, -1)[4, 3, 2, 1, 0]
a[1::-1]First two items, reversed(1, -1, -1)[1, 0]
a[:-3:-1]Last two items, reversed(4, 2, -1)[4, 3]
a[-3::-1]All except the last two, reversed(2, -1, -1)[2, 1, 0]
a[4:1:-1]Down from 4, not reaching 1(4, 1, -1)[4, 3, 2]
a[10:20]Past the end: clamps(5, 5, 1)[]
a[-100:100]Both sides clamp(0, 5, 1)[0, 1, 2, 3, 4]
a[3:1]Empty, not reversed(3, 1, 1)[]
a[::0]Zero step—compile error

Why clamp

Indexing traps on a missing position, and the Bytes methods return OutOfRange. Slices clamp instead because that is Python’s documented rule, which users already rely on: line[:80] truncates, and a[-3:] takes at most three. A trapping slice would turn correct Python habits into crashes. Clamping is total, so clamping alone never causes an error. The existing slice methods of lists and text already clamp.

The step

A zero step selects no well-defined positions. Python raises ValueError, and Plenty’s range traps today. Neither suits Plenty’s reliability goal: a zero computed elsewhere, such as n // k, must not stop the program where a slice uses it. So the step’s type cannot hold zero. It is the nonzero integer type of issue #584, called NonZero[i64] here; its exact construction API is decided in that issue. range steps use the same type.

  • An integer literal step converts at compile time: xs[::2], xs[::-1] and xs[1:10:3] need no conversion.
  • A literal 0, or a constant that evaluates to zero, is a compile error.
  • A step that is a run-time i64 needs the explicit, fallible conversion that issue #584 defines, and the program handles the zero case where the value is created. A NonZero[i64] value can be passed directly.
  • An omitted step is 1.

The check happens once, where the value is created, not at each slice. A slice has no zero-step trap and no zero-step error, and its type stays Result[_, AllocError]. Slice syntax with a written step depends on issue #584; slices without a step do not.

Results by receiver

The receiver may be an owned value, a named place, a field, an element, or a & or &mut reference. A slice only observes it: the source is unchanged, and a mutable receiver is borrowed shared for the duration of the copy.

Receiverx[a:b:c] typeSelected values are duplicated by
list[T]Result[list[T], AllocError]copy() semantics; share() for Shared[T]
strResult[str, AllocError]Copying the selected scalars’ UTF-8
std.bytes.BytesResult[Bytes, AllocError]Copying bytes
bytesResult[Bytes, AllocError]Copying bytes out of read-only data
tuple[...]Result[tuple[...selected], AllocError]copy() semantics per component
clist[T]Result[clist[T], AllocError]Copying column values

Every slice returns a Result, including empty slices, tuple slices of copyable components such as (1, 2, 3)[1:], and short strings that happen not to allocate. One rule is easier to learn and to keep in mind than a list of exceptions, and code does not change type when a bound changes. The compiler may remove work it can prove unnecessary, such as the error path of a tuple slice that cannot allocate.

Other receivers are rejected: dict, set, deques, ranges, generators, Shared[list[T]] handles (slice &*handle once views exist, or index it), and user classes. There is no __getitem__-style hook for slices yet.

Lists

Each selected element is duplicated by its type’s copy:

  • Scalars, @copy classes and copyable aggregates are copied inline.
  • str elements share their immutable storage, as copy() does.
  • Affine elements that copy() accepts, such as nested lists, dictionaries, sets, tuples of those, and classes without __del__, are deep-copied. This is what makes rows[1:]? work for the common nested data.
  • Shared[T] elements are duplicated with share(): a count increment, no allocation. The handles in both lists name the same values.
  • Aggregates that contain Shared fields, such as (str, Shared[Log]), are copied with their handles shared once copy() does that (issue #582). Until then they are rejected, as copy() rejects them.
  • Any other element type is a compile error naming the type. These include Bytes and other resources, generators, closures with captures, types with __del__, and aggregates that contain any of them.

For an owned temporary receiver, as in load()[1:]?, the implementation may move the selected elements instead of copying them and drop the rest. Types whose destruction is observable are rejected anyway, so the only difference is fewer allocations. The existing .slice(start, stop) method keeps that transfer as its contract and stays the way to take affine, non-copyable elements out of a temporary.

Generic code is checked per specialization, so xs[1:]? in a function over &list[T] reports a non-copyable T at the instantiation, naming the concrete type.

Strings

Positions are Unicode scalars, as for len(text) and text[i]; they are not bytes and not grapheme clusters. "Aé🙂Z"[::-1] is "Z🙂éA", and a combining mark is reversed separately from its base.

A step-1 string slice copies. It does not share the source’s storage:

  • A str is one word pointing at a header followed by its UTF-8 bytes (see value representations). A substring that shares storage needs an offset and length, either a second word or a new header kind. Every string operation, hash and comparison would pay to check for it.
  • A shared substring keeps its whole source alive. A ten-character field from a 100 MB file would hold the file. Java changed String.substring to copy in JDK 7u6 for this reason, and Go substrings have the same retention cost. That conflicts with low memory use.
  • The string design chose one public str with no owning/view distinction. A view belongs to &s[a:b], a separate type that borrows.

So the rule is the same for every str slice: it returns Result[str, AllocError]. Two cases allocate nothing as an optimization, not as a contract: results of at most seven UTF-8 bytes are stored inline (the runtime’s try_build already does this), and a whole-string step-1 slice may retain the source’s immutable storage instead of copying it.

Bytes and bytes

buffer[a:b:c] on a Bytes returns a new Bytes with exact capacity, like the slice method. The syntax clamps; the methods keep the range rule, under which an out-of-range bound returns BytesError.OutOfRange. Index syntax already follows list rules rather than method rules, so this is consistent within Bytes. Because clamping removes OutOfRange, the error is AllocError, not BytesError. Code whose error type is only BytesError keeps using slice(start, stop) or widens its error type.

A bytes value refers to read-only data with its length in a header, so a sub-range cannot be another bytes without a new representation, and new read-only data cannot be created at run time. A bytes slice therefore returns an owned Bytes: b"GET /"[0:3]? is the buffer GET. This is the same copy as std.bytes.from_bytes(value)? followed by a slice, done in one allocation. Using the result does not need import std.bytes; naming its type does, as f-strings produce a std.fmt.FormatError without an import.

Tuples

The result type depends on the bounds, so tuple slice bounds and step must be integer literals, optionally negated (t[-2:]), or omitted. Any other bound is rejected: tuple slice bounds must be integer literals; the result type depends on them. The positions are computed at compile time with the arity as the length:

Source, t: (i64, str, list[i64])Type
t[1:]Result[(str, list[i64]), AllocError]
t[:1]Result[(i64,), AllocError]
t[::-1]Result[(list[i64], str, i64), AllocError]
t[3:]Result[(), AllocError]

An empty selection has the unit type (), which is also how Python spells the empty tuple. Components are duplicated with copy() semantics, so the list[i64] component is deep-copied and the source tuple is unchanged. Tuple indexing originally accepted only nonnegative literals; issue #583 made it accept negative literals too, so indices and slice bounds follow the same rule.

Columnar lists

items[a:b:c] on a clist[T] returns a new clist[T] holding the selected rows. Fields are scalars, so no element copy can fail; only the new column storage can. For a step other than one, each column is gathered with a stride. Whole-clist copy is unsupported today, so items[:]? becomes the way to duplicate one.

Evaluation, borrowing and &

A slice evaluates the receiver place, then start, stop and step, left to right, each once. It then normalizes the bounds and copies under a shared loan of the receiver. Bounds may read the receiver, as in xs[len(xs) - 2:]?, because the bounds are evaluated before the copy and only shared loans are involved.

& and &mut directly applied to a slice are rejected, with or without ? and with or without parentheses:

&xs[1:3]      # error: borrowed slice views are not implemented (B11/B39);
&xs[1:3]?     #        bind the copy first: `part = xs[1:3]?`, then `&part`
&mut xs[1:]   # error: likewise

&xs[1:3] is the planned view. Accepting &xs[1:3]? now as a borrow of a temporary copy would leave two spellings one character apart, one copying and one not, once views land. Rejecting both now keeps the view syntax free and the cost of every accepted form plain. These rejections are phase-guard tests that name issue #581, B11 and B39.

Costs and allocation failure

For k selected elements, a slice does O(1) bound normalization and O(k) copying, plus the cost of each element’s copy:

ReceiverWorkAllocations
list[T], copyable TO(k) row copies or retainsList header, plus rows when k > 0; as slice()
list[T], affine TO(k) plus each element’s deep copyAs above, plus each copy’s allocations
str, ASCIIO(k)One, or none when the result fits inline
str, other textOne scan over scalars to the farthest selected position; O(length) at worst, no auxiliary storageOne, or none when the result fits inline
Bytes, bytesO(k)As Bytes.slice()
TuplesEach selected component’s copyOnly those of the components’ copies
clist[T]O(k) per columnAs clist[T].with_capacity(k)

The count k is known before copying, so the result is reserved once with its exact size; no slice grows its result. A str slice of non-ASCII text needs the selected scalars’ byte length before allocating, so it scans the selected region twice.

If an allocation fails, the slice returns Err(AllocError.OutOfMemory). Every value already copied into the partial result is dropped, last copied first, and the partial result is freed. The source is never changed, so retrying or recovering needs no repair. k never exceeds the source length, so a slice never reports CapacityOverflow.

Copying a slice only to iterate it wastes the copy. Lessons should teach for i in range(1, len(xs)): or a pipeline over the source for that, until a view can be iterated with for x in &xs[1:].

Diagnostics

SourceMessage
ys: list[i64] = xs[1:3]slice returns Result[list[i64], AllocError]; use ? to propagate allocation failure, matching the collection-literal message
buffers[1:]?, buffers: list[Bytes]slicing list[Bytes] copies each element, and Bytes cannot be copied; hint: transfer from an owned temporary with .slice(start, stop), or build a new list from explicit copies
xs[::0]slice step cannot be zero
xs[::k], k: i64slice step must be NonZero[i64]; hint: the conversion from issue #584
xs[a:b] = ys, xs[a:b] += ysslice assignment is not supported; hint: assign elements by index, or build a new list
&xs[1:3], &mut xs[1:3]borrowed slice views are not implemented yet; hint: bind the copy first
t[i:], t: tupletuple slice bounds must be integer literals; the result type depends on them
m[1:2, 3]a slice cannot have several dimensions
d[1:2], d: dictdict does not support slicing
xs[1.5:]The existing index-type diagnostic, naming i64

The current “this resource cannot be copied” message does not name the type. The slice diagnostic names the element type and the receiver type, since the element is not written in the slice expression.

Out of scope

FeatureWhat it would need
Slice assignment a[1:3] = valuesPython lets a step-1 assignment change the length, which can allocate. An assignment statement has no value for ?, so this needs a fallible method such as xs.splice(start, stop, values)? or a statement-level failure rule. A stepped assignment requires equal lengths. The right side’s elements would move in.
Deletion del a[1:3]Plenty has no del. A remove_range(start, stop) method needs no allocation and can be added independently.
slice(start, stop, step) valuesA prelude slice type with Option[i64] fields, and subscripts that dispatch on the operand’s type; today they accept integers and dictionary keys. A range has required, typed bounds and is a sequence of positions; a slice is unresolved until applied to a length. s.indices(length) would return the normalized range.
Borrowed views &xs[a:b], &mut xs[a:b] (issue #581)A view type (address, length, stride) under the existing loan rules. Local-only views can follow the dictionary key-view precedent (issue #396); storing and returning them needs B11 stored references. &mut xs[a:b] would allow in-place sorts and fills of a sub-range. String views need byte offsets; Bytes sub-range views are B39’s remaining item.
Ranges, deques, generators, user classesSlicing a range could return a range without allocation; user classes need a hook design. Each needs its own use case.

Prior art

  • Python copies for list, str, bytes and tuple slices, clamps, and allows negative steps. memoryview and NumPy slices are views. This proposal takes Python’s results and adds the visible ?.
  • Rust spells views &v[a..b]: no copy, no negative positions, no step in the syntax (.step_by on iterators), and out-of-range bounds panic. v.get(a..b) returns an Option, and .to_vec() copies. Plenty’s planned &xs[a:b] follows Rust’s choice that a borrow is spelled with &.
  • Go slices s[a:b] share the backing array, so append through one slice can overwrite elements visible through another. Out-of-range bounds panic, and there are no negative positions or steps. Its three-index form s[a:b:c] sets capacity, not a step. Plenty rejects shared mutable aliasing, so it cannot copy this design.
  • Mojo keeps Python’s a[start:stop:step] syntax through __getitem__ with a Slice argument. Its List returns a new list, while Span and StringSlice borrow.
  • Nim writes inclusive a[1..3] and counts from the end with a[^2..^1]. It copies into a new seq and raises on out-of-range bounds; toOpenArray borrows without copying.

Slice syntax implementation plan

Status: Implemented in one change rather than the sequence below. This page planned the slice syntax proposal (issue #579). Its code fences show proposed syntax and are not compiled. File names refer to local main at f681a0d5.

Reviewable slices

Each slice is one local PR. Each updates the reference pages, cheat sheet rows and tutorial lessons for what it ships, following the development workflow.

  1. Parsing, diagnostics and copyable lists. Add Expression::Slice { base, start, stop, step } in src/frontend.rs; the postfix loop that builds Expression::Index recognizes a top-level :. Reject slice assignment targets (Statement::SetIndex and augmented assignment), commas, & and &mut applied to a slice, and unsupported receivers, with the messages in the proposal. Generalize the runtime’s slice_bounds (plenty-runtime/src/aggregates.rs) to the stepped algorithm, and extend try_list_slice to copy strided rows of copyable elements, including str retains. The step uses the nonzero integer type, so S1 depends on issue #584 for written steps; slices without a step can land first if #584 is not ready.
  2. Lists of affine elements. Reuse the runtime’s try_copy for each selected element, and share() for Shared[T]. Elements with nested Shared fields work once issue #582 lands. Check copyability per specialization, with a diagnostic that names the element type. Owned temporaries may transfer selected elements instead of copying them.
  3. Strings. Extend try_slice in plenty-runtime/src/strings.rs to steps: byte offsets for ASCII text, one bounded scan for other text, an exact-size result. Retain the source for a whole-string step-1 slice.
  4. Bytes and bytes. Lower Bytes slices to the native storage copy used by Bytes.slice, without its range rule. Make bytes slices produce a Bytes, loading std.bytes when the program does not import it.
  5. Tuples and columnar lists. Compute tuple positions in the checker from literal bounds and build the result tuple type. Add a strided column copy for clist[T].

The first slice alone covers most uses (xs[1:3]? on lists of numbers or strings). Slices 2 to 5 depend on the first but not on each other.

Tests

Extend tests/test_slicing.rs, which holds the existing slice method tests:

  • Normalization model. A Rust model of PySlice_AdjustIndices inside the test, compared against compiled programs for list lengths 0 to 6, every bound from -8 to 8 and omitted, steps -3 to 3 except zero, and the extremes i64.MIN and i64.MAX for bounds and steps. The table of examples in the proposal is a separate rstest case list, with results checked against CPython when it was written.
  • Runtime unit tests in plenty-runtime for the generalized slice_bounds, including a step of i64.MIN and lengths 0 and 1.
  • Element duplication. Nested lists, dicts and tuples of lists are independent after the slice; mutating the copy leaves the source unchanged. Shared[Mutex[i64]] elements are shared: a change through one list’s handle is visible through the other.
  • Allocation failure under the runtime-checks feature, following every_list_slice_allocation_failure_preserves_borrowed_sources: for every allocation budget of a nested-list slice, the result is Err(AllocError.OutOfMemory) or the full slice, and the source prints unchanged. Under --all-features the allocation-checks runtime confirms at exit that partial results were freed.
  • Evaluation order. Bounds with printing side effects run once, left to right, after the receiver; bounds may read the receiver.
  • Parsing. TABLE[n:] on a module constant works while issue #562 is open; conditional and call expressions as bounds; xs[1:3]?[0] and len(xs[1:3]?).
  • Diagnostics. One plenty-error style case per row of the proposal’s diagnostics table. The &xs[a:b] rejection is a phase-guard test with a comment naming issue #581, B11 and B39, to be replaced when views land.
  • Step type. Literal steps convert at compile time; literal and constant 0 are compile errors; a plain i64 step is a compile error whose hint shows the issue #584 conversion; a converted run-time step slices correctly.
  • Strings. ASCII and mixed text, negative steps, combining marks, results of seven and eight UTF-8 bytes (inline versus heap), and the whole-string case.
  • Tutorial and reference fences, run by test_tutorial:: and the design example tests.

Documentation and tutorial

A new reference page, design/language/04-expressions/slices.md, holds the syntax, the normalization algorithm, the examples table, the step type and the reserved & form. Receiver pages link to it and state their result types and costs:

PageChange
design/facilities/01-builtins/lists.md#slicesLead with xs[a:b:c]?; keep slice() as the transfer from an owned temporary
design/facilities/01-builtins/text-building.mdAdd text[a:b:c]? to the table; remove “Slice syntax and steps are unsupported”
design/stdlib/bytes.mdReplace “There is no slice syntax” with the clamping syntax and its AllocError; keep the method range rule
design/language/03-types/bytes.mdReplace “no slicing” with the Bytes result
design/language/03-types/tuples.mdLiteral-bound slices and their types
design/facilities/01-builtins/clists.mdRow slices; items[:]? as the duplicate
design/examples/cheatsheet.mdUse name[1:3]? and items[1:3]? in the examples; replace “There is no slice syntax”; change the Python row to xs[1:3] → xs[1:3]? (“allocates a copy”)
design/examples/collections/list-slices.md, design/examples/text-files/text-23-slice-text-by-unicode-scalar-position.mdShow the syntax first, then the methods
design/toolchain/02-architecture/support.mdList slice syntax

Tutorial:

  • New lesson “Take part of a list or text” in tutorial/04-collections/, after Transfer or copy ownership, because a slice is a copy. It teaches xs[a:b]?, omitted bounds, negative positions, [::-1], clamping, and the cost of copying; it shows nested lists being deep-copied, and recommends a range loop or pipeline over the source when the copy is only iterated.
  • Build text with recoverable allocation mentions text[a:b]? as another allocating text operation.
  • Byte buffers keeps frame.slice(1, 3)? where the function’s error type is BytesError, and notes the syntax.

Review other lessons and stdlib/std/*.plenty for .slice( calls and manual sub-range loops that the syntax states more clearly, following the tutorial guidelines. The standard library has 45 .slice( calls; migrate only where the result is clearer, as a separate PR.

Status updates when the work lands: this proposal’s status blocks, the string design note “slice syntax remains a B12 candidate”, and the proposals index.

Backlog note

The backlog is edited only after merge. Proposed B39 wording, replacing “Slice syntax is deliberately omitted: it would hide a copy.”:

Slice syntax (buffer[a:b:c]?, issue #579) copies with Python’s clamping and returns Result[Bytes, AllocError]; bytes slices produce a Bytes. See [slices](../../slice-syntax/design/language/04-expressions/slices.md). Remaining: borrowed sub-range views &buffer[a:b] (issue #581; needs B11 stored references).

B12 lists “borrowed sub-range views (B11/B39)”; add that their spelling is &xs[a:b].

Development workflow

The backlog is the sole list of future work and priorities. Issues, claims, and local PR reviews live in the shared issues.db; the repository’s AGENTS.md and CLAUDE.md describe the worktree and review workflow.

Keep implementation, reference contracts, and executable lessons aligned in the same change. Reference pages describe implemented rules and current limits; proposals preserve rationale and alternatives. Update a backlog entry when its scope changes or completes.

Native tests should check observable output, diagnostics, evaluation order, mutation, ownership, borrowing, and cleanup. Allocating operations need failure and partial-state cleanup checks. Use runtime allocation checks, sanitizers, or Miri where they exercise the changed boundary.

Build a complete local installation

Install just, a stable Rust toolchain with its rust-docs component, Python 3, and the native C compiler/linker and binutils (readelf, nm). From the checkout, run:

just                       # same as just build
just compiler              # compiler only
just engine dataframe      # one engine and its redistribution notices
just engine linalg
just test                  # all-features workspace suite, with a timing report
just docs                  # all three published-site checks
just ci                    # every CI check this Linux host can run
just release-check         # just ci plus the Docker release build and smoke tests
just --list

The default builds release plenty and plentyc, then both optional engines, sequentially. The first engine builds are substantial; later builds reuse Cargo artifacts. target/release/ contains both executables, both shared libraries, and dataframe-notices/ and linalg-notices/ with checksummed metadata and licenses for --dist. Missing Rust license files can be installed with rustup component add rust-docs. Cargo may need network access for uncached registry and pinned Git dependencies.

Add the checkout’s absolute target/release directory to PATH. For Fish:

fish_add_path /path/to/plenty/target/release

Rebuilding updates this installation; moving the checkout or cleaning its artifacts removes it. The compiler recipe explicitly selects target/, even if CARGO_TARGET_DIR is set. Engine caches stay under target/dataframe-engine/ and target/linalg-engine/. Ordinary cargo build --release --locked remains compiler-only. Existing Python engine build commands remain available for custom output paths and job counts.

These are local host builds, not portable release archives. Local packaging retains architecture, SONAME, export, dependency and RPATH checks, but records the engine’s actual required glibc symbol version rather than enforcing 2.28. It also records build_kind: local and whether the checkout is dirty (excluding the shared issues register); this is provenance information, not an attestation. The compiler and generated applications can require newer system libraries too. The explicit --local-install packaging mode does not produce release archives or enforce the portable GLIBC/GCC symbol baseline; the release workflow never selects it. Portable releases keep the strict baseline audit.

Do not run builds against the same installation concurrently. Recipes stop on the first failure; the complete installation is not a transaction. Notices are staged before replacement, and --dist rejects a library whose checksum no longer matches its notices after an interrupted update. Re-run the recipe to repair that installation. Existing notices directories are generated output and are replaced, not a location for application-owned files.

Check timing reports

just test runs the full all-features workspace suite, including doctests, with compilation timed separately from test execution. It saves Markdown and JSON reports under a unique target/check-timings/suite-*/ directory, and retains Cargo’s compilation HTML under target/cargo-timings/. Output stays on the terminal and the command returns failure if compilation or tests fail. The execution phase includes Cargo’s short freshness check and doctest builds; this is command-level attribution, not a compiler or per-test profiler.

python3 -B scripts/check_timings.py suite --profile ci
python3 -B scripts/check_timings.py report target/check-timings/suite-CURRENT/timings.json --baseline target/check-timings/suite-EARLIER/timings.json

just ci also saves each invocation’s step logs, steps.tsv, timings.json and timings.md under target/local-ci/run-*/, including failed steps. It continues to later checks after a failure. These timings use Bash’s integer seconds; the suite wrapper uses a monotonic clock. Both add negligible work at command boundaries and retain separate runs for comparison.

Direct cargo test remains available but does not save this report. Reports record the revision and host; note profile, cache warmth, competing builds, Cargo lock waits and environment overrides when comparing runs. JSON command rows record the suite’s actual Cargo arguments. Measurements observe ordinary work; they do not reserve the host or claim controlled benchmark results.

For CI job/step reports and historical comparisons, see CI timings.

Dependency versions

The compiler executable and runtime share a tracked workspace Cargo.lock. Fresh checkouts and worktrees use its dependency versions; CI builds and tests with --locked so a stale or missing lockfile fails instead of resolving a new dependency graph. Use --locked for local validation too.

Update dependencies deliberately with cargo update, and review the resulting lockfile changes with the code that needs them. Changes to Cranelift or regalloc2 require rerunning object-identity checks and affected compiler performance comparisons. Record the dependency versions with benchmark results so results from different dependency graphs are not compared as if only Plenty changed.

Build profiles and CI

The workspace uses Rust edition 2024. Development and release builds disable incremental compilation; workspace code keeps line tables for backtraces, while dependencies have no debug information. See CI and repository configuration for the build profiles, jobs, caches, timings, and source highlighting.

For the local CI-profile full suite, run:

cargo nextest run --locked --workspace --all-features --cargo-profile ci

Running tests

cargo test --locked --workspace --all-features runs the full suite: the tests CI runs in shards under nextest, plus the doctests. When several workers share a machine, add -j 3 -- --test-threads=3.

CI’s lint job also gates cargo fmt --all -- --check, cargo clippy --locked --workspace --all-targets -- -D warnings, and python3 -B scripts/check_links.py book/src on stable Rust. Run them before committing.

The integration tests in tests/test_*.rs are modules of one test binary, tests/integration.rs, so all of them share one pool of test threads and the binary is linked once. Select one file’s tests by module, or one test by path:

cargo test --locked --test integration test_std_url::
cargo test --locked --test integration -- --exact test_std_url::rfc_3986_examples_split_into_components

Cargo builds separately for each set of --workspace and feature flags, so add the full suite’s flags to reuse its build. List a new test file in tests/integration.rs; every_test_file_is_listed fails until it is. Editing any test file recompiles the whole binary.

Most tests compile a Plenty program and link it with cc. The workspace’s .cargo/config.toml sets COMPILER_PATH for the processes Cargo starts, so where mold is installed as /usr/libexec/mold/ld, cc links these programs with mold, several times faster than GNU ld. Elsewhere cc uses its default linker. CI installs mold for this. The setting does not change the compiler’s default cc driver or the Rust linker; COMPILER_PATH= cargo test ... links with the system default. Tests must not depend on one linker’s output, such as the exact symbol table. The file also sets PLENTY_LINKER_THREADS=1, so each test’s link runs mold on one thread (Native linking): many links run at once, and mold’s default thread per core made them compete for the cores. PLENTY_LINKER_THREADS= cargo test ... restores mold’s default.

The same file points PLENTY_CACHE_DIR at target/plenty-cache, so every test run of a worktree reuses the compiled-function cache from the runs before it and never writes to ~/.cache. A fresh cache must not change any result: PLENTY_CACHE=off cargo test ... runs without it, and a test that exercises the cache uses a directory of its own.

cargo-nextest is optional locally; CI uses it. cargo nextest run --locked --workspace --all-features runs each test in its own process and does not run doctests (cargo test --locked --workspace --all-features --doc does). Its -j sets test threads, not build jobs: when sharing a machine, use --build-jobs 3 --test-threads 3. Because each test is its own process, a static lock in a test file no longer keeps two tests apart: tests that must not overlap belong to a test group in .config/nextest.toml, as the timing tests of tests/test_compile_scaling.rs do. --profile ci is CI’s profile: it runs every test after a failure and writes target/nextest/ci/junit.xml.

Runnable documentation

tests/test_tutorial.rs reads tutorial and design/ pages in SUMMARY.md order. A plenty fence has an output fence and runs both through compile-and-run and an explicitly compiled binary. A plenty fragment fence is an explanatory excerpt, such as a signature shape, and is not run. A plenty-error fence has an error substring and must fail without executing effects. A plenty-file companion module remains on the same page as its example. Every book page must appear in SUMMARY.md. The examples run as 16 tests in parallel, each taking every 16th example in reading order; a failure names the example’s page and line.

book/theme/plenty-highlight.js registers the highlight.js grammars for these fences: Plenty source for plenty, plenty fragment, plenty-error and plenty-file, compiler diagnostics with highlighted `code` for error, and plain text for output. book/theme/plenty-highlight.css colours the classes the bundled themes leave unstyled. Its keyword, literal, type and built-in lists follow the compiler; tests/test_book_highlighting.rs fails when a reserved word or lower-case prelude name is missing from them, or when they name one the compiler no longer defines.

Proposal fences are illustrative and are not executed by the tutorial suite. Use text fences for proposed syntax; check implemented behavior with a complete program or an executable lesson.

Before handing off changes to book content, navigation, assets, configuration, or docs build/check scripts, run the same checks as documentation CI:

python3 -B scripts/check_links.py book/src
python3 -B -m unittest discover -s scripts -p 'test_*.py'
python3 -B scripts/build_docs_site.py

Use the mdBook version pinned in book/releases.json; see Documentation site for installation. Record the complete-site result in the PR. If a check is blocked, state which one and why rather than reporting documentation validation as complete. The assembled-site check includes generated chapter and print links and version navigation; a plain mdbook build is only a preview, not a substitute.

For moved or changed executable lessons, also run cargo test --locked --test integration test_tutorial::. Prose-only changes need the listed docs checks, not a compiler rebuild.

The link checker uses Python’s standard library. It checks local file targets in inline links, images, and reference definitions, ignoring code and comments. It does not check remote URLs, HTML links, or heading anchors. To check only a changed area, pass a directory or Markdown file relative to book/src, for example python3 -B scripts/check_links.py book/src proposals.

Documentation site

Install the pinned tools and preview the book locally:

rustup toolchain install 1.99.0 --profile minimal
cargo +1.99.0 install mdbook --version 0.5.4 --locked
mdbook serve book --open

python3 -B scripts/build_book.py builds the checkout into book/book; python3 -B scripts/build_docs_site.py builds and checks the complete versioned site in book/site. Do not use mdbook test for Plenty fences; the tutorial integration tests execute them.

See documentation deployment for site previews, publication, permissions and recovery, and documentation versions for release admission and retention.

Reverse-proxy tests

Run the nginx and Caddy integration tests when needed; they require Docker and curl:

PLENTY_DOCKER_TESTS=1 cargo test --locked --test integration test_std_http_server_proxy::

Without the variable, these tests report a skip and pass. With it, a missing Docker daemon or curl fails them. See reverse-proxy tests for container cleanup and CI image maintenance.

Quick reproductions

The compiler expects a complete program with a main entry point. Put probe statements inside it; for allocation or printing, propagate failures:

def main() -> Result[(), AllocError | IoError]:
    xs = [1, 2, 3]?
    print(xs)?
    Ok(())

An infallible body can use def main() -> ():. The fragment wrapper in tests/support/mod.rs is test fixture construction. For repeated probes, build once with cargo build --release, then use target/release/plenty FILE from that worktree; rebuild when compiler or runtime sources change.

Measure compilation cost when changing checking or lowering; the architecture guide describes reproducible performance evidence. The opt-in compiler profiler subdivides lowering, ownership checks and native compilation. Use compile_bench --workload check|scale|aot to isolate a workload, and --no-warmup when the report should contain only the requested repetitions. Normal tests never start a profiling session automatically.

CI and repository configuration

For contributor commands, see the development workflow.

Rust build profiles

Both workspace crates (plenty and plenty-runtime) use Rust edition 2024. The embedded runtime archives built directly by build.rs use the same edition; vendored third-party crates retain their upstream editions.

Development and release builds disable incremental compilation. Workspace sources (plenty and plenty-runtime) retain line tables for backtraces with file names and line numbers; third-party dependencies emit no debug information. Tests inherit the development settings, and benchmarks inherit the release settings. These profiles are configured in the workspace’s .cargo/config.toml so they take precedence over user-wide Cargo profile settings.

CI build caches and timings

The Cargo workflow (.github/workflows/ci.yml) runs on stable Rust for every push to main and every PR. It runs the all-feature suite, doctests and a default-feature build on both ubuntu-24.04 and ubuntu-24.04-arm; function and build cache keys include runner architecture. Optional engines are built and exercised independently. Its jobs run in parallel:

  • test: four shards of the full all-features suite under cargo-nextest, split with --partition hash:K/4 so a test stays in the same shard from run to run. The Docker proxy tests are excluded; they run in proxy_tests. Each shard uploads its JUnit report (junit-shard-K) with per-test durations.
  • doctests: builds default features once, then runs the doctests, which nextest does not run.
  • lint: format, clippy and book links.
  • proxy_tests: the reverse-proxy tests.

Optional-engine jobs use runtime-checks for both module tests and tutorial examples, reusing the compiler, embedded runtime and integration test binary. Cargo also builds the CLI for the integration target, so the sales-report application uses that binary without a separate compiler build. just ci dataframe linalg follows the same rule locally. The doctests job still verifies a default-feature compiler build separately.

The shards build with the ci Cargo profile, defined in .cargo/config.toml: the dev profile with the plenty crate at opt-level = 1 and dependencies at opt-level = 2, keeping debug assertions and overflow checks. Nearly every test compiles a Plenty program, so suite time is mostly the compiler’s own run time, and an optimized compiler shortens it by about a third. Building takes longer, so the local default stays dev, which rebuilds faster after an edit. For a local full-suite run, cargo nextest run --locked --workspace --all-features --cargo-profile ci uses CI’s build; its artifacts live in target/ci/. The doctests and proxy_tests jobs run few tests per build and stay on dev. The nightly workflow also stays on dev, the profile cargo test uses locally, so the full suite still runs weekly with an unoptimized compiler.

Each shard compiles the tests itself. A shared build job with cargo nextest archive would compile once, but the shards would then wait for that job’s setup, compilation and upload, so it saves runner time, not wall time.

Jobs that run tests install mold from Ubuntu’s package and fail unless cc links with /usr/libexec/mold/ld (scripts/ci_check_mold.sh); see Running tests for why. A weekly workflow, .github/workflows/nightly.yml, runs the full all-features suite with cargo test on nightly Rust, also on demand through workflow_dispatch.

Dependency cache

Third-party Rust dependencies are cached separately for each job. The key includes the compiler, lockfile, manifests, Cargo configuration and compiler environment. Only successful main runs save caches, and only the first shard saves the shards’ shared dependency cache; PRs restore the base branch’s caches without filling storage with per-PR copies. Workspace artifacts and incremental state are not retained. The workspace profiles also keep compilation outputs small. The nightly workflow keeps no cache: each nightly update invalidates compiled dependencies.

Optional-engine jobs keep compiler dependencies in target/ and engine dependencies in plenty-dataframe/target/ or plenty-linalg/target/. These cache paths must not overlap: the cache action cleans each target with its workspace’s dependency list, so cleaning a parent target would discard nested engine-only dependencies. The engine build receives the matching explicit --target-dir; its installed library still goes to target/engine/. The baseline-isolated-v2 engine cache key starts a new cache population; the first run is cold, and hosted timings must confirm reuse on later runs.

Embedded-runtime Rust dependencies

The test shards also retain the 27 vendored Rust rlibs that build.rs compiles for both embedded runtimes. They are independent of runtime source, Cargo profile and allocation-check mode. A separate cache outside target/ survives workspace-artifact cleanup. Only a successful main run’s first shard saves it; both architectures have separate entries.

The build script validates a SHA-256 key over the vendored source tree, build logic, lockfile, target, Rust compiler version and target standard-library contents. It checks each restored rlib’s content before reuse and publishes complete entries atomically. Native C archives and the two runtime archives still rebuild: native headers/SDKs and runtime source need different inputs. Custom PLENTY_RUNTIME_RUSTFLAGS bypass reuse because flags may name extra files that the key cannot infer. Unreadable or symlinked key inputs also bypass reuse; they do not prevent an otherwise valid compiler build.

Locally, entries live under the Cargo profile’s plenty-runtime-vendor-cache/ directory and survive runtime-source and feature-only rebuilds. PLENTY_RUNTIME_VENDOR_CACHE=/absolute/path shares entries across profiles or isolated targets; PLENTY_RUNTIME_VENDOR_CACHE=off disables reuse. Delete the directory to reclaim entries after a dependency or toolchain change. Cache status and dependency-stage duration appear in verbose Cargo build output; saved Cargo timings still report the whole build script.

A local 2026-11-06 probe reduced this dependency stage from 18.95 seconds to 0.46 seconds, removing all 27 vendored rustc invocations on a hit. Whole build-script samples were 82.16 seconds before caching, 82.23 seconds with a cold cache and 108.21 seconds with a warm cache: native/runtime work varied on the shared machine. Those samples establish a dependency-stage saving, not an end-to-end speedup. Hosted restore/save overhead and CI savings remain to be measured. The native and runtime archive stages remain outside this cache.

Compiled-function cache

Each shard also keeps the function entries of the compiled-function cache, target/plenty-cache/fn, in its own GitHub Actions cache. The dependency cache deletes that directory, so it has its own restore and save steps. Their keys depend on the compiler and Cranelift versions and the compiled IR, so entries stay valid across commits. Program entries, target/plenty-cache/program, key on the compiler binary’s modification time, never hit in a new CI build, and are not kept.

The cache needs no maintenance. A cache hit only reads its entry, so scripts/ci_fn_cache.sh mark gives every restored entry an old access time, and the next read updates it. After a successful main run, scripts/ci_fn_cache.sh prune deletes the entries the run did not read or write. The saved cache is therefore the shard’s working set for that commit: it grows only with the suite, and entries for replaced code generation leave it at the next save. The cache key ends with a digest of the remaining entry names, so an unchanged working set is not saved again. A save deletes the cache it replaces, so storage holds about one cache per shard. If reads do not update access times on the runner, mark warns and nothing is pruned or saved. GitHub removes a cache unused for seven days; the next run is then cold, as is the first run after the shard count changes.

Timings

The Cargo workflow ends with a Check timing report job, also after failed checks. Its job summary shows jobs ranked by duration and a table of step names aggregated across shards and architectures, with counts, seconds, status and share of measured step time. The check-timings artifact retains the Markdown, normalized JSON and raw Actions timestamps for 30 days. The reporting job itself is excluded. Cancellation can prevent a report from being produced.

The same small helper can report on an existing run of any workflow. It reads job/step timestamps through an authenticated gh CLI, including all pages and only the latest run attempt; it does not download logs or rerun checks:

python3 -B scripts/check_timings.py ci --repo OWNER/REPO --run RUN_ID
python3 -B scripts/check_timings.py report target/check-timings/ci-CURRENT/timings.json --baseline target/check-timings/ci-EARLIER/timings.json

Use the output directories printed by the commands. Comparisons show the absolute delta for every step name; a changed number of steps is marked rather than treated as a slowdown. Job and step status remains visible when checks fail. A report collected during a run is partial; unfinished jobs have no completed duration. Raw timestamps remain in jobs.json and run metadata in run.json for further inspection.

Elapsed span (first job start to last job finish) and summed runner-step time are separate: jobs run concurrently, and runner time cannot be summed into workflow latency. Initial queue time is outside the span. Step timestamps have second resolution and do not attribute scheduling gaps between steps. Compare the same workflow, architectures, shard counts and cache conditions; cache restore/save and setup/post-action costs are included as their own steps. These reports locate growing process steps; existing JUnit and Cargo artifacts provide more detail when needed.

Test compilation is a separate step from execution. The cargo-timings-* artifacts retain Cargo’s HTML compilation reports for seven days, including reports available after a failed run: cargo-timings-tests from the first shard, cargo-timings-build from the default-feature build, cargo-timings-proxy and cargo-timings-nightly.

Documentation caches only the pinned mdBook executable, keyed by its version, Rust version, runner OS, architecture and Ubuntu image. Every run checks its version before building. Cache hits skip both Rust setup and cargo install. Superseded PR runs are cancelled; main documentation deployments are not interrupted. Release packaging keeps its isolated, pinned build environment.

The first run for a compiler or cache key remains cold. Compare the restore, compile, test and cache-save step durations on hosted runs before attributing a speedup; cached dependencies do not eliminate compiling Plenty itself.

GitHub source highlighting

Plenty is its own language. Until GitHub supports it through Linguist, the root .gitattributes selects Python highlighting for every .plenty file, including nested modules. This is approximate coloring: Plenty-specific syntax can be miscolored, and a file viewer may label the file as Python. The linguist-detectable=false attribute excludes these files from repository language statistics so they do not count as Python; genuine Python files keep their normal classification. The files remain visible in diffs.

This override applies only to repositories containing the rule. It does not provide editor integration or guarantee GitHub code navigation or search classification. Keep Markdown’s plenty fences and the book’s custom highlighter; they are independent of GitHub’s source viewer. Once native Plenty support reaches GitHub, remove both temporary attributes and recheck source highlighting and language statistics.

Reverse-proxy tests

tests/test_std_http_server_proxy.rs runs std.http.server behind nginx and Caddy (see reverse proxies). They need Docker and curl, so they run only when asked:

PLENTY_DOCKER_TESTS=1 cargo test --locked --test integration test_std_http_server_proxy::

Without the variable each test prints that it is skipped and passes, so the ordinary suite needs no Docker and does not slow down. With it, a missing Docker daemon or curl fails the tests. The first run pulls the two pinned images (about 70 MB each). Each test starts its own containers with --network host on free loopback ports, names them plenty-proxy-test-<pid>-<n> with the label plenty-test=proxy, and removes them when it ends, also on failure; nothing else is stopped or pruned. A test process killed outright leaves its containers running; list them with docker ps --filter label=plenty-test=proxy.

CI runs them in the proxy_tests job of .github/workflows/ci.yml on GitHub’s ubuntu-latest, which has Docker and curl. The job pulls nginx:1.30.6-alpine3.24 and caddy:2.11.7-alpine in an earlier step, so pulls are not timed with the tests; when the tests change an image tag, change the job’s pull step to match.

Documentation site deployment

The public book defaults to development until the first published release is admitted in book/releases.json. After that, the root serves the selected stable release. Development documentation always follows main; admitted releases live under /plenty/VERSION/. Every page identifies its version and offers a version menu. Matching chapters are preserved when switching versions; missing chapters fall back to that version’s homepage. The links work without JavaScript; JavaScript also preserves the current query and heading fragment when the chapter exists.

.github/workflows/docs.yml builds every push to main, PR targeting main (including forks), and manual dispatch. Only a main push or main dispatch can deploy. It does not publish packages or push branches. Release admission and retention are described in versioned documentation.

See the development workflow for pinned tools and local preview commands.

For only the current checkout, python3 -B scripts/build_book.py writes book/book. The complete site helper writes book/site. Both show Development and its source commit, explicitly marking local changes. Each version’s docs-version.json records the full revision and Cargo package version; the package version alone does not identify a development build. Labels are static HTML. Raw mdbook serve previews content without deployment metadata. To preview the complete site, mount book/site at /plenty/ on a static HTTP server; configure unknown paths to serve its root 404.html as GitHub does.

The build needs no Plenty compiler. Do not use mdbook test for Plenty fences; the tutorial integration tests execute those examples. The output checker verifies rendered local links stay inside the artifact and reach existing files, and checks search, print, 404 and custom highlighting assets. Source checking alone cannot catch a malformed SUMMARY.md that leaves a chapter unrendered. Neither checker validates external URLs or heading anchors.

Book links must target published pages or assets. A relative link to a repository file outside book/src is rejected by the source checker even when it exists, because mdBook does not publish that file. Refer to such files with a repository-relative code path (for example, experiments/rust-crates/README.md), or include the needed content in a book page. Run the complete site helper after adding links; a successful mdbook build alone does not validate generated links, including those in print.html.

The Pages artifact is only book/site, which remains ignored. It contains all admitted versions together, so development updates preserve older books. Each build receives its own site-url prefix for nested 404 assets. Check navigation, anchors, search, print, version switching and Plenty highlighting under each prefix; repeat those checks on the public site after deployment.

First publication and recovery

After review, merge and authorization to publish, an administrator must:

  1. Check account/repository visibility and organization policies allow Pages and the pinned actions. Choose Settings > Pages > Build and deployment > Source: GitHub Actions. The workflow does not enable Pages automatically.
  2. Set the github-pages environment’s deployment branch policy to main only, preserving any required reviewers.
  3. Run Documentation from current main, approve its environment if required, and inspect the deployed URL and source commit in the run summary.

Build jobs have only contents: read; PRs do not configure/upload Pages or use the deployment environment. The deploy job has pages: write, id-token: write and contents: read for a main-tip API check. It uses the built-in token, with no PAT. Actions are pinned by SHA; the upload action also pins its nested action.

Deployment jobs share one concurrency group and do not cancel a running deployment. After acquiring that slot and any environment approval, a run checks the live main SHA and skips if superseded. Queue ordering is not guaranteed: rerun current main if its pending deployment was replaced by an older run that then skipped. A commit arriving after the check can briefly leave the previous revision live until the next successful run; serialization prevents an older deployment finishing after a newer deployment. API failures fail closed.

Failed checks or deployments leave the last successful site available. Rerun current main for transient failures; fix or revert bad content through review and deploy the resulting main commit. Do not roll back by dispatching an old branch or deleting the site. Tool/action upgrades require another complete book build and theme/navigation check. See the official Pages workflow guide and mdBook HTML configuration.

Linux AArch64 development

The source tree implements native aarch64-unknown-linux-gnu alongside x86_64-unknown-linux-gnu, and releases publish its compiler archive, with both engines, built and tested on native ARM runners as described in releasing. This page describes the emulation harness used to develop the port and the evidence it recorded. That evidence does not establish minimum kernel versions, page sizes other than 4 KiB, or weak-memory behavior on other ARM hardware. Platform support describes the implemented contract; future work remains in B18.

Backend and C ABI probe

Install QEMU user-mode tools and Python 3, with Cargo’s locked dependencies available locally. scripts/aarch64_probe.py provisions a checksum-pinned Bootlin GCC/glibc toolchain, generates ARM ELF with the pinned Cranelift backend, links a C fixture and executes it on an emulated Cortex-A53:

ARM_TOOLS="$PWD/target/aarch64-toolchain"
python3 scripts/aarch64_probe.py --provision "$ARM_TOOLS"
cargo fetch --locked --manifest-path scripts/aarch64-probe/Cargo.toml
CARGO_TARGET_DIR="$PWD/target/aarch64-probe-build" \
    python3 scripts/aarch64_probe.py --toolchain "$ARM_TOOLS" \
    --out "$PWD/target/aarch64-probe-output"

Provisioning refuses to replace an existing directory. The archive is aarch64--glibc--stable-2025.08-1.tar.xz, SHA-256 dfb47eee874eef9e8a7fc042eee4e0a183f444b6bcde6a82fef8f009918389c9. The probe prints tool versions and ELF relocations. It exercises native C calls, i64 values, u128 arguments/results, direct and indirect Tail calls with spilled arguments, a large inline-probed frame, data addresses and a recorded UDF trap. The expected trap runs in a child process. Keep the tool/version output with qualification results; the toolchain’s libc is a development input, not evidence for the release’s glibc 2.28 floor. This Bootlin GCC also injects 4096-byte maximum/common ELF page sizes through its linker specs. Its linked binaries qualify the 4 KiB test environment only. Use native release tools and test the resulting ELF layout before claiming support for larger kernel page sizes; do not carry those overrides into releases.

Build the ARM compiler on x86

Cross-building the Rust compiler is a development facility. It does not add a user-facing Plenty cross-compilation mode. Install the matching Rust target and supply both Rust and C build tools:

rustup target add aarch64-unknown-linux-gnu
export CC_aarch64_unknown_linux_gnu="$ARM_TOOLS/bin/aarch64-buildroot-linux-gnu-gcc"
export AR_aarch64_unknown_linux_gnu="$ARM_TOOLS/bin/aarch64-buildroot-linux-gnu-ar"
export CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER="$CC_aarch64_unknown_linux_gnu"
cargo build --locked --target aarch64-unknown-linux-gnu --bins
qemu-aarch64 -cpu cortex-a53 \
    -L "$ARM_TOOLS/aarch64-buildroot-linux-gnu/sysroot" \
    target/aarch64-unknown-linux-gnu/debug/plenty --print-target

Native build helpers resolve CC_TARGET, CC_target_with_underscores, TARGET_CC, then CC (and the equivalent AR variables). Without an override, cross builds choose the architecture-prefixed GNU tools, never host cc. Both embedded runtime archives, native dependencies and test/fuzz helper objects must match the selected Cargo target. Use a disk-backed CARGO_TARGET_DIR with adequate space; the compiler and runtime are large enough to exhaust a small tmpfs. Limit CARGO_BUILD_JOBS when sharing a machine with other builds.

A Cargo runner wraps the Rust test process only. Plenty integration tests also spawn a compiler, linker and generated programs, so setting a QEMU runner alone is insufficient. Use an explicit subprocess harness or run inside an ARM guest. Keep task scratch files outside the sysroot. In this harness, a task-owned sysroot/tmp directory interfered with QEMU path handling: three SQLite/KV cases failed before removing it and passed afterward with unchanged binaries. Preserve the original results and test artifacts before correcting harness inputs.

Full ARM Linux guest

A full-system guest exercises an ARM kernel’s signal contexts, filesystem flags and stack behavior. The initial environment used QEMU 10.2.2, Cortex-A53 with TCG, two virtual CPUs and 2 GiB RAM, and Debian 13 genericcloud ARM64, kernel 6.12.113 with 4 KiB pages, glibc 2.41 and native GCC 14.2. Its base image was downloaded from Debian cloud images and verified against the official sibling SHA512SUMS:

debian-13-genericcloud-arm64.qcow2
71aa5d5dbcb3d9ecfe96bc3f6dd41b6805a0a41c852204a32747d230b64b95912297fb453b11a6031fd54d84a0a0a430e137d07e6986dc9ebc9d41483be003c4

The latest URL can change: preserve the verified base image and checksum, or obtain the same image from a dated archive. A different checksum is a new qualification environment, not permission to bypass verification. Create a 16 GiB qcow2 overlay with qemu-img create -f qcow2 -F qcow2 -b BASE work.qcow2 16G. Create meta-data containing instance-id: plenty-arm and local-hostname: plenty-arm on separate lines. Create this user-data, replacing the key placeholder with your SSH public key:

#cloud-config
users:
  - name: plenty
    groups: sudo
    shell: /bin/bash
    sudo: "ALL=(ALL) NOPASSWD:ALL"
    ssh_authorized_keys:
      - ssh-ed25519 REPLACE_WITH_YOUR_PUBLIC_KEY
package_update: true
packages: [gcc, libc6-dev, binutils, make, pkg-config, curl, git, python3]

Build the seed with genisoimage -output seed.iso -volid cidata -joliet -rock user-data meta-data. Keep private keys and guest credentials outside the repository.

With AAVMF firmware installed, boot from the directory containing those files:

qemu-system-aarch64 -machine virt -cpu cortex-a53 -accel tcg -smp 2 -m 2048 \
    -bios /usr/share/AAVMF/AAVMF_CODE.fd \
    -drive file=work.qcow2,if=virtio,format=qcow2 \
    -drive file=seed.iso,if=virtio,format=raw,readonly=on \
    -netdev user,id=net0,hostfwd=tcp:127.0.0.1:22264-:22 \
    -device virtio-net-pci,netdev=net0 -display none \
    -serial file:console.log -monitor none -daemonize
ssh -p 22264 plenty@127.0.0.1

Copy the exact source revision and matching ARM binaries/test executables into the guest, preserving executable permissions. Tests that compile C fixtures need the guest’s native cc, and tests that launch plenty need the matching compiler at the path captured when the test executable was built. Record uname -a, getconf PAGESIZE, ldd --version, cc --version, Rust/QEMU versions, compiler revision and feature flags with the results. A native build inside the guest avoids captured host paths; it needs Rust and sufficient disk/RAM as well.

Qualification boundaries

The isolated C/backend probe and all-feature runtime suites passed under QEMU. Compiler workflow qualification combines a broad all-feature user-mode run with exact tests in the full ARM guest. Guest execution covers checked traps and stack overflow, signal-host coexistence, actor ownership, native process and environment contracts, relocated/offline toolchains, persistent SQLite/KV and required fuzz tests. The guest uses native tools where user-mode subprocesses would otherwise find x86 tools or lose their ARM loader settings.

Raw user-mode failures are retained rather than reported as a clean run. Exact guest reruns preserve the assertions, including missing-program errors and fatal signal output. A tutorial allocation imbalance observed in user mode did not reproduce in the complete unchanged guest shard; its cause remains unproven. An earlier integration harness itself exited on SIGILL without a core, before several qualification fixes. Focused reruns did not reproduce it, and local issue #592 records the missing original binary and diagnostic evidence without attributing a cause. Use the ARM fault capture procedure to preserve a recurrence before rebuilding. Local PR #706 records revisions, totals and reconciled results; its retained artifacts are now under the primary worktree’s ignored target/portability-pr706/worker-qualification/ and target/portability-pr706/aarch64-vm/evidence/ directories.

Seven tests are explicitly ignored: the large specialization-budget resource case, the external SQLite-vec extension trial, the systemd/playpen memory-limit case, and four runtime timing/scaling benchmarks. Additionally, 13 dataframe real-engine/tutorial cases and eight Docker proxy cases return early without their opt-ins. Their libtest ok entries do not establish execution coverage. Dataframe all-target compilation passed; full dataframe execution remains unrun locally. FFI tests ran without optional Valgrind instrumentation.

The required nine libFuzzer tests passed in the guest with native clang and its matching runtime/interceptor archives. The supported engine-build helper’s SONAME-bearing linalg package passed all six enabled integration tests and the optional tutorial test. That local package requires GLIBC 2.39; it does not qualify the release floor. The release configuration retains glibc 2.28 with GNU ld for ARM, avoiding the pinned ARM mold binary’s glibc 2.30 requirement.

The emulated Cortex-A53 advertises AES, PMULL, SHA1, SHA2 and CRC32, but no LSE. Known-answer crypto and compression checks also passed with a temporary getauxval interposer reporting only mandatory FP/ASIMD features. This exercises runtime fallback dispatch; it does not emulate the physical absence of crypto instructions or change the compiler’s generic ARMv8-A baseline.

User-mode QEMU cannot establish kernel signal coexistence or ARM’s real weak-memory concurrency behavior by itself. Full-system TCG adds kernel coverage but is still emulation. Native hardware stress, additional page sizes and release-baseline clean-container execution must each be recorded before claiming them as supported. Native ARM CI is configured for the compiler/runtime suite, both optional engines and the Docker reverse-proxy tests. Run 41210234842 on 2026-10-31 passed all of those jobs at revision 4c07ea01, including all four ARM integration shards, default-feature build/doctests and strict lint checks. These native results supplement the emulation evidence above; they do not establish minimum deployment versions, additional page sizes or release-package compatibility.

For the complete developer gate, run the locked workspace suite with all features, default-feature tests, formatting, clippy and tutorial/reference examples, as described in development workflow. Run cold, warm and cache-off compilation, C/library/Tail tests, checked traps and stack overflow, actors, filesystem no-follow cases, crypto/compression known answers, packages, runtime extraction, LSP and relocated distributions. Record missing services, libFuzzer/sanitizer tools and ignored tests explicitly. Release packaging and native runner gates are described in releasing.

ARM fault diagnostics

Use this procedure when the ARM integration harness itself exits on a signal. The Linux AArch64 guide describes the execution environment and qualification limits. Local issue #592 records a historical SIGILL whose exact executable and core were not retained; a successful later run cannot identify its cause. Do not infer the faulting test from the last printed ok: parallel tests can still be running when that line appears.

Preserve the run before rebuilding

Stop other writers to the task’s build and cache directories. Create a fresh, absolute evidence directory outside the sysroot and build output. Preserve:

  • The exact integration ELF, matching plenty/plentyc, runtime archives, optional engines, and split debug files (.dwo). Copy bytes, not hard links to artifacts a build can overwrite. Record SHA-256 hashes and ELF build IDs.
  • The source commit, Cargo.lock, feature flags, build command, tracked changes (git diff --binary HEAD) and any untracked source inputs. A revision alone does not describe a dirty build. Keep the source checkout available: tests can embed its absolute paths and read fixtures from it.
  • The complete toolchain/sysroot and tool-wrapper scripts, including symlink targets, plus QEMU and debugger versions. Retain the shared libraries reported by the debugger. A hash without the corresponding file is not a preserved artifact.
  • Cache contents before execution, and a separate working copy for the rerun. Keep the post-run working copy too. Do not clear or reuse the original cache.
  • The exact command, working directory, CPU affinity, test threads, environment overrides, inherited core limits and raw stdout/stderr. Record opt-ins and skipped tests; a passing count alone does not establish coverage.

Before a rerun, verify the ELF’s --list output, captured source/compiler paths, subprocess execution and linker setup. An explicit QEMU command wraps only the initial process: it does not make nested ARM executables runnable. Preserve existing binfmt or guest configuration as part of the run description. If that configuration is gone, restoring it is a new experiment, not an exact replay.

Capture the guest fault with GDB

This method captures guest registers and memory without changing host-wide kernel.core_pattern or relying on the shell’s core limit. It was checked with QEMU 10.2.2 and an AArch64-capable GDB 18.1. The GDB stub changes timing, so retain ordinary-run evidence as well. Use a short, task-owned Unix socket path; local sandbox policies may require permission to create/connect to that socket.

Set these absolute paths for the preserved run (the values are examples):

ARM_EVIDENCE=/tmp/plenty-arm-fault-001
ARM_ELF="$ARM_EVIDENCE/integration"
ARM_SYSROOT="$ARM_EVIDENCE/toolchain/aarch64-buildroot-linux-gnu/sysroot"

From the original test working directory, launch the saved ELF with the original environment, affinity and test arguments, adding QEMU’s -g option. For a four-thread Cortex-A53 run the command has this form:

qemu-aarch64 -cpu cortex-a53 -L "$ARM_SYSROOT" \
    -g "$ARM_EVIDENCE/gdb.sock" "$ARM_ELF" \
    --test-threads=4 --nocapture > "$ARM_EVIDENCE/run.log" 2>&1

It waits for a debugger. In another terminal, open the same ELF with an AArch64-capable gdb (some distributions call it gdb-multiarch):

gdb -q -iex 'set debuginfod enabled off' "$ARM_ELF"

Enter these GDB commands, substituting the absolute evidence paths. GDB command files do not expand shell variables such as $ARM_EVIDENCE.

set pagination off
set sysroot /tmp/plenty-arm-fault-001/toolchain/aarch64-buildroot-linux-gnu/sysroot
set logging file /tmp/plenty-arm-fault-001/gdb.txt
set logging overwrite off
set logging enabled on
target remote /tmp/plenty-arm-fault-001/gdb.sock
handle SIGILL stop print pass
continue

When the debugger reports SIGILL, capture before continuing:

info registers
x/8i $pc
x/32bx $pc
thread apply all bt full
info sharedlibrary
info proc mappings
generate-core-file /tmp/plenty-arm-fault-001/guest.core

Some QEMU/GDB combinations do not expose process mappings or core generation. Retain their errors along with the register, instruction and backtrace output; do not claim a core was captured unless it exists and can be reopened. Even a readable core can lack mappings or debugging information. For a generated core, check it independently against the saved ELF:

gdb -q -iex 'set debuginfod enabled off' "$ARM_ELF" "$ARM_EVIDENCE/guest.core"

Compare info registers pc sp and x/i $pc with the live capture. Preserve the loaded objects and sysroot alongside the core. After capture, continue delivers the stopped signal normally; Plenty uses intentional illegal instructions for checked traps, so a SIGILL stop alone does not prove a defect. Record whether it is handled or terminates the process. The stub observes the attached process; a subprocess’s failure needs its own capture.

Interpret and minimize

Identify the instruction and containing ELF/object before attributing a cause. Compare with the compiler’s registered trap sites and the runtime signal handler. Keep the first captured failure, then reduce test filters and thread counts in separate runs with copied inputs. Compare cache-off, fresh-cache and preserved cache runs explicitly. Debugger timing or a serial pass does not disprove a race.

Reconcile the failing revision with intervening compiler/runtime fixes. Compare the minimized case in user-mode QEMU, the ARM Linux guest and native ARM when available, preserving the same assertions. Record non-reproduction and missing environments honestly; diagnostic improvements do not resolve an unexplained historical crash.

Documentation versions

The documentation site is assembled from an explicit manifest, book/releases.json, and the current checkout. There are no published versions in the initial manifest: a Git tag alone does not establish a release.

PathDocumentation
/plenty/Selected stable release, or development while stable is null
/plenty/dev/Current main
/plenty/VERSION/Admitted release from its pinned source commit

Development pages show Development, their source commit, and any local changes. Released pages show Plenty VERSION. Each book includes a version menu and docs-version.json; versions.json at the site root lists the entire catalog. Selecting another version retains the chapter if it exists there, otherwise opens the homepage. A heading may not exist in an older chapter; the browser still opens that chapter even when the fragment has no target.

Redirect pages declared in [output.html.redirect] in book/book.toml are excluded from version-banner decoration; their destination pages carry the version label. Content pages must still have the expected mdBook layout.

Admit a published release

After a stable GitHub Release is published successfully:

  1. Verify the published tag and record git rev-parse 'vX.Y.Z^{commit}'. Do not admit drafts, prereleases, or a tag whose release job failed.
  2. Add a manifest entry with version, tag, and the full commit hash. For example, the structure is { "version": "0.2.0", "tag": "v0.2.0", "commit": "FULL_40_CHARACTER_COMMIT_HASH" }. Set stable to the version that should be the default; never infer it from main’s Cargo version.
  3. Keep all previously admitted entries. Archive URLs are permanent; do not rewrite tag/commit pins. The helper rejects moved tags, version mismatches, duplicates and unsafe version names. Removing an entry would remove its deployed archive, so retention changes require an explicit review decision.
  4. Run python3 -B scripts/build_docs_site.py. It uses the manifest’s exact mdBook version, currently 0.5.4. When changing that pin, also change the workflow installation pin and validate every archive with the new renderer.
  5. Check version switching, search, custom highlighting, print, nested 404 handling, and existing root links. Submit the manifest change through the normal local PR workflow. Its eventual push to main rebuilds the site; publishing a release by itself does not change the documentation default.

Historical books use their own prose, configuration, highlighting and included files from the pinned tree. The renderer and navigation layer come from the current site builder. Thus source revisions stay fixed, while presentation can receive fixes. No old compiler or repository script runs during extraction; mdBook may execute preprocessors declared in a historical book configuration, so review such additions when admitting releases.

The v0.1.1 tag predates a table-of-contents indentation fix. If that version is eventually published and admitted, its entry needs summary_indent_repairs with tutorial/11-concurrency/stop-on-signal.md and tutorial/11-concurrency/bounded-io.md. This explicitly recorded, rendering-only repair restores omitted chapters; it does not replace historical prose. The build fails if the repair no longer matches. Metadata records applied repairs.

The helper builds all books in temporary directories and checks their HTML links and assets before replacing book/site. Includes can refer outside book/, so releases are extracted from their complete tracked tree. Only generated books enter the artifact; no source trees or issues database are uploaded. Logged mdBook errors are fatal even if the renderer exits successfully.

The first stable admission deliberately changes the root’s content from development to stable. Existing chapter paths found in stable continue to work. Development-only paths get small forwarding pages to /plenty/dev/, preserving queries and fragments with JavaScript and offering an ordinary link without it. The builder creates these for development pages present at build time; it is not a historical redirect registry for subsequently deleted or renamed chapters.

GitHub supplies the site’s root 404 page for missing URLs in any version. JavaScript forwards known version prefixes to their corresponding 404 page; without JavaScript the root 404 and version menu remain usable. No server-side rewrite rules, new domain settings, token, or extra Pages environment are needed.

Local admission experiments can use --manifest /tmp/releases.json and --output /tmp/docs-preview without changing the reviewed manifest. The output directory must not exist, or must contain a previous builder’s versions.json. Follow the deployment recovery procedure for failures. No archive is retained solely in short-lived Actions artifacts: each deployment rebuilds the complete admitted catalog from Git.

Local semantic search trial

examples/semantic_search.plenty is a native Plenty CLI for a separate SQLite search database. It uses sqlite-vec for exact cosine-distance search and built-in FTS5 for keyword search, combining their top 30 candidates with reciprocal rank fusion and returning up to ten chunks. All parsing, validation, hashing, database operations, transactions and ranking are driven by Plenty. Only model inference runs in Python, through scripts/search_embed.py and FastEmbed. Plenty owns the CLI, chunking and document lifecycle.

For the repository corpus, use the setup and synchronization wrapper, which scans documentation, source docs and the local issue register and removes deleted documents. This page covers the underlying interface and individual document operations. No generated database or model is committed.

Setup

Use the compiler built from this checkout and the pinned sqlite-vec 0.1.9 shared library for x86-64 Linux. One way to obtain the extension is its Python wheel. Install FastEmbed in the same environment:

uv venv /tmp/plenty-search-env
uv pip install --python /tmp/plenty-search-env/bin/python sqlite-vec==0.1.9 fastembed==0.9.0
export PLENTY_SQLITE_VEC="$(/tmp/plenty-search-env/bin/python -c 'import sqlite_vec; print(sqlite_vec.loadable_path())')"
cargo run --locked --bin plenty -- --compile examples/semantic_search.plenty -o /tmp/plenty-search

The executable loads that library through Connection.load_extension. Use only a trusted extension. SQLite’s SQL loading function remains disabled. Storage-only commands do not need a Python process.

Search document text

Run these commands from the checkout root. Setup downloads the pinned BAAI/bge-small-en-v1.5 quantized ONNX model (about 67 MB); inference thereafter uses only the local cache and CPU. The helper pins the model artifact revision, uses 384-dimensional vectors, and refuses input beyond its 512-token limit.

export SEARCH_PYTHON=/tmp/plenty-search-env/bin/python
export SEARCH_CACHE=/tmp/plenty-search-model-cache
"$SEARCH_PYTHON" scripts/search_embed.py --setup "$SEARCH_CACHE"
export SEARCH_MODEL="$("$SEARCH_PYTHON" scripts/search_embed.py --identity)"
/tmp/plenty-search "$PLENTY_SQLITE_VEC" /tmp/search.db "$SEARCH_MODEL" init 384
/tmp/plenty-search "$PLENTY_SQLITE_VEC" /tmp/search.db "$SEARCH_MODEL" index docs:sqlite SQLite book/src/design/stdlib/sqlite.md "$SEARCH_PYTHON" scripts/search_embed.py "$SEARCH_CACHE"
/tmp/plenty-search "$PLENTY_SQLITE_VEC" /tmp/search.db "$SEARCH_MODEL" find "How do I undo changes when a transaction fails?" "$SEARCH_PYTHON" scripts/search_embed.py "$SEARCH_CACHE"

index ID TITLE TEXT_FILE PYTHON SCRIPT CACHE reads one UTF-8 text document. Plenty hashes its title and text before inference; unchanged input returns immediately. Changed documents are split into 400-character windows with 64-character overlap, then embedded in one subprocess invocation. This simple trial chunker can split sentences and Markdown structures; corpus-aware chunking remains follow-on work. Empty files keep a document with zero chunks. All replacement happens in one transaction after successful inference; a failed subprocess, invalid response or wrong vector dimension leaves the old document.

find QUERY PYTHON SCRIPT CACHE embeds the query and runs hybrid search. The process receives JSON text and returns JSON vectors; it never opens a database or reads source files. There is no shell interpolation of paths or queries. The helper reports errors on stderr, which Plenty preserves in its error. Each indexing/query invocation loads the model anew, trading latency for a simple setup without a background service.

Use show ID, status and delete ID with the same extension, database and model arguments. Reindex after editing a file; explicitly delete its stable ID after deleting it. Nothing modifies the original source or issues.db.

Vector input and document operations

Every invocation starts with EXTENSION DB MODEL. Use a new database path, separate from the authoritative issues.db. MODEL identifies the exact model, revision, normalization and chunking configuration; changing it requires a new database. The CLI rejects a different identity or initialization dimension.

The following example uses synthetic two-dimensional vectors to test storage; they are not embeddings of the example text.

/tmp/plenty-search "$PLENTY_SQLITE_VEC" /tmp/search-trial.db fixture-v1 init 2

Save this JSON as /tmp/search-chunks.json:

[
  {"text": "Owned resources are released on scope exit.", "embedding": [1.0, 0.0]},
  {"text": "Borrows cannot outlive their owners.", "embedding": [0.9, 0.1]}
]
/tmp/plenty-search "$PLENTY_SQLITE_VEC" /tmp/search-trial.db fixture-v1 put docs:ownership "Ownership" /tmp/search-chunks.json
/tmp/plenty-search "$PLENTY_SQLITE_VEC" /tmp/search-trial.db fixture-v1 show docs:ownership
/tmp/plenty-search "$PLENTY_SQLITE_VEC" /tmp/search-trial.db fixture-v1 status

put replaces all chunks belonging to that document in one transaction. Its hash covers the title, chunk text and supplied vectors. Identical input prints unchanged. Parsing happens before the transaction; if insertion fails (for example, a vector has the wrong dimension), the previous document remains. An empty chunk array keeps the document with no searchable chunks.

Save [1.0, 0.0] as /tmp/search-query.json, then query:

/tmp/plenty-search "$PLENTY_SQLITE_VEC" /tmp/search-trial.db fixture-v1 query /tmp/search-query.json "resource cleanup"
/tmp/plenty-search "$PLENTY_SQLITE_VEC" /tmp/search-trial.db fixture-v1 delete docs:ownership

Query words are quoted individually and joined with OR for FTS5. Empty words give vector-only results. Results show document ID/title, zero-based chunk number, fused score and text. Vectors must come from the same model and preprocessing as the stored documents; the CLI cannot infer this from numbers. Supply finite, nonzero vectors suitable for cosine similarity.

delete removes the document, its chunks and embedded vectors through a foreign key cascade; triggers remove the associated FTS5 entries. It never deletes a source file. The trial uses ordinary SQLite BLOB columns and sqlite-vec distance functions, so vector ownership follows normal relational deletion.

Validation

The deterministic integration test uses the actual extension and fake inference and checks insertion, unchanged input, search, rollback on malformed dimensions, replacement, deletion, Unicode chunking, skipped inference and subprocess failure preserving the previous document:

PLENTY_SQLITE_VEC="$PLENTY_SQLITE_VEC" cargo test --locked --test integration test_std_sqlite::sqlite_vec_search_trial -- --ignored

The test is explicitly ignored in the default suite because the extension is an optional external dependency. The ordinary SQLite tests exercise extension loading with a locally compiled fixture and check FTS5 without any download.

Search the repository

The local semantic index covers tracked Markdown documentation, source documentation, and all local issue/PR titles and descriptions. It is a separate, rebuildable database; issues.db remains authoritative. The CLI, extraction, chunking, invalidation and queries run in Plenty. Python/FastEmbed performs only embedding inference. See the search interface for the model identity, storage schema and individual-document commands.

Setup and use

From a checkout, install the pinned dependencies/model and build the executable:

bash scripts/search.sh setup
bash scripts/search.sh sync
bash scripts/search.sh find 'How do borrowed values interact with worker threads?'
bash scripts/search.sh status
bash scripts/search.sh check
bash scripts/search.sh show repo:doc:book/src/design/stdlib/sqlite.md

Setup needs uv, Cargo and a C linker, plus network access for the initial packages and model. Subsequent indexing/search uses the local model. build rebuilds just the executable after changing the Plenty source. The wrapper uses .search/ in the primary worktree for search.db, semantic-search, venv/ and models/. These generated files are ignored by Git. Model weights and the environment are never part of an index snapshot.

The default corpus is the primary checkout, even when running the wrapper from a task worktree. Set PLENTY_SEARCH_ROOT to index another checkout, or PLENTY_SEARCH_HOME to use a separate index/environment. The issues register must always be the primary worktree’s absolute issues.db path; the Plenty program validates it against git worktree list.

Coverage and lifecycle

check is a deterministic freshness audit. It recomputes SHA-256 hashes of each document’s title and extracted text, compares them with the hashes stored in documents, and lists added, changed and deleted IDs. It opens the search database read-only, never invokes inference and does not modify embeddings. Exit status is zero when fresh and nonzero when stale or on an operational error (the diagnostic distinguishes these). This lets scripts/agents check freshness without relying on a model’s judgment. Only indexed text matters: changing implementation code without changing extracted documentation need not invalidate its embeddings. A rename is a deletion plus an addition.

  • repo:doc:PATH: tracked Markdown, including the book and root/project docs. Vendor trees and the duplicate BACKLOG.md alias are excluded.
  • repo:source:PATH: Rust //////! documentation with declaration context; Python and Plenty module/function/class triple-quoted docstrings; and Plenty module headers and comment blocks preceding declarations. Source is grouped by file so its embeddings can be replaced/deleted together. The lightweight lexical extractor skips strings/test fixtures; it is not a general AST parser. Rust block doc comments and #[doc = ...] attributes are not extracted.
  • repo:issue:N and repo:pr:N: titles and descriptions from a read-only SQLite snapshot, across all statuses. Discussion comments and PR code/diffs are not read. Code snippets already contained in descriptions remain part of that text.

sync scans Git’s tracked paths (including staged new files and working-copy edits), skips missing tracked files, and reads issues/PRs in one transaction. Untracked files are excluded. It compares each document’s extracted title/text hash before inference, groups changed documents, and embeds at most 128 chunks per subprocess call. Each document replacement is transactional. Successful batches survive an interruption and are reused on the next run.

Only after the whole scan and all embeddings succeed does synchronization remove missing repo: documents. Explicitly indexed IDs outside that prefix are preserved. It then compacts the database. Run one sync at a time. A failed run does not prune missing documents, and any document whose replacement fails retains its previous chunks. sync_info records the source checkout, its HEAD and the last successful completion time; working-copy edits are also indexed, so HEAD alone is not a content fingerprint.

Run sync after document edits/deletions or issue/PR updates. Search results can be stale between runs. The current 400-character chunker can split paragraphs and Markdown structures; search is a discovery aid, so read the original source before relying on a result. A different model/chunker identity requires rebuilding the database.

Tests

After setup/build, run the extraction/synchronization tests against the real SQLite extension and a deterministic fake inference process:

PLENTY_SEARCH_BIN="$PWD/.search/semantic-search" \
PLENTY_SQLITE_VEC="$(.search/venv/bin/python -c 'import sqlite_vec; print(sqlite_vec.loadable_path())')" \
python3 -B scripts/test_search_index.py

The tests cover extraction, skipped inference, changed docs/issues, deletion, failed inference, stale-register rejection, manual document preservation, read-only freshness auditing, foreign keys and FTS cleanup. They need no model download and skip unless both environment variables are set.

Analyzing emitted code

Use this procedure to make generated programs faster: find the loop that dominates a benchmark, compare its machine code with C’s, and account for every extra instruction before changing the compiler. Profiling the compiler itself is covered in Profiling the compiler.

The worked example is local PR #917, whose kernels, scripts and results are in experiments/kernels/ (see its README.md). It took fannkuch-redux from 3.16x to 1.69x C’s instruction count.

Ground rules

  • Measure optimized builds. Builds are optimized by default; never time or count a --fast-compile build, which runs Cranelift at opt_level=none and skips call-site inlining.
  • Count instructions, not time. perf stat -e instructions:u and per-pass counts read from assembly do not depend on machine load. Wall-clock numbers on a shared machine are noise; take them last, alternating builds in one session, and compare only within that session.
  • Pin parallel programs to physical cores (taskset -c 0-7 on the reference laptop), so a busy machine does not change thread placement.
  • Check outputs. Every build in a comparison prints identical output.
  • Prove, never assume. A bounds, overflow or index-normalization check can be removed only when the compiler proves at compile time that it cannot fail. The safety requirement is that no out-of-bounds access or unreported overflow can happen, not that every access runs a check. Checks that can fail keep their trap kind, position and message.

Procedure

1. Find the hot loop

Count the whole program, then attribute by function:

plenty --no-strip --compile app.plenty -o /tmp/app
perf stat -e instructions:u taskset -c 0-7 /tmp/app ARGS
valgrind --tool=callgrind --callgrind-out-file=/tmp/cg.out /tmp/app SMALL_ARGS
callgrind_annotate /tmp/cg.out | head -30

perf record followed by perf annotate shows the hot instructions inside a function. --no-strip keeps the symbols.

2. Isolate kernels

Write each hot loop as a standalone Plenty function and a C function that does the same work with the same types, as experiments/kernels/kernels.plenty and kernels.c do. Keep one loop per function, pass data as views or pointers plus lengths, and give the C version no information Plenty’s lacks (for example, no restrict unless Plenty’s borrows prove the same thing).

3. Get both disassemblies

gcc -O2 -fno-tree-loop-distribute-patterns -c kernels.c -o /tmp/kc.o
objdump -d --no-show-raw-insn /tmp/kc.o > /tmp/c.asm

plenty --dump ops,clif,clif-opt,asm kernels.plenty -o /tmp/k

-fno-tree-loop-distribute-patterns stops gcc replacing copy loops with memmove, which would hide the loop. Use --dump-filter NAME to narrow the dump. See Inspecting generated code for the stages; clif-opt and asm are what runs.

A dump keeps every function body, while a compiled program drops the standalone bodies of functions inlined at every call site. Compare the inlined code in its caller when that matters.

4. Count instructions per pass

experiments/kernels/loops.py reads either disassembly and prints, for each innermost loop, the instructions one pass runs on its hot path. Its README shows how to normalize --dump asm output for it. Record the counts in a table: one row per kernel, one column per build.

5. Attribute every extra instruction

Line up the two hot paths and give each extra Plenty instruction one cause:

CauseTypical shape (x86-64)Usual fix lives in
Bounds checkcmp + jae to a cold blockPlenty: prove the index in range, reuse a dominating check, hoist
Negative-index joinlea of index + len and a second cmpPlenty: prove the index nonnegative
Overflow checkadd/seto/test/jne, or joPlenty: prove no overflow (loop counters, range facts)
Header or length reloadmov from the list owner each passPlenty: row caching across the loop
Register movemov reg, reg with no other effectUsually regalloc2; check whether a value passed to a cold call causes it
Spill or reloadmov to or from the stackregalloc2, or too many live values from Plenty’s IR
Loop layoutunconditional jmp back to the headerPlenty: loop rotation
Call overheadcall of a small helperPlenty: inlining
View or address constructionisplit, base + offset each passPlenty: build once outside the loop
Useful workPresent in C tooNothing

Sum the causes per kernel. A cause that accounts for many instructions across kernels is the next target.

6. Test IR shapes before changing the compiler

Copy the kernel’s clif-opt into a .clif file and change one shape per variant: rotation, check placement, trap form (trapz against a cold call), what a cold block receives, address arithmetic, block parameters. Compile each with Plenty’s release flags and count it:

cargo build --release --manifest-path experiments/kernels/harness/Cargo.toml \
  --target-dir target/kernel-harness
target/kernel-harness/release/plenty-kernel-harness FILE.clif \
  | python3 experiments/kernels/loops.py

Record a variant table (letter, change, count). The best variant shows what Cranelift can reach; the gap between it and C is the backend’s part. The Cranelift IR reference and the ISLE reference describe the IR and the lowering rules. For the pinned version, read the mid-end rules (opts/*.isle) and x64 lowering (lower.isle) in the cranelift-codegen source under ~/.cargo/registry/src/.

7. Decide where the fix lives

  • Plenty’s frontend or lowering: facts that remove checks, IR shapes that Cranelift lowers better. Implement these.
  • Cranelift mid-end or lowering rules, regalloc2: write the finding up with the variant that demonstrates it, for an upstream report. Do not patch pinned dependencies in a Plenty PR.

8. Implement and verify

  • Add tests that fail on the old code: CLIF or asm dump tests asserting the check is gone or the shape changed, plus behavior tests for the cases that must still trap (negative, out of range, near i64.MAX).
  • Recount the kernel table, then the whole program, then compile time: compile_bench 1000 10 --workload aot with PLENTY_CACHE=off and a few larger programs compiled with --compile, counted with perf stat -e instructions:u. Report run-to-run spread.
  • Run the full suite.

9. Record

Put the kernel table, the attribution, the variant table and the whole-program counts in the PR description. Keep kernels and scripts in experiments/ so the comparison can be rerun. File an issue for each remaining cause you did not fix.

Report template

## Kernel: NAME (source: FILE:LINE)

| Build | Instructions per pass |
|---|---|
| C (gcc VERSION -O2) | |
| Plenty before (COMMIT) | |
| Plenty after (COMMIT) | |

| Cause | Before | After | Notes |
|---|---|---|---|
| Bounds checks | | | |
| Negative-index joins | | | |
| Overflow checks | | | |
| Header reloads | | | |
| Register moves | | | |
| Spills | | | |
| Loop layout | | | |
| Calls | | | |
| Useful work (also in C) | | | |

| Variant | IR change | Count |
|---|---|---|
| a | as dumped | |

Whole program (instructions:u): before, after, C.
Compile time (instructions:u): compile_bench aot, other programs; spread.
Fixed here: ... Upstream: ... Remaining issues: #...

Pitfalls

  • An unoptimized build makes every comparison wrong; check the dump index’s optimize line.
  • Cranelift GVN merges identical pure instructions, such as iconst and stack_addr, and hoists them to a dominator, so a variant that duplicates them may compile to the same code.
  • A hot value passed to a cold call’s arguments can add moves on the hot path, because regalloc2 keeps the value in the call’s fixed argument register. PR #917 passes list-index failure operands through a stack slot for this reason.
  • Iterating a collection directly (for x in xs) has no bounds check; indexing in a loop has one unless the compiler proves it unnecessary.

Benchmarks Game suite

benchmarks/game/ holds ten programs from the Computer Language Benchmarks Game with a Plenty version of each, so that changes to generated code can be measured against fixed programs and a recorded baseline. It is a regression reference: it records where Plenty stands, and a slower run after a compiler change is a finding to explain. To make a program faster, follow Analyzing emitted code.

Layout

Each benchmark directory contains:

PathContents
benchmark.jsonProvenance, site build and command lines, local build command, thread count, sizes, Plenty variants with any compiler flags, and status (ready or blocked with blocked_reason)
reference/The reference program, copied verbatim from its site page
expected/Output at the small check size, from the site’s downloadable output files
plenty/The Plenty programs: the idiomatic one, and any parallel, data-layout or unchecked variant

The reference programs are distributed under the Benchmarks Game’s revised BSD license, reproduced in benchmarks/game/LICENSE-benchmarks-game.txt. Each manifest names the program’s page and contributors.

BenchmarkReferenceThreadsPlenty variantsStatus
fannkuch-reduxC #11idiomatic; parallel (examples/fannkuch_redux.plenty, 8 workers)ready
n-bodyRust #31idiomatic; clist (bodies in a clist); unchecked (--unchecked assume)ready
spectral-normC #81idiomaticready
mandelbrotC #81idiomaticready
fastaRust #31idiomaticready
k-nucleotideRust #17idiomatic (1 thread); parallel (7 pool workers)ready
reverse-complementC #51idiomaticready
binary-treesC #11idiomaticready
pidigitsC #1 (GMP)1noneblocked: no arbitrary-precision integers (backlog B38, issue #481)
regex-reduxRust #19idiomatic, with std.regex (1 thread)ready

Each reference is one C or Rust program from the site. None is marked with an asterisk, the site’s mark for a program that uses a different algorithm, so each benchmark has one idiomatic Plenty program that follows the reference’s algorithm. Where the reference uses threads and the Plenty program does not, the table says so; compare CPU time (user_s) as well as wall time for those.

Running

The runner, benchmarks/game/bench.py, uses only the Python standard library. It needs the release compiler (cargo build --locked --release), clang, rustc and cargo for the references, and optionally perf, taskset and GNU time. regex-redux’s reference is a Cargo package that uses the regex version pinned in the workspace lockfile; it builds offline from the local registry.

python3 -B benchmarks/game/bench.py list
python3 -B benchmarks/game/bench.py check                      # build all, verify check sizes
python3 -B benchmarks/game/bench.py run --mode quick n-body    # one benchmark, small size
python3 -B benchmarks/game/bench.py run --mode full --perf --output results.json \
    --compare benchmarks/game/baseline.json
python3 -B benchmarks/game/bench.py compare benchmarks/game/baseline.json results.json --metric instructions

Builds, generated inputs and outputs go under target/bench-game/. run:

  1. builds the references with the commands in their manifests, which match the site’s flags except -march=native (-C target-cpu=native) instead of the site’s ivybridge;
  2. builds each Plenty variant with plenty --compile and the variant’s flags;
  3. checks every program’s output at the check size against expected/;
  4. generates the fasta inputs for k-nucleotide, reverse-complement and regex-redux with the fasta reference;
  5. runs each program --repeats times (default 3), pinned with taskset -c 0,2,4,6,8,10,12,14 (one thread per physical core on the reference laptop), recording wall time, CPU time and peak RSS, and reports the median and minimum;
  6. compares each Plenty program’s output at the timed size with the reference’s output from the same run;
  7. with --perf, adds one perf stat run for instructions:u and cycles:u;
  8. writes JSON with the commit, tool versions, build commands and machine.

n-body and spectral-norm outputs compare within 1e-8; the others must be byte-identical.

compare (or run --compare) reports each Plenty program whose metric grew by more than --threshold (default 0.10) and exits 1 if any did. Metrics are time (median wall time), ratio (median relative to the reference in the same run, which transfers better between machines) and instructions (the least noisy on one machine). Benchmarks run at different sizes are skipped.

Sizes and baseline

quick sizes finish in about half a minute in total. full sizes are the site’s except reverse-complement, which uses the 25,000,000 fasta input shared with k-nucleotide (about 250 MB) instead of the site’s 100,000,000 (about 1 GB). benchmarks/game/baseline.json holds a full run on the reference laptop (AMD Ryzen 7 7840HS, performance governor):

BenchmarkSizeReferencePlentyRatio
binary-trees2111.11 s50.38 s4.54
fannkuch-redux1224.64 s29.21 s1.19
fannkuch-redux (parallel, 8 threads)1224.64 s3.30 s0.13
fasta25,000,0001.49 s4.17 s2.81
k-nucleotide25,000,00011.50 s (7 threads)60.63 s5.27
mandelbrot16,00012.49 s12.65 s1.01
n-body50,000,0001.27 s42.89 s33.67
pidigits10,000blockedblocked
regex-redux5,000,0000.74 s (9 threads)59.70 s80.66
reverse-complement25,000,0000.36 s6.39 s17.91
spectral-norm5,5001.26 s3.78 s3.00

Times are median wall-clock seconds. The k-nucleotide and regex-redux references are multithreaded while the idiomatic Plenty programs run one thread; user_s in the JSON gives CPU time (36.2 s for the k-nucleotide reference, 0.65 s for regex-redux).

k-nucleotide’s parallel variant postdates this baseline and has no entry in it yet; record it with the next full run. It counts the seven frame lengths on seven pool workers, as the reference does, and is the suite’s threaded workload for reference-count changes (issue #751): each job owns a Shared handle to the sequence, and its dictionary, with heap-string keys for the 12- and 18-byte frames, returns through a future and is released on the main thread.

n-body’s clist variant also postdates the baseline. It stores the bodies in a clist[Body], one column per field, instead of a list[Body], and tracks the cost of indexed clist field access in loops (issue #759): at 1,000,000 steps it runs 5.66 billion instructions against 2.95 billion for the list[Body] program and 0.23 billion for the reference.

n-body’s unchecked variant also postdates the baseline. It is the idiomatic program with the index arithmetic of energy and advance in unchecked: suites, built with --unchecked assume, so it measures what integer overflow checks cost. List bounds checks remain, since no build setting removes them. At 1,000,000 steps it runs 2.85 billion instructions against 2.95 billion for the idiomatic program.

Blocked benchmarks

A benchmark whose Plenty program needs a missing feature keeps its reference, expected output and sizes, with "status": "blocked" and the reason. The runner prints the reason instead of timing it, and tests/test_benchmarks_game.rs asserts it stays blocked with a phase-guard comment naming the backlog item. pidigits waits on std.bigint (backlog B38, issue #481). Its reference also needs GMP’s gmp.h and -lgmp.

Investigating a regression

dump writes the Plenty program’s optimized Cranelift IR and assembly, and the reference’s disassembly, under target/bench-game/dump/NAME/:

python3 -B benchmarks/game/bench.py dump n-body --filter advance

From there, follow Analyzing emitted code and its emitted-code agent skill (.agents/skills/emitted-code/SKILL.md): isolate the hot loop as a kernel next to an equivalent C kernel, as in experiments/kernels/, and attribute each extra instruction before changing the compiler. run --perf --metric instructions confirms the result on the whole program.

Tests

tests/test_benchmarks_game.rs compiles every Plenty variant without optimization and checks it at the check size, so a compiler change that breaks a benchmark fails cargo test. It does not time anything or build the references. benchmarks/game/test_bench.py tests the runner:

cargo test --locked --test integration test_benchmarks_game::
python3 -B -m unittest discover -s benchmarks/game -p 'test_*.py'

Adding a benchmark

  1. Copy the reference program verbatim from its site page into reference/ and its site output into expected/.
  2. Write benchmark.json following an existing one, then the Plenty program.
  3. Add the benchmark to ready_benchmarks! (or BLOCKED) in tests/test_benchmarks_game.rs and to the table above.
  4. Run bench.py check, then a full run, and update baseline.json.

Profiling the compiler

For fresh-process cold, unchanged and edited builds, see Measuring module edits. That workload complements the fixed compile_bench baseline and validates cached objects against uncached ones.

The profile Cargo feature reports, per compiler phase, the call count, time percentiles, total time and, with profile-alloc, the bytes and number of allocations. It uses hotpath, pinned to one version because its report formats change between releases. The feature is off by default; without it no hotpath code is compiled and the compiler binary is unchanged.

Running a profile

The profile is recorded only when PLENTY_PROFILE is set, and is printed when the process exits:

cargo build --release --features profile-alloc --example compile_bench
PLENTY_CACHE=off PLENTY_PROFILE=1 taskset -c 0,2,4,6,8,10,12,14 \
    target/release/examples/compile_bench 1000 10 --aot

The same works for the plenty binary, for example PLENTY_PROFILE=1 target/release/plenty --emit-object FILE -o OUT. Build with --features profile for time alone. Building with either feature replaces the release build of the compiler in target/; pass --target-dir to keep both.

VariableEffect
PLENTY_PROFILERecord and print a profile. Unset, the instrumented functions record nothing.
HOTPATH_OUTPUT_PATHWrite the report to this file instead of stdout.
HOTPATH_OUTPUT_FORMATtable (default), json, or json-pretty.
HOTPATH_ALLOC_CUMULATIVEtrue counts a function’s allocations with those of the functions it calls.
PLENTY_CACHE=offCompile every function; otherwise cached functions skip Cranelift.

The compiler turns hotpath’s live-metrics HTTP server off unless HOTPATH_METRICS_SERVER_OFF is set.

To see what the compiler generated rather than how long it took, use plenty --dump: it writes each function’s operations, Cranelift IR before and after optimization, and assembly, with an index sorted by code size. To make generated programs faster, follow Analyzing emitted code.

Selecting a benchmark workload

For generated-program dispatch rather than compiler phase attribution, see match dispatch measurements.

The first two compile_bench arguments are generated units (not lines) and repetitions. A unit contains a protocol, class, enum and functions. Choose one workload per invocation to avoid mixing source sizes and checking with AOT:

cargo build --locked --release --features profile --example compile_bench
PLENTY_CACHE=off PLENTY_PROFILE=1 HOTPATH_OUTPUT_PATH=check.txt \
    target/release/examples/compile_bench 1000 10 --workload check --no-warmup
PLENTY_CACHE=off PLENTY_PROFILE=1 HOTPATH_OUTPUT_PATH=scale.txt \
    target/release/examples/compile_bench 1000 10 --workload scale --no-warmup
PLENTY_CACHE=off PLENTY_PROFILE=1 HOTPATH_OUTPUT_PATH=aot.txt \
    target/release/examples/compile_bench 1000 10 --workload aot --no-warmup
SelectionWork performed
--workload checkCheck the requested unit count only.
--workload scaleCheck four times the requested unit count only; the metadata shows requested_units and the actual units checked. No ratio is printed without a base check.
--workload aotCompile and link the requested unit count only, including its frontend checks.
--workload check-and-scale (default)Check both sizes and print the scaling ratio.
--workload all or legacy --aotCheck both sizes and also compile/link the base size.

Without --no-warmup, each selected workload runs once before its timed repetitions. That warm-up is excluded from benchmark medians but included in the profiler report. With --no-warmup, ten repetitions mean ten calls of the selected workload. Source generation and benchmark setup are still within the outer main measurement. workload and warmup appear in the printed metadata; --help lists the options. Conflicting selections and invalid counts fail before profiling starts.

All workloads also work without profiling. Disable the cache deliberately for cold-build measurements; for warm-cache comparisons use an isolated PLENTY_CACHE_DIR, retain it between runs, and omit PLENTY_CACHE=off. Cache hits skip Cranelift but still build IR and assemble the object. compile_bench never uses program objects; pass --no-program-cache to plenty for the same warm-function measurement, since an unchanged program otherwise skips the frontend and code generation. These modes do not collect a whole-test-suite profile.

Reading the report

Each row is one measured function: the coarse phases of the compiler pipeline, such as modules::parse, frontend::lower_function, ownership::check, op::check, codegen::build_user_function, output::compile, codegen::emit_object and toolchain::link.

Additional rows subdivide the larger phases:

ParentSubphase rows and boundaries
output::compileoutput::cranelift_compile: Cranelift compilation or cache lookup/compilation, including cache bookkeeping; output::collect_code: copy machine code and convert relocations after success; output::clear_context: clear reusable compiler state, including after a compilation error. These do not expose individual Cranelift passes.
ownership::checkownership::prepare_futures, ownership::initialize_state, ownership::check_moves (outer sequence and final release), and ownership::check_loans.
ownership::check_loansFor bodies with loans: ownership::loan_graph (CFG, reachability and loan tables), ownership::loan_liveness (fixed point), and ownership::validate_loans (conflict checks and diagnostics). The no-loan scan remains in the parent.
frontend::lower_functionfrontend::setup_function (initial state, parameters and parameter loans; for a foreign function, its whole lowering), frontend::lower_function_body (the outer body-lowering call), and frontend::finish_function (return inference and final operation-body construction).
frontend::lowerAlso reports generics::expand_data and tail_calls::validate; both can return early when there is no relevant work.

Recursive ownership walks and nested expression/block lowering are not annotated on every visit. A loan-bearing function produces one graph/liveness/validation measurement per ownership check, regardless of CFG nesting. Function lowering can recursively specialize or re-lower a function during inference; those nested function-level measurements remain inclusive.

  • Time is inclusive: frontend::lower contains every lower_function call.
  • % Total is relative to the whole run. Functions that run on every core, such as ownership::check (inside op::check), build_user_function and output::compile, sum the time of all threads and can exceed 100%.
  • Allocation rows are exclusive by default: bytes allocated by the function itself and unmeasured functions it calls, not by measured functions below it. Adding subphase rows moves allocation attribution from parents to children; compare sums with the same instrumentation, or deliberately use cumulative allocation mode. Parent time still includes its children: never sum all rows as independent costs.
  • A function measured per call costs about 40 ns, so only phase-level functions are measured. To find hot functions inside a phase, use a sampling profiler such as perf record on a build without the feature.
  • Allocation counting adds work to every allocation: in one measurement it added about 2% to Cranelift compilation. Compare times from a profile build.
  • The compiler executables and compile_bench allocate with mimalloc, and profile-alloc counts allocations on top of it. Integration tests call the compiler in-process with the system allocator, so their timings do not show the executables’ allocator behavior.
  • For thread contention, compare system time and voluntary context switches (/usr/bin/time -f '%e %U %S %w %c %M') as well as wall time. Those counters include child processes: compile_bench --workload aot and --compile count the linker’s threads too, while plenty --emit-object counts only the compiler.
  • hotpath’s CPU sampling mode (hotpath-cpu) is not enabled: it needs samply and relaxed kernel settings, and perf already samples the compiler without a feature.

On 2026-10-18 (AMD Ryzen 7 7840HS, eight physical cores, PLENTY_CACHE=off), compile_bench -- 1000 10 --aot gave the same medians with the feature off, built but not recording, recording time, and recording allocations: the fastest of three runs was 646, 648, 644 and 662 ms for the AOT build. Noise from other load was larger than these differences. A clean release build of the compiler with the feature compiled 27 more crates and took 10 s longer (91 s against 81 s with -j 3). Those measurements predate the finer subphase annotations; measure overhead again when evaluating short-function costs and confirm optimizations on a feature-disabled build. Use sampling to identify Cranelift passes and hot leaf functions rather than adding per-operation rows.

cargo test --all-features builds the feature too. The instrumented functions then record nothing, because no test sets PLENTY_PROFILE except tests/test_profile.rs, which checks the report.

Other benchmarks and historical results

Function storage and list-row measurements

Historical optimization measurements; see Profiling the compiler for the profiling guide.

Reusing function storage

The B19 follow-up measured on 2026-10-22 reuses the cleared Cranelift Function already held by each emission worker. Ordinary user functions take its buffers before building IR and return them through the existing compilation path. Cache hits and misses both clear function state. Generated adapters and generator bodies still start with fresh IR; there is no additional pool or Cranelift patch. Larger buffers can remain until that worker finishes.

On the AMD Ryzen 7 7840HS, pinned to eight physical cores (taskset -c 0,2,4,6,8,10,12,14), a release profile-alloc build running compile_bench 1000 1 --workload aot --no-warmup reported these exclusive allocations (the profiler’s KB/MB units):

AllocationBeforeReused storage
build_user_function, cache disabled, per call20.3 KB4.9 KB
Same phase, all 7,001 calls138.5 MB33.4 MB
Same phase, warm function cache138.5 MB33.5 MB
Whole run, warm function cache642.4 MB535.5 MB

Function construction allocated about 76% less; total warm-cache allocation fell about 17%. These replace the older 34.7 KB/function observation in B19; other compiler improvements had already reduced that baseline. The whole-program cache is disabled by compile_bench, so a warm run still builds each function’s IR. Baseline and changed 1,000-unit program objects were byte-identical.

Feature-disabled release binaries were then run in three alternating pairs on the same cores, with no other agent builds or tests running. The default Plenty compilation mode was used; times include checking, emission, runtime extraction and linking. Disabled-cache and warm-cache processes used five timed repetitions after one warm-up; each cold-cache process used a new empty directory and one repetition without warm-up. Values below are medians of the three process results; RSS is peak resident memory from /usr/bin/time.

UnitsFunction cacheBefore, msReuse, msBefore, MiBReuse, MiB
1,000Disabled438402116.3117.1
1,000Cold52753494.997.1
1,000Warm307313120.1120.0
5,000Disabled2,3172,261404.1405.8
5,000Cold2,2632,385384.6384.4
5,000Warm1,6201,799402.5402.6

Latency differences changed sign between paired runs. For example, the three 5,000-unit warm-cache pairs changed by +1.6%, +24.2% and -0.4%; these noisy runs establish no consistent speedup. Peak memory stayed similar on these many-function workloads. The reason to retain this small change is its allocation reduction without an extra pool or dependency.

A separate warm-cache perf stat -e instructions,cycles comparison pinned to one core (taskset -c 0, 1,000 units, three repetitions plus warm-up) counted 11.811 billion user-space instructions before and 11.532 billion after (-2.4%); cycles were 9.190 and 9.151 billion. This single pair supports lower compiler work, while adding no reliable latency claim.

Inline scalar list rows

Issue #509 found std.diff’s Myers loop spending most of its time in runtime calls: on 3,000 dissimilar lines, perf attributed 30% of cycles to plenty_list_scalar_get, 27% to the Entries::get it called and 19% to plenty_list_scalar_set, against 21% in the std.diff functions. The access forms were borrowed &list[i64] parameters indexed in while loops and list fields of a &mut record read and written in the same loop. Only for loops over an unchanged borrowed parameter hoisted a row view, so these forms called the runtime per element.

Codegen now reads and writes borrowed scalar list rows inline through the fixed offsets in plenty-runtime/src/list_layout.rs (collections). Screening came first: --dump asm of std.diff.common_prefix shows each read as three loads from the owner, the index normalization, an unsigned compare and the row load, with a cold call to plenty_trap_list_index; afterwards perf attributes 94% of the cycles to std.diff functions and none to list helpers. User-space instructions and cycles (perf stat, one run, pinned to the eight physical cores):

BenchmarkInstructions before / afterCycles before / after
dis.plenty (3k dissimilar lines, Myers)33.31 / 12.85 billion7.79 / 3.49 billion
bench.plenty (all cases)81.10 / 38.73 billion17.90 / 9.24 billion
large.plenty (50k lines, 150 edits)7.51 / 7.18 billion2.59 / 2.37 billion

Best of three interleaved runs of bench.plenty’s five-repetition minimums (load average 0.5): dissimilar 3k Myers 265 to 100 ms, patience 265 to 100 ms; repetitive 3k 7.8 to 3.9 ms for both; similar 3k Myers 2.03 to 1.94 ms; large 50k Myers 38.4 to 36.2 ms, patience unchanged at 54 ms. Edit counts are identical. The large case is dominated by splitting and interning lines. The benchmark programs are outside the repository; issue #509 names them.

Compile cost is small: --compile of bench.plenty rose from 418.9 to 423.0 million instructions (1.0%), its executable’s .text grew 644 bytes, and compile_bench 1000 10 --workload aot stayed at 76.2 billion instructions. std.diff.middle_snake grew from 3,320 to 4,168 bytes of machine code.

Match dispatch measurements

Historical dispatch experiments; see Profiling the compiler for the profiling guide and the emission reference for current behavior.

Dense-match dispatch

The 2026-10-22 B19 experiment at 9c994af0 measures dense integer and enum dispatch with 4, 16, 64 and 256 arms against the then-production comparison chains. A temporary Cranelift Switch patch produces br_table; the patch is preserved as the original experiment. The production policy addresses issue #395. Reproduction, samples and generated-fixture methodology are in examples/dense_match/.

On a Ryzen 7 7840HS, x86-64 Linux, Cranelift 0.131.1, pinned to one CPU, median wall times over seven shuffled rounds for two million dispatches were:

ArmsInteger uniform: chain / switchEnum uniform: chain / switchInteger 90%-last: chain / switch
413.1 / 15.7 ms23.0 / 26.9 ms8.5 / 9.9 ms
1616.7 / 18.4 ms27.2 / 29.2 ms11.3 / 10.5 ms
6423.1 / 18.7 ms33.9 / 30.8 ms22.6 / 10.0 ms
25648.1 / 19.4 ms84.0 / 31.2 ms68.0 / 11.0 ms

The same-session baseline and prototype runs interleave. Times include process startup, all-arm correctness checks, the loop and output; enums also include borrowed list indexing. They are not pure dispatch latency or integer-versus-enum ABI comparisons. A Python oracle checks every sequence’s checksum; integer fallbacks include the first out-of-range value and u64.MAX.

The 256-arm enum 90%-last case fell from 111.4 to 14.6 ms. First-arm-heavy cases show why distribution matters: switches regressed at 4/16/64 arms for both integers and enums; at 256 they improved from 11.7 to 9.2 ms and 18.2 to 14.6 ms. A balanced Plenty integer decision tree was slower than chains for uniform 4/16/64-arm inputs (14.7/22.9/30.2 ms), and slower than the switch at 256 (37.9 ms). Neither balanced trees nor switches should replace every chain.

At 256 arms, isolated dispatch size fell from 5,515 to 3,656 bytes for integers and 7,763 to 3,688 for enums, including table data/padding. At four arms it grew from 91 to 128 and 131 to 152 bytes. Optimized CLIF instruction counts at 256 fell from 1,284 to 521 and 1,549 to 534. Cold compile/link medians for the full uniform fixtures were 69/65 ms (integer chain/switch) and 95/99 ms (enum); these include validation/setup code and linking, with only three samples in separate batches. They establish no reliable compile-time gain.

Production policy

The production lowerer now selects a jump table for at least 256 distinct contiguous integer/tag values, in any source order (issue #395). Smaller and sparse sets retain chains. The threshold deliberately leaves the mixed 64-arm results alone; it is not claimed to be optimal for every CPU or distribution. Regression checks cover all eight integer widths, negative/offset ranges, signed extrema, unsigned maxima, distant 64-bit fallbacks, reversed cases, alternatives, nested payload ordering, owned/borrowed enum payloads and tail exits. Exhaustive enum matching retains the invalid-tag trap. The emission contract describes index normalization and bounded table storage.

The harness also offers --kinds work: each arm computes a different wrapping multiply/add over a runtime state argument. This avoids drawing the entire conclusion from constant-return arms, while remaining a synthetic workload. It does not establish a benefit for any existing application. No source syntax or tutorial idiom changes follow from this optimization.

A follow-up paired run against the chain compiler at 70aa0a3e validated the production policy on the same host and CPU affinity, using two million dispatches and seven shuffled rounds (672 checked timed executions):

256-case workloadUniform chain / table90%-first chain / table90%-last chain / table
Integer55.6 / 21.0 ms12.2 / 10.5 ms79.7 / 12.1 ms
Enum100.1 / 35.1 ms19.7 / 15.6 ms127.7 / 17.8 ms
Runtime multiply/add57.6 / 22.1 ms13.5 / 10.5 ms82.1 / 11.8 ms

Isolated dispatch bytes, including table data/padding, fell from 5,515 to 3,646 for integers, 7,763 to 3,674 for enums and 7,623 to 5,754 for runtime arithmetic. All 26 optimized-IR/assembly dump files for the smaller matches and unchanged tree controls are byte-identical between compilers. Full samples, compiler/source hashes and dump indexes are preserved in production-results.json. Cold build samples were collected in separate batches with other builds active during the baseline batch; they establish no compile-time gain. The machine was not reserved or frequency-locked, so small timing differences in unchanged controls are noise rather than evidence for tuning smaller matches. This run does not measure allocation or peak memory, nor justify a broader sparse-table policy.

String match dispatch

A match with at least 16 distinct str literals before its fallback calls plenty_str_match once and branches through a br_table (issue #565). The runtime binary-searches a read-only table of the literals, ordered by byte length and then bytes; each entry maps to the first source arm with that literal. Smaller matches keep the plenty_str_eq chain. The table stores 32-bit offsets followed by the literal bytes, so it needs no load-time relocations in a PIE.

The 2026-10-30 measurements at 1756617d used a Ryzen 7 7840HS, x86-64 Linux and Cranelift 0.137, pinned to the eight physical cores. Each program matches the first n sorted IANA zone names (a seeded sample below 597) and four misses: a longer string, a shorter prefix, the empty string and an unknown name. Inputs are split from one literal at run time, so no lookup compares literal addresses. Times are medians and ranges of five shuffled rounds of two million lookups, including process start:

ArmsUniform: chain / tableFirst arm only: chain / tablefind bytes: chain / table
532.5 / 32.9 ms (chain kept)18.3 / 17.1 ms283 / 283
1558.5 / 59.6 ms (chain kept)17.8 / 17.7 ms693 / 693
1659.7 [59.4–60.5] / 49.2 [48.6–52.1] ms18.2 / 28.0 ms734 / 357
50123.8 [123.0–125.0] / 65.1 [63.5–66.2] ms17.7 / 30.9 ms2,128 / 831
5971,078.8 [1,077.8–1,087.6] / 110.6 [109.4–112.3] ms18.7 / 46.3 ms24,555 / 8,491

At 597 arms a uniform lookup falls from about 530 to 46 ns. The table data adds 4(n + 1) bytes plus the literal bytes. Optimized CLIF instruction counts fall from 115 to 55 at 16 arms and from 3,601 to 1,217 at 597. Cold compile times were within noise of each other (66–71 ms at 597 arms).

A run that forced the table for every size found the chain faster for uniform inputs below about 12 arms (5 arms: 33.8 / 38.1 ms), so 16 is where the table clearly pays. When the first arm matches every input, the chain always wins, by about 5 ns per lookup at 16 arms and 14 ns at 597: the chain stops after one comparison, while the search always takes about log2(n) steps. Order arms by frequency only in a small match; a large match no longer depends on order. A same-length comparison by 8-byte big-endian words was slower than the byte-slice comparison, which calls memcmp, and was not kept.

Serial pipeline measurements

Historical measurements; see Profiling the compiler for the profiling guide.

Serial pipeline callbacks

scripts/bench_pipelines.py compares an inline range/filter/map/fold loop with fused pipelines using named callbacks, closure literals, and immutable scalar captures. All variants do the same checked arithmetic: filter even values, square each accepted value modulo 1,000, and add the results. A second pair compares a loop with a closure-literal map whose body is a conditional expression, against its own inline loop. Run:

python3 -B scripts/bench_pipelines.py target/debug/plenty --items 100000000 --runs 7

The compiler executable can be a debug build: the harness requests optimized native output with --release, disables caches, checks every result, interleaves executables and reverses their order on alternate rounds. Compilation and cold checking of 100 separate functions are measured separately. This is a repeatable measurement tool, not a timing-sensitive unit test.

Issue #399 recorded the original named-callback pipeline at 1.906 times its inline-loop baseline. A fresh pre-change run on this host measured 0.160906 s for the inline loop and 0.283777 s for the pipeline (1.764 times, medians of seven executions over 100 million items). The direct callback calls survived fusion. Bounded scalar leaf substitution removes those calls, while SSA mirrors of immutable scalar captures expose constant divisors to native optimization.

Only straight-line scalar leaf bodies with at most 24 meaningful operations qualify. Capture mirrors follow local assignments and joins; exclusive borrowing disables them for the affected local. Checked arithmetic keeps the original callee source location. Debug code and larger or effectful callbacks retain their existing behavior. Broader callback substitution needs separate evidence.

On the same host, a quiet post-change run over 100 million items (seven interleaved executions per variant) measured:

VariantNative medianRelative to inline loopCold check, 100 functionsSingle-program compile
Inline loop0.156573 s1.0000.011199 s0.046002 s
Named callbacks0.155743 s0.9950.011768 s0.043120 s
Closure literals0.169209 s1.0810.023402 s0.042156 s
Scalar captures0.154756 s0.9880.023753 s0.044352 s

All executions returned 24500000000; all three pipeline forms meet the proposal’s within-10% runtime target for this workload. Closure literals still cost about 8% here, and checking 100 closure-heavy functions costs roughly twice the named-function variant. Single-program compilation is one observation per variant, not evidence of a compilation speedup. The machine was not frequency- locked or CPU-pinned; small runtime differences around parity are noise. These results do not promise the same ratios for larger, branching or mutable callbacks.

Closure-literal splicing

Closure literals written in a chain are now lowered in the consumer’s loop rather than as hidden functions (B75 stage 2). Before timing, --dump ops showed no ClosureNew or ClosureCall in the fused functions, and the optimized asm of the filter/map/fold closure variant has no call inside the loop. The branching variant’s asm has 107 instructions against 108 for its inline loop.

Both compilers were measured on this host back to back, pinned with taskset -c 0,2,4,6,8,10,12,14, over 100 million items with seven interleaved executions per variant:

VariantBefore: medianBefore: ratioAfter: medianAfter: ratio
Inline loop0.155813 s1.0000.157660 s1.000
Named callbacks0.157610 s1.0120.156564 s0.993
Closure literals0.159477 s1.0240.156964 s0.996
Scalar captures0.159048 s1.0210.156939 s0.995
Branching loop0.223247 s1.0000.222619 s1.000
Branching closure0.234639 s1.0510.225732 s1.014

Cold checks of 100 functions, medians of seven runs:

VariantBeforeAfter
Inline loop0.003607 s0.004034 s
Named callbacks0.003654 s0.003728 s
Closure literals0.005867 s0.005057 s
Scalar captures0.007005 s0.004893 s
Branching loop0.004116 s0.003629 s
Branching closure0.005514 s0.004041 s

All runs returned 24500000000 and 16850000993. Closure literals now run within 1.5% of their inline loops. Checking closure-heavy functions fell from about 1.6 to 1.4 times the named variant; the remainder is per-literal capture and splice lowering. Cold-check numbers are one series per compiler and vary by roughly 10% between runs.

Fluent generators and explicit conversion

scripts/bench_pipeline_generators.py measures the same filter/map/fold workload with range and generator sources, local fused consumption, explicit .generator() conversion. The compiler executable requests optimized native output, disables caches, checks equal results, and alternates execution order as in the callback benchmark:

python3 -B scripts/bench_pipeline_generators.py target/debug/plenty --items 5000000 --runs 7

A quiet run on this host measured the following medians. Every execution returned 1225000000.

VariantNative median
Inline range loop0.010011 s
Local fused range pipeline0.009781 s
Inline generator loop0.042060 s
Generator source through iter(...)0.044602 s
Direct fluent generator source0.044454 s
Range pipeline converted with .generator()0.027202 s
Generator-source pipeline converted with .generator()0.065896 s
std.iter chain with the same generator source (removed module; see the discussion after the table)0.105389 s

Direct fluent syntax has the same cost as wrapping the source in iter(...) (0.997 times here), and costs 1.057 times the inline generator loop. Explicit conversion adds a resumable frame: the converted range pipeline costs 2.781 times local consumption. With the same generator source, the converted pipeline took 0.625 times the runtime of a nested std.iter adapter chain. That row was measured at commit 57982e0f; std.iter has since been removed, and the script no longer runs it. Internal filter/map adapters are fused into one frame; resuming the source generator and the converted frame still has a cost. Keep local consumers fused when the stream does not need to escape.

The existing 100-million-item callback benchmark was also rerun with seven interleaved executions. Named callbacks, closure literals and scalar captures measured 0.999, 1.066 and 1.000 times the inline loop respectively, retaining the within-10% target. Compiler tests separately check a single converted frame with inlined scalar callbacks, and zero tracked allocations while constructing, moving, resuming and dropping an owned scalar pipeline. These timings are observations on a machine without frequency locking or CPU pinning; they do not establish universal ratios or allocation behavior for heap-owning source items.

Parallel pipeline measurements

Historical measurements; see Profiling the compiler for the profiling guide.

Parallel pipelines

scripts/bench_par_pipelines.py compares a serial pipeline, the equivalent pool.map/reduce_tree calls, and the same pipeline after .par(&pool) (B75 stage 4). Cases are map+collect, filter+count, sum over i64 lists, i64 ranges and f64 lists, and reduce. The cheap callback is x * 3 + 1; the expensive one runs 100 rounds of a linear congruential step. The pool column for filter+count maps the predicate to 0/1 and sums serially; for the sums and reduce it is reduce_tree over pool.map. Run:

taskset -c 0,2,4,6,8,10,12,14 python3 -B scripts/bench_par_pipelines.py target/release/plenty

The taskset mask keeps the eight-worker pool on the eight physical cores of this laptop; more than ten busy threads drop its clock to about 1.1 GHz. Each program builds its input outside the timed region, times one terminal with the monotonic clock per repetition, discards the first repetition and prints a checksum; every variant of a case must agree. The script interleaves the variants over three rounds of four timed repetitions and reports medians in milliseconds. The machine was not otherwise idle (load average 2 to 5 from other work), so differences under about 10% are noise.

Screening. Before timing, --dump asm of the chunk workers showed the same per-item calls as the serial loop (plenty_list_scalar_get and, when collecting, plenty_collection for the append) for map, filter and count. The signed-sum worker folded into a seven-field tuple and called signed_step per item, with two memmove calls for the tuple state; a serial microbenchmark of that fold over ten million items took 400 ms against 20 ms for the same arithmetic in local variables and 6 ms for sum. The worker now runs the chunk in a loop over locals, tracking the exact running total as a wrapping i64 plus a wrap count. Combining collected chunks now appends each chunk’s storage with extend instead of moving items one at a time.

The first round of measurements found two losses: splitting a list moved each item on the calling thread with a list append (about 37 ns per item), and .par reduce collected every item, then ran reduce_tree, which submitted one job per pair. Three changes followed:

  • A list source was split by an internal list operation that copied each chunk’s contiguous rows in one bulk copy. Issue #455 replaced this with borrowed chunks, described in this report.
  • pool.reduce_tree gives one job an aligned run of adjacent inputs, a power of two chosen so there is about one run per worker, and the job reduces it serially through the same rounds. The remaining run results are combined by pair jobs. Because each round maps index i to i / 2, an aligned run is exactly a subtree of the fixed tree, so the grouping is unchanged; a test compares it with a serial level-by-level evaluation for lengths 0 to 70 and larger odd and even lengths, with 1, 2, 3 and 8 workers.
  • .par reduce now groups like sum: each chunk reduces its items left to right in its worker, and the chunk results are reduced in input order. No list of all items is collected. The worker’s loop makes the same direct calls per item as the serial pipeline (plenty_list_scalar_get, the callback and f).

Borrowed chunks (issue #455). A list chunk is now a span of the source’s rows: the job receives the first row’s address, the row size and the item count, and reads (or, for owned affine items, takes) items in place. The calling thread copies nothing; it allocates one small list of chunk bounds. This relies on every terminal draining its jobs before it returns. --dump asm of the filter+count worker showed the per-item loop with no calls (an imul and a load per row, then the inlined predicate); before, it called plenty_list_scalar_get per item, and the split called the runtime copy.

The first measurement after that change regressed filter+count and the sums at 100,000 items (0.46 to 1.24 times serial): the owned source was now dropped on the calling thread, and dropping a list walked and cleared every row even when the items own nothing (about 3.4 ns per row). A list of items that own no heap storage now frees its buffer without the walk. That also makes the serial pipelines, which consume the list, faster.

Results on an AMD Ryzen 7 7840HS, eight workers, with borrowed chunks and the drop change (load average 5 to 6 from other work):

CaseCallbackItemsSerialpoolparpar/serialpar/pool
map+collectcheap1,0000.0411.0820.0421.030.04
map+collectcheap100,0005.21388.0153.3130.640.04
map+collectcheap10,000,000567.2599362.107328.2010.580.04
map+collectheavy1,0000.3401.1420.1290.380.11
map+collectheavy100,00034.43397.3239.0940.260.09
map+collectheavy1,000,000331.989982.12074.5920.220.08
filter+countcheap1,0000.0061.0350.0132.190.01
filter+countcheap100,0000.637104.2340.0820.130.00
filter+countcheap10,000,00084.4309071.53031.8550.380.00
filter+countheavy1,0000.2710.7170.0800.300.11
filter+countheavy100,00027.91990.1433.9020.140.04
filter+countheavy1,000,000274.661883.82534.6120.130.04
sum i64cheap1,0000.0050.8090.0122.250.01
sum i64cheap100,0000.54489.3180.0890.160.00
sum i64cheap10,000,00080.11610037.63436.0100.450.00
sum i64heavy1,0000.2820.8120.0890.320.11
sum i64heavy100,00028.724108.0876.4200.220.06
sum i64heavy1,000,000282.463969.47040.3240.140.04
sum i64 rangecheap1,0000.0010.8100.01623.210.02
sum i64 rangecheap100,0000.06890.4650.0791.160.00
sum i64 rangecheap10,000,0006.5859071.5215.7210.870.00
sum i64 rangeheavy1,0000.3080.8920.1000.330.11
sum i64 rangeheavy100,00027.91991.4614.8820.170.05
sum i64 rangeheavy1,000,000276.006929.20138.8520.140.04
sum f64cheap1,0000.0050.8360.0101.820.01
sum f64cheap100,0000.54795.4490.0460.080.00
sum f64cheap10,000,00075.0719511.75430.2470.400.00
sum f64heavy1,0000.1030.8270.0340.330.04
sum f64heavy100,00010.58392.9451.6730.160.02
sum f64heavy1,000,000104.262919.34213.3210.130.01
reducecheap1,0000.0050.8500.0112.090.01
reducecheap100,0000.59690.5440.0640.110.00
reducecheap10,000,00076.8979112.80929.4580.380.00
reduceheavy1,0000.2700.7720.0760.280.10
reduceheavy100,00027.30788.8553.8670.140.04
reduceheavy1,000,000272.071897.81234.3520.130.04

Before, at main 9e3970d5 with the bulk-copy split, measured just before in the same conditions (load average 4 to 6):

CaseCallbackItemsSerialpoolparpar/serialpar/pool
map+collectcheap1,0000.0510.7630.0440.860.06
map+collectcheap100,0006.01698.5222.8380.470.03
map+collectcheap10,000,000641.97211471.189540.4340.840.05
map+collectheavy1,0000.3480.6680.1370.390.20
map+collectheavy100,00035.87398.5386.9670.190.07
map+collectheavy1,000,000358.0551089.04871.7630.200.07
filter+countcheap1,0000.0100.8120.0262.620.03
filter+countcheap100,0001.115134.4470.7380.660.01
filter+countcheap10,000,000136.06915679.961262.1021.930.02
filter+countheavy1,0000.3221.1820.1050.330.09
filter+countheavy100,00031.387145.9788.4570.270.06
filter+countheavy1,000,000312.9111371.32563.9290.200.05
sum i64cheap1,0000.0100.9210.0333.310.04
sum i64cheap100,0000.980132.0050.8210.840.01
sum i64cheap10,000,000129.79912216.945248.0141.910.02
sum i64heavy1,0000.3180.7760.0900.280.12
sum i64heavy100,00029.457114.1886.8680.230.06
sum i64heavy1,000,000317.6421362.14869.6370.220.05
sum i64 rangecheap1,0000.0022.1400.05427.750.03
sum i64 rangecheap100,0000.169156.2960.1640.970.00
sum i64 rangecheap10,000,0009.37911315.1015.7530.610.00
sum i64 rangeheavy1,0000.2650.8100.0830.310.10
sum i64 rangeheavy100,00026.57392.5175.2450.200.06
sum i64 rangeheavy1,000,000268.563895.40937.1410.140.04
sum f64cheap1,0000.0090.7690.0171.890.02
sum f64cheap100,0000.92292.6120.4660.510.01
sum f64cheap10,000,000113.0429285.398208.3121.840.02
sum f64heavy1,0000.1210.8920.0470.390.05
sum f64heavy100,00011.289100.9613.1310.280.03
sum f64heavy1,000,000114.782950.89419.8750.170.02
reducecheap1,0000.0090.7770.0212.240.03
reducecheap100,0001.021105.9880.6940.680.01
reducecheap10,000,000119.9679456.671202.3081.690.02
reduceheavy1,0000.3130.8340.0830.270.10
reduceheavy100,00028.51591.6814.2810.150.05
reduceheavy1,000,000279.319911.11639.3720.140.04

Findings.

  • With cheap callbacks over ten million list items, .par now takes 0.38 to 0.58 times the serial time (1.7 to 1.9 before). At 100,000 items it takes 0.08 to 0.16 times serial for filter+count, the sums and reduce, and 0.64 for map+collect, which still appends each result.
  • With expensive callbacks, .par runs at 0.13 to 0.26 times the serial time from 100,000 items, as before.
  • At 1,000 cheap items the job overhead still exceeds the work (1.8 to 2.3 times serial). A range source over cheap items is unchanged.
  • reduce keeps per-chunk grouping; pool.reduce_tree keeps aligned runs.

Remaining work. Early stop for parallel any/all; user-facing scoped pool jobs (a deferred B22 extension) could reuse the drain-before-return guarantee.

Warm script startup measurements

Issue #699 investigated fresh-process launches of an unchanged no-op script. The expensive work was extracting the bundled runtime before an executable-cache lookup that did not need it. Deferring that extraction until a miss reduced the measured shebang median from 16.35 ms to 6.56 ms, without bypassing source, toolchain or native-input validation. These are local measurements, not startup guarantees for other machines or toolchains.

Workload and environment

Measured on 2026-11-03: AMD Ryzen 7 7840HS, 16 logical CPUs, x86-64 Fedora Linux 7.2.11-200.fc44, glibc 2.43, GCC 16.2.1 and mold 2.40.4. Both compiler binaries were Plenty 0.8.8, built with Rust 1.99.1 using cargo build --locked --release --bin plenty, without profiling features. The baseline was bd407f0525ab8a4da277de543ba60da3b3001b54; the changed compiler used the same sources with deferred runtime extraction. The script was:

#!/usr/bin/env -S plenty script
def main() -> Result[(), IoError]:
    Ok(())

The benchmark used a temporary working directory, an isolated private cache, tmpfs temporary outputs, and a fixed environment containing only HOME, PATH, PLENTY_CACHE_DIR and TMPDIR. PATH contained a benchmark-local plenty symlink and /usr/bin:/bin. Redirection/injection variables were absent. Both program-object and executable hits were confirmed before and after timing; the linker was not rerun in the timed launches. This establishes the cause of the reported high system time, but does not reconstruct the original fish session’s compiler build, full environment or exact timing.

Repeated untraced results

Each workload had ten untimed warm-ups and 100 fresh-process samples per compiler. Workload order was shuffled within each repetition. Wall time includes Python’s subprocess launch/wait and output collection; child user/system times come from getrusage(RUSAGE_CHILDREN). Diagnostics, compilation, cache population and tracing were separate. No fish timing overhead is included.

WorkloadBefore median (p10–p90), msAfter median (p10–p90), ms
plenty script hello.plenty15.984 (15.237–17.161)6.189 (5.686–6.705)
Executable env/shebang script16.346 (15.597–17.111)6.562 (6.097–7.075)
Equivalent AOT executable0.679 (0.524–0.781)0.687 (0.526–0.783)
plenty --version1.152 (0.933–1.319)1.118 (0.900–1.285)
/usr/bin/true0.543 (0.440–0.616)0.521 (0.425–0.599)

For direct invocation, median child user/system CPU fell from 2.615/13.114 ms to 2.657/3.324 ms. For shebang invocation it fell from 2.747/13.330 ms to 2.628/3.741 ms. Removing runtime extraction saved about 9.8 ms per launch; shebang/PATH handling was a small part of the original delay.

A reverse-order repeat on the final tree (another 100 samples per workload) measured 17.010 versus 6.650 ms for shebang execution and 16.516 versus 6.179 ms for direct invocation. The improvement persisted when the changed compiler ran first. Raw reports include compiler SHA-256 identities.

Attribution and remaining work in a hit

Separate strace -f -c and strace -f -tt -T runs confirmed the mechanism:

  • Before: write a 17,401,052-byte runtime archive into the temporary directory, then remove it after restoring a cached executable. One detailed trace measured 7.78 ms for the write and 2.10 ms for its removal. These traced syscall times are illustrative; the untraced before/after difference above measures the overall saving.
  • After: neither open/write nor removal of that archive occurs on a hit. The object (10,048 bytes) and restored private executable (121,104 bytes) are still materialized. The native-validation trace retained 447 statx and 222 readlink calls; it was not replaced with a stale-path shortcut.
  • Both versions invoke GCC for -print-prog-name=ld.mold and -print-search-dirs, then launch/wait for the application. Independent untraced probe measurements (100 samples after ten warm-ups) had medians of 0.782 and 0.854 ms. Those probes remain necessary under the current toolchain eligibility contract.
  • Compiler startup/dynamic loading has the approximately 1.1 ms --version control; the AOT child has the approximately 0.7 ms control. The rest of a roughly 6.2 ms direct hit includes runner/header/project setup, private locks, source/native dependency checks, object/executable restoration and cleanup. These independent medians form a rough cost budget, not additive phase instrumentation. Syscall attribution was sufficient to select this change; no compiler-phase profiler was needed.

Cold misses still extract and link the runtime. Wrappers, redirected toolchains, explicit outputs, hooks/fuzz builds and other unsupported executable-cache configurations retain their existing linking behavior. The compiler identity already includes the embedded runtime; deferring its extraction changes no cache key or manifest. Existing shared-library runtime files remain explicit inputs. The regression makes the runtime extraction path a directory: a real warm hit must succeed without writing it, while an edited-object miss must fail extraction and leave no executable to run. Cache invalidation, private executable restore, argument/exit handling and locking retain their existing tests and contracts.

Reproduction

Preserve a baseline release compiler before rebuilding the changed compiler, then run the helper from this checkout on an otherwise quiet machine:

python3 -B scripts/measure_script_startup.py /absolute/before/plenty /absolute/after/plenty \
    --runs 100 --warmup 10 --output /tmp/script-startup.json

The helper creates the script, AOT control, isolated cache and fixed environment, checks hits outside timing, prints distributions and writes raw wall/CPU samples to JSON. It refuses to benchmark a toolchain that cannot confirm executable hits. It does not change CPU frequency policy or pin a core. Repeat in reverse compiler order when checking for drift. Do not run compilation or tracing alongside the timed comparison.

For syscall diagnostics, create a separate working directory containing the same hello.plenty, warm it twice with the exact same fixed environment, then use strace -f -c or strace -f -tt -T. Keep trace output outside that working directory, since its identity is a watched linker input. Never report traced wall time as an ordinary launch sample. For a single fixed compiler, for example:

env -i HOME="$HOME" PATH=/usr/bin:/bin PLENTY_CACHE_DIR=/tmp/script-trace-cache \
    /absolute/plenty script --cache-stats hello.plenty
# Repeat to confirm hits, then trace with the same environment:
env -i HOME="$HOME" PATH=/usr/bin:/bin PLENTY_CACHE_DIR=/tmp/script-trace-cache \
    strace -f -c -o /tmp/script-syscalls.txt /absolute/plenty script hello.plenty

Measuring module edits

scripts/bench_module_edits.py compares complete builds in fresh compiler processes. It generates an application with user modules, an edited dependency and a generic function; by default it also imports std.http.server, using only a small part of it. It does not change the historical compile_bench template.

cargo build --locked --release --features profile --bin plenty
taskset -c 0,2,4,6,8,10,12,14 python3 scripts/bench_module_edits.py \
    /path/to/baseline/plenty target/release/plenty \
    --modules 10 --functions 100 --runs 7 --profile --json /tmp/module-edits.json

The compiler paths can name separate worktree builds or saved copies. Choose CPUs available on the measurement machine and pause concurrent builds and tests. Each repetition uses a fresh cache directory per compiler, alternates compiler order, and keeps source paths fixed throughout the comparison.

By default, timings include frontend checking, object emission and process startup, but not linking. The script also compiles and runs each source variant once per compiler to check observable results. Options include:

  • --release: measure optimized executable builds, including linking, and compare executable bytes. This covers cross-module scalar substitution; the CLI does not expose optimized object-only emission.
  • --import-std '': run a control without the broad standard import.
  • --profile: run separate, untimed profiles after the first repetition. A profile build records timing rows; profile-alloc also records allocations. Do not mix instrumented reports with ordinary latency samples.
CaseWork and correctness check
disabledCache disabled, including the whole-program cache.
populateFirst build with an empty cache. The program entry is initially only a marker.
function_warmSecond build: function cache entries exist; this build stores the program object.
unchangedThird build: requires a whole-program hit and the same object bytes as disabled.
bodyChange one ordinary function body; requires a program miss and byte identity with a fresh uncached build.
interfaceAdd a parameter and update its caller; requires the same invalidation and identity checks.
genericChange a generic body without changing its signature; requires invalidation of its instance and the same identity check.

Each edit starts from the base program with a current cached base object. The function cache retains earlier compiled variants, as in normal editing. JSON records every sample, cache statistics, artifact size and SHA-256, compiler binary hashes, CPU affinity, and GNU time peak RSS in KiB. Timings use a monotonic clock.

Interpreting results

Native emission omits unreachable standard definitions only after checking; this is not incremental frontend checking. Applications using most of an imported library may benefit less. See native emission and the separate per-module cache design.

In a 2026 measurement, the emission change improved the broad-import workload but showed no benefit for a control without standard imports. These historical results are workload-specific, not a general performance guarantee.

Module-edit emission measurements

These historical measurements assess issue #400’s unreachable-standard-definition emission pass, not current compiler performance. See the module-edit benchmark for the workload and how to reproduce a comparison, and native emission for the pass’s contract. It runs after checking and retains user definitions, exports, destructors and generators, following explicit calls and callback adapters. Conservative roots may retain functions execution does not need.

Measurements

2026-10-23, AMD Ryzen 7 7840HS, eight physical cores pinned as in the benchmark example. Baseline 42a4a67c and the reachability change used release compiler builds with the profile feature inactive during timings. Seven alternating samples per compiler, with agent builds and tests paused, ten user modules of 100 functions, and the default broad std.http.server import:

Object buildBefore, median msAfter, median msBefore / after peak RSS, KiB
Cache disabled73.9354.7361,228 / 37,360
Empty cache77.9454.5862,952 / 38,176
Warm functions60.3651.8146,008 / 35,836
Unchanged program5.844.489,636 / 8,616
One body edit60.8850.6845,448 / 35,860
Interface and caller edit60.9052.4147,192 / 37,556
Generic body edit61.4051.9747,316 / 37,500

The base object shrank from 1,368,960 to 408,744 bytes. A separate timing profile counted 1,558 native user-function builds before and 1,004 after; cold backend time fell from 30.7 to 7.9 ms. Lowering still took about 21 ms, which explains why this change does not eliminate the edited-build frontend cost. The isolated pass took about 0.7 ms, including release of discarded operations.

Separate profile-alloc builds of the same workload reported total allocated bytes of 285.8 → 133.7 MB with caching disabled and 125.1 → 89.2 MB after a body edit (hotpath’s MB units). These are allocation traffic, not peak RSS. Parsing and body-lowering allocation remained 21.4 and 19.7 MB respectively; most of the reduction came from avoiding native construction and compilation. Allocation instrumentation timings are excluded from the latency table.

Repeating the matrix with --release (optimized executables, linking included) gave 106.12 → 87.87 ms cold and 95.11 → 82.18 ms after a body edit. Unchanged linked builds were 36.23 → 35.32 ms: linking dominates this path. The executable size stayed 3,283,000 bytes because the linker already discarded unused code. Every measured cached object or executable matched its uncached counterpart; body, interface and generic-body edits all missed the program cache and produced the expected changed result.

The same 1,000-function workload with --import-std '' produced identical object sizes before and after. Its cold median was 19.19 → 19.90 ms and body-edit median 18.93 → 19.02 ms; these short runs establish no benefit for that control. The pass exits after its candidate scan when there are no standard definitions. These are workload-specific observations, not a general compilation-speed guarantee or a substitute for measuring an application that uses most of a library.

Per-module cache investigation

The 2026-10-24 bounded B19/B17 investigation supports keeping the current function/program caches. A scalar-only separate-compilation spike proved reuse and invalidation at an explicit boundary, but was slower than compiling the same computation normally. It is not a production cache implementation.

Frontend reuse remains promising for imports with substantial bodies, especially the standard library. It requires a real lowering boundary; independently packaging today’s modules as C libraries is not a useful shortcut. This report records evidence and constraints. Scheduling remains in the backlog.

Current compiler measurements

Local main at 00d32a13, AMD Ryzen 7 7840HS, Linux x86-64, Rust 1.99.0, release compiler with profile compiled in but inactive during timings. Compiler SHA-256: 61875f0816906d4c5d2577da75bb7f918ad9a26115029bc772d188176d6afc07. Seven repetitions, physical cores 0,2,4,6,8,10,12,14, other agent builds paused. The existing module-edit harness generates ten modules with 100 independent functions each, a caller and an entry point. Its broad-import variant imports std.http.server; the scalar control omits that import.

Object buildBroad import, median msScalar control, median msBroad import peak RSS, KiB
Cache disabled45.5017.1435,348
Populate empty cache47.7317.6636,264
Warm functions44.5415.5633,788
Unchanged program3.994.068,096
Ordinary body edit44.0016.1734,044
Interface and caller edit46.7916.3535,804
Generic body edit44.6716.6635,588

These timings include compiler startup and object writing, but exclude native linking. Every cached artifact matched its uncached counterpart byte for byte, and every source variant executed with its expected output. No-change builds hit the program cache; all three edits missed it. This is a synthetic workload, not a forecast for arbitrary applications.

Separate timing profiles after a body edit recorded:

PhaseBroad importScalar control
Parse all modules8.35 ms / 25 modules2.20 ms / 12 modules
Resolve names3.32 ms1.16 ms
Lower program19.28 ms2.66 ms
Lower functions, inside program lowering10.91 ms / 1,629 functions1.42 ms / 1,004 functions
Independent operation check2.21 ms0.49 ms

Nested profile rows are not additive. The broad-import frontend contains much more reusable work than the scalar control. Avoiding all this work is only an optimistic ceiling: loading, dependency validation, merging and changed-module checking still cost time.

A separate profile-alloc build reported 119.8 MB allocated with caching disabled and 74.5 MB after a body edit (hotpath’s displayed MB units). Parsing allocated 10.4 MB, name resolution 4.8 MB and function-body lowering 19.6 MB after that edit. These are exclusive allocation traffic counters, not retained memory or RSS. Allocation-instrumented latency is excluded from all timing tables. This leaves meaningful allocation work for a frontend cache to avoid, but artifact decoding would introduce allocations of its own.

Five additional broad-import --release repetitions measured optimized executable builds, including linking: 79.77 ms uncached, 33.77 ms unchanged, 73.49 ms body edit, 71.78 ms interface edit and 77.73 ms generic edit. Cached executables again matched uncached executables. Reusing frontend artifacts alone does not eliminate the link floor.

Actual reusable-artifact spike

experiments/module-cache/run.py generates ten scalar modules, each with a single exported i64 function and 100 independent private functions. A separate caller uses the first module. It caches existing static-library outputs using compiler-binary, source, module-name and imported-interface hashes. Integrity checks reject damaged artifacts. A second mode extracts only each archive’s plenty_exports.o, leaving the final application to supply the runtime.

This intentionally uses existing C export/import support rather than modifying the compiler. Interfaces are generated from the fixed fixture, not inferred by a new parser. There are no generic imports, classes, borrowed values, strings, closures or cross-module substitution at the artifact boundary. It proves that an ordinary scalar body edit can reuse callers; it does not prove a general Plenty interface is just its written signatures.

Five repetitions per mode, same compiler and CPU affinity. Each repetition has an empty artifact cache. Existing Plenty caches are disabled to isolate artifact reuse. The ordinary comparator builds the same scalar computation and independent functions through normal imports, also with caching disabled. All rows below include native linking and use default code optimization, unlike --release above. Source generation and execution checks are outside the timings.

BuildFull archive cache, msExtracted object cache, msOrdinary compilation, ms*
Cold1,222.611,236.9748.89
Unchanged191.4996.52—
Ordinary body edit279.73197.8055.36
Interface and caller edit379.55307.3450.31

*Ordinary comparator from the extracted-object run. The independently repeated archive-run comparator was 48.71 / 50.37 / 50.33 ms respectively. These modes were run in separate batches; small differences are not an optimization claim.

An ordinary body edit rebuilt only module m0; its caller and the nine other modules were hits. A signature edit rebuilt m0 and the caller. Rebuilding all artifacts afresh produced the same executable bytes as reuse for each edit. The program produced 42, 43 and 44 in the base, body and interface cases. Corrupting every cached artifact forced rebuilding and restored correct output.

Full archives occupied 174,083,980 bytes for the base program; extracted objects occupied 335,712 bytes. The former repeats the library runtime for each module. Hashing those archives on hits is part of the measurement. Extraction removes that storage/read cost but still creates archives on misses. The extracted mode’s peak compiler-child RSS was about 55 MiB cold/edited and 32 MiB unchanged, versus about 37 MiB for ordinary compilation. This excludes the Python driver’s RSS and is not a total allocation count. Object and archive modes have different linker retention, so executable byte identity is checked within a mode, not between modes or against ordinary compilation.

Repeated compiler processes, export adapters, archive construction and linking outweigh avoided checking in this small workload. Do not adopt this mechanism as the module cache. The experiment does not estimate the speed of a proper in-process lowered-artifact cache, which avoids these costs.

Required boundary and invalidation

The implementation still lowers a whole resolved program. frontend::lower creates aliases, signatures, generic instances and pending work; lowering shares the string heap and records information used by subsequent functions. The module loader preserves dependency order, import resolution and source identities. Persisting a function’s operations while ignoring these outputs is insufficient.

The small check_dependencies.py probe changes only &mut pair.x to &mut pair.y in an imported function. Its declaration stays select(pair: &mut Pair) -> &mut i64, yet an unchanged caller that mutates pair.y changes from accepted to a conflicting-borrow diagnostic. Reverting the body restores acceptance. This directly exercises frontend::references::returned_fields: inferred projections are interface facts.

DependencyWhat must invalidate or be reconstructed
Source/import resolutionModule text, canonical package/module identity, import target changes, compiler identity/settings
Public declarationsSignatures, nominal layouts, aliases, visibility, protocols and generated methods
Body-derived semanticsReturned-reference projections and any effects/capture facts consumed elsewhere
Generic instantiationDefining body, type arguments, constraints and transitive instantiated dependencies
Native substitutionAny body embedded in another function, even if its signature is unchanged
Stored operationsDeterministic string/type/instance identities, source locations and ownership diagnostic origins

A conservative first cache may depend on complete imported source digests; this sacrifices reuse but avoids claiming interface precision prematurely. Precision can follow after an explicit module result can be built and replayed against fresh imported environments. Generic definitions/instances and embedded native bodies need separate dependencies from ordinary declaration hashes. Corrupt/version-mismatched artifacts must miss, and invalidated diagnostics must remain in program order. Existing cache-on/off identity and import-resolution tests remain acceptance gates.

The smallest credible compiler change is therefore to isolate and test a module lowering result without persistence first. Measure replay against normal lowering on an actual std-heavy application before adding serialization. The earlier syntax-tree/copy investigation already argues against inserting a parsed-tree cache instead. This investigation does not authorize a broad redesign or promise a speedup.

Reproduction

Build plenty with cargo build --locked --release --features profile --bin plenty. Run scripts/bench_module_edits.py with --runs 7 --profile, then repeat with --import-std ''; use --runs 5 --release for the optimized executable check. Each command requires a --json PATH. Run the experiment with:

python3 experiments/module-cache/run.py target/release/plenty --runs 5 --json /tmp/archives.json
python3 experiments/module-cache/run.py target/release/plenty --runs 5 --objects --json /tmp/objects.json
python3 experiments/module-cache/check_dependencies.py target/release/plenty

Pin CPUs with taskset as above and avoid concurrent compilation. JSON records every sample, compiler hash, affinity, outputs, reuse decisions and artifact sizes. The existing module-edit harness additionally records separate phase profiles and cache statistics. The spike is single-process orchestration in a temporary directory; it is not a hardened shared cache or a general-purpose module parser.

For allocation counts, rebuild with --features profile-alloc and run the module-edit harness with --runs 1 --profile; keep those results separate from timing runs. No production Rust code changed, so validation here is the executed native experiments, edit matrix, Python syntax checks and Markdown link checks.

The Plenty agent skill

The plenty skill teaches coding agents (Claude Code, Codex, Cursor and others that read the Agent Skills format) to write Plenty. It is for people writing Plenty programs; the repository search skill is for work on this repository. This page covers where the skill lives, how it is published, and how to trial it.

Layout

PathContents
.agents/skills/plenty/The skill: SKILL.md and hand-written references/
.claude/skills/plentySymlink to .agents/skills/plenty, so Claude Code loads it here
scripts/agent_skill_repo/Files that exist only in the published repository: plugin manifests and README
scripts/sync_agent_skill.pyAssembles the published repository
scripts/skill_trial.pyRuns fresh agents against the published skill

Agents working in this checkout use the skill directly. In the checkout, its links point at book pages such as the syntax cheat sheet and the signature index, so it never holds a copy that could fall behind the book. Copies are made only when publishing.

tests/test_tutorial.rs runs every executable fence in the skill with the same fence kinds as the book, so a language change that breaks a skill example fails the suite. scripts/test_sync_agent_skill.py checks that the skill’s links resolve and that the published tree is self-contained.

When a skill passage works around a local issue, an HTML comment names it, for example <!-- issue #575 -->. Remove the workaround when the issue is closed. The published copy drops these comments.

Keeping the guidance current

When a language or toolchain change lands, review SKILL.md and its handwritten references alongside the cheat sheet. Correct obsolete advice and remove workarounds for resolved issues. Keep common error-prevention rules in the entrypoint; link to reference contracts for specialized features instead of copying their full specification. The skill’s references/discovery.md guide routes migration tasks to existing modules and asks for versioned minimal reproductions before declaring a capability absent. Applied ownership examples distinguish ordinary copying, explicit class copying and shared content; package examples teach module visibility and target coverage.

Check both the checkout skill and the assembled tree: a working book link can lose its destination when publishing if that page is not bundled. The language reference is bundled so agents can follow arithmetic, ownership, generic and protocol contracts without this checkout. Run the sync tests and documentation checks; run the executable skill examples when changing code fences. These checks validate examples and packaging; the trial runs separately measure how agents use the guidance.

Publishing

The published repository, plentylang/skills, has one tag per Plenty release, vX.Y.Z, matching the compiler. To assemble it from a release checkout:

just skill-sync ../skills          # or: python3 -B scripts/sync_agent_skill.py ../skills

The script refuses to replace a directory that is not a skills tree, keeps .git in a clone, and replaces everything else. It:

  • copies .agents/skills/plenty/ to skills/plenty/, naming the compiler version from Cargo.toml (--version overrides it);
  • copies the cheat sheet, language reference, package and script pages, every standard library page, and the builtins, I/O, concurrency and editor-setup pages into skills/plenty/references/;
  • rewrites links between copied pages, and reduces links to other pages to their text, until the book has a published URL;
  • drops issue notes and contributor-only passages, then recalculates signature-index module line ranges against the published text;
  • adds the README and the .claude-plugin/ marketplace and plugin manifests, with the version filled in.

Commit the result in the published repository and tag it vX.Y.Z.

Contributor-only passages

Some book passages address compiler maintainers, such as how to update a vendored crate or regenerate the signature index. Enclose them in markers so the published skill omits them:

<!-- contributors-only -->
To update one, replace its directory with the new release, run
`cargo update -p CRATE --precise VERSION`, and rerun the tests.
<!-- /contributors-only -->

Each marker is a line of its own. The markers are invisible in the book. Unbalanced markers stop the sync.

Trial runs

A trial gives fresh agents only the published skill and the compiler, asks them to write three small programs, and checks the results:

just skill-trial                           # all tasks, the agent's default model
just skill-trial wordfreq --model sonnet   # one task, a chosen model

The recipe builds the release compiler first. Trials need the claude CLI, signed in, and spend API credit (at most --max-budget-usd per task, default 3). They are not run in CI. python3 -B scripts/skill_trial.py --help lists every option.

TaskAsks forChecks
wordfreqA CLI counting words in a file, with error and usage exit statusesOutput, exit status 1 naming a missing file, exit status 2 for usage
inventoryClasses, mutation through &mut self, and an error enumExact output
packagesA library with tests and an executable depending on it, using std.jsonplenty test passes; exact output of plenty run

Each task runs claude -p in a new directory under the system temporary directory, never inside a checkout, because Claude Code reads CLAUDE.md from parent directories. The agent gets:

  • the published tree as a plugin (--plugin-dir), as /plugin install would provide it;
  • plenty on PATH, as a symlink to target/release/plenty;
  • no user or project settings, CLAUDE.md, MCP servers or session history;
  • the file tools, Bash and Skill, without permission prompts.

Because the agent can run shell commands, it could still read files elsewhere. The prompt tells it not to, and the report lists any tool call that mentions this repository.

Reading the results

Each run writes target/skill-trial/RUN/:

FileContents
report.mdSummary table, then one section per task
report.jsonThe same data, for comparing runs
tasks/NAME/prompt.mdThe prompt sent
tasks/NAME/transcript.jsonlThe agent’s full stream-json transcript
tasks/NAME/files/The sources the agent left
skill/The published tree the agents used

The summary table records:

  • Result: whether every check passed.
  • Skill used: whether the agent loaded the skill or read its files. A no means the skill’s description did not trigger.
  • plenty runs: compiler invocations the agent made.
  • Failed runs: invocations that exited nonzero, including command-line misuse.
  • Diagnostics: each compiler error the agent hit.
  • Leaks: tool calls that reached outside the task and the skill.

Each task’s section lists failed checks with expected and actual output, every diagnostic, and the failed commands with their last lines of output.

To improve the skill, read the diagnostics and failed commands first. Each one the skill could have prevented points at a missing rule, a misleading example, or a diagnostic worth improving. Then search the transcript for guesses: places where the agent searched the references and did not find an answer. Results vary between runs and models, so compare several runs before changing the skill for a single failure.

Releasing Plenty

A release is one vX.Y.Z tag on main. The maintainer creates and pushes it with cargo-release; the tag push starts .github/workflows/release.yml, which builds, tests, and publishes the GitHub Release. Users install it as described in installing a release.

Only the maintainer releases, after the changes to ship have been reviewed and merged under the local PR policy. Agents and PR branches never run cargo release --execute, create release tags, or push.

Versions

PackageVersion policy
plentyThe language release version. cargo release bumps it in Cargo.toml and Cargo.lock
plenty-runtimePrivate and embedded in the compiler. Keeps its own version; plenty-runtime/release.toml excludes it, so it is never bumped or tagged
plenty-dataframe-engineSeparately built workspace; its crate version is internal. Release archives bundle it under the Plenty version on Linux, macOS ARM64 and Windows x86-64, with the ABI generation in the shared-library filename
plenty-linalg-engineSeparately built faer workspace with the same release-version and ABI-generation policy

plenty --version reports the plenty package version. Tags are vX.Y.Z, or vX.Y.Z-PRE for a pre-release; asset names use the same version.

The root Cargo.toml is the source for the current compiler version. The CLI, LSP server information, source-cache paths, compilation fingerprints, and the minimum compiler version written by plenty new/plenty init use Cargo’s CARGO_PKG_VERSION at build time. Release scripts read the same manifest to name archives and fill the packaged README; packaging rejects a compiler whose reported version disagrees. The Python engine packager reads [package].version directly with tomllib, without invoking Bash with a native Windows path; check-version.sh remains the shell interface for archive/tag checks. Rebuild executables after bumping the manifest.

Do not globally replace older version strings. The private runtime has an independent version; new user packages start at 0.1.0; example manifests, minimum-version requirements, test fixtures, dependency pins, and historical documentation references describe their own versions. Compiler-output examples use X.Y.Z placeholders so they do not require edits on every release.

The documentation release catalog is separate from a version bump. Admit a version only after its release is published, preserving earlier archive pins. Development labels identify their source revision automatically; archived labels come from the selected release.

Tools

release.toml was tested with cargo-release 1.1.6:

cargo install cargo-release --version 1.1.6 --locked

The release build itself needs no local tools: it runs on GitHub Actions. To rehearse it, Docker is enough (see checking locally).

Prepare a clean release checkout

cargo-release refuses a dirty tree and commits with git commit -a. The primary checkout often has a modified issues.db, the shared issue register described in CLAUDE.md, which must not be reset, stashed, or committed to make a release. Release from a separate clone of main instead, whose origin is the primary checkout’s origin:

PRIMARY=/path/to/plenty            # the primary checkout
git clone --branch main "$PRIMARY" ~/plenty-release
cd ~/plenty-release
git remote set-url origin "$(git -C "$PRIMARY" remote get-url origin)"
git fetch origin
git status --short --branch    # must be clean and level with origin/main

Bring main level with origin/main first, through the usual review flow, so the release push is a fast-forward. Before releasing, run the full checks on that commit:

just release-check

This runs every Linux CI step, both optional engines, the Docker reverse-proxy tests, license notices for every release target, the documentation checks, and scripts/release/local-check.sh --test. It continues past a failing step, writes each step’s log to target/local-ci/, and prints a summary. scripts/local_ci.sh --help lists the steps; just ci STEP... reruns selected ones. The macOS and Windows release jobs, and the host’s other Linux architecture, still run only in CI.

Release

Preview, then execute:

cargo release patch                # dry run: prints the bump, commit, tag, push
cargo release patch --execute      # bump, commit "Release Plenty X.Y.Z", tag vX.Y.Z, push
LevelExample from 0.3.0
patch0.3.1
minor0.4.0
major1.0.0
rc, beta, alpha0.3.1-rc.1, then 0.3.1-rc.2
releaseDrop the pre-release: 0.3.1-rc.2 becomes 0.3.1
X.Y.ZExactly that version

The dry run changes nothing. The configuration rejects branches other than main, never publishes to crates.io, and pushes the commit and tag together (git push --atomic).

Afterwards, bring the release commit into the primary checkout without touching issues.db:

cd "$PRIMARY"
git fetch ~/plenty-release main --tags
git merge --ff-only FETCH_HEAD

Then publish the matching coding-agent skill to plentylang/skills, tagged with the same version:

git clone git@github.com:plentylang/skills.git ~/plenty-skills   # once
cd "$PRIMARY"
just skill-sync ~/plenty-skills
cd ~/plenty-skills
git add -A
git commit -m "Plenty X.Y.Z"
git tag vX.Y.Z
git push --atomic origin main vX.Y.Z

The v0.8.9 glibc verification records downloaded-artifact symbol audits and glibc 2.28 execution evidence for both Linux architectures.

What the workflow does

JobWork
Verify tagRequires the tagged commit to be on main and the tag to equal v plus the Cargo.toml version (scripts/release/check-version.sh)
TestThe full test suite and release-script tests on native Ubuntu x86-64 and ARM64 runners, with the pinned Rust
Build and testFor each architecture in its pinned glibc 2.28 image: installs the pinned Rust, builds both executables with --locked, runs the test suite, audits the compiler, then builds/audits both engines and packages them in the compiler archive
Smoke testFor each target and its pinned distribution images: installs only the documented prerequisites, disconnects the network, and exercises all archives, including app-only engine relocation with the compiler/original engine removed (smoke-in-container.sh)
Build and test (experimental)On macos-15 and windows-2025 with the pinned Rust: builds both executables, runs the platform-boundary and compiler/runtime unit tests, audits the executables (check-native.py), and packages the archive (build-native.sh), including the validated dataframe dylib on macOS or DLL on Windows
Smoke test (experimental)On a fresh runner of the same kind, with Rust, Cargo and PLENTY* variables removed from the environment: checks the archive, then type-checks, compiles and runs a program from paths with spaces, before and after moving the installation (smoke-native.py)
PublishTag pushes only. Creates a draft release with every target’s archives and checksums, then publishes it (publish.sh)

Native Ubuntu test runners and the x86-64 baseline build install the SHA-256-verified mold version pinned in scripts/release/toolchain.env, without requiring root. They check a linked program’s .comment section before running the suite, so a missing linker fails early rather than silently changing executable sizes. The ARM glibc 2.28 build and its local rehearsal use the GNU linker already pinned by the build image: upstream’s ARM mold 3.0.0 binary imports gettid@GLIBC_2.30 and cannot run in that image. This preserves the existing glibc floor. Size budgets measured with mold require care when comparing GNU ld outputs, which can retain additional unused runtime code and panic strings. The clean distribution smoke tests also exercise system linkers; mold is not a release-installation dependency.

Builds use the tagged source, not moving main, a host build, or a cache from another environment. Publication waits for every job to pass. Pre-release tags (vX.Y.Z-PRE) publish GitHub pre-releases that are not marked latest. Runs for the same tag are serialized. Only the publish job can write repository contents, through the workflow token; no crates.io or other credential is used.

scripts/release/target.sh defines each target and scripts/release/matrix.py turns the list into job matrices:

TargetRunnerArchives
x86_64-unknown-linux-gnuubuntu-24.04compiler with dataframe and linalg engines (.tar.gz)
aarch64-unknown-linux-gnuubuntu-24.04-armcompiler with dataframe and linalg engines (.tar.gz)
aarch64-apple-darwin (experimental)macos-15compiler with dataframe dylib (.tar.gz), macOS 11 or later
x86_64-pc-windows-msvc (experimental)windows-2025compiler with dataframe DLL (.zip), Windows 10 or later

Each target publishes one asset, plenty-VERSION-TRIPLE.EXT, and its checksum. release_engines in target.sh lists the engines bundled for a target. Linux bundles both ELF engines; Windows bundles the dataframe DLL, built and validated natively by build-native.sh. macOS bundles the dataframe dylib, also built and validated natively, with a macOS 11 deployment minimum, @rpath/libplenty_dataframe_v1.dylib identity, system-only dependencies and verified ad-hoc signature. Native archive modes distinguish the two compiler executables (0755) from engine libraries and notices (0644). Windows packaging audits x86-64 PE headers, Windows/MSVC dependencies and all facade exports. The native clean smoke tests check a Parquet query, compiler relocation, --dist application relocation and failure without the engine. Native archives are not notarized or signed with a distribution identity; macOS executables and the dataframe dylib carry linker-generated ad-hoc signatures, and the Windows build links the dynamic MSVC runtime that the documented Build Tools prerequisite installs. archive.py writes those archives with sorted entries, fixed owners and modes, and SOURCE_DATE_EPOCH timestamps. The release notes mark them experimental. The ARM harness records the emulated evidence gathered before native ARM runners. A Linux archive contains bin/libplenty_NAME_v1.so beside the compiler and bin/NAME-notices/ with licenses, dependency notices and engine.json identifying the release/source, target and library checksum. build.sh builds each engine and package.sh --engine NAME=LIBRARY installs it through package-engine.py; the archive README covers using and shipping the engines. package-engine.py NAME rejects an incorrect SONAME, embedded search paths, unsupported native dependencies/symbol versions, and missing symbols required by that checkout’s stdlib facade. This catches stale engines with the same ABI-generation name. The workflow builds all components from the tagged source; it never substitutes an engine downloaded from another release.

Normal compiler builds remain independent of Polars and faer, and the compiler links neither; programs load an engine only when they reach it. The release engine is built once for reuse when compiling and running many applications. The engine’s native dependency graph supplies its own notices; it is not a copy of the compiler’s dependency inventory.

The release build environment

scripts/release/toolchain.env pins every input that is not in the lockfile:

  • BUILD_IMAGE and BUILD_IMAGE_AARCH64: architecture-matched quay.io/pypa/manylinux_2_28_x86_64 and manylinux_2_28_aarch64 images by digest. They provide AlmaLinux 8 with glibc 2.28 and gcc-toolset, so C/Rust objects are built against that baseline. ELF symbol auditing and execution in the oldest smoke environment must confirm the result for each target.
  • RUST_VERSION, architecture-specific checksummed rustup-init, and mold.
  • SMOKE_IMAGES and SMOKE_IMAGES_AARCH64: separate pinned inventories. x86 retains its six distributions; ARM uses AlmaLinux 8 and Debian 13. A distribution without a matching ARM image is not silently treated as tested.

Update these deliberately: change the pin, run scripts/release/local-check.sh, and update the tested distribution list in scripts/release/README.dist.md and the installation page when it changes. check-elf.sh fails the build if a change raises the glibc requirement.

Recovering from a failed release

Never move or delete a pushed release tag, and never change a published release.

  • A job failed before publishing, for example a flaky test or an unavailable image: rerun the failed jobs. No release exists yet, or only a draft.
  • Publishing was interrupted: rerun the publish job. publish.sh finds the draft, replaces its assets, and publishes it, without creating a second release.
  • A rerun after publishing: succeeds without changes when both published checksums match, and fails otherwise, including a missing engine checksum.
  • The tagged source is wrong: fix it on main through review and release the next version. Delete an unpublished draft for the bad tag in the GitHub interface if one remains; the tag stays.

Dry runs

Run the workflow manually to test it before tagging. A manual run builds and smoke-tests the selected targets from the chosen branch and never publishes; the publish job runs only for tag pushes. It skips the tag checks and reads the version from Cargo.toml.

gh workflow run release.yml --ref BRANCH -f targets=all    # or linux, native
gh run watch

Or use Actions → Release → Run workflow in the GitHub interface. The built archives remain downloadable from the run’s artifacts for 30 days.

Checking locally

scripts/release/local-check.sh runs the build and smoke jobs with Docker, without tagging, uploading, or publishing. It builds in the pinned image into target/release-container-ARCH/, separate from host builds, and smoke-tests the archives in that target’s pinned distributions. By default it selects the host architecture; PLENTY_RELEASE_TARGET explicitly selects one of the two triples. The native release build refuses a mismatched host rather than quietly relying on emulation. For example, on an ARM host:

PLENTY_RELEASE_TARGET=aarch64-unknown-linux-gnu scripts/release/local-check.sh --test

The engine builder accepts --target TRIPLE and --validate; a successful cross-build alone does not validate engine execution. Options:

OptionEffect
--testAlso run the full test suite in the build image
--skip-smokeOnly build, audit, and package

scripts/release/smoke-test.sh ARCHIVE runs the archive checks on the host. bash scripts/release/smoke-engine.sh ARCHIVE dataframe checks the bundled engine’s files and modes, optional linkage, discovery beside the compiler, a Parquet query, --dist relocation without the compiler, and the missing runtime library diagnostic. smoke-engine.sh ARCHIVE linalg runs the same checks with a linear solve. smoke-in-container.sh runs both after smoke-test.sh. Containers are labelled org.plenty.release-build and org.plenty.release-smoke and removed when the check ends.

Licenses in the archive

See release archive licenses for inventories and maintenance requirements.

Linux release glibc verification

Issue #730 asked whether the Linux release baseline was glibc 2.28 or 2.30. The v0.8.9 published archives support glibc 2.28 on both x86-64 and AArch64. The release text’s 2.28 requirement needs no correction. This audit was performed on 2026-11-05; it does not qualify unpublished builds.

Published artifact evidence

The downloaded v0.8.9 archives matched their published SHA-256 files:

ArchiveSHA-256
plenty-0.8.9-x86_64-unknown-linux-gnu.tar.gz249526b6536b503227e9b9e176398e74a0becddb8eb3db14a3d6e3b53134015a
plenty-0.8.9-aarch64-unknown-linux-gnu.tar.gz370bb7541ab3ebd2026ca0cbeb036814231bf32728296634c5f9153812c4c2da

For each architecture, scripts/release/check-elf.sh passed for bin/plenty, bin/plentyc, bin/libplenty_dataframe_v1.so and bin/libplenty_linalg_v1.so. All eight files have a highest referenced GLIBC version of 2.28 and GCC symbol version of 4.2.0. The 2.28 reference is statx, a weak undefined symbol. No file references GLIBC_2.30 or a later version. The audit also checked ELF architecture, loader, allowed dynamic libraries, absence of RPATH/RUNPATH and engine SONAMEs. Symbol inspection alone is not an execution test or proof that earlier glibc versions are supported.

Baseline execution

The published archives were tested with the existing scripts/release/smoke-in-container.sh harness, using the pinned AlmaLinux 8 images from scripts/release/toolchain.env:

ArchitectureImage manifest digestExecution
x86-64sha256:8b469a3a78515e8a18ea8fc727e6a3679e1d0c5ba6f58d5b30be3d0d9b86cffeNative x86-64 container
AArch64sha256:29fdedeed8a6707558c9e03b5d3e8f09eac5307cdd0811218d7d55a2cc9ae522ARM64 container under QEMU user emulation

Both containers reported AlmaLinux 8.10 and glibc 2.28. The harness installed only the documented gcc tar gzip prerequisites, disconnected the network, and ran the extracted archives without Rust or a source checkout. It checked both compiler version commands, compiled and executed a program using SQLite and SHA-256, ran a generated project, and repeated the checks after relocating the installation and installing a second copy. The dataframe and linalg smokes additionally exercised their native engines, optional linkage and app-only --dist relocation. All three unmodified smoke scripts passed on x86-64.

The stock ARM compiler smoke first passed compilation, execution and the project check, then failed its missing-linker diagnostic assertion. Under QEMU 10.2.2, invoking absent cc reported linker driver `cc` failed (exit status: 127) rather than the expected installation hint. A temporary copy of smoke-test.sh accepted either diagnostic for that one negative test; the nonzero-exit requirement and all other checks stayed unchanged. The repeated compiler smoke and both unmodified engine smokes passed. This adaptation is not a change to the release harness or evidence that the original ARM diagnostic assertion passed.

These are libc/userspace checks on host Linux 7.2.11-200.fc44.x86_64. They do not qualify the oldest supported kernel, native ARM concurrency or other ARM page sizes. They establish the supported 2.28 baseline for these artifacts; they do not claim that every older version fails or every library operation was exercised.

Current source and generated programs

The source inspected was main commit fefe4bae (the audit worktree base); the published tag was v0.8.9, locally resolved to f65fdda439f8af2e15289c441f5aa626ab59c808. Current build.sh requires glibc 2.28, both release build images are pinned manylinux_2_28 images, and check-elf.sh rejects release binaries above 2.28. This audit did not rebuild current main and does not substitute for its release qualification. The Linux build, ELF audit and smoke scripts used here are unchanged between that tag and the audit base.

The 2.30 reference in the release workflow explanation concerns the upstream ARM mold 3.0.0 build tool: its binary imports gettid@GLIBC_2.30. The ARM baseline build uses its image’s GNU linker instead. Mold is not shipped in the Plenty archive and is not an installation dependency. This tool requirement does not raise the compiler’s glibc baseline.

Programs link against the compiling machine’s C library and startup objects. A program built on a newer distribution may require a newer glibc even when its compiler runs on 2.28. Build on the oldest supported deployment system, and audit and execute the resulting program there. Programs using dataframe or linalg also need the matching engine; the shipped engines passed this 2.28 audit. Locally rebuilt or substituted engines have their own requirements. See installation requirements and native linking.

Repeating the check

Download both archives and their checksum files for the release being checked. From the download directory, verify each checksum, extract the archive, then change to the source checkout and run its audit for each target. Set target to x86_64-unknown-linux-gnu or aarch64-unknown-linux-gnu, and root to the absolute path of the extracted archive directory:

export PLENTY_RELEASE_TARGET="$target"
bash scripts/release/check-elf.sh "$root/bin/plenty" "$root/bin/plentyc"
for engine in dataframe linalg; do
    library="libplenty_${engine}_v1.so"
    bash scripts/release/check-elf.sh --engine "$library" "$root/bin/$library"
done

Run smoke-in-container.sh IMAGE ARCHIVE with each pinned baseline image and its matching archive. On an x86-64 host with ARM binfmt/QEMU already configured, set DOCKER_DEFAULT_PLATFORM=linux/arm64 for the ARM invocation. Keep the checksum file beside the archive. Record the release identity, hashes, image digests, audit output and smoke exit statuses; do not infer artifact compatibility solely from the source configuration.

Release archive licenses

Plenty is licensed MIT OR Apache-2.0 (license in both manifests). The archive ships LICENSE-MIT and LICENSE-APACHE from the repository root and THIRD-PARTY-NOTICES.md, generated by scripts/release/third_party_notices.py from cargo metadata. It lists every crate linked into the compiler or its embedded runtime (normal dependencies only; build scripts and procedural macros do not ship) with each crate’s own license files, plus the vendored SQLite amalgamation and Brotli codec, and the third-party code and data that standard library modules and the runtime embed. VENDORED_SOURCES lists source copied from a published package with its license files kept beside the copy: plenty-runtime/src/unicode/vendor holds unicode-normalization 0.1.25 (MIT OR Apache-2.0). EMBEDDED lists the rest: std.markdown derives from commonmark.js 0.31.2 (BSD-2-Clause) and carries tables generated from the Unicode Character Database 16.0.0 and the WHATWG HTML named character references (BSD-3-Clause for portions in source code), and the unicode-normalization tables derive from the Unicode Character Database 17.0.0 (Unicode License v3). std.diff derives its Myers and patience algorithms from the similar crate 3.2.1 (Apache-2.0). Their reviewed license texts are bundled in scripts/release/licenses/; update the entry and its text together when the upstream version changes. The compiler notices fail when a directory named vendor, or a subdirectory of plenty-runtime/native, has no notice: a child of a vendor directory named after a shipped crate (NAME-VERSION, as in plenty-runtime/vendor) is covered by that crate’s entry, and any other vendored directory needs a VENDORED_SOURCES or native-library entry. The Rust standard library’s notices come from the build toolchain’s share/doc/rust. The script fails when a shipped crate has no license file or reviewed fallback, so a new dependency’s license is reviewed before it ships. For iced-x86 1.21.0, whose published crate omits its license file, the generator uses a bundled copy from the upstream commit recorded in the crate’s .cargo_vcs_info.json. The fallback matches the package name, version, and declared license and includes the source URL in the notices. yaxpeax-arm 0.5.0, the A64 assembly decoder, declares 0BSD but omits a license file; its pinned fallback supplies the standard SPDX 0BSD terms without inventing a copyright notice. Both fallbacks identify their package version and source URL. Dependency upgrades require a fresh review if they still omit the license file; notice generation does not download fallback text during the release build.

For each engine’s NAME-notices/, third_party_notices.py --component NAME walks its own manifest/lockfile. The faer fork and its backend are Git dependencies pinned in the linalg lockfile, with their upstream notices; missing workspace-root licenses in the dependency family have version-pinned, reviewed fallbacks. Offline builds need the pinned Git checkout as well as Cargo’s registry cache. For dataframe, alloc-stdlib shares its repository’s BSD license with alloc-no-stdlib. polars-parquet-format 0.1.0 declares MIT AND Apache-2.0 but omits top-level license files; a version/license-pinned fallback preserves its upstream parquet-format-safe license texts and the published package’s author attribution. Its Thrift sources retain Apache Software Foundation license headers. The bundled fallback identifies its upstream revision, and dependency upgrades require review again.