Columnar lists

clist[T] stores each field of T in a contiguous column of native-width values. Choose it for repeated scans of a few fields in many rows. Ordinary list[T] remains useful for general objects and whole-record access.

T must be a fully defined, nonempty class containing only bool, signed or unsigned 8/16/32/64-bit integers, f32, or f64. Custom destructors and fallible initializers are rejected. Strings, nested classes, enums, nullable fields, collections and references are not supported. Creating a clist or copying or removing a row requires access to every field; field selection respects module privacy.

Construction and mutation

Capacity growth allocates replacement storage, copies each column, and publishes the new storage only after success. Failure preserves length, capacity and all existing values. Negative or overflowing capacity requests return AllocError.CapacityOverflow. Capacity may exceed the requested size.

AppendError.OutOfMemory(row) and AppendError.CapacityOverflow(row) preserve the complete submitted row for retry. Successful append moves its scalar fields into the columns without rerunning an initializer. An append within reserved capacity allocates nothing. The empty owner and all live rows are cleaned up on every ordinary exit; moving a clist transfers its storage.

Field replacement evaluates the RHS before its indices, following ordinary assignment. Enclosing owner indices precede the row index, and each is evaluated once. Augmented assignment evaluates these indices, reads its current value, then evaluates the RHS under a shared owner loan before writing. Out-of-bounds indices trap at the source position; copy_row and remove return Nothing instead.

remove accepts an i64 index; negative indices count from the end. An invalid index leaves the owner unchanged. The returned row is an independent ordinary T, assembled without rerunning its initializer. Removal shifts later rows left in every column. clear makes all columns empty. Both require mutable access, allocate nothing, and retain capacity for later appends. Neither changes the owner pointer or column starts. Removing a row requires access to every field, just like copy_row; clearing does not expose any field.

A slice reserves exactly the selected row count, then copies each column; a step-1 slice copies each column as one block. It reads every field, so it needs the same access as copy_row, and the receiver must be a named clist.

Column iteration and ownership

T.field is a compile-time selector accepted only by column. It is not a string lookup or a general reflection value. Each column plan borrows the entire clist until consumed. While that loan is live, reserve, append, remove, clear, field writes, movement and destruction of the owner are rejected. Several shared column plans can be zipped together. Their lengths and row order agree.

Columns yield scalar values, not &Field references. The source remains borrowed, but each element is loaded directly from its native-width buffer. Ordinary Plenty references address tagged value slots; treating packed bytes as those slots would be invalid. Use scalar callback parameters with existing map, filter, fold, zip and other local pipeline operations. Recreate a column plan for a second traversal. Plans cannot escape into stored values or suspended generators. Column iteration is currently rejected inside generator functions, even if the scan would finish before a yield.

items[i], &items[i], packed field references and direct row-method calls are unsupported: there is no contiguous T at that row position. copy_row creates an ordinary owned T on which methods can run; writes to that copy do not write back. Whole-clist copy, mutable column views and general row iteration are not part of this initial interface. Equality compares row values; formatting reports clist(len=N) without materializing rows.

Layout and cost

The current implementation uses one allocation for an owner header and all columns. Every column contains tightly packed elements (bool uses one byte). Capacity rounding aligns column starts; this is an internal layout, not an Arrow, Parquet, C ABI or serialization contract. Growth copies all columns, and can temporarily hold both old and new allocations.

Ordered removal moves (len - normalized_index - 1) values in each column, so its cost grows with the number of following rows and the total native row width. Removing the last row only gathers that row. Repeated removal near the front can be quadratic; this is not a bulk filtering operation. clear takes constant time because all fields are primitives with no destructors. It does not erase old bytes, shrink storage, or allocate; subsequent appends overwrite the newly live rows. Neither operation uses tombstones.

Column loops load the pointer and length before iteration. Checked arithmetic and floating-point rules are unchanged. Indexed field reads and writes use checked native-width loads and stores. copy_row checks its index once, then gathers the known fields directly into fresh ordinary row storage without runtime schema dispatch. Each access observes the current owner, so growth and mutation between accesses are safe. Whole-row workloads can still cost more than ordinary lists: gathering separated fields and moving the reconstructed object have costs of their own. The access measurements compare these paths with the initial implementation.

Automatic SIMD counts

With the default optimized profile, a column of i32 or u32 followed by one filter and count() uses four-lane SIMD when the predicate compares its borrowed argument against a constant or immutable by-value capture of the same type (==, !=, <, <=, >, >=). Either operand order is recognized. The predicate may also join two such comparisons with and, for example *x >= low and *x < high. Captured bounds may be runtime values; the closure retains its own snapshot. This bounded pattern accepts at most two captures, both of the column’s scalar type. Named functions and closures are recognized. For example: items.column(T.score).filter(def(x: &i32) -> bool: *x > 5i32).count(). The ordinary scalar program remains valid with --fast-compile.

The compiler proves this small pattern from the checked operations; it requires no user annotation, temporary collection, allocation, aligned column start or lane-count bookkeeping. Full groups use unaligned-safe vector loads; the scalar loop handles the remaining zero to three rows. Counts use 64-bit accumulators, including for columns larger than a 32-bit lane counter could represent. The generated instructions use the existing x86-64 SSE2 baseline, so this slice needs neither a newer CPU nor runtime feature dispatch.

Borrowed or mutable captures, or, more than two comparisons, checked arithmetic in predicates, additional adapters, other scalar types and other terminals retain scalar execution. Only total comparisons qualify: eagerly combining their SIMD masks cannot introduce effects or traps from a skipped and operand. Side effects, traps and floating-point order are unchanged. This is a bounded optimization, not general auto-vectorization; parallel scans remain unimplemented. See the validation and measurements.

See the tutorial and the design rationale and measurements.