std.toml

import std.toml parses TOML 1.0 documents into a value tree and writes a tree back as TOML text, under explicit limits, with one error type, TomlError.

For editing human-maintained files, document_text and document create a Document that preserves the source layout. Use the value tree writer below for generated files.

The module is checked Plenty code with no native primitives of its own. It reads and writes std.bytes buffers and std.io streams, and is available on the supported x86-64 Linux GNU target.

import std.toml
from std.toml import Toml

def main() -> Result[(), AllocError | IoError | std.toml.TomlError]:
    bounds = std.toml.limits()
    config = std.toml.parse_text("title = \"demo\"\n\n[server]\nport = 8080\nhosts = [\"a\", \"b\"]\n", &bounds)?
    match &config:
        case Toml.Table(entries):
            for entry in entries:
                match &entry.value:
                    case Toml.Text(text):
                        print(entry.key, "is", text)?
                    case Toml.Table(fields):
                        print(entry.key, "has", len(fields), "entries")?
                    case _:
                        print(entry.key, "is another type")?
        case _:
            print("not a table")?
    print(std.toml.encode_text(&config, &bounds)?.rstrip()?)?
    Ok(())

title is demo
server has 2 entries
title = "demo"

[server]
port = 8080
hosts = ["a", "b"]

Accepted text

The parser accepts TOML 1.0 and rejects everything else, including TOML 1.1 additions.

Tables follow the definition rules of the specification:

A header may name a table that an earlier header created implicitly, such as [a] after [a.b], once. A header path that passes through an array of tables continues in its last table.

Value tree

pub enum Toml:
    Bool(bool)
    Int(i64)
    Float(f64)
    Text(str)
    DateTime(DateTime)
    Array(list[Toml])
    Table(list[Entry])

pub class Entry:
    pub key: str
    pub value: Toml

pub enum DateTimeKind:
    OffsetDateTime
    LocalDateTime
    LocalDate
    LocalTime

pub class DateTime:
    pub kind: DateTimeKind
    pub year: i64
    pub month: i64
    pub day: i64
    pub hour: i64
    pub minute: i64
    pub second: i64
    pub nanosecond: i64
    pub offset_minutes: i64

A document is a Toml.Table. Tables keep their entries in the order each key first appeared in the text, including keys added later by a header, so [a.b] before [a] gives a the entry b first. An array of tables is an Array of Table values. DateTime fields that its kind does not use are 0; offset_minutes is minutes east of UTC, and Z is 0.

read_value reads chunks of up to 4096 bytes, retries Interrupted, and treats a zero-length read as end of input. It reads at most one byte past max_input before reporting InputTooLarge.

Writing

The writer produces text that parses back to the same values:

• Each table lists its plain entries as key = value lines first, then each table entry as a [path] section and each non-empty array whose elements are all tables as [[path]] sections. A section header is omitted for a non-empty table whose entries are all sections. Inside one group, entries keep their order. The plain-first order is required: a plain key written after a header would belong to that header’s table.
• Other arrays and tables nested in them are written inline on one line.
• Keys are bare when they are non-empty and use only bare-key bytes; otherwise they are basic strings.
• Strings are basic strings. ", \, and control characters are escaped (\b \t \n \f \r, otherwise \u00XX in uppercase hex); every other scalar is written as UTF-8.
• Floats are the shortest text that parses back to the same value, as str.from, with .0 added when that text has no point or exponent; infinities and NaN are inf, -inf, and nan.
• Date-times use T, Z for offset 0, and the shortest fraction that keeps the nanoseconds.

Comments, blank lines, key quoting, number spellings, and inline-or-section choices of the source are not kept. The writer rejects a root that is not a table (NotATable), a table with a repeated key (DuplicateKey), and a DateTime whose fields are out of range or whose unused fields are not 0 (InvalidDateTime).

Limits

pub class Limits:
    pub max_depth: NonNegative[i64]
    pub max_input: NonNegative[i64]
    pub max_output: NonNegative[i64]

limits() returns Limits(64, 67108864, 67108864). The fields are public; adjust a copy. A literal converts on assignment; a computed value needs NonNegative.try_from(value)?. Every function takes &Limits and traps when max_depth is above 1024.

The depth limit also bounds the parser’s and writer’s recursion, so neither a document nor a tree built by hand can overflow the stack.

Input cursor advances consume validated bytes, and lookahead beyond the integer representation is treated as beyond the input. Bounded cursor, digit and calendar arithmetic uses explicit opt-outs with local range proofs. Stream reads check the full allocation extent, including the extra byte used to detect oversize input, before resizing or invoking the reader; an unrepresentable extent reports Allocation(AllocError.CapacityOverflow). Existing parser, output-size and atomic document-edit errors remain unchanged.

Errors

pub class TomlError:
    pub fault: TomlFault
    pub offset: i64
    pub line: i64
    pub column: i64

For parsing, offset is the byte offset from 0, and line and column start at 1. Only \n ends a line. A column is one Unicode scalar, not one byte. If a one-based line or column cannot fit in i64, the fault is InputTooLarge at the still-valid byte offset, with both line and column 0 meaning unavailable. Coordinates are not saturated or wrapped. A syntax fault points at the offending byte. A fault about a whole key or header (DuplicateKey, DuplicateTable, KeyConflict, and a DepthExceeded that a key or header causes) points at the first byte of its key or [. A DepthExceeded that an array or inline table causes points at its opening bracket, and a fault about a number or date-time at the value’s first byte. UnexpectedEnd points one past the last byte. An allocation failure before scanning starts points at offset 0, line 1, column 1; one while assembling the value after a successful scan points one past the last byte.

For writing and for stream failures, offset is the number of bytes produced, written, or read when the call failed, and line and column are 0. NotATable reports 0, 0, 0.

Partial trees and buffers are released before an error returns.

Ownership and threads

A Toml owns its whole tree; moving it moves everything and dropping it releases everything. There is no shared or global state, so trees can be moved into a worker thread. parse copies its source and keeps no reference to it.

Toml is recursive, so the builtins that require a finite layout walk reject it: ==, copy, print, and str.repr do not accept a Toml. Compare and display trees through encode_text. DateTime is an ordinary class and can be printed and compared.

Costs

The parser takes runs of plain string, comment, and bare-key bytes with one std.bytes scan each. A table lookup goes through an index keyed by parent and name, so a table with many keys costs the same per key as a small one. Measured with a release compiler on x86-64 Linux, parsing and then writing a 2,688,896-byte document of 20,000 [[item]] tables and one table of 20,000 keys takes about 2 seconds, close to std.json on input of the same size and shape.

Limits of this module

• The value-tree writer normalizes formatting; use Document for edits that preserve unrelated source text.
• No event or streaming interface: the whole document and its tree are held in memory. TOML’s tables can be extended anywhere in a file, so no part of the tree is final before the end of input.
• Integers are i64 and floats f64, as TOML 1.0 requires of implementations; a larger integer is an error, never rounded.
• No tree equality, path lookup, or typed decoding helpers; a typed reader is written by hand over the tree with find.
• TOML 1.1 additions (newlines and trailing commas in inline tables, \e, \xHH, optional seconds) are rejected.