std.json

import std.json decodes and encodes JSON (RFC 8259) under explicit limits, with one error type, JsonError. It has two levels:

• Events: a resumable Decoder and a validating Encoder that never hold the document. Typed codecs are ordinary functions over them.
• Value tree: Json, built from and written through the same events.

The module is checked Plenty code with no native primitives of its own. It reads and writes std.bytes buffers and std.io streams, and is available on the supported x86-64 Linux GNU target.

import std.bytes
import std.json
from std.json import Json

def main() -> Result[(), AllocError | IoError | std.bytes.BytesError | std.json.JsonError]:
    bounds = std.json.limits()
    value = std.json.parse_text("{\"name\": \"Ada\", \"langs\": [\"en\", \"fr\"]}", &bounds)?
    match &value:
        case Json.Object(members):
            for member in members:
                match &member.value:
                    case Json.Text(text):
                        print(member.key, "is", text)?
                    case Json.Array(items):
                        print(member.key, "has", len(items), "items")?
                    case _:
                        print(member.key, "is another type")?
        case _:
            print("not an object")?
    mut out = std.json.encoder(&bounds)?
    out.start_object()?
    out.key("id")?
    out.integer(7)?
    out.key("person")?
    out.value(&value)?
    out.end_object()?
    print(out.take()?.to_text()?)?
    Ok(())

name is Ada
langs has 2 items
{"id":7,"person":{"name":"Ada","langs":["en","fr"]}}

Accepted text

The decoder accepts exactly RFC 8259 and one document per input.

The encoder writes compact text: no whitespace, members in call order. In strings it escapes ", \, and bytes below 32 (\b \f \n \r \t, otherwise \u00xx in lowercase); every other scalar, including / and non-ASCII, is written as UTF-8.

Limits

pub class Limits:
    pub max_depth: NonNegative[i64]
    pub max_input: NonNegative[i64]
    pub max_output: NonNegative[i64]
    pub duplicate_keys: DuplicateKeys   # Reject or Keep

limits() returns Limits(64, 67108864, 67108864, DuplicateKeys.Reject). The fields are public; adjust a copy. A literal converts on assignment; a computed value needs NonNegative.try_from(value)?. Every constructor and whole-document function takes &Limits; decoder and encoder trap when max_depth is above 1024, so the whole-document functions built on them do too.

The depth limit also bounds the encoder’s recursion over a Json tree, so a tree built by hand cannot overflow the stack.

Input and output counters are bounded by the remaining declared budget before addition. Taking or writing output does not reset that budget. Bounded cursor, hex-digit, surrogate and UTF-8 state arithmetic uses explicit, locally justified unchecked: scopes; malformed values still return the JSON faults below, not arithmetic errors. No public arithmetic error type is added.

Errors

pub class JsonError:
    pub fault: JsonFault
    pub offset: i64
    pub line: i64
    pub column: i64

For decoding, offset is the byte offset from 0, and line and column start at 1. Only \n ends a line. A column is one Unicode scalar, not one byte. A syntax fault points at the offending byte; a fault found after a token was read (DuplicateKey, NumberOutOfRange) points at the token’s first byte. UnexpectedEnd points one past the last byte. Position arithmetic is checked: if the next byte or ASCII run would make an offset, line, or column unrepresentable as i64, decoding returns InputTooLarge at the current position without consuming that byte or run. This also applies when max_input itself is i64.MAX; the one-based line and column need their own representation checks.

For encoding, offset is the number of bytes produced when the call failed and line and column are 0. A failed constructor reports 0, 0, 0.

After any error, a decoder or encoder is poisoned: every later call reports InvalidState. Partial trees and buffers are released before the error returns.

Events

pub enum Event:
    Pending
    End
    Null
    Bool(bool)
    Number(str)
    Text(str)
    Key(str)
    StartArray
    EndArray
    StartObject
    EndObject

Number holds the validated source text, so a codec chooses the type and loses nothing: i64.parse, u64.parse, f64.parse, or the text itself. Text and Key hold the unescaped string.

Decoder

decoder(bounds: &Limits) -> Result[Decoder, JsonError].

next returns Pending when every fed byte has been examined and the document is incomplete; feed more or call finish. After finish it returns End once the document and trailing whitespace are consumed, and End again on every later call. A number at the end of the fed input stays pending until the next byte or finish shows where it ends. Input may be split anywhere, including inside an escape or a UTF-8 sequence.

read_event[R: Reader](source: &mut Decoder, reader: &mut R) -> Result[Event, JsonError] drives a decoder from a stream. It reads up to 4096 bytes whenever next is Pending, retries Interrupted, treats a zero-length read as end of input, and never returns Pending. Do not mix it with feed on one decoder.

A typed codec reads the events it expects and uses reject for values it will not accept:

def coordinate(decoder: &mut std.json.Decoder) -> Result[i64, JsonError]:
    match decoder.next()?:
        case Event.Number(text):
            match i64.parse(text):
                case Ok(value):
                    Ok(value)
                case Err(_):
                    Err(decoder.reject(JsonFault.NumberOutOfRange))
        case _:
            Err(decoder.reject(JsonFault.UnexpectedByte))

Encoder

encoder(bounds: &Limits) -> Result[Encoder, JsonError]. Every writing method takes an exclusive receiver and returns Result[(), JsonError].

The encoder accepts only call sequences that form one document: a key outside an object, a value where a key belongs, a mismatched close, or a second top-level value is InvalidState. It does not check member names for duplicates.

write_pending[W: Writer](source: &mut Encoder, writer: &mut W) -> Result[(), JsonError] writes every pending byte with std.io.write_all and keeps the buffer’s capacity. It does not flush. If the writer fails, the bytes it accepted are removed and the rest stay pending.

Value tree

pub enum Json:
    Null
    Bool(bool)
    Int(i64)
    Float(f64)
    Text(str)
    Array(list[Json])
    Object(list[Member])

pub class Member:
    pub key: str
    pub value: Json

A number without a fraction or exponent becomes Int and must fit i64; otherwise the document is rejected with NumberOutOfRange, never rounded. Any other number becomes Float, the nearest f64, and must be finite: 1e309 is NumberOutOfRange, while 1e-400 is 0.0. -0 is Int(0); -0.0 keeps its sign. Float values are written with a fraction or exponent (100.0, 1e21), so a tree keeps its Int/Float distinction through a round trip.

Objects keep members in source order, including duplicates under Keep. find is a linear scan; build a dict when an object is searched often.

Ownership and threads

A Json owns its whole tree; moving it moves everything and dropping it releases everything. Json, Decoder, and Encoder are single-owner values with no shared or global state. They can be moved into a worker thread, and the module’s functions pass the ordinary worker-effect checks. One decoder or encoder cannot be used from two threads at once, since every operation needs an exclusive loan.

Sources are shared loans: feed and parse copy what they need and keep no reference to the caller’s bytes. Events own their strings.

Json is recursive, so the builtins that require a finite layout walk reject it: ==, copy, print, and str.repr do not accept a Json. Compare and display trees through encode_text.

Costs

The decoder and the tree builder use no recursion. Encoder.value recurses once per nesting level, at most max_depth (1024) deep.

An event loop that drains next to Pending before each feed, or uses read_event, holds only the current chunk, the current token, and the depth stack, whatever the document size. A tree costs more than its text: each value is an enum cell and each string and container its own allocation.

The decoder takes a run of unescaped ASCII string content, and the encoder a run of bytes that need no escape, with one std.bytes scan each. Structural bytes, whitespace, numbers, literals, escapes, and non-ASCII string bytes still go through a checked single-element Bytes call. Measured with a release compiler on x86-64 Linux, pinned to one core, on an 8,388,872-byte document of 34,393 objects (687,862 events):

That is roughly 2 to 5 megabytes per second. The remaining time is the calls made per token and per event, not byte access.

Limits

• Compact output only: no pretty-printing or key sorting.
• One document per input: no JSON Lines or concatenated documents. Split the input and use one decoder per document.
• The tree holds integers as i64 and other numbers as f64. Larger integers and exact decimals are available only as Event.Number text.
• write_value builds the whole encoding before writing. Use an Encoder with write_pending for bounded output memory.
• No derived codecs: a typed codec is written by hand over events.
• No tree equality or path lookup helpers.