Build and normalize bounded text

std.text offers a reusable UTF-8 builder. Its limit counts bytes: é takes two bytes and 🙂 takes four. A failed append leaves the old contents intact.

import std.text
import std.unicode

def main() -> Result[(), AllocError | IoError | std.text.TextError]:
    mut message = std.text.builder(32)?
    message.append("Hello, ")?
    message.append("é🙂")?
    print(message.to_text()?, message.byte_len())?
    message.clear()
    message.append("again")?
    print(message.finish()?)?
    words = ["one", "two", "three"]?
    print(std.text.join(&words, " / ", 32)?)?
    print(str.repr(std.text.hard_wrap("é🙂abc", 2, 32)?)?)?
    print(std.unicode.version())?
    print(std.unicode.nfc("é", 16)? == "é")?
    print(std.unicode.nfkc("ﬃ", 16)?)?
    Ok(())

Hello, é🙂 13
again
one / two / three
"é🙂\nab\nc"
17.0.0
True
ffi

Use to_text() for a snapshot when keeping the builder, and finish() when done. finish consumes it even if allocating the final string fails; to_text keeps it available for retry. Both currently copy the output bytes.

hard_wrap counts Unicode scalars, preserves LF, and can split a combining sequence; it does not measure terminal columns or find word boundaries. Normalization has a separate scalar limit covering the input and decomposed intermediate. For example, decomposing é needs room for two scalars even though NFC can compose them back to one. Every operation remains fallible: use ? or match TextError to handle allocation, invalid limits, and output that exceeds them.

Normalization never happens automatically. NFC treats canonically equivalent spellings consistently; NFKC also removes compatibility distinctions such as the ligature above. Choose it only when that loss is intended. Byte decoding still rejects malformed UTF-8 before normalization; embedded NUL is ordinary text. See std.text and std.unicode for costs and complete contracts.

For prose, use fill to wrap at word boundaries. shorten collapses whitespace and truncates to one line; dedent removes a common space/tab margin. These helpers use default input and output caps of 1 MiB, which you can change with max_input_bytes and max_bytes.

import std.text

def main() -> Result[(), IoError | std.text.TextError]:
    print(std.text.fill("one two three four", 9)?)?
    print(std.text.shorten("one two three four", 13)?)?
    print(std.text.indent(&std.text.dedent("    first\n      second")?, "> ")?)?
    Ok(())

one two
three
four
one two [...]
> first
>   second

Widths count scalars rather than display columns. Use a reusable TextWrapper when you need indentation, tab options, or a maximum number of lines. Wrap each paragraph separately: the default settings treat input newlines as spaces.

Keyboard shortcuts

Plenty

Build and normalize bounded text