Encode bytes and pack numbers

Binary formats store integers at fixed widths in a stated byte order, and text channels carry binary data as hex or base64. std.encoding does both over std.bytes buffers.

import std.bytes
import std.encoding
from std.encoding import Endian

def main() -> Result[(), AllocError | IoError | std.encoding.EncodingError | std.bytes.BytesError]:
    mut frame = std.bytes.empty()?
    std.encoding.append_u16(&mut frame, 513u16, Endian.Big)?
    std.encoding.append_u32(&mut frame, 7u32, Endian.Little)?
    frame.extend(&std.bytes.from_text("hi")?)?
    print(std.encoding.hex_text(&frame)?)?
    print(std.encoding.base64_text(&frame)?)?
    print(std.encoding.base64url_text(&frame[:7])?)?
    print(std.encoding.unpack_u16(&frame, 0, Endian.Big), std.encoding.unpack_u16(&frame, 0, Endian.Little))?
    std.encoding.pack_u32(&mut frame, 2, 8u32, Endian.Little)
    print(std.encoding.unpack_u32(&frame, 2, Endian.Little), frame.len())?
    Ok(())

0201070000006869
AgEHAAAAaGk=
AgEHAAAAaA
513 258
8 8

Every integer function names its width and takes the byte order as an argument; there is no default order. Endian.Big stores the most significant byte first. The same two bytes read as 513 or 258 depending on the order you state. append_ grows the buffer by the integer’s width. pack_ overwrites bytes that already exist and never grows it. unpack_ reads at an offset.

Mixed integer and float fields

Use f32 or f64 when the format specifies IEEE 754 binary32 or binary64. The same append, overwrite, and offset rules apply. A numeric cast to an integer changes the value; these codecs preserve its float representation:

import std.bytes
import std.encoding
from std.encoding import Endian

def main() -> Result[(), AllocError | IoError | std.encoding.EncodingError]:
    mut record = std.bytes.with_capacity(14)?
    std.encoding.append_u16(&mut record, 1u16, Endian.Big)?
    std.encoding.append_f32(&mut record, 1.5f32, Endian.Big)?
    std.encoding.append_f64(&mut record, -2.5, Endian.Little)?
    print(std.encoding.hex_text(&record)?)?
    print(std.encoding.unpack_u16(&record, 0, Endian.Big))?
    print(std.encoding.unpack_f32(&record, 2, Endian.Big))?
    print(std.encoding.unpack_f64(&record, 6, Endian.Little))?
    Ok(())

00013fc0000000000000000004c0
1
1.5
-2.5

Widths count bytes, so the fields begin at offsets 0, 2, and 6. Floats need no alignment padding. The float packing reference explains signed zero, infinities, NaNs and exact bit preservation.

The codecs take a byte view: &buffer for a whole buffer, &buffer[start:stop] for part of one, or a b"..." literal. The _text forms return a str; hex_encode, base64_encode, and base64url_encode return the encoded bytes as a new buffer. base64 uses + and / and always pads with =. base64url uses - and _ and never pads.

Ordinary decoding accepts exactly one form and reports what is wrong with anything else. Nothing is skipped or repaired, including spaces and line breaks:

import std.bytes
import std.encoding
from std.encoding import Endian

def main() -> Result[(), AllocError | IoError | std.bytes.BytesError | std.encoding.EncodingError]:
    print(std.encoding.base64_from_text("aGk=")?.to_text()?)?
    print(str.repr(std.encoding.hex_from_text("abc"))?)?
    print(str.repr(std.encoding.hex_from_text("0g"))?)?
    print(str.repr(std.encoding.base64_from_text("aGk"))?)?
    print(str.repr(std.encoding.base64_from_text("aG k"))?)?
    print(str.repr(std.encoding.base64_from_text("aGl="))?)?
    print(str.repr(std.encoding.base64url_from_text("aGk="))?)?
    Ok(())

hi
Result[std.bytes.Bytes, std.encoding.EncodingError].Err(std.encoding.EncodingError.InvalidLength)
Result[std.bytes.Bytes, std.encoding.EncodingError].Err(std.encoding.EncodingError.InvalidByte(1))
Result[std.bytes.Bytes, std.encoding.EncodingError].Err(std.encoding.EncodingError.InvalidLength)
Result[std.bytes.Bytes, std.encoding.EncodingError].Err(std.encoding.EncodingError.InvalidByte(2))
Result[std.bytes.Bytes, std.encoding.EncodingError].Err(std.encoding.EncodingError.NonCanonical(2))
Result[std.bytes.Bytes, std.encoding.EncodingError].Err(std.encoding.EncodingError.InvalidPadding(3))

The number in an error is the index of the offending byte. aGl= is NonCanonical because its last symbol carries bits that belong to no byte: aGk= is the only accepted text for hi. Reading four bytes from a three-byte buffer traps. Validate untrusted record lengths before unpacking; Cursor field reads return Truncated when input can legitimately end before a field is complete.

Decoded bytes are not text. Call to_text() on the result when you expect UTF-8; it validates and can fail on its own.

Base32 and protocol-specific forms

Choose the encoding the receiving protocol expects. Base32 uses uppercase letters and digits 2 through 7, with padding. For a protocol requiring a different alphabet or no padding, build a validated encoding once and reuse it:

import std.bytes
import std.encoding

def main() -> Result[(), IoError | AllocError | std.bytes.BytesError | std.encoding.EncodingError]:
    data = std.bytes.from_text("hello")?
    print(std.encoding.base32_text(&data)?)?
    lower = std.encoding.custom("abcdefghijklmnopqrstuvwxyz234567", Nothing)?
    text = lower.text(&data)?
    print(text, lower.from_text(&text)?.to_text()?)?
    print(std.encoding.base64mime_from_text("aG Vs\r\nbG8=")?.to_text()?)?
    Ok(())

NBSWY3DP
nbswy3dp hello
hello

base32hex offers the alternate alphabet 0–9, A–V. For email body transfer, base64mime_text wraps padded base64 at 76 columns using CRLF between lines. Only the MIME decoder skips ASCII space, tab, CR and LF; other invalid characters, incorrect padding and nonzero trailing bits still fail. It does not parse MIME headers or append a final CRLF. Ordinary base64_from_text continues to reject all whitespace.

See the encoding reference for every operation, its errors, and its costs.