std.csv

import std.csv reads and writes delimited records over std.io byte streams. A Reader fills one reusable Record per call from a fixed buffer. A Writer encodes one record at a time. Neither loads a whole file, and every size is bounded by the Dialect.

The module is checked Plenty over std.bytes and std.io, with no native calls of its own. It is available on the supported x86-64 Linux GNU target.

import std.bytes
import std.csv
import std.io
from std.csv import Newline

def main() -> Result[(), AllocError | IoError | ParseError | std.bytes.BytesError | std.csv.CsvError | OverflowError[i64]]:
    text = "item,price\r\npen,3\r\n\"book, large\",12\r\n"
    mut input = std.csv.reader(std.io.cursor(std.bytes.from_text(text)?), &std.csv.rfc4180(), 4096)?
    mut record = std.csv.record()?
    input.read(&mut record)?
    mut total = 0
    while input.read(&mut record)?:
        total += i64.parse(&record.text(1)?)?
    mut dialect = std.csv.rfc4180()
    dialect.newline = Newline.Lf
    mut output = std.csv.writer(std.io.stdout(), &dialect)?
    output.texts(&["rows", "total"]?)?
    output.texts(&[str.from(input.records() - 1)?, str.from(total)?]?)?
    output.flush()?
    Ok(())

rows,total
2,15

Format

The grammar is RFC 4180 with a configurable delimiter and quote byte. There is one accepted form; nothing is trimmed, skipped, or repaired.

• A record ends at LF, CRLF, or CR. The last record may end at end of input.
• Fields are separated by the delimiter. a, has two fields, the second empty.
• An empty line is one record with one empty field. Empty input has no records.
• A field that starts with the quote byte is quoted and ends at the next unpaired quote. Inside it, the delimiter, CR, and LF are data, and two quotes are one quote byte.
• A closing quote must be followed by the delimiter, a record end, or end of input.
• A quote byte anywhere else in an unquoted field is an error.
• Spaces are data, including those next to a delimiter or quote.

There are no headers, comments, or escape characters other than the doubled quote. The first record is an ordinary record.

Dialect

Dialect is a class with public fields. std.csv.rfc4180() returns the values in the third column; change fields before creating a reader or writer.

delimiter and quote must be different ASCII bytes other than CR and LF, each limit at least one, and Width.Exactly(count) at least one. Otherwise reader and writer return InvalidDialect. A reader accepts all three record ends whatever newline says, and ignores quoting.

Fields are bytes. The module never decodes them on its own: Record.text validates one field as UTF-8 when asked, and a writer emits the bytes it is given. Other encodings are read with Record.bytes and converted by the program.

Errors

Position locates the byte a reader stopped at:

A limit fault points at the first byte that does not fit. Record numbers and field indexes in the other variants use the same bases.

Reading

reader takes ownership of inner, copies the dialect, and allocates one buffer of exactly capacity bytes, which never grows. capacity is Positive[i64], so 0 is a compile error. inner needs the Reader method of std.io. The reader does its own buffering, so inner needs no BufReader.

read retries Interrupted and continues across short reads; a record may span any number of buffer fills. After False, the record is empty and every later call returns False without reading the stream again.

Record, line and byte positions are semantic i64 counters, not saturating diagnostics. If the next position is unrepresentable, reading returns Stream(StreamError.OutOfRange), empties the record and stops the reader. Record number i64.MAX is still processable when the preceding count is i64.MAX - 1. A failing line increment leaves the parser cursor before that newline; CRLF still counts as one line, inside and outside quotes. The inner stream may already have read ahead: parser consumption is distinct from transport progress. Buffer fills are capped by the remaining representable byte span; at zero room, a reader can refuse before an EOF probe.

A Width error is reported after the whole record has been read: record holds it, and the next read continues with the following record. Every other error empties record and stops the reader; later calls return Stopped. A stopped reader cannot resynchronize, because a malformed quote leaves no reliable record boundary.

inner is a public field, for calls such as input.inner.close(). Reading it directly skips bytes the reader already holds.

Records

A Record stores the unescaped fields end to end in one buffer and one list of field ends. A reader overwrites it, keeping its capacity.

An index outside 0 <= index < len() traps before reading or appending. The message names Record.start or Record.stop and index outside record; bytes, text, and append_to validate through those position methods.

data() is the borrowed view. It and the offsets describe the current contents only. Reader.read needs &mut Record, so the compiler rejects a program that still holds the view across the next read. text, texts, bytes, and append_to return or fill storage the caller owns, which survives any number of reads.

Writing

writer takes ownership of inner, which needs the Writer methods of std.io, and copies the dialect.

A record in progress lives in the writer’s buffer and reaches the stream only at end_record, as one run of writes that retries Interrupted and short counts. A record with one empty field is written as two quote bytes, so that it is not an empty line. end_record with no fields is EmptyRecord.

Failures leave a defined state:

• Rejected and Allocation from field or text add nothing; the record in progress keeps its earlier fields.
• Width from end_record discards the record in progress. Nothing of it was written. With Width.First, the first finished record sets the count.
• A transport Stream error from end_record means the record is finished and counted, and its undelivered bytes are held. The next field, end_record, or flush delivers them first and fails again if the stream still refuses.
• record and texts discard the record in progress on any error. A Stream error still leaves the finished record held.

Position exhaustion is distinct from that ordinary transport retry: after record i64.MAX is finished, field, text and end_record reject an unrepresentable next record as Stream(StreamError.OutOfRange) before encoding or implicitly draining any pending output. Already-finished pending bytes and their count remain intact; explicit flush still delivers them. No counter wraps, saturates, or adds a public arithmetic error type. Internal field/content and pending-buffer arithmetic uses validated remaining budgets and held-buffer bounds; scan endpoints clamp room before adding it to a cursor.

Nothing is written when a Writer is dropped. End the last record and call flush. The writer does not close inner; it is the public field inner.

Ownership and threads

A reader or writer owns its stream and has one user at a time: every operation that moves data takes &mut self. A Record is an ordinary owned value, independent of the reader that filled it, and can be moved to another thread. The module holds no global state and adds no synchronization.

Costs

Reading examines each input byte once and copies field bytes into the record in runs. Writing examines each field byte once to decide quoting, and once more when quoting to double quote bytes. Both are linear in the bytes processed. Field access by index is constant time.

The reader and writer find the next delimiter, quote, or line break with the std.bytes scans, so the work in Plenty code is per field and per quote, not per byte. Measured with a release compiler on x86-64 Linux, pinned to one core, reading 64 MB from standard input into one reused Record:

Short fields are bounded by the cost of the calls made per field, about 600 ns, not by scanning.

A Record holds at most max_record bytes of content and max_fields offsets. A reader consumes at most 2 * max_record + 3 * max_fields + 2 bytes for one record before returning it or a limit fault. A writer holds at most one finished record that the stream refused and one record in progress, each at most that many bytes. Storage grows geometrically, so capacity can be up to twice the content.

Limits

• Only x86-64 Linux GNU is supported.
• There is no lenient mode: stray quotes, and data after a closing quote, are errors, and a reader does not continue after them.
• There are no header mapping, typed columns, comment lines, or whitespace trimming. Convert fields with the type’s parse method.
• Only the UTF-8 byte-order mark is recognized, and only with skip_bom. A writer never emits one. Other encodings need conversion by the program.
• delimiter and quote are single ASCII bytes.
• A stream error that could be retried, such as EAGAIN on a nonblocking descriptor, stops the reader like any other.
• End of input is final: a reader does not look for data appended later.
• Writer.text copies the text once to reach its bytes. Use field with a reused Bytes to avoid the copy.