Documentation

A shared UTF-8 byte buffer plus n+1 row offsets (base row r spans bytes [offsets!r, offsets!(r+1))); validity lives in the column's bitmap. ptSel is an optional selection layer letting a gatherjoinsort result share the buffer.

ptCanonicalSel marks a selection that is a canonical dictionary encoding: equal byte slices always map to the same base row (codes). Set by dictionary compaction; preserved by gather/take over an already-canonical selection (a row keeps its code); False for a gather over an unselected base, where two logical rows can select different but equal-byted base rows. Grouping keys on codes directly when it holds.

Row byte-offsets, physically Int32 when every value fits (total buffer bytes < 2^31) and Int otherwise. Values are non-negative byte positions.

A selection layer mapping logical rows to base rows; -1 marks an invalid/null row. Int32 when the base row count fits.

Offset at index i, widened to Int.

Number of offset entries (row count + 1).

Base row for logical row i (may be -1).

Build a contiguous packed payload (no selection): the freeze-path shape.

mkPackedContiguous from offsets already produced at Int32 width.

Narrow an Int offset vector when the final offset (total bytes) fits.

Narrow an Int base-row vector (-1 sentinels allowed) when the base row count fits in Int32.

Reindex a packed payload by a selection vector, sharing the byte buffer; logical row i becomes base row indices!i. A negative or out-of-range index decodes to the empty slice. Composes with an existing selection; canonicality survives composition (a kept row keeps its code) but not a first selection over the unselected base.

Take the first k logical rows, sharing the byte buffer via a capped selection layer. O(k), no byte copy or decode — cheap take/display on a large packed column.

The shared buffer + contiguous n+1 offsets when the payload is the unselected base; a selected (gathered) payload returns Nothing (its rows are non-contiguous). Lets contiguous consumers skip the selection indirection.

Row count: length sel when selected, else length offsets - 1.

Raw byte slice for logical row i: (buffer, offset, length). The hot accessor.

On-demand single Text for row i, using the same validate-or-lenient decode as the freeze path so output is bit-identical.

Byte-wise equality of two slices. UTF-8 is injective on valid scalar sequences and lenient decode is deterministic, so this agrees with Text's == on the decoded values.

Unsigned byte-lexicographic comparison (memcmp semantics). For well-formed UTF-8 this matches compare exactly, since UTF-8 byte order equals codepoint order for all valid scalars.