std/buffer

A Buffer builds bytes from pieces, and a ByteReader reads them back in order, as binary file formats and network protocols need:

import "std/buffer"

Buffer out = buffer()
write_u16_be(out, 0xCAFE)
write_uleb128(out, 300)
append_str(out, "hi")
bytes message = build(out)              // b"\xca\xfe\xac\x02hi"

ByteReader input = reader(message)
u16 tag = read_u16_be(input)?
u64 size = read_uleb128(input)?
bytes rest = read_exact(input, 2)?

Building:

Function	Result
`buffer()`, `buffer_with_capacity(n)`	an empty buffer; the second has room for `n` bytes before it grows
`append(buf, data)`	adds bytes
`append_byte(buf, byte)`	adds one byte, from 0 to 255
`append_str(buf, text)`	adds the UTF-8 bytes of text
`buffer_length(buf)`	how many bytes it holds
`build(buf)`	the bytes so far; the buffer can go on growing

Numbers of fixed size are written with write_u8, write_i8, and for 16, 32, and 64 bits write_u16_le, write_u16_be, write_i16_le, write_i16_be, and so on, and floats with write_f32_le, write_f32_be, write_f64_le, and write_f64_be. _le puts the least significant byte first (little-endian) and _be the most significant (big-endian, the order of network protocols). Floats are stored in their IEEE 754 form.

Numbers of variable size take fewer bytes when they are small:

Function	Form
`write_uleb128(buf, n)`	unsigned LEB128, seven bits a byte, the lowest first
`write_sleb128(buf, n)`	signed LEB128
`write_quic_varint(buf, n)`	a QUIC variable-length integer (RFC 9000): 1, 2, 4, or 8 bytes, up to `QUIC_VARINT_MAX`, 2^62 - 1

Reading, from reader(data):

Function	Result
`read_u8(r)`, `read_u16_be(r)`, ..., `read_f64_le(r)`	the number, as written by the matching `write_`
`read_uleb128(r)`, `read_sleb128(r)`, `read_quic_varint(r)`	a number of variable size
`read_exact(r, n)`	the next `n` bytes, shared with the data
`read_until(r, byte)`	the bytes before the next `byte`, which is read too
`peek(r)`	the next byte, which stays to be read
`remaining(r)`, `at_end(r)`	how many bytes are left, and whether none are

Each read returns a Result: when the bytes run out it gives an ERROR of kind "end_of_data" and leaves the position where it was, and a LEB128 number too long for 64 bits gives one of kind "invalid_data".

A Buffer is a Writer and a ByteReader a Reader, so functions written for those work with them too.