std/encoding

Bytes and text in the encodings that files, URLs, and protocols use. Decoding gives an ERROR of kind "invalid_data" that says what is wrong and where.

import "std/encoding"

print(base64_encode(b"hola"))                     // aG9sYQ==
print(base64_decode("aG9sYQ")?)                   // b"hola"
print(percent_encode("a b/c", "path"))            // a%20b/c
print(form_encode([FormField("q", "x y"), FormField("n", "1")]))  // q=x+y&n=1
bytes wide = encode_text("añ", "utf-16le")?
print(decode_text(wide, "utf-16le")?)             // añ

Function	Result
`hex_encode(data)`, `hex_encode(data, upper: true)`	two hexadecimal digits a byte
`hex_decode(text)`	the bytes, from digits in either case; spaces and line breaks are left out unless `strict: true`
`base64_encode(data)`, `base32_encode(data)`	base64 and base32 (RFC 4648), padded with `=` unless `padding: false`
`base64url_encode(data)`	base64 with `-` and `_`, unpadded unless `padding: true`
`base64_decode(text)`, `base32_decode(text)`, `base64url_decode(text)`	the bytes
`percent_encode(text, part)`	`%XX` for each byte that a part of a URL cannot hold as it is
`url_encode(text)`	`percent_encode(text)`: everything but letters, digits, and `- . _ ~` as `%XX`
`url_decode(text)`, `url_decode(text, plus: true)`	each `%XX` as its byte, and `+` as a space with `plus`
`form_encode(fields)`, `form_decode(text)`	a form body or a query string, from and to `FormField(name, value)` items
`form_values(fields, name)`, `form_value(fields, name)`	the values of the fields with a name, and the first of them
`encode_text(text, encoding)`	the bytes of text in an encoding, after its byte-order mark with `bom: true`
`decode_text(data, encoding)`, `decode_text_lossy(data, encoding)`	the text, without the byte-order mark; the lossy form puts U+FFFD where the data is wrong
`is_text_encoding(name)`	whether the two above know an encoding
`detect_encoding(data)`, `decode_detected(data)`	the encoding a file seems to be in, and its text read so

Decoding is lenient unless strict: true: spaces and line breaks are left out, the padding may be missing, base32 takes lowercase letters, and base64 and base64url take each other's characters. Strict decoding takes the text only as the encoder writes it, with the bits after the last byte zero.

The parts of percent_encode are "component", the default, which keeps only letters, digits, and - . _ ~, as a value inside a path or a query needs; and "path", "segment", "query", "fragment", and "userinfo", which keep what RFC 3986 allows there, so / stays in a path and & and = in a query. Forms are written as application/x-www-form-urlencoded: a space is +, and every byte but letters, digits, and * - . _ is %XX; a name may be in several fields.

The text encodings are "utf-8", "utf-16le", "utf-16be", "utf-32le", "utf-32be", "latin-1" (also "iso-8859-1"), and "windows-1252" (also "cp1252"), in any case; an unknown name is an ERROR of kind "invalid_argument". A character that Latin-1 or Windows-1252 does not have cannot be encoded, and U+0000, which text cannot hold, cannot be decoded. detect_encoding goes by the byte-order mark, then by the zero bytes that ASCII text in UTF-16 and UTF-32 has, then gives "utf-8" for valid UTF-8 and "windows-1252" for the rest.