# `Arrow.Buffer`

Encoders and decoders between in-memory values and Arrow's raw buffer
layout.

The Arrow columnar format spec describes the per-type set of buffers (a
validity bitmap, value buffer, optional offset buffer, etc.). This module
packs and unpacks each of those buffers in isolation. The higher-level
`Arrow.Array.*` structs hold the packed buffers directly, so the JSON
reader/writer and (eventually) the IPC reader/writer can share this code.

## Conventions

- All multi-byte integers and floats are little-endian.
- Bitmaps (validity, Bool values) are LSB-first: slot `i` is bit `i mod 8`
  of byte `floor(i / 8)`, with the least-significant bit being slot 0.
- For validity bitmaps, `1` means valid, `0` means null.
- When a buffer is materialised for the IPC body, callers are expected to
  pad it to an 8-byte boundary using `pad_to_alignment/2`.

# `primitive_kind`

```elixir
@type primitive_kind() ::
  :int8
  | :int16
  | :int32
  | :int64
  | :uint8
  | :uint16
  | :uint32
  | :uint64
  | :float32
  | :float64
```

# `pack_bool_values`

```elixir
@spec pack_bool_values([0 | 1 | boolean()]) :: binary()
```

Packs a list of `0 | 1 | true | false` into a Bool *values* bitmap. Slots
whose validity bit is 0 may carry any value; callers typically pass `0`.

    iex> Arrow.Buffer.pack_bool_values([true, false, true, true])
    <<0b00001101::8>>

# `pack_int32_offsets`

```elixir
@spec pack_int32_offsets([non_neg_integer()]) :: binary()
```

Builds an `int32` offsets buffer from a list of per-slot byte lengths.

The returned buffer has `length(byte_lengths) + 1` entries, starting at `0`
and ending at the sum.

    iex> Arrow.Buffer.pack_int32_offsets([3, 2, 0])
    <<0::little-signed-32, 3::little-signed-32,
      5::little-signed-32, 5::little-signed-32>>

# `pack_int64_offsets`

```elixir
@spec pack_int64_offsets([non_neg_integer()]) :: binary()
```

`int64` analogue of `pack_int32_offsets/1`, used by `LargeUtf8`,
`LargeBinary`, and `LargeList`.

# `pack_primitive`

```elixir
@spec pack_primitive([number()], primitive_kind()) :: binary()
```

Packs a list of numeric values into a little-endian fixed-width buffer.

`kind` is one of the atoms in `t:primitive_kind/0`. `:int64`-like atoms map
to signed packing; `:uint*` to unsigned; `:float32` and `:float64` to IEEE
floats. Null slots may carry any in-range placeholder (typically `0`).

    iex> Arrow.Buffer.pack_primitive([1, 2, 3], :int32)
    <<1::little-signed-32, 2::little-signed-32, 3::little-signed-32>>

# `pack_validity`

```elixir
@spec pack_validity([0 | 1 | boolean()]) :: {binary(), non_neg_integer()}
```

Packs a list of `0 | 1 | true | false` flags into Arrow's LSB-first validity
bitmap. Returns the bitmap *padded to a whole number of bytes* plus the null
count.

    iex> {bitmap, null_count} = Arrow.Buffer.pack_validity([1, 0, 1, 1, 0])
    iex> null_count
    2
    iex> bitmap
    <<0b00001101::8>>

# `pad_to_alignment`

```elixir
@spec pad_to_alignment(binary(), pos_integer()) :: binary()
```

Pads `binary` with zero bytes up to the next multiple of `alignment` bytes
(default 8). Used at IPC body buffer boundaries.

    iex> Arrow.Buffer.pad_to_alignment(<<1, 2, 3>>)
    <<1, 2, 3, 0, 0, 0, 0, 0>>

# `padding_size`

```elixir
@spec padding_size(non_neg_integer(), pos_integer()) :: non_neg_integer()
```

Returns the number of pad bytes needed to reach the next alignment boundary.

# `slice_variable`

```elixir
@spec slice_variable(binary(), binary(), non_neg_integer()) :: [binary()]
```

Slices a variable-length value buffer using a list of `int32` offsets,
returning one binary per slot.

    iex> offs = Arrow.Buffer.pack_int32_offsets([1, 2, 0, 3])
    iex> Arrow.Buffer.slice_variable(offs, "abcdef", 4)
    ["a", "bc", "", "def"]

# `slice_variable_large`

```elixir
@spec slice_variable_large(binary(), binary(), non_neg_integer()) :: [binary()]
```

`int64` analogue of `slice_variable/3`, used by `LargeUtf8` and
`LargeBinary`.

# `unpack_bool_values`

```elixir
@spec unpack_bool_values(binary(), non_neg_integer()) :: [0 | 1]
```

Unpacks a Bool values bitmap into a list of `0 | 1`.

# `unpack_int32_offsets`

```elixir
@spec unpack_int32_offsets(binary(), non_neg_integer()) :: [integer()]
```

Unpacks an `int32` offsets buffer of `length + 1` entries.

# `unpack_int64_offsets`

```elixir
@spec unpack_int64_offsets(binary(), non_neg_integer()) :: [integer()]
```

Unpacks an `int64` offsets buffer of `length + 1` entries.

# `unpack_primitive`

```elixir
@spec unpack_primitive(binary(), primitive_kind(), non_neg_integer()) :: [number()]
```

Unpacks a fixed-width buffer into a list of values.

    iex> Arrow.Buffer.unpack_primitive(<<1::little-signed-32,
    ...>                                  2::little-signed-32,
    ...>                                  3::little-signed-32>>, :int32, 3)
    [1, 2, 3]

# `unpack_validity`

```elixir
@spec unpack_validity(binary() | nil, non_neg_integer()) :: [0 | 1]
```

Unpacks a validity bitmap into a list of `0 | 1` slots.

    iex> Arrow.Buffer.unpack_validity(<<0b00001101::8>>, 5)
    [1, 0, 1, 1, 0]

---

*Consult [api-reference.md](api-reference.md) for complete listing*