# `Arrow.Ipc.Stream`

Encodes and decodes the Arrow IPC streaming format.

An Arrow stream is a sequence of length-prefixed *messages*, each of
which carries a FlatBuffers-encoded `Message` metadata table and,
optionally, a raw body of column buffers.

## Frame layout per message

    continuation marker (u32 0xFFFFFFFF, 4 bytes)
    metadata length     (i32 little-endian, 4 bytes)
    metadata bytes      (variable, padded to 8-byte boundary)
    body bytes          (variable, already 8-byte aligned per buffer codec)

An end-of-stream marker is the continuation marker followed by a zero
length: `0xFFFFFFFF 0x00000000`. Some pre-0.15 producers omitted the
continuation marker — `decode/1` accepts both forms.

## Stream composition

A well-formed Arrow stream is:

    <Schema message>
    <DictionaryBatch message>   (zero or more)
    <RecordBatch message>       (zero or more)
    <EOS>

DictionaryBatch messages are decoded into the result's `dictionaries`
registry (delta dictionaries are rejected). Footer interactions belong
to the file format — see `Arrow.Ipc.File`, which reuses the frame
builders exposed here.

## Errors

`decode/1` returns `{:ok, payload}` or `{:error, %Arrow.DecodeError{}}`
with kind `:unsupported` (the input uses a feature this library
deliberately rejects) or `:malformed` (the input is corrupt, truncated,
or internally inconsistent). `decode!/1` raises the same error.

# `payload`

```elixir
@type payload() :: %{
  schema: Arrow.Schema.t(),
  dictionaries: %{optional(non_neg_integer()) =&gt; Arrow.Array.t()},
  batches: [Arrow.RecordBatch.t()]
}
```

Everything a decoded stream carries: schema, dictionary registry, batches.

# `decode`

```elixir
@spec decode(binary()) :: {:ok, payload()} | {:error, Arrow.DecodeError.t()}
```

Parses a complete Arrow IPC stream binary into `{:ok, %{schema:,
dictionaries:, batches:}}` or `{:error, %Arrow.DecodeError{}}`.

# `decode!`

```elixir
@spec decode!(binary()) :: payload()
```

Like `decode/1`, but returns the payload directly and raises
`Arrow.DecodeError` on failure.

# `dictionary_batch_message_frame`

```elixir
@spec dictionary_batch_message_frame(non_neg_integer(), Arrow.Array.t()) ::
  {iodata(), non_neg_integer(), non_neg_integer()}
```

Builds a single DictionaryBatch message frame from a dictionary id
and its values array. Returns iodata + framing-inclusive metadata
length + body length.

# `encode`

```elixir
@spec encode(
  Arrow.Schema.t(),
  [Arrow.RecordBatch.t()],
  %{optional(non_neg_integer()) =&gt; Arrow.Array.t()}
) :: binary()
```

Builds a complete Arrow IPC stream from a schema, optional dictionaries
registry, and zero or more record batches.

# `eos`

```elixir
@spec eos() :: binary()
```

End-of-stream marker: continuation marker followed by a zero length.

# `record_batch_message_frame`

```elixir
@spec record_batch_message_frame(Arrow.RecordBatch.t()) ::
  {iodata(), non_neg_integer(), non_neg_integer()}
```

Builds a single RecordBatch message frame: continuation + length +
padded metadata + body. Returns iodata, the framing-inclusive metadata
length, and the body length.

Exposed for `Arrow.Ipc.File`, same reason as `schema_message_frame/1`.

# `schema_message_frame`

```elixir
@spec schema_message_frame(Arrow.Schema.t()) :: {iodata(), non_neg_integer()}
```

Builds a single Schema message frame: continuation + length + padded
metadata. No body. Returns iodata and the framing-inclusive metadata
length (i.e. `8 + padded_metadata`).

Exposed for `Arrow.Ipc.File`, which needs to record per-message file
offsets in the Footer's Block list.

# `validate_dictionaries!`

```elixir
@spec validate_dictionaries!(Arrow.Schema.t(), %{
  optional(non_neg_integer()) =&gt; Arrow.Array.t()
}) :: :ok
```

Validates the schema ↔ dictionaries registry pairing before encoding:
every dictionary-encoded field in `schema` must have an entry for its
id in `dictionaries`, or the output would reference a dictionary that
is never written (spec-invalid). Raises `ArgumentError` otherwise.

The converse direction (registry id with no referencing field) is
checked where the DictionaryBatch messages are emitted.

Exposed for `Arrow.Ipc.File`, which builds its frames directly.

---

*Consult [api-reference.md](api-reference.md) for complete listing*