Expand description
The real Databricks Arrow-Flight adapter (behind the databricks-sdk
feature). It implements ZerobusTransport/TransportFactory over the
databricks-zerobus-ingest-sdk ZerobusArrowStream.
The version bridge (ADR-0065): thalweg is on Arrow 58, the SDK on Arrow 59.
Batches cross as Arrow IPC bytes via ingest_ipc_batch, so no Arrow-59
array/batch type is ever handled here — only the stream’s schema is rebuilt
into the SDK’s Arrow-59 Schema at open time (to_sdk_schema).
Replay-on-failure is delegated to the SDK’s own recovery (.recovery(true)
on the stream): the SDK reconnects and re-ingests unacknowledged records
internally, so the sink’s pool-level rebuild only needs to reopen a fresh
stream — it never re-submits batches (see pool::StreamPool::rebuild).
Structs§
- Databricks
Factory - Opens Zerobus Arrow-Flight streams against a Databricks workspace.
Functions§
- to_
sdk_ schema - Rebuild our Arrow-58 schema into the SDK’s Arrow-59
Schema. Only the column types are mapped; the array data never crosses the boundary (it goes as IPC bytes), so this covers the sink-supported scalar/temporal set.