Expand description
FFI boundary — both directions of foreign interop, isolated in one audited crate. Scaffold only.
Two halves, one lint exception:
-
Inbound — Arrow C Data Interface (BUILT NOW). A trusted, in-process, zero-copy handoff: a producer in another language (
PyArrow, Go, C++, Java) passes anArrowSchema*+ArrowArray*/ArrowArrayStream*, and Thalweg reads the producer’s memory directly — no copy, no serialisation, no sandbox. Fills the trusted/in-process cell that neither WASM (untrusted, in-process) nor Flight (out-of-process) covers. It is a data HANDOFF, not an extension mechanism: logic still goes through WASM or Flight. Exposed as a Source producing the shared record stream, with locator support. -
Outbound — stable C ABI (DEFERRED, design-for-don’t-build). The mirror of the predecessor’s exported-symbol ABI that let Java/Python facades drive the pipeline. Reserved here so the trait shapes stay C-ABI-compatible, but not implemented in this phase.
§Why unsafe lives here and only here
The C Data Interface is raw pointers with manual release-callback lifetime discipline — use-after-free if mishandled. The workspace forbids unsafe; this crate downgrades it to deny-with-audited-exceptions. The unsafe surface is minimised, wrapped in a safe API, and never leaks: no raw pointer crosses this crate’s boundary. This is the single audited exception, named in the purity story alongside zstd-encode and gssapi.
§Extension mechanism matrix (for reference)
| in-process | out-of-process | |
|---|---|---|
| trusted | Arrow C Data Interface (here) | Flight sidecar |
| untrusted | WASM | (Flight+isolation) |