Expand description
resolved validation spec (schema + quality rules) with precedence and most-restrictive merge; contract-format-agnostic
Quality rules, precedence, and most-restrictive merge remain scaffold. The
neutral structural schema hub (schema) is implemented — it is the
format-agnostic shape that the cross-format schema bridges (protobuf ⇄ ODCS,
and later JSON Schema / Avro) convert through. See ADR-0064.
§Uniqueness declarations gate content dedup
Content-hash dedup (see twg-connector-core) drops a record whose payload has
been seen before, which is only safe where payloads are genuinely unique. Two
distinct events with identical bodies — heartbeats, timestamp-less clicks —
would otherwise be silently collapsed into one.
So the contract is the gate. A stream may enable content dedup only where the resolved contract declares a natural key or an explicit uniqueness guarantee, and the declared fields are what get hashed rather than the whole payload where a subset is available. A stream with no such declaration cannot enable it, and config validation refuses rather than warns — a warning about silent data loss is not a control.
Modules§
- schema
- Neutral, format-agnostic structural schema model — the hub every schema format (protobuf, ODCS, JSON Schema, Avro, …) converts to and from, so N formats cost N adapters rather than N². This carries shape (named objects, their fields, cardinality, leaf types, cross-object references); quality rules and precedence/merge remain the resolved-spec concern of the rest of this crate.