feat(pj_base): attach_source_record runtime-host slot for the M3 source cache (0.28.0) - #189
Open
facontidavide wants to merge 4 commits into
Open
feat(pj_base): attach_source_record runtime-host slot for the M3 source cache (0.28.0)#189facontidavide wants to merge 4 commits into
facontidavide wants to merge 4 commits into
Conversation
One provider->host call, stated once at download start: the canonical descriptor JSON of the reproducible request a source answers. The host copies the bytes and derives the record identity itself (the plugin never supplies an identity, per the descriptor_import doctrine), enabling the host-driven transparent source cache. Byte-identical re-attach is idempotent; different bytes fail; failure is a contract failure, never a trust verdict, and never affects ingest. Old hosts read as no-caching via PJ_HAS_TAIL_SLOT; the C++ wrapper reports the absence explicitly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The record is stored verbatim and keyed on the host's own digest (an internal keyspace — never required to agree with a provider identity scheme); the unspoofable datum is the provider id, taken from the binding. Refusal is allowlist-semantics (unknown fields are an error), not a field denylist. Last attach before the first push_message wins and the host may stage the record until its ingest transaction commits, so a replacing reload or in-place refill never silently loses it. Thread tag matches push_message ([stream-thread]). DatasetIngestHostView gains the forward; guide/architecture docs list the slot; tests add the unbound-host and short-struct_size-with-stale-pointer gates. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The SDK half of PJ4's M3 source cache (PJ4
docs/RECORDING_AND_CACHING.md§3.4): one optional runtime-host tail slot so a provider can state, once per download, "this dataset is request X" — and the host does everything else (capture, cache, restore).The slot
C++:
DataSourceRuntimeHostView::attachSourceRecord(std::string_view), forwarded onDatasetIngestHostViewso toolboxes use their canonical facade.[stream-thread]; bytes copied during the call; absent slot (older host) reads as "no caching" viaPJ_HAS_TAIL_SLOTwith an explicit wrapper error.Contract highlights
(dataset, identity, descriptor_json)sketch — a plugin-supplied identity would allow aliasing one request's artifact onto another's, and a host cannot canonicalize under a provider-declared field policy anyway.false+PJ_error_t), never a trust verdict, never affecting ingest.push_messageon this context wins; attaching after ingest begins is the error. Byte-exact re-attach is idempotent (matches PJ4's byte-equal trust rule).Mechanics
Precedent-exact tail-slot addition (the 0.21
set_host_infopattern): offset/size pinned by the ABI-layout sentinels test, both-direction compatibility (old plugin / old host),baseline.abiuntouched per MINOR policy.VERSION0.27.0 → 0.28.0; CHANGELOG entry with the offset line; docs updated (data-source-guide.mdmethod table,ARCHITECTURE.mdtail-slot history).Tests
85/85 on both legs (RelWithDebInfo and Debug+ASAN);
attach_source_record_test5/5: string fidelity across the ABI, host-error propagation, absent-slot detection, short-struct_sizegating, unbound-view refusal.Review provenance: one adversarial combined review (ABI mechanics verified precedent-exact; all findings were contract-text and are applied in the second commit).
Downstream note: PJ4 and pj-official-plugins each need a one-line designated-initializer addition when they bump to 0.28 (
-Werror=missing-field-initializers).🤖 Generated with Claude Code