diff --git a/docs/fdx/FDX1_WORKSPACE_AND_CONTRACTS.md b/docs/fdx/FDX1_WORKSPACE_AND_CONTRACTS.md new file mode 100644 index 0000000..da1e8a7 --- /dev/null +++ b/docs/fdx/FDX1_WORKSPACE_AND_CONTRACTS.md @@ -0,0 +1,358 @@ +# FDX1 — Workspace split and domain contracts + +Status: planned + +Owner: Datasheet-cli + +Depends on: FDX0 contract shape agreed; implementation may use pinned Ferrodoc fixtures while FDX0 finishes + +## Objective + +Separate reusable electronics semantics from the existing CLI/remote-LLM/source-client code and define the stable serialized contracts that every later FDX phase uses. + +FDX1 changes architecture, not extraction quality. It must preserve existing user-facing commands unless a command is explicitly documented as migrated/deprecated. + +## Deliverable 1: workspace conversion + +Create: + +```text +crates/datasheet-core +crates/datasheet-compiler +crates/datasheet-model +tools/datasheet-train +apps/datasheet-cli +``` + +Responsibilities: + +### `datasheet-core` + +Dependency-light durable types only: + +- subject hints/handles; +- predicate term references; +- specification regimes; +- applicability/conditions; +- evidence anchors; +- quantity/value candidate references; +- `ClaimBundle/v1`; +- `DesignConstraintBundle/v1`; +- versioned serialization/schema generation. + +It must not depend on HTTP clients, LLM providers, PDF runtimes, PyTorch, Foundry persistence or source implementations. + +### `datasheet-compiler` + +Owns: + +- Ferrodoc DocumentIR adapter; +- `DatasheetSketch/v1`; +- deterministic candidate extraction; +- ontology/predicate retrieval interfaces; +- Quantitas integration; +- source vocabulary mappings; +- weak-label compiler; +- claim assembly/validation. + +### `datasheet-model` + +Owns: + +- model manifest; +- feature schema; +- tokenizer/model artifact loading; +- RTen inference; +- calibration; +- model output validation; +- optional backend trait if later benchmarks justify non-RTen inference. + +### `datasheet-train` + +Training/export/evaluation tooling. Python/PyTorch is allowed and preferred if it reduces complexity. It emits immutable ONNX/model manifests; production does not embed Python. + +### `apps/datasheet-cli` + +Moves the existing command application and provider/source clients here. Existing Gemini extraction remains an explicit remote path/fallback. + +## Deliverable 2: licensing/API boundary + +The current binary is GPL-3.0-only. Decide and document licensing for new reusable library crates before FDX1 merges. Preferred default for ecosystem reuse is `MIT OR Apache-2.0` for new libraries while the application may remain GPL, but implementation must make the final explicit decision rather than accidentally inheriting ambiguous licensing. + +Do not change ownership/license of copied existing code without tracking its provenance. + +## Deliverable 3: versioned `EvidenceAnchor` + +Define a Datasheet-side serialized reference compatible with Foundry FLS5 without importing Foundry internals: + +```text +EvidenceAnchor/v1 { + source_pdf_sha256 + document_ir_logical_sha256 + page_id + region_id? + evidence_ids[] + selectors[] + source_geometry[] + geometry_quality[] +} +``` + +Selectors initially include: + +```text +whole_evidence +text_range { evidence_id, start, end } +table_cell { table_region_id, row, column } +``` + +The adapter validates every anchor against the pinned DocumentIR before emitting a bundle. + +## Deliverable 4: `DatasheetSketch/v1` + +Define the compact electronics-oriented representation derived from DocumentIR. It is not a second document IR; it is a deterministic semantic candidate index that always points back to Ferrodoc evidence. + +Logical shape: + +```text +DatasheetSketch/v1 { + schema_version + source_pdf_sha256 + document_ir_logical_sha256 + producer + document_family_id + + subjects[] + sections[] + blocks[] + tables[] + quantities[] + symbols[] + packages[] + template_signatures[] +} +``` + +### Subject candidate + +```text +SubjectCandidate { + id + kind: part | family | package_variant | unknown + raw_designation + manufacturer_hint? + normalized_search_key? + evidence_anchor +} +``` + +Normalization is a search/index aid, not identity proof. + +### Section candidate + +```text +SectionCandidate { + id + heading_text? + heading_anchor? + parent_section_id? + block_ids[] + table_ids[] + deterministic_type_candidates[] +} +``` + +### Table candidate + +```text +SketchTable { + id + source_region_id + heading_path[] + rows + columns + cells[] + structural_signature + reconstruction_quality +} +``` + +Each cell references the original Ferrodoc table cell/source spans. + +### Quantity candidate + +```text +QuantityCandidate { + id + raw_text + parsed_shape + raw_unit? + quantitas_quantity? + quantity_kind_hint? + dimensionality? + anchor + parse_warnings[] +} +``` + +Initial shapes: + +```text +scalar +range +inequality +tolerance +set +text_unparsed +``` + +Do not invent values from context. Parsing transforms exact source text into a candidate representation. + +### Symbol candidate + +```text +SymbolCandidate { + id + raw_text + normalized_symbol? + anchor +} +``` + +### Package candidate + +Retain raw package/code/variant terms and anchors; package identity is not resolved merely from a string match. + +## Deliverable 5: specification regimes + +`datasheet-core` defines explicit regimes compatible with Foundry's assertion model: + +```text +absolute_maximum +recommended_operating +guaranteed +tested +characterized_typical +application_guidance +informative +derived +inferred +measured +unknown +``` + +Do not flatten absolute maximum and recommended operating values into the same property without regime. + +## Deliverable 6: `ClaimBundle/v1` + +A bundle contains candidate/accepted-by-extractor claims, not Foundry promotion decisions. + +```text +ClaimBundle/v1 { + schema_version + source_pdf_sha256 + document_ir_logical_sha256 + producer + subject_hints[] + claims[] + warnings[] +} + +ClaimCandidate { + local_id + subject_candidate_id + predicate_term_id + regime + value_candidate_id + condition_candidate_ids[] + package_variant_candidate_ids[] + evidence_anchors[] + extraction_confidence + grounding_status +} +``` + +`grounding_status` must fail closed if selected candidate IDs/anchors cannot be resolved. + +The serialized bundle must preserve enough raw/normalized value detail for Foundry to store the original spelling plus Quantitas-backed normalized quantity without duplicating the DocumentIR. + +## Deliverable 7: predicate vocabulary interface + +Define an interface over versioned predicate vocabulary-as-data: + +```text +PredicateTerm { + id + canonical_name + description + aliases[] + symbol_aliases[] + expected_quantity_kind? + expected_shapes[] + regime_constraints? + category_scope? +} +``` + +The compiler accepts an ontology snapshot/digest as input. It does not compile one fixed global Rust enum containing every electrical property. + +Known application predicates may still have typed Rust adapters where useful, but the model/compiler boundary is vocabulary-driven. + +## Deliverable 8: producer identity + +Every durable compiler result binds: + +```text +code revision/version +configuration digest +DocumentIR schema/version +ontology snapshot digest +Quantitas registry digest/version when used +model digest when used +``` + +Wall-clock run IDs do not enter deterministic semantic identity. + +## Deliverable 9: CLI compatibility + +Preserve current remote commands while adding explicit local/introspection commands conceptually equivalent to: + +```text +datasheet sketch +datasheet compile characteristics <...> +datasheet explain <...> +datasheet model inspect +``` + +Exact CLI shape may evolve, but local deterministic/model extraction must be distinguishable from remote Gemini extraction. + +## Tests + +Required: + +- workspace builds with dependency boundaries enforced; +- `datasheet-core` has no network/PDF/model runtime dependencies; +- `DatasheetSketch` round-trips deterministically; +- every sketch evidence anchor resolves against fixture DocumentIR; +- quantity candidate preserves original text after normalization; +- absolute-max and recommended-operating regimes remain distinct; +- `ClaimBundle` rejects missing candidate/evidence references; +- ontology snapshot digest participates in derivation identity; +- adding a predicate term does not require changing the serialized `ClaimBundle` schema; +- existing CLI smoke cases continue to work or have an explicit migration test. + +## Acceptance criteria + +FDX1 is complete when: + +1. Datasheet-cli is a buildable workspace with the declared reusable boundaries; +2. a pinned Ferrodoc fixture deterministically produces `DatasheetSketch/v1`; +3. quantity/evidence candidates round-trip with exact source anchors; +4. `ClaimBundle/v1` is serialization-stable enough for Foundry FLS6 integration; +5. predicate vocabulary is data-driven rather than a fixed model-output enum; +6. existing remote extraction remains available as a separate application path; +7. no Foundry persistence/domain types are imported into reusable Datasheet core/compiler crates. + +## Landed + +Record final crate/license choices, schema versions, Ferrodoc/Quantitas pins and compatibility commands after implementation. diff --git a/docs/fdx/FDX2_DETERMINISTIC_COMPILER.md b/docs/fdx/FDX2_DETERMINISTIC_COMPILER.md new file mode 100644 index 0000000..d5f2d54 --- /dev/null +++ b/docs/fdx/FDX2_DETERMINISTIC_COMPILER.md @@ -0,0 +1,294 @@ +# FDX2 — Deterministic datasheet compiler baseline + +Status: planned + +Owner: Datasheet-cli + +Depends on: FDX1; FDX0 evidence contract available through a pinned Ferrodoc revision/fixture + +## Objective + +Build a useful, evidence-grounded electronics extractor without machine learning. This establishes the production fallback and the qualification baseline that every later learned component must beat. + +Initial scope is deliberately narrow: + +```text +Absolute Maximum Ratings +Recommended Operating Conditions +Electrical Characteristics +Thermal Information +``` + +## Deliverable 1: DocumentIR -> `DatasheetSketch` + +Implement the deterministic adapter from pinned Ferrodoc DocumentIR into the FDX1 sketch. + +Required steps: + +```text +validate IR + digest + | +build heading/section ancestry + | +index tables/paragraphs/source spans + | +extract subject/package/symbol candidates + | +parse exact quantity candidates through Quantitas-backed registry + | +compute template/structural signatures + | +produce canonical DatasheetSketch +``` + +If table/cell geometry is coarse, preserve the quality and lower extraction confidence; do not invent precise anchors. + +## Deliverable 2: quantity lexer/parser + +Implement deterministic parsing over exact cell/span text. + +Support at least: + +```text +3.3 V +60 nA +-0.3 V +1.8 to 5.5 V +1.8–5.5 V +<= 5.5 V +5.5 V max +±2 % +25 °C +-40 °C to 125 °C +180 °C/W +10 kΩ +1 MHz +``` + +Keep raw source spelling and parse structure separately. + +Represent symbolic/relative expressions that cannot safely canonicalize, for example: + +```text +VDD + 0.3 V +VOUT - 0.5 V +0.7 * VDD +``` + +as structured/raw candidates rather than pretending they are fixed scalar quantities. + +A failed parse remains explicit and does not erase source text. + +## Deliverable 3: section/regime rules + +Implement conservative heading/section aliases for the first task family. + +Examples: + +```text +absolute maximum ratings +absolute maximum rating +maximum ratings + +recommended operating conditions +recommended operating condition +operating conditions + + electrical characteristics +dc characteristics +ac characteristics +dc/ac characteristics + +thermal information +thermal characteristics +package thermal data +``` + +Aliases are versioned data/configuration, not scattered string comparisons. + +Map only sufficiently strong headings to a regime. Ambiguous generic headings such as `Characteristics` should remain candidates until table/header context resolves them. + +## Deliverable 4: table-header role inference + +Recognize common generic schemas: + +```text +Parameter | Symbol | Conditions | Min | Typ | Max | Unit +Parameter | Test Conditions | Min | Typ | Max | Units +Symbol | Parameter | Conditions | Minimum | Typical | Maximum +Parameter | Conditions | Rating | Unit +Parameter | Min | Max | Unit +``` + +Infer roles: + +```text +parameter +symbol +condition +min +typ +max +unit +rating/package +notes +unknown +``` + +Use header ancestry/merged cells from Ferrodoc where available. + +Rules must abstain when multiple assignments are equally plausible. + +## Deliverable 5: row compiler + +Compile one structured row into deterministic candidates: + +```text +row text/cells + | + +--> parameter label + +--> symbol candidate + +--> condition/application candidates + +--> min/typ/max/rating quantity candidates + +--> unit inheritance + +--> footnote references + +--> package/variant scope +``` + +Do not collapse min/typ/max into one scalar. Preserve the semantic role of each value. + +A row may yield multiple `CandidateClaim` objects when multiple values genuinely express separate constraints. + +## Deliverable 6: unit inheritance and cell conventions + +Datasheets frequently place units in a dedicated column or heading and use blank/dash cells. + +Implement explicit table-level conventions derived from headers/body consistency: + +```text +unit column inheritance +unit embedded in each value +blank means unspecified +em dash/hyphen means unspecified when table convention establishes it +`N/A` / `NA` explicit unavailable token +``` + +Never turn a blank/dash into numeric zero. + +Retain which convention produced each interpretation. + +## Deliverable 7: footnote/applicability linking + +Use Ferrodoc evidence/reading order and textual markers to connect: + +```text +row marker -> table footnote +header marker -> table/column footnote +section note -> scoped condition candidate +``` + +Emit condition/applicability candidates with exact evidence anchors. + +Do not apply a footnote globally if its marker scope is ambiguous. + +## Deliverable 8: predicate candidate retrieval baseline + +Build an auditable non-neural predicate retriever over the versioned vocabulary. + +Signals: + +```text +normalized parameter text alias +exact/normalized symbol alias +quantity kind/dimensionality +section/regime compatibility +component/category prior when supplied +``` + +Return top-K candidate term IDs with per-signal reasons. + +Do not require one exact winner. Ambiguous rows remain candidate sets for FDX3/FDX5. + +## Deliverable 9: deterministic claim assembly + +For candidates passing configurable confidence gates, assemble `ClaimBundle/v1` by selecting only existing evidence/value IDs. + +The deterministic compiler may emit: + +```text +accepted_by_extractor +ambiguous +unsupported +invalid_grounding +``` + +These are extraction statuses, not Foundry promotion decisions. + +## Deliverable 10: explanations + +Every row/claim exposes deterministic reasons, for example: + +```text +section heading matched recommended_operating@v1 +header schema matched parameter|symbol|condition|min|typ|max|unit +quantity parsed as electric potential +symbol alias matched VDD +predicate alias matched supply voltage +value selected from table r=12,c=5 +condition selected from r=12,c=2 + footnote 3 +``` + +This explanation becomes useful both for debugging and as FDX3 labeling-function evidence. + +## Deliverable 11: protected deterministic baseline + +Create an evaluation harness before FDX3/FDX5 changes the system. + +Metrics: + +```text +section/regime precision/recall +header/cell-role accuracy +quantity parse accuracy +predicate top-1/top-k accuracy +condition-span accuracy +claim exact match +claim evidence-anchor validity +ungrounded output count +abstention/coverage +wall time / peak memory where practical +``` + +Treat a missing prediction as missing work, not success. + +## Tests + +Required: + +- supported quantity forms preserve exact raw text and canonical meaning; +- symbolic relative values remain symbolic rather than guessed scalars; +- dash/blank never becomes zero; +- absolute max and recommended operating sections cannot silently merge regimes; +- min/typ/max roles survive serialization; +- unit inheritance is deterministic and fails closed on conflicting units; +- ambiguous header schema abstains; +- footnote marker resolves only within defensible scope; +- predicate dimension filters reject impossible kinds; +- claim bundle cannot contain a value not present in sketch candidates; +- every claim anchor validates against pinned IR; +- repeated compilation of the same inputs produces byte-identical canonical outputs. + +## Acceptance criteria + +FDX2 is complete when: + +1. a retained real analog/power datasheet corpus produces useful grounded claims with no learned model; +2. all emitted values come from exact deterministic candidates; +3. rule explanations make every accepted/abstained row auditable; +4. protected baseline metrics are frozen for later comparison; +5. ambiguous/unsupported rows are retained for FDX3/FDX5 rather than guessed; +6. the compiler consumes only generic Ferrodoc evidence and does not bypass it with its own PDF parser. + +## Landed + +Record corpus identity, Quantitas/ontology snapshots, supported forms, baseline metrics and known failure clusters after implementation. diff --git a/docs/fdx/FDX3_WEAK_SUPERVISION.md b/docs/fdx/FDX3_WEAK_SUPERVISION.md new file mode 100644 index 0000000..9b10a23 --- /dev/null +++ b/docs/fdx/FDX3_WEAK_SUPERVISION.md @@ -0,0 +1,443 @@ +# FDX3 — Weak-supervision and training-data compiler + +Status: planned + +Owner: Datasheet-cli semantic compiler; Foundry persists immutable observations/votes/datasets + +Depends on: FDX2 deterministic candidates; Foundry FLS2 structured-fact transport and FLS4 silver lineage boundary + +## Objective + +Generate a large, diverse, auditable training corpus without paying an LLM to read every PDF. + +FDX3 treats structured source facts, document structure, template regularity and independent engineering representations as noisy labeling functions. Labelers vote or abstain on factorized candidate decisions. The compiler retains every vote and combines them into probabilistic/reviewable training labels. + +## Core principle + +Do not supervise free-form JSON generation. + +Supervise these factors separately: + +```text +section/table semantic type +cell/column role +predicate link +specification regime +subject/package/variant scope +applicability/condition evidence +``` + +The final `ClaimBundle` remains deterministic assembly over selected candidates. + +## Deliverable 1: source-fact input contract + +Consume Foundry `StructuredFactObservation/v1` or an equivalent serialized fixture contract containing: + +```text +source/publisher/transport identity +source artifact + exact record locator +manufacturer / MPN / family / package hints +external namespace/schema version/field key/name +raw value/raw unit/value-shape hint +category path +rights snapshot +``` + +The Datasheet compiler does not require a live source API to reproduce labels; retained source observations are sufficient. + +## Deliverable 2: source vocabulary maps + +Define a versioned mapping from source-native fields to predicate candidates: + +```text +SourceVocabularyMap/v1 { + id + external_namespace + external_schema_version + external_field_key + external_field_name? + + predicate_term_id + expected_quantity_kind? + expected_value_shapes[] + regime_hint? + category_scope[] + + provenance + mapping_version +} +``` + +Mappings may originate from deterministic schema metadata, human review or a compact LLM taxonomy-mapping task. They are not source-plugin assertions. + +One reviewed mapping is intentionally reusable across every record carrying that stable source field. + +## Deliverable 3: training candidate identity + +A `TrainingCandidate` is one factorized decision over immutable sketch/source inputs. + +```text +TrainingCandidate/v1 { + id + task + document_ir_logical_sha256 + datasheet_sketch_digest + local_candidate_ids[] + source_fact_record_ids[] + ontology_snapshot_digest + document_family_id + template_family_id? + manufacturer_key? +} +``` + +Candidate identity is content/producer derived. Physical lake row location is not identity. + +## Deliverable 4: `LabelVote/v1` + +Every labeling function produces an auditable vote: + +```text +LabelVote/v1 { + id + training_candidate_id + task + + labeler_id + labeler_version + + decision: vote | abstain + label_json? + abstain_reason? + + input_record_ids[] + lineage_groups[] + reason_codes[] + + producer +} +``` + +Do not store only an aggregate confidence. Downstream debugging/training must be able to reproduce which labelers agreed or conflicted. + +## Deliverable 5: lineage groups + +Define source-lineage metadata so copies of one upstream value do not become fake independent evidence: + +```text +LineageGroup/v1 { + id + publisher_key? + upstream_dataset_fingerprint? + source_schema? + observation_epoch? + derivation_parent_ids[] +} +``` + +Two distributor observations with indistinguishable upstream provenance may still be useful for commercial corroboration, but they do not receive the same independence bonus as unrelated publishers/representations. + +## Deliverable 6: generic document labelers + +Implement these labeler identities first: + +### `section.heading.v1` + +Votes for section/regime from high-precision heading aliases. Abstains on generic/ambiguous headings. + +### `section.table_header_schema.v1` + +Votes for table semantic/schema compatibility from column/header pattern. + +### `cell.min_typ_max_header.v1` + +Votes cell/column roles from explicit Min/Typ/Max/Minimum/Typical/Maximum headers. + +### `quantity.dimension.v1` + +Filters/votes predicate compatibility from Quantitas dimensionality/quantity kind. It is normally a constraint/filter, not enough by itself to identify a specific predicate. + +### `predicate.symbol_alias.v1` + +Votes predicate candidates from exact/normalized symbol aliases in the ontology snapshot. + +### `predicate.row_name_alias.v1` + +Votes from parameter-name aliases/fuzzy normalized forms under strict thresholds. + +### `alignment.value_unique.v1` + +Given a structured source fact with normalized value/dimension, votes for the unique document row/value candidate matching that value inside a plausible section. Abstains if zero or multiple plausible matches remain. + +### `alignment.range.v1` + +Matches source min/max ranges to document endpoint candidates with compatible dimension and regime. + +### `template.cluster.v1` + +Votes table/column semantics only for a previously reviewed high-purity template family. + +### `applicability.footnote_link.v1` + +Votes condition/applicability evidence from explicit row/header footnote markers and defensible local scope. + +### `scope.package.v1` + +Votes package applicability when source package and document package candidate agree under a reviewed normalization/mapping. + +### `scope.part.v1` + +Votes subject scope from exact observed MPN/family evidence around the section/table. + +### `value.null_dash_convention.v1` + +Votes that blank/dash means unspecified only after table-level convention is established; never emits numeric zero. + +### `ranking.regime_conflict.v1` + +Creates a hard training negative/ranking preference when an otherwise matching value occurs in the wrong specification regime. This is not a negative Foundry assertion. + +### `revision.structure_stability.v1` + +Boosts structural/template confidence when the same normalized table structure recurs across related manufacturer document revisions. Related revisions stay in one evaluation family. + +## Deliverable 7: TI labelers + +Implement stable IDs: + +```text +ti.identity_gate.v1 +ti.field_map.v1 +ti.value_align_unique.v1 +ti.range_align.v1 +ti.unit_kind.v1 +ti.package_align.v1 +``` + +Behavior: + +1. establish defensible MPN/family identity between the structured record and datasheet before technical votes count; +2. map stable TI parametric field identity through `SourceVocabularyMap`; +3. normalize the source value/range/unit; +4. find dimension/regime-compatible document candidates; +5. vote only when the alignment is unique or otherwise high precision; +6. retain TI API and TI datasheet as shared-publisher lineage rather than pretending they are independent authorities. + +## Deliverable 8: DigiKey labelers + +Implement: + +```text +digikey.identity_gate.v1 +digikey.parameter_id_map.v1 +digikey.parameter_text_alias.v1 +digikey.value_align_unique.v1 +digikey.category_prior.v1 +digikey.package_scope.v1 +digikey.datasheet_link.v1 +``` + +Prefer stable source parameter IDs over reinterpreting free-form names on every record. + +`category_prior` may rerank plausible predicates but must not override incompatible dimensionality/evidence. + +A DigiKey datasheet URL is document linkage evidence, not manufacturer authority by itself. + +## Deliverable 9: Mouser labelers + +Initial scope reflects what the retained source adapter actually supplies: + +```text +mouser.mpn_identity.v1 +mouser.datasheet_link.v1 +mouser.category.v1 +mouser.package.v1 +mouser.lifecycle.v1 +mouser.replacement.v1 +``` + +Do not invent electrical-parametric labelers unless a retained Mouser source schema actually exposes the required fields. + +## Deliverable 10: generic manufacturer/PIM labelers + +Implement: + +```text +manufacturer.field_map.v1 +manufacturer.value_align.v1 +manufacturer.range_align.v1 +manufacturer.package.v1 +manufacturer.variant_scope.v1 +manufacturer.order_code.v1 +manufacturer.family_scope.v1 +``` + +These consume reviewed source vocabulary mappings for REST/GraphQL/PIM/BMEcat/ETIM/catalog field identities. + +A field mapping may be reused across many parts/documents but remains versioned so later taxonomy corrections do not rewrite history. + +## Deliverable 11: EDA supervision + +Consume serialized observations from first-party/other EDA parsers without parsing those formats inside Datasheet-cli. + +Initial labelers: + +```text +eda.pin_number_set.v1 +eda.pin_name_by_number.v1 +eda.pin_count.v1 +eda.package_code.v1 +eda.pad_count.v1 +eda.pin_to_pad.v1 +eda.body_dimensions.v1 +eda.model_package.v1 +``` + +Authority/lineage is part of the vote: manufacturer EDA can receive a different prior than a community library, but even community observations can be valuable alignment signals. + +These tasks may be persisted before their corresponding learned heads are implemented. + +## Deliverable 12: CMSIS/SVD supervision + +Implement only semantics those sources defensibly provide: + +```text +cmsis.device_identity.v1 +cmsis.cpu_core.v1 +cmsis.memory.v1 +cmsis.document_link.v1 +svd.peripheral.v1 +svd.register.v1 +``` + +Do not stretch device metadata into electrical-property claims it cannot support. + +## Deliverable 13: cross-source combiners + +Implement these explicit vote producers: + +```text +agreement.independent.v1 +agreement.structured_document.v1 +agreement.subject_triangulation.v1 +conflict.source.v1 +``` + +### Independent agreement + +Boost only after lineage deduplication. + +### Structured/document agreement + +A structured fact matching exact document evidence is strong alignment supervision even when both ultimately come from the same manufacturer; its strength is evidence localization, not independent authority. + +### Subject triangulation + +Combine independent identifier/package/document signals to improve which part/family/variant a table applies to. + +### Source conflict + +Retain the conflicting candidates and emit abstention/review priority. Never silently select one value solely because more transports copied it. + +## Deliverable 14: label combination + +Start with an auditable calibrated weighted combiner: + +```text +hard gates + | +high-precision votes + | +lineage deduplication + | +calibrated weights from reviewed development set + | +probabilistic target + entropy/disagreement +``` + +Define `TrainingLabel/v1`: + +```text +TrainingLabel/v1 { + id + training_candidate_id + task + target_json + posterior_probability? + supervision_class + input_vote_ids[] + combiner_version + review_id? +} +``` + +Supervision classes: + +```text +deterministic_high +weak_consensus +teacher_reviewed +human_gold +``` + +Later evaluate a Snorkel-style generative label model that estimates labeler accuracies/correlations from overlap, but do not make an opaque combiner a prerequisite for the first corpus. + +## Deliverable 15: active-learning priority + +Compute a queue score from: + +```text +vote disagreement/entropy +model uncertainty when FDX5 exists +new manufacturer +new template family +new predicate/alias +unusual unit/value syntax +OCR/refinement damage +source conflict +rare package/variant scope +high Foundry enrichment priority +``` + +This queue becomes the input to FDX5 teacher/human review rather than sampling PDFs uniformly. + +## Rights enforcement + +Training rights are hard gates. + +The compiler may derive ordinary extraction/validation findings from a source when permitted while refusing to export the same observation into a training dataset if `may_train` is false. + +A `TrainingLabel`/`TrainingExample` retains the exact rights snapshots contributing to its eligibility decision. + +## Tests + +Required: + +- every labeler has retained positive/negative/abstention fixtures; +- identity gates block technical votes for mismatched subjects; +- value alignment abstains on duplicate matching values; +- dimensional incompatibility blocks a predicate vote; +- wrong-regime matching value can be a ranking negative without becoming a negative engineering assertion; +- blank/dash never becomes zero; +- lineage-deduplicated agreement differs from naive source-count agreement; +- source conflict is retained and does not force a winner; +- a source vocabulary mapping version change creates new derivation identity; +- label combination is deterministic for fixed votes/calibration; +- training-rights forbidden observations cannot produce exportable training examples; +- all votes trace to exact source/sketch/evidence records. + +## Acceptance criteria + +FDX3 is complete when: + +1. at least two materially different structured source families produce retained supervision votes on a real datasheet corpus; +2. one stable source vocabulary mapping supervises many records without per-PDF LLM calls; +3. every aggregate training label can be decomposed back into exact votes and source/evidence inputs; +4. cross-source agreement is lineage-aware; +5. conflicts/ambiguity increase review priority rather than hallucination pressure; +6. a reviewed development sample provides measured precision/calibration for major labelers; +7. the resulting training labels are large enough to begin FDX4/FDX5 while remote LLM labeling remains optional. + +## Landed + +Record source schemas, vocabulary-map versions, labeler precision/coverage/abstention, rights exclusions, vote counts and active-learning queue composition after implementation. diff --git a/docs/fdx/FDX4_CORPUS_AND_EVALUATION.md b/docs/fdx/FDX4_CORPUS_AND_EVALUATION.md new file mode 100644 index 0000000..e680d47 --- /dev/null +++ b/docs/fdx/FDX4_CORPUS_AND_EVALUATION.md @@ -0,0 +1,326 @@ +# FDX4 — Template propagation, synthetic corpus, and protected evaluation + +Status: planned + +Owner: Datasheet-cli + +Depends on: FDX3 label compiler + +## Objective + +Turn weak supervision into a leakage-resistant, diverse training/evaluation corpus and multiply reviewed work through table-template propagation and synthetic DocumentIR/Sketch generation. + +FDX4 must make it difficult for a model to look good merely by memorizing one manufacturer's recurring document layout. + +## Deliverable 1: template signatures + +Compute deterministic table/section signatures from generic structure plus limited source identity where appropriate: + +```text +normalized section-heading path +column count +normalized column-header tokens +row/column span pattern +header-depth pattern +symbol-column presence +unit-column presence +condition-column presence +x/width ratio buckets +row density/geometry ratios +manufacturer/template hints when available +``` + +The signature is a clustering feature, not proof that two tables have identical semantics. + +## Deliverable 2: template-family clustering + +Cluster recurring structures within and across manufacturers. + +Required outputs: + +```text +template_family_id +member table IDs +representative/medoid IDs +structural purity metrics +semantic label distribution where reviewed +outlier score +``` + +A review/teacher can label the representative of a high-purity cluster and propagate the mapping only under an explicit versioned `template.cluster` labeling function. + +Outliers are not forced into the propagated label. + +## Deliverable 3: template review workflow + +Review one representative with context: + +```text +section heading/path +normalized headers +several representative rows +candidate column roles +sample source anchors +cluster size/purity/outliers +``` + +Review result can define: + +```text +table semantic type +column roles +regime constraints +known manufacturer/template scope +``` + +One reviewed template should be able to supervise thousands of structurally identical tables without one LLM call per document. + +## Deliverable 4: synthetic sketch/IR generator + +Generate exact labeled examples at the domain-structure level before paying to render PDFs. + +Synthetic families should vary: + +```text +column order +missing Min/Typ/Max fields +merged headers +multi-line parameter names +multi-line conditions +row/column spans +footnotes +package-specific columns +variant-specific columns +unit column vs embedded units +µ/u normalization +Ω/Ohm variants +scientific notation +inequalities +± tolerances +ranges +dash/blank/NA conventions +symbolic relative values such as VDD+0.3V +repeated identical numeric values in different regimes +similar predicate names with identical dimensions +``` + +Generated truth includes exact: + +```text +section/regime +cell roles +predicate term +value candidate +condition/applicability spans +subject/package scope +``` + +Synthetic data augments real data; it does not replace held-out real evaluation. + +## Deliverable 5: rendered/degraded synthetic subset + +Render a minority of synthetic families into PDFs/documents and feed them back through Ferrodoc to expose downstream training to reconstruction noise. + +Controlled variations may include: + +```text +font changes +kerning/spacing changes +vector vs raster text +low DPI +rotation +compression +blur/noise +broken/absent ToUnicode mapping +multi-column pages +ruling-line and whitespace-only tables +``` + +Every rendered/degraded derivative shares the parent `document_family_id` so related variants cannot cross train/test partitions. + +## Deliverable 6: real corpus family identity + +Define document families using conservative signals such as: + +```text +exact PDF/text hash ancestry +manufacturer + product family +document revision lineage +near-duplicate normalized page/table hashes +known template lineage +source document-version relations +``` + +If uncertain whether two revisions are related, prefer keeping them in the same family to reduce leakage. + +## Deliverable 7: `TrainingExample/v1` + +Export examples as immutable references rather than giant flattened prompt blobs: + +```text +TrainingExample/v1 { + example_id + task + + document_ir_logical_sha256 + datasheet_sketch_digest + candidate_ids[] + target_json + training_label_id + + document_family_id + template_family_id? + manufacturer_key? + + ontology_snapshot_digest + rights_snapshot_ids[] + producer +} +``` + +A dataset manifest records exact shard artifacts, example count, split policy, ontology/schema revisions and input parent batches. + +## Deliverable 8: split policy + +Required logical slices: + +```text +train +development +document-family-held-out +template-held-out +manufacturer-held-out +final-blind-test +``` + +The first four may overlap in the sense of evaluation tags only if the manifest makes their policy explicit; the final blind test must be protected from model/labeler iteration. + +Random row-level split is not a valid primary evaluation. + +No document family may cross training and any family-held-out/final partition. Related synthetic degradations remain together. + +## Deliverable 9: trusted reviewed corpus + +Build a claim-level trusted set rather than manually annotating entire PDFs unnecessarily. + +Review should verify: + +```text +subject scope +section/regime +predicate +min/typ/max/value role +exact raw value/unit +condition/applicability evidence +evidence anchor +``` + +Prefer a few thousand diverse, carefully reviewed claims spanning manufacturers/templates over tens of thousands of shallow whole-PDF labels. + +Teacher-generated labels are not automatically `human_gold`. Preserve their supervision class separately. + +## Deliverable 10: hard-negative generation + +Generate difficult ranking negatives from real candidate sets: + +```text +same quantity dimension, wrong predicate +same numeric value, wrong section/regime +same symbol, different device context +same predicate, wrong package/variant +nearby row, wrong value candidate +same source field, wrong part identity +``` + +These are training negatives for candidate selection. They do not become negative real-world assertions. + +## Deliverable 11: benchmark metrics + +Report each factor separately: + +```text +section/table-type precision/recall +cell-role accuracy +predicate retrieval recall@K +predicate rerank top-1/MRR +regime precision/recall +condition/applicability span F1/exact match +subject/package scope accuracy +complete claim exact match +evidence-anchor validity +grounding validity +abstention/coverage +calibration (ECE/Brier where appropriate) +``` + +Also report end-to-end quality by: + +```text +manufacturer +template family +document type/revision family +native vs OCR/refined evidence +supervision class +``` + +## Deliverable 12: safety gates + +Qualification treats these as hard failures: + +```text +model/compiler value not present in deterministic candidate set +invalid evidence anchor +wrong source document identity +silent absolute-max/recommended-operating regime collapse +training/test family leakage +use of training-rights-forbidden example +``` + +Coverage is not improved by suppressing these failures from the denominator. + +## Deliverable 13: FDX2 baseline freeze + +Run the FDX2 deterministic compiler on every protected slice before training the FDX5 model. + +The evaluation artifact records: + +```text +compiler commit/configuration +ontology/Quantitas snapshots +Ferrodoc version/IR schema +corpus manifest +per-case outputs +metric code digest +results +``` + +FDX5 must compare on identical cases. + +## Tests + +Required: + +- related revisions cannot cross family split; +- synthetic derivatives inherit parent family; +- template propagation refuses low-purity/outlier clusters; +- dataset builder rejects rights-forbidden examples; +- manifest reproduces exact shard/example identities; +- final blind cases are inaccessible to ordinary training export path; +- hard negatives reference valid alternative candidates but are never emitted as database negations; +- metrics count failures/missing predictions honestly; +- deterministic baseline report is immutable/content-identifiable. + +## Acceptance criteria + +FDX4 is complete when: + +1. a real weakly supervised corpus is frozen with reproducible family/template/manufacturer splits; +2. at least one reviewed table template propagates supervision to many real tables with measured audited precision; +3. a synthetic structure corpus covers the known first-task edge cases and a rendered subset has passed through Ferrodoc; +4. a trusted claim-level evaluation set exists independently of weak-label training targets; +5. FDX2 baseline reports are frozen on all protected slices; +6. no detected family/template leakage or rights violation remains in the qualified dataset. + +## Landed + +Record dataset manifests, split policy/version, template families, reviewed corpus size/composition, synthetic families and frozen baseline reports after implementation. diff --git a/docs/fdx/FDX5_COMPACT_MODEL.md b/docs/fdx/FDX5_COMPACT_MODEL.md new file mode 100644 index 0000000..2ae07b4 --- /dev/null +++ b/docs/fdx/FDX5_COMPACT_MODEL.md @@ -0,0 +1,502 @@ +# FDX5 — Compact IR-native model and active teacher + +Status: planned + +Owner: Datasheet-cli + +Depends on: FDX4 frozen corpus/baselines + +## Objective + +Train and qualify a small local model over `DatasheetSketch` candidates, export it to a reproducible ONNX bundle, run it through a Rust inference path, and use remote LLM/VLMs only for active-learning/fallback cases where deterministic/compact-model evidence is insufficient. + +The first model must not consume page pixels in the ordinary path and must not generate arbitrary engineering values. + +## Initial architecture decision + +Start with an ELECTRA-Small-class encoder rather than a document VLM or large generative model. + +Reference shape: + +```text +~14M parameters +12 transformer layers +hidden size 256 +embedding size 128 +4 attention heads +``` + +This is a starting implementation target, not a permanent model brand dependency. Preserve a feature/model interface that permits replacing the encoder after measured comparisons. + +Also benchmark one small distilled 6-layer/~384-hidden encoder class as a challenger if export/runtime support is straightforward. Do not proliferate architectures before the baseline experiment exists. + +## Why text/IR rather than pixels + +Ferrodoc has already paid the generic document-understanding cost: + +```text +PDF -> native/OCR evidence -> geometry -> regions -> tables -> reading order +``` + +The FDX model solves semantic linking over compact candidate windows: + +```text +what kind of table/section is this? +which cell is min/typ/max/condition? +which predicate does this row mean? +which regime applies? +which subject/package/variant applies? +which evidence spans are conditions? +``` + +A page-image encoder would duplicate generic layout work, increase compute and weaken the exact evidence-selection boundary. + +## Deliverable 1: canonical model feature schema + +Define `datasheet-model-features/v1`. + +Each candidate window contains text tokens plus structural features available before the model's target decision. + +### Text serialization + +Conceptual window: + +```text +[SECTION] +Recommended Operating Conditions + +[HEADERS] +Parameter | Symbol | Conditions | Min | Typ | Max | Unit + +[ROW-1] +... + +[ROW] +Supply voltage | VDD | - | 1.8 | - | 5.5 | V + +[ROW+1] +... + +[FOOTNOTE] +1. ... + +[SUBJECT] +raw part/family/package hints +``` + +Default target sequence length: 128 tokens. + +Allow 256 for complex rows/tables. Longer context requires an explicit benchmark showing benefit rather than silently increasing all inference cost. + +### Structural embeddings/features + +Encode at least: + +```text +region kind +table row index bucket +table column index bucket +header/body role candidate +normalized x bucket +normalized y bucket +page-position bucket +native/OCR/reconciled evidence source +geometry-quality class +quantity-kind/dimensionality hint +unit/value-shape hint +``` + +Use embeddings/projections compatible with the selected encoder. Do not serialize every numeric coordinate as verbose text if a bounded structural feature is more stable. + +All feature computation is deterministic and versioned. + +## Deliverable 2: shared encoder and factor heads + +Use one encoder for the first task family with small heads for: + +```text +section/table semantic type +cell/column role +specification regime +applicability/condition selection +subject/package/variant relationship +``` + +Heads output probabilities over candidate-local labels/relationships, not final database facts. + +## Deliverable 3: predicate retrieval instead of fixed classifier + +Foundry/Datasheet predicates are vocabulary-as-data. Do not implement a permanently fixed `Linear(hidden, N_predicates)` as the primary linker. + +Use a two-stage design: + +```text +row/context encoder + | + v +candidate embedding + | + +---- retrieve top-K predicate terms from ontology snapshot + | + v + compact reranker + | + v + predicate term ID +``` + +Predicate term text includes: + +```text +canonical name +description +aliases +symbol aliases +expected quantity kind/dimension +expected value shapes +regime/category constraints +``` + +Apply deterministic compatibility filters before neural ranking, especially dimensionality and impossible regime/category constraints. + +Adding a new predicate term should be representable by adding vocabulary data and embeddings. Retraining may improve ranking but is not required to represent the new term. + +## Deliverable 4: predicate bi-encoder + +Train contrastive representations so matching row/context and predicate definitions are near each other. + +Hard negatives should dominate over random negatives: + +```text +same dimension, wrong property +same symbol family, wrong context +same English noun, wrong regime +same numeric value, wrong row +same device category, adjacent property +``` + +Cache predicate embeddings by ontology snapshot digest. + +## Deliverable 5: compact reranker + +Evaluate two options on the protected development set: + +1. cross-encode the row window plus each top-K predicate description with the same/smaller encoder; +2. a cheaper MLP/bilinear reranker over row/predicate embeddings plus deterministic compatibility features. + +Prefer the cheaper option unless the cross-encoder produces a material held-out precision gain. + +`K` begins small (for example 8-16) and is benchmark-selected. + +## Deliverable 6: multi-task training objective + +Conceptual loss: + +```text +L = + λ_pred_retrieval * contrastive_predicate_loss + + λ_pred_rerank * predicate_ranking_loss + + λ_cell * cell_role_loss + + λ_regime * regime_loss + + λ_section * section_loss + + λ_applicability * applicability_loss + + λ_scope * subject_scope_loss + + λ_consistency * structural_consistency_loss +``` + +Do not choose weights by intuition alone. Record them in the training config and tune only on the development partition. + +## Deliverable 7: supervision weighting + +Consume FDX3 probabilities/classes rather than flattening all labels to equal truth. + +Examples: + +```text +human_gold full weight +teacher_reviewed high weight, separately measured + deterministic_high high weight after audited precision gate +weak_consensus posterior/confidence weighted +low-confidence omit or low weight +``` + +Exact weights are configuration, not schema semantics. + +## Deliverable 8: optional domain-adaptive pretraining + +First train/evaluate using the pretrained encoder unchanged except supervised fine-tuning. + +Only if error analysis shows vocabulary/context limitations, run additional domain-adaptive pretraining over large unlabeled/weakly labeled datasheet text/table serialization. + +Potential objectives: + +```text +ELECTRA-style replaced-token detection +masked language modeling if using a non-ELECTRA challenger +``` + +Domain pretraining uses the same family split discipline: protected evaluation/final-blind content is excluded. + +Do not block the first model on this step. + +## Deliverable 9: no-generation output contract + +Inference produces a versioned result containing selected local IDs: + +```text +ModelDecision/v1 { + model_bundle_digest + feature_schema_digest + ontology_snapshot_digest + + predicate_candidate_id + value_candidate_id + regime_id + condition_candidate_ids[] + subject_candidate_id + + per-factor probabilities + calibration_status +} +``` + +Validation then checks every selected ID against the exact sketch/candidate set. + +There is no free-form numeric `value` field. + +## Deliverable 10: training/export tool + +`tools/datasheet-train` supports reproducible operations conceptually equivalent to: + +```text +dataset validate +train +export onnx +evaluate +compare +quantize +manifest +``` + +The training run records: + +```text +git commit +dataset manifest/digest +ontology snapshot +feature schema +tokenizer +base model identity +random seeds +optimizer/scheduler +batch/accumulation +sequence lengths +loss weights +epoch/step counts +hardware/software environment facts needed for reproduction +``` + +Training may use Python/PyTorch. Generated ONNX/model artifacts are the production boundary. + +## Deliverable 11: model bundle + +Produce an immutable bundle: + +```text +model.onnx +model.int8.onnx? +tokenizer.json +feature-schema.json +ontology-snapshot.json +calibration.json +manifest.json +evaluation-report.json +``` + +Manifest binds digests for every component plus: + +```text +training dataset manifest +code revision +training config digest +metric/evaluator version +license/provenance +``` + +Do not activate a model if tokenizer/feature/ontology compatibility does not match the manifest. + +## Deliverable 12: Rust inference through RTen + +Use RTen as the first production inference runtime because it is Rust-native, accepts ONNX and is already used in the broader Ferrodoc ecosystem. + +`datasheet-model` must provide: + +```text +bundle validation +model loading +feature encoding +dynamic/batched inference +output decoding +probability calibration +grounding validation +resource measurement hooks +``` + +Keep the backend behind a narrow internal trait if doing so is cheap, but do not add ONNX Runtime/native dependencies until a benchmark demonstrates a need. + +## Deliverable 13: quantization + +Evaluate int8/uint8 quantization after the fp32 model is qualified. + +Report: + +```text +bundle bytes +resident memory +p50/p95 latency +batch throughput +per-factor accuracy/precision delta +complete-claim delta +``` + +Admit quantized inference only if quality loss stays below a declared threshold and there is a material deployment benefit. + +Do not assume quantization is faster on every CPU. + +## Deliverable 14: training compute gate + +The first model must be trainable on one ordinary single-GPU development machine rather than requiring distributed training. + +Initial targets: + +```text +~500k-2M factorized training examples if corpus provides them +128 token default windows +256 token complex windows +2-4 supervised epochs as an initial search range +full fine-tuning, not LoRA by default +``` + +These are experiment starting points, not acceptance claims. Record actual GPU memory, wall time and examples/sec. + +LoRA/adapters are unnecessary complexity for a ~14M parameter first model unless experiments show a specific benefit. + +## Deliverable 15: model qualification + +Compare against FDX2 deterministic baseline on identical FDX4 protected cases. + +Required slices: + +```text +overall development +document-family-held-out +template-held-out +manufacturer-held-out +native evidence +OCR/refined evidence +``` + +Metrics remain factorized and end-to-end. + +Hard gates: + +1. zero tolerated ungrounded/generated values; +2. 100% syntactically valid evidence-anchor/candidate references for accepted outputs; +3. no detected split leakage; +4. high-confidence precision must meet a declared threshold before automatic candidate acceptance; +5. a learned model must materially improve at least one declared quality/coverage frontier over FDX2 without unacceptable regressions elsewhere. + +A model may be retained as a negative experiment without being activated. + +## Deliverable 16: active teacher + +Teacher priority score combines: + +```text +weak-label disagreement +student entropy/calibration uncertainty +new manufacturer +new template family +new predicate/alias +unusual unit/value form +source conflict +OCR/refinement damage +rare package/variant scope +Foundry enrichment priority +``` + +Teacher input is compact by default: + +```text +source structured facts +section/table/row sketch +candidate predicate definitions +candidate value/evidence IDs +weak-label votes and disagreement +student decision/probabilities +``` + +Teacher output chooses candidate IDs/labels or `ABSTAIN`. It does not return arbitrary engineering values. + +Batch multiple rows from one table/request when practical. + +## Deliverable 17: teacher accounting and distillation loop + +Record: + +```text +provider/model +prompt/schema version +input/output artifact digests where policy permits +token counts +monetary cost +candidate count +accepted labels +review corrections +trigger reasons +``` + +Metrics: + +```text +tokens per accepted teacher label +cost per accepted teacher label +teacher-label audited precision +fraction of corpus requiring teacher +student improvement after adding teacher labels +``` + +Teacher-reviewed labels feed the next frozen dataset version; they never mutate an old dataset in place. + +## Tests + +Required: + +- deterministic feature encoding for fixed sketch/schema; +- predicate embedding cache invalidates on ontology/model/feature changes; +- dimensional filters prevent impossible predicate retrieval; +- output IDs must exist in candidate set; +- model cannot serialize a free-form value; +- bundle refuses mismatched tokenizer/feature/ontology artifacts; +- same model/input produces stable inference within backend determinism guarantees; +- protected split evaluation uses identical cases as baseline; +- quantized bundle records distinct digest/evaluation; +- teacher output containing arbitrary values or invalid IDs is rejected; +- teacher abstention remains abstention rather than negative truth. + +## Acceptance criteria + +FDX5 is complete when: + +1. one compact local model is frozen with reproducible training/evaluation artifacts; +2. Rust/RTen inference produces grounded candidate selections on real retained datasheets; +3. model quality is compared against FDX2 on family/template/manufacturer held-out slices; +4. no accepted output can introduce an ungrounded engineering value; +5. CPU model size/latency/throughput are measured; +6. optional quantization is admitted only on measured quality/resource tradeoffs; +7. active teacher can label ambiguous candidate decisions without whole-PDF upload in the normal case; +8. teacher cost/coverage is measurable and the resulting labels enter a new immutable dataset version. + +## Landed + +Record final encoder/runtime architecture, model/dataset manifests, training compute, protected metrics, CPU/quantization benchmarks, calibration gates and teacher economics after implementation. diff --git a/docs/fdx/FDX6_PRODUCTION_CASCADE.md b/docs/fdx/FDX6_PRODUCTION_CASCADE.md new file mode 100644 index 0000000..c24304d --- /dev/null +++ b/docs/fdx/FDX6_PRODUCTION_CASCADE.md @@ -0,0 +1,304 @@ +# FDX6 — Production cascade and Foundry integration + +Status: planned + +Owner: Datasheet-cli runtime/integration with Foundry FLS6 + +Depends on: FDX5 qualified local model; Foundry FLS5 document plane and FLS4 silver/promotion boundary + +## Objective + +Move the FDX compiler/model into the real Foundry enrichment path as a progressive, budget-aware cascade. Cheap deterministic/local extraction runs broadly; expensive table/OCR refinement and remote teacher/VLM work run only on ambiguous/high-value cases. + +## Deliverable 1: serialized Foundry boundary + +Stabilize the production serialization for: + +```text +DatasheetSketch/v1 +ClaimBundle/v1 +DesignConstraintBundle/v1 +ModelDecision/v1 +ExtractionTrace/v1 +``` + +Foundry consumes serialized contracts and exact Artifactum/IR references. It does not import Datasheet internal model/training implementation types. + +## Deliverable 2: progressive pass contract + +Expose explicit stages equivalent to: + +```text +P0 inspect +P1 cheap document reconstruction +P2 local datasheet compile/model +P3 targeted document refinement +P4 remote teacher/fallback +P5 human review handoff +``` + +Datasheet-cli directly owns P2 and P4 semantic behavior; Ferrodoc owns P1/P3 document evidence; Foundry owns scheduling/prioritization/persistence. + +Each stage consumes immutable identities and produces a new deterministic/provenance-bearing artifact or explicit observational remote result. + +## Deliverable 3: P2 local extraction API + +Provide a library/CLI operation accepting: + +```text +pinned DocumentIR +ontology snapshot +Quantitas registry snapshot +qualified model bundle (optional; deterministic-only mode supported) +compiler configuration +``` + +and returning: + +```text +DatasheetSketch +ClaimBundle candidates +ExtractionTrace +escalation recommendations +``` + +The operation is offline/network-free when no remote teacher is requested. + +## Deliverable 4: escalation reasons + +Use stable reason codes, initially: + +```text +insufficient_geometry +ambiguous_table_structure +quantity_parse_failure +predicate_low_margin +regime_conflict +subject_scope_ambiguous +source_conflict +weak_label_disagreement +model_low_confidence +unseen_template +unseen_manufacturer +unsupported_formula_or_graph +``` + +A recommendation includes target page/region IDs and required capability where possible. + +## Deliverable 5: targeted P3 refinement + +Translate eligible escalation reasons into Ferrodoc refinement requests rather than rerunning the whole PDF. + +Example: + +```text +ambiguous_table_structure + -> table region R17 + -> request stronger table recognition only for R17/page 6 + -> new pinned IR generation + -> re-run affected DatasheetSketch/claim derivations +``` + +Recompute only descendants whose semantic inputs changed. Unrelated accepted claims remain cache hits. + +## Deliverable 6: remote P4 fallback + +Remote teacher/fallback receives compact structured context by default. + +Allowed request modes: + +```text +candidate_selection +source_vocabulary_mapping +template_review +ambiguous_row_review +compact_region_interpretation +whole_pdf_vlm_last_resort +``` + +`whole_pdf_vlm_last_resort` must require explicit policy and be separately metered. + +Teacher results retain provider/model/prompt/schema/token/cost provenance and must pass the same grounding validator before normal claim candidates are emitted. + +## Deliverable 7: policy profiles + +Expose useful runtime profiles conceptually: + +```text +bulk-cheap +interactive +high-confidence +training-harvest +``` + +Example intent: + +### `bulk-cheap` + +P0-P2 broadly, no remote calls, P3 only under cheap explicit bounds. + +### `interactive` + +Allow targeted P3 and small P4 request when a user is waiting for one requested property. + +### `high-confidence` + +Escalate important disagreements/low-margin candidates before recommending promotion. + +### `training-harvest` + +Prioritize novel/disagreeing examples for teacher/review labels rather than maximizing immediate claim coverage. + +Foundry remains the final budget/policy authority. + +## Deliverable 8: local model hot path + +Load the qualified model bundle once per worker/process where possible. + +Support row/table batching while preserving per-candidate trace identity. + +Measure cold-load and warm throughput separately. Do not repeatedly load the model for each row/PDF. + +## Deliverable 9: cache/derivation boundaries + +Separate cache identities for: + +```text +DocumentIR generation +DatasheetSketch +quantity/candidate index +rule extraction +model decisions +ClaimBundle assembly +refined region IR +teacher result +``` + +A new promotion rule does not invalidate extraction. A new ontology snapshot invalidates predicate-link descendants but should not require re-OCR. A new model invalidates model decisions/claim descendants but reuses deterministic sketch/candidates. + +## Deliverable 10: incremental document revisions + +Where Foundry/Ferrodoc can identify unchanged page/table evidence across document revisions, reuse lower-level extraction artifacts according to their stable content identity. + +Do not assume two visually similar revisions are byte/semantic identical; reuse only content-addressed sub-results whose inputs actually match. + +## Deliverable 11: confidence/calibration routing + +Apply model calibration only to the model factors it was evaluated for. + +A final extraction confidence remains a vector/structured explanation including: + +```text +identity evidence +source authority/lineage +structural extraction confidence +predicate model probability/margin +regime confidence +grounding validity +source agreement/conflict +``` + +Do not collapse everything into an unexplained scalar that Foundry treats as promotion truth. + +## Deliverable 12: feedback into training + +Production generates new immutable observations: + +```text +novel template +new manufacturer/source field +model disagreement +teacher correction +human correction +source conflict +promotion rejection reason +cross-validation finding +``` + +FDX6 exports eligible records to the next training-candidate derivation. Retraining is explicit and produces a new dataset/model version; production corrections never mutate old model provenance. + +## Deliverable 13: quality/drift monitoring + +Track by model/compiler version: + +```text +candidate volume +accepted-by-extractor coverage +abstention/escalation reasons +P3 refinement rate +P4 remote rate/cost +invalid grounding count +predicate/regime confidence distributions +new template/manufacturer rates +source-conflict rates +sampled audited precision +``` + +Trigger reevaluation/retraining proposals on meaningful drift, not automatically on every new document. + +## Deliverable 14: first end-to-end proving run + +Run the full path on the initial analog/power corpus: + +```text +source structured facts +manufacturer PDFs +Ferrodoc DocumentIR +DatasheetSketch +FDX2 rules +FDX5 model +selective P3 cases +limited P4 active teacher cases +Foundry Silver ClaimBundle ingestion +sample promotion/cross-validation +``` + +Attach exact version/digest identities and cost/resource metrics. + +## Deliverable 15: remote-dependency removal from default commands + +Make the ordinary extraction command choose the local path by default once it is qualified for a supported task family. + +Preserve explicit remote commands/options for: + +```text +unsupported task family +user-requested remote extraction +last-resort fallback +teacher/review workflow +``` + +Do not silently send source documents to a remote model. + +## Tests + +Required: + +- P2 local operation runs with network disabled; +- escalation identifies exact target region where possible; +- P3 changes only affected downstream derivations; +- unmodified claims remain cache/reuse hits after targeted refinement; +- P4 response with invented value fails grounding validation; +- whole-PDF remote mode requires explicit policy; +- remote cost/token trace is retained; +- model bundle is reused across batched documents rather than loaded per row; +- ontology/model changes invalidate only appropriate descendants; +- production feedback creates new immutable training candidates rather than mutating historical labels; +- deterministic-only mode remains available if the learned model is unqualified/unavailable. + +## Acceptance criteria + +FDX6 is complete when: + +1. Foundry can run supported datasheet enrichment through the local Datasheet compiler/model path without a remote model; +2. ambiguous tables/regions can selectively escalate through Ferrodoc refinement; +3. remote teacher/fallback is an explicit measured minority path on the proving corpus; +4. every emitted claim value remains grounded to immutable source/evidence candidates; +5. model/compiler/dataset/ontology identities make every candidate extraction reproducible; +6. production feedback can generate the next immutable training dataset version; +7. cost/latency/coverage/quality metrics demonstrate the cascade's actual economics; +8. the existing full-PDF Gemini path is no longer required for the supported first task family. + +## Landed + +Record Foundry/Ferrodoc pins, proving-run manifests, P0-P5 rates, CPU performance, remote cost, extraction quality and feedback/retraining outputs after implementation. diff --git a/docs/fdx/README.md b/docs/fdx/README.md new file mode 100644 index 0000000..b8995a1 --- /dev/null +++ b/docs/fdx/README.md @@ -0,0 +1,113 @@ +# FDX datasheet extraction and model-training program + +Status: planned + +Owner: Datasheet-cli for FDX1-FDX6 + +Upstream dependency: Ferrodoc FDX0 + +Production integration: Foundry FLS2/FLS4/FLS5/FLS6 + +## Goal + +Turn Datasheet-cli from a PDF-to-remote-LLM command into a reusable electronics-specific document compiler with a cheap deterministic path, weak-supervision/data compiler, compact local model and explicit remote-teacher fallback. + +The existing remote Gemini extraction commands remain useful as a fallback/teacher and compatibility surface. They stop being the architectural center of the project. + +## Program map + +| Phase | Purpose | +|---|---| +| FDX0 (Ferrodoc) | evidence-grade native geometry, table cells and selective refinement | +| [FDX1](FDX1_WORKSPACE_AND_CONTRACTS.md) | workspace split, domain contracts, `DatasheetSketch/v1` | +| [FDX2](FDX2_DETERMINISTIC_COMPILER.md) | deterministic electrical-table compiler and qualification baseline | +| [FDX3](FDX3_WEAK_SUPERVISION.md) | source-taxonomy mappings and weak-label compiler | +| [FDX4](FDX4_CORPUS_AND_EVALUATION.md) | template propagation, synthetic corpus, leakage-safe benchmark | +| [FDX5](FDX5_COMPACT_MODEL.md) | compact IR-native model, RTen inference and active teacher | +| [FDX6](FDX6_PRODUCTION_CASCADE.md) | production cascade, Foundry integration, retraining/evaluation loop | + +## Repository boundary + +Datasheet-cli owns electronics interpretation: + +```text +Ferrodoc DocumentIR + | + v +DatasheetSketch + | + +--> deterministic candidate compiler + +--> weak-supervision compiler + +--> compact model + +--> teacher/review tooling + | + v +ClaimBundle / DesignConstraintBundle +``` + +It does not own: + +- manufacturer/source acquisition policy; +- Foundry canonical identities or promotion state; +- Artifactum storage internals; +- generic PDF/OCR/table engines; +- a competing unit/dimensional registry. + +Foundry supplies immutable observations/rights/lake manifests and consumes serialized claim bundles. Ferrodoc supplies immutable DocumentIR/evidence. Quantitas supplies dimensional/unit semantics. + +## Workspace target + +The current project is a single binary crate containing remote LLM extraction, PDF handling and multiple data-source clients. FDX1 turns it into a small workspace with boundaries that correspond to real reuse/compile/runtime needs: + +```text +. +├── crates/ +│ ├── datasheet-core/ +│ ├── datasheet-compiler/ +│ └── datasheet-model/ +├── tools/ +│ └── datasheet-train/ +├── apps/ +│ └── datasheet-cli/ +├── docs/fdx/ +└── prompts/ +``` + +Do not create more crates merely to mirror every module. Training may use Python/PyTorch where that is the shortest path; production inference and domain contracts remain Rust. + +## Core safety invariant + +A learned model does not emit arbitrary engineering values. The compiler first enumerates deterministic candidates from exact DocumentIR/source evidence, then the model chooses candidate IDs/relationships. + +Expected learned output resembles: + +```text +predicate_candidate_id +value_candidate_id +regime_id +condition_candidate_ids[] +subject_candidate_id +``` + +Deterministic code constructs the final claim. A numeric/string value without an exact selected evidence/source candidate is invalid. + +## First task family + +Do not port every existing prompt at once. The first qualification corpus is analog/power semiconductor characteristics: + +```text +Absolute Maximum Ratings +Recommended Operating Conditions +Electrical Characteristics +Thermal Information +``` + +This task family has strong table structure, abundant structured-source supervision and immediate Foundry value. + +After the loop is proven, add pinouts, package/footprint semantics, power sequencing, high-speed constraints, boot configuration, reference designs and application circuits as additional task families. + +## Qualification philosophy + +Every learned component competes against a deterministic baseline on a protected case set. Family/template/manufacturer leakage is prohibited. Precision, grounding validity, evidence-anchor validity, resource cost and coverage remain separately visible. + +Remote LLM/VLM usage is measured as an escalation tier and should concentrate on new source vocabularies, new templates and high-disagreement examples rather than one request per PDF.