Skip to content

Add model training and export commands - #2

Open
Kaushik-Kumar-CEG wants to merge 1 commit into
gsoc/package-foundationfrom
gsoc/training-pipeline
Open

Add model training and export commands#2
Kaushik-Kumar-CEG wants to merge 1 commit into
gsoc/package-foundationfrom
gsoc/training-pipeline

Conversation

@Kaushik-Kumar-CEG

Copy link
Copy Markdown
Collaborator

Adds the model training and ONNX export modules.

  • trains the DeBERTa BIOES/CRF model from the generated dataset
  • keeps the best validation checkpoint
  • validates dataset records, truncation, and saved weights
  • installs train-required-phrase-model and export-required-phrase-model
  • adds focused tests, docs, and one ML CI job

This moves the training work from aboutcode-org/scancode-toolkit#5222 into this package.

Checks: 78 tests pass with ML dependencies, and the base package suite, Ruff, docs, and package builds pass.

Package the DeBERTa BIOES and CRF training pipeline with strict dataset and checkpoint validation. Add ONNX export, focused tests, documentation, and installed command entry points.

Signed-off-by: Kaushik Kumar <kaushikrjpm10@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant