Update ort and ort-genai package - #934
Open
baijumeswani wants to merge 3 commits into
Open
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
There was a problem hiding this comment.
Pull request overview
Updates ONNX Runtime dependencies and standardizes native package consumption on the CPU package.
Changes:
- Bumps ORT to 1.28.0 and ORT GenAI to 0.15.0.
- Replaces Foundry/GPU-specific ORT packages with
Microsoft.ML.OnnxRuntime. - Updates pipeline prefetching, packaging, and documentation.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
sdk_v2/js/script/install-native.cjs |
Uses the standard ORT package. |
sdk_v2/deps_versions.json |
Updates shared dependency versions. |
sdk_v2/cpp/nuget/pack.py |
Updates ORT package help text. |
sdk_v2/cpp/nuget/Microsoft.AI.Foundry.Local.Runtime.nuspec |
Changes the runtime dependency. |
sdk_v2/cpp/cmake/FindOnnxRuntime.cmake |
Simplifies standard ORT acquisition. |
.pipelines/v2/templates/steps-prefetch-nuget.yml |
Updates package prefetching. |
.pipelines/v2/templates/steps-build-windows.yml |
Updates parameter documentation. |
.pipelines/v2/templates/steps-build-linux.yml |
Removes GPU-package prefetching. |
.pipelines/v2/sdk_v2-pipeline-plan.md |
Documents CPU-only Linux builds. |
.pipelines/v2/sdk_v2-js-pipeline-plan.md |
Documents the standard ORT package. |
.pipelines/foundry-local-packaging.yml |
Synchronizes pipeline version pins. |
Comments suppressed due to low confidence (1)
.pipelines/v2/templates/steps-prefetch-nuget.yml:132
- After removing the GPU package, this list contains only two packages, so “All four packages” is now incorrect in the bash branch as well.
declare -a entries=(
"genai:Microsoft.ML.OnnxRuntimeGenAI.Foundry:${{ parameters.genaiVersion }}"
"ort:Microsoft.ML.OnnxRuntime:${{ parameters.ortVersion }}"
)
Comment on lines
82
to
84
| @{ key = 'genai'; id = 'Microsoft.ML.OnnxRuntimeGenAI.Foundry'; version = '${{ parameters.genaiVersion }}' }, | ||
| @{ key = 'ort'; id = 'Microsoft.ML.OnnxRuntime.Foundry'; version = '${{ parameters.ortVersion }}' } | ||
| @{ key = 'ort'; id = 'Microsoft.ML.OnnxRuntime'; version = '${{ parameters.ortVersion }}' } | ||
| ) |
…512 FP nondeterminism The exact digit a 0.5B greedy model emits is not stable across CPU ISAs: ORT 1.28 routes FP32 GQA single-token decode through a new flash/online-softmax GEMV kernel whose softmax reductions dispatch to AVX-512F variants on AVX-512 hosts (e.g. AMD EPYC 9V74) vs AVX2 elsewhere, giving a different FP accumulation order. A few-ULP delta flips the greedy argmax between near-tie digit tokens (CI: turn 2 5->3, turn 4 6->2). This is mathematically-equivalent FP reordering, not an SDK bug. Keep all mechanics assertions strict (finish reason, token counts, turn count, and the turn-3==turn-2 rewind-determinism check, which is bit-stable on a single host). Emit the exact-digit expectations as non-fatal GTEST_LOG_(WARNING) warnings, and document the cause plus less-invasive alternatives (ORT-level determinism test, wider-margin prompt, ORT_GQA_DISABLE_FLASH_ATTENTION=1) in a comment.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.