feat: add Core AI provider package#219
Draft
grabbou wants to merge 11 commits into
Draft
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
grabbou
commented
Jun 17, 2026
| - Exported `.aimodel` bundles from `apple/coreai-models` | ||
| - The Apple Core AI Models Swift Package linked to the host app target | ||
|
|
||
| ## Swift Package Setup |
Contributor
Author
There was a problem hiding this comment.
This is to be verified whether that setup process is sound.
grabbou
commented
Jun 17, 2026
| - `CoreAILM` for `coreAI.languageModel(...)` and sessions | ||
| - `CoreAIDiffusion` for `coreAI.imageModel(...)` | ||
| - `CoreAISegmentation` for `coreAI.segmenter(...)` | ||
| - `CoreAIObjectDetection` for `coreAI.objectDetector(...)` |
Contributor
Author
There was a problem hiding this comment.
I think we can turn this into config plugin that takes options of what to use, unless already the case.
Contributor
Author
There was a problem hiding this comment.
And by default, I think we can link all.
grabbou
commented
Jun 17, 2026
| @@ -0,0 +1,35 @@ | |||
| # Starter Models | |||
|
|
|||
| Source this list from `apple/coreai-models/python/src/coreai_models/model_registry.py` when updating the package catalog. Prefer iOS examples. Use macOS only when the registry has no iOS entry. | |||
Contributor
Author
There was a problem hiding this comment.
This preface is unnecessary
grabbou
commented
Jun 17, 2026
| | ------------------------------------------------------------ | ----------------------------- | ----------- | ------------------------------ | -------------------------------------------------------------- | | ||
| | Language sessions, text generation, structured output, tools | `qwen3-0.6b` | iOS + macOS | Implemented | Best first Core AI session demo. | | ||
| | Better iOS language quality | `qwen2.5-1.5b-instruct` | iOS + macOS | Implemented | Good second example after the session API works. | | ||
| | Larger iOS language validation | `qwen3-4b` | iOS + macOS | Implemented | Useful for memory and specialization testing. | |
Contributor
Author
There was a problem hiding this comment.
Here i think better this table to group by type, models, platform + link to model file as TBD, we willadd links later, i assume there somewhere available.
For things not avialable/not-exposed, we do not list them at all.
artus9033
changed the title
Contributor
Author
|
BTW @artus9033 this is ready for you to be taken into non-draft. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a new
@react-native-ai/core-aipackage for Apple Core AI model bundles exported fromapple/coreai-models.The package now follows the same light shape as the other React Native AI providers:
coreAIexposes AI SDK-compatible model adapters: language, image generation, text embeddings, and transcription.CoreAIexports the raw native TurboModule for Core AI-specific calls such as model inspection, specialization, task execution, and raw function execution.model.prepare()andmodel.createSession(...).core.tsfacade and no package-level static model catalog. Starter model data stays in docs/example code for now; real apps should load available models dynamically.Native Notes
React Native installs this package through CocoaPods, but Apple Core AI helpers live in Swift Package products. The host iOS app still needs to add
https://github.com/apple/coreai-modelsthrough SPM and link the products it uses:CoreAILMCoreAIDiffusionCoreAISegmentationCoreAIObjectDetectionFor Expo this should be handled by a config plugin that patches the generated Xcode project. Bare React Native apps can add the package in Xcode or with a deterministic project patching script.
Model Management TBD
Core AI does not load remote URLs directly. Apple’s API expects a local
.aimodelor.aimodelcURL, then specializes that local asset withAIModel(contentsOf:)orAIModel.specialize(contentsOf:).A follow-up should add a first-class model-management path that downloads remote model bundles into app storage, returns a local
source: { type: 'file', uri }, optionally specializes before first use, and persists bookmark data for specialized cached assets. The downloader and storage pieces should share helpers with other React Native AI runtimes where practical, instead of duplicating provider-specific plumbing.Validation
bun run --cwd packages/core-ai typecheckswiftc -parse packages/core-ai/ios/CoreAIError.swift packages/core-ai/ios/CoreAIImpl.swiftbun run lintbun run --cwd website buildbun run buildbun run typecheckCaveats
The Swift file parses locally, but full iOS compilation with Apple Core AI products linked still needs to be verified in an Xcode 27/iOS 27 app target. Embeddings, ASR, depth, super-resolution, classification, specialization persistence, remote model download/bookmark restore, and raw tensor execution still need real device/runtime validation before we claim they are production-ready.