Introduction A collection of on-device AI primitives for React Native with first-class Vercel AI SDK support. Run AI models directly on users' devices for privacy-preserving, low-latency inference without server costs.
#Why On-Device AI? Privacy-first: All processing happens locally—no data leaves the device Instant responses: No network latency, immediate AI capabilities Offline-ready: Works anywhere, even without internet Zero server costs: No API fees or infrastructure to maintain #DevTools The AI SDK Profiler plugin captures OpenTelemetry spans from Vercel AI SDK requests and surfaces them in Rozenite DevTools. DevTools are runtime agnostic, so they work with on-device and remote runtimes.
Learn more in the DevTools documentation.
#Available Providers #Apple Intelligence Native integration with Apple's on-device AI capabilities through @react-native-ai/apple:
Text Generation - Apple Foundation Models for chat and completion Embeddings - NLContextualEmbedding for semantic search and similarity Transcription - SpeechAnalyzer for fast, accurate speech-to-text Speech Synthesis - AVSpeechSynthesizer for natural text-to-speech Production-ready with instant availability on supported iOS devices.
#Llama Engine Run any GGUF model from HuggingFace locally using llama.rn through @react-native-ai/llama:
Text Generation - High-performance GGUF inference with full streaming support Model Management - Automatic downloading and caching from HuggingFace Customizable - Support for context size, threads, and GGUF-specific parameters Cross-platform parity with optimized performance on both iOS and Android.
#MLC Engine (Work in Progress) Run any open-source LLM locally using MLC's optimized runtime through @react-native-ai/mlc:
Support for popular models like Llama, Mistral, and Phi Cross-platform compatibility (iOS and Android) NOTE MLC support is experimental and not recommended for production use yet.
#JSON UI Build UIs from tool-calling models with @react-native-ai/json-ui:
Tool-based spec — Model calls tools to add/set/delete nodes and props GenerativeUIView — Renders the spec in React Native; override styles or supply a custom node renderer Small-model friendly — Designed for on-device models with limited context See the JSON UI docs for setup and API.
#Google (Coming Soon) Support for Google's on-device models is planned for future releases.
Get started by choosing the approach that fits your needs! Polyfills Several functions that are internally used by the AI SDK might not be available in the React Native runtime depending on your configuration and the target platform.
#Expo First, install the following packages:
npm install @ungap/structured-clone @stardazed/streams-text-encoding
Then create a new file in the root of your project with the following polyfills:
polyfills.js import { Platform } from 'react-native' import structuredClone from '@ungap/structured-clone'
if (Platform.OS !== 'web') { const setupPolyfills = async () => { const { polyfillGlobal } = await import('react-native/Libraries/Utilities/PolyfillFunctions')
const { TextEncoderStream, TextDecoderStream } =
await import('@stardazed/streams-text-encoding')
if (!('structuredClone' in global)) {
polyfillGlobal('structuredClone', () => structuredClone)
}
polyfillGlobal('TextEncoderStream', () => TextEncoderStream)
polyfillGlobal('TextDecoderStream', () => TextDecoderStream)
}
setupPolyfills() }
export {}
Make sure to import this file early in your app's entry point (e.g., at the top of your App.tsx or index.js):
import './polyfills'
#Bare React Native For projects not using Expo, you'll need additional stream polyfills since Expo provides some of these out of the box.
First, install the following packages:
npm install @ungap/structured-clone @stardazed/streams-text-encoding web-streams-polyfill
Then create a new file in the root of your project with the following polyfills:
polyfills.js import { Platform } from 'react-native' import structuredClone from '@ungap/structured-clone' import { TransformStream, ReadableStream, WritableStream, } from 'web-streams-polyfill'
if (Platform.OS !== 'web') { const setupPolyfills = async () => { const { polyfillGlobal } = await import('react-native/Libraries/Utilities/PolyfillFunctions')
const { TextEncoderStream, TextDecoderStream } =
await import('@stardazed/streams-text-encoding')
if (!('structuredClone' in global)) {
polyfillGlobal('structuredClone', () => structuredClone)
}
if (!('TransformStream' in global)) {
polyfillGlobal('TransformStream', () => TransformStream)
}
if (!('ReadableStream' in global)) {
polyfillGlobal('ReadableStream', () => ReadableStream)
}
if (!('WritableStream' in global)) {
polyfillGlobal('WritableStream', () => WritableStream)
}
polyfillGlobal('TextEncoderStream', () => TextEncoderStream)
polyfillGlobal('TextDecoderStream', () => TextDecoderStream)
}
setupPolyfills() }
export {}
Make sure to import this file early in your app's entry point (e.g., at the top of your App.tsx or index.js):
import './polyfills' AI SDK Profiler DevTools A Rozenite plugin that captures Vercel AI SDK spans and surfaces them in Rozenite DevTools. Inspect requests, inputs, outputs, provider metadata, and latency.
AI SDK Profiler preview
DevTools are runtime agnostic, so they work with on-device and remote runtimes.
#Installation npm install @react-native-ai/dev-tools
#Rozenite setup This plugin requires Rozenite to be installed and enabled in your app. Follow the Rozenite getting started guide to install and configure it: https://www.rozenite.dev/docs/getting-started
#Quick Start #1. Initialize the tracer and DevTools hook import { getAiSdkTracer, useAiSdkDevTools, } from '@react-native-ai/dev-tools';
export function App() { useAiSdkDevTools(); return ; }
#2. Enable AI SDK telemetry on requests import { generateText } from 'ai'; import { openai } from '@ai-sdk/openai'; import { getAiSdkTracer } from '@react-native-ai/dev-tools';
const tracer = getAiSdkTracer({ serviceName: 'my-app', });
await generateText({ model: openai('gpt-4o-mini'), prompt: 'Write a short story about a cat.', experimental_telemetry: { isEnabled: true, tracer, functionId: 'unique-identifier-where-the-call-comes-from', }, });
#3. Open DevTools Start your development server and open Rozenite DevTools. Select AI SDK Profiler to inspect the spans. Getting Started The Apple provider enables you to use Apple's on-device AI capabilities with the Vercel AI SDK in React Native applications. This includes language models, text embeddings, and other Apple-provided AI features that run entirely on-device for privacy and performance.
#Installation Install the Apple provider:
npm install @react-native-ai/apple
While you can use the Apple provider standalone, we recommend using it with the Vercel AI SDK for a much better developer experience. The AI SDK provides unified APIs, streaming support, and advanced features. To use with the AI SDK, you'll need v5 and required polyfills:
npm install ai
#Requirements React Native New Architecture - Required for native module functionality NOTE Different Apple AI features have varying iOS version requirements. Check the specific API documentation for compatibility details.
#Running on Simulator To use Apple Intelligence with the iOS Simulator, you need to enable it on your macOS system first. See the Running on Simulator guide for detailed setup instructions.
#Basic Usage Import the Apple provider and use it with the AI SDK:
import { apple } from '@react-native-ai/apple'; import { generateText } from 'ai';
const result = await generateText({ model: apple(), prompt: 'Explain quantum computing in simple terms' }); Generating You can generate response using Apple Foundation Models with the Vercel AI SDK's generateText or generateObject function.
#Requirements iOS 26+ - Apple Foundation Models is available in iOS 26 or later Apple Intelligence enabled device - Device must support Apple Intelligence #Text Generation import { apple } from '@react-native-ai/apple'; import { generateText } from 'ai';
const result = await generateText({ model: apple(), prompt: 'Explain quantum computing in simple terms' });
#Streaming import { streamText } from 'ai'; import { apple } from '@react-native-ai/apple';
const { textStream } = await streamText({ model: apple(), prompt: 'Write me a short essay on the meaning of life' });
for await (const delta of textStream) { console.log(delta); }
NOTE Streaming objects is currently not supported.
#Structured Output Generate structured data that conforms to a specific schema:
import { generateObject } from 'ai'; import { apple } from '@react-native-ai/apple'; import { z } from 'zod';
const schema = z.object({ name: z.string(), age: z.number().int().min(0).max(150), email: z.string().email(), occupation: z.string() });
const result = await generateObject({ model: apple(), prompt: 'Generate a user profile for a software developer', schema });
console.log(result.object); // { name: string, age: number, email: string, occupation: string }
#Tool Calling Enable Apple Foundation Models to use custom tools in your React Native applications.
#Important Apple-Specific Behavior Tools are executed by Apple, not the Vercel AI SDK, which means:
No AI SDK callbacks: maxSteps, onStepStart, and onStepFinish will not be executed Pre-register all tools: You must pass all tools to createAppleProvider upfront Empty toolCallId: Apple doesn't provide tool call IDs, so they will be empty strings #Setup All tools must be registered ahead of time with Apple provider. To do so, you must create one by calling createAppleProvider:
import { createAppleProvider } from '@react-native-ai/apple'; import { generateText, tool } from 'ai'; import { z } from 'zod';
const getWeather = tool({
description: 'Get current weather information',
inputSchema: z.object({
city: z.string()
}),
execute: async ({ city }) => {
return Weather in ${city}: Sunny, 25°C;
}
});
const apple = createAppleProvider({ availableTools: { getWeather } });
If you want to change the tools at runtime, you can do it as follows:
const apple = createAppleProvider({ availableTools: { getWeather } }); const model = apple();
model.updateTools({ getWeather, getDate });
#Basic Tool Usage Then, generate output like with any other Vercel AI SDK provider:
const result = await generateText({ model: apple(), prompt: 'What is the weather in Paris?', tools: { getWeather } });
#Inspecting Tool Calls You can inspect tool calls and their results after generation:
const result = await generateText({ model: apple(), prompt: 'What is the weather in Paris?', tools: { getWeather } });
// Inspect tool calls made during generation console.log(result.toolCalls); // Example: [{ toolCallId: '<< redacted >>', toolName: 'getWeather', input: '{"city":"Paris"}' }]
// Inspect tool results returned
console.log(result.toolResults);
// Example: [{ toolCallId: '<< redacted >>', toolName: 'getWeather', result: 'Weather in Paris: Sunny, 25°C' }]
#Tool calling with structured output You can also use experimental_output to generate structured output with generateText. This is useful when you want to perform tool calls at the same time.
const response = await generateText({
model: apple(),
system: Help the person with getting weather information.,
prompt: 'What is the weather in Wroclaw?',
tools: {
getWeather,
},
experimental_output: Output.object({
schema: z.object({
weather: z.string(),
city: z.string(),
}),
}),
})
#Supported features We aim to cover most of the OpenAI supported formats, including the following:
Objects: z.object({}) with nested properties Arrays: z.array() with minItems and maxItems constraints Strings: z.string() Numbers: z.number() with minimum, maximum, exclusiveMinimum, exclusiveMaximum Booleans: z.boolean() Enums: z.enum([]) for string and number values The following features are currently not supported due to underlying model limitations:
String formats: email(), url(), uuid(), datetime() etc. Regular expressions: Due to a Unions: z.union(), z.discriminatedUnion() #Availability Check Always check if Apple Intelligence is available before using the provider:
import { apple } from '@react-native-ai/apple';
if (!apple.isAvailable()) { // Handle fallback logic return; }
#Available Options Configure model behavior with generation options:
temperature (0-1): Controls randomness. Higher values = more creative, lower = more focused maxTokens: Maximum number of tokens to generate topP (0-1): Nucleus sampling threshold topK: Top-K sampling parameter You can pass selected options with either generateText or generateObject as follows:
import { apple } from '@react-native-ai/apple'; import { generateText } from 'ai';
const result = await generateText({ model: apple(), prompt: 'Write a creative story', temperature: 0.8, maxTokens: 500, topP: 0.9, });
#Direct API Access For advanced use cases, you can access the native Apple Foundation Models API directly:
#AppleFoundationModels import { AppleFoundationModels } from '@react-native-ai/apple'
// Check if Apple Intelligence is available const isAvailable = AppleFoundationModels.isAvailable()
// Generate text responses const messages = [{ role: 'user', content: 'Hello' }] const options = { temperature: 0.7, maxTokens: 100 }
const result = await AppleFoundationModels.generateText(messages, options) Embeddings Generate text embeddings using Apple's on-device NLContextualEmbedding model with the AI SDK.
#Overview This provider uses Apple's NLContextualEmbedding to generate contextual text embeddings entirely on-device. This is Apple's implementation of a BERT-like transformer model integrated into iOS 17+, providing privacy-preserving text understanding capabilities.
#Model Architecture NLContextualEmbedding uses a transformer-based architecture trained with masked language modeling (similar to BERT). Apple provides three optimized models grouped by writing system:
Latin Script Model (20 languages): English, Spanish, French, German, Italian, Portuguese, Dutch, and others - produces 512-dimensional embeddings Cyrillic Script Model (4 languages): Russian, Ukrainian, Bulgarian, Serbian CJK Model (3 languages): Chinese, Japanese, Korean Each model is multilingual within its script family, enabling cross-lingual semantic understanding. The models are compressed and optimized for Apple's Neural Engine, typically under 100MB when downloaded.
#Requirements iOS 17+ - NLContextualEmbedding requires iOS 17 or later #Usage #Single Text import { embed } from 'ai' import { apple } from '@react-native-ai/apple'
const { embedding } = await embed({ model: apple.textEmbeddingModel(), value: 'Hello world', })
console.log(embedding)
#Multiple Texts import { embedMany } from 'ai' import { apple } from '@react-native-ai/apple'
const { embeddings } = await embedMany({ model: apple.textEmbeddingModel(), values: ['Hello world', 'How are you?', 'Goodbye'], })
console.log(embeddings)
#Language Support The embeddings model supports multiple languages. You can specify the language using ISO 639-1 codes or full names when creating the model:
const model = apple.textEmbeddingModel({ language: 'fr' })
await embed({ model, value: 'Bonjour', })
For list of all supported languages, check Apple documentation.
NOTE By default, the embeddings model will use device language.
#Preparing the Model Apple's NLContextualEmbedding requires downloading language-specific assets to the device. While the provider automatically prepares assets when needed, you can call prepare() ahead of time for better performance:
const model = apple.textEmbeddingModel({ language: 'en' })
// Call prepare() ahead of time to optimize first inference latency await model.prepare()
// Now embeddings will be faster on first use const { embedding } = await embed({ model, value: 'Hello world' })
TIP Calling prepare() ahead of time is recommended to avoid delays on first use. If not called, the model will auto-prepare when first used, but a warning will be logged.
When you call prepare(), the system first checks if the required assets are already present on the device. If they are, the method resolves immediately without any network activity.
NOTE All language models and assets are stored in Apple's system-wide assets catalog, separate from your app bundle. This means zero impact on your app's size. Assets may already be available if the user has previously used other apps, or if system features have requested them.
#Direct API Access For advanced use cases, you can access the embeddings API directly:
#AppleEmbeddings import { AppleEmbeddings } from '@react-native-ai/apple'
// Get embedding model information const modelInfo: EmbeddingInfo = await AppleEmbeddings.getInfo(language: string)
// Prepare language assets await AppleEmbeddings.prepare(language: string)
// Generate embeddings const embeddings = await AppleEmbeddings.generateEmbeddings( values: string[], language: string ): Promise<number[][]>
export interface EmbeddingInfo { hasAvailableAssets: boolean dimension: number languages: string[] maximumSequenceLength: number modelIdentifier: string revision: number scripts: string[] }
#Benchmarks Performance results showing processing time in milliseconds per embedding across different text lengths:
Device Short (~10 tokens) Medium (~30 tokens) Long (~90 tokens) iPhone 16 Pro 19.19 21.53 33.59 Each category is tested with 5 consecutive runs to calculate reliable averages and account for system variability.
강아지를 찾습니다 골든 리트리버 아이보리 경상북도 의성군 봉양면 봉호로14