Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 47 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,11 +45,12 @@ receiving the current app's telemetry stream.

## Available Providers

| Provider | Built-in | Platforms | Runtime | Description |
| --------------- | -------- | ------------ | ------------------------------------------------------------------- | ---------------------------------------------------------- |
| [Apple](#apple) | ✅ Yes | iOS | [Apple](https://developer.apple.com/documentation/FoundationModels) | Apple Foundation Models, embeddings, transcription, speech |
| [Llama](#llama) | ❌ No | iOS, Android | [llama.rn](https://github.com/mybigday/llama.rn) | Run GGUF models via llama.rn |
| [MLC](#mlc) | ❌ No | iOS, Android | [MLC LLM](https://github.com/mlc-ai/mlc-llm) | Run open-source LLMs via MLC runtime |
| Provider | Built-in | Platforms | Runtime | Description |
| ------------------- | -------- | ------------ | ------------------------------------------------------------------- | ---------------------------------------------------------- |
| [Apple](#apple) | ✅ Yes | iOS | [Apple](https://developer.apple.com/documentation/FoundationModels) | Apple Foundation Models, embeddings, transcription, speech |
| [Core AI](#core-ai) | ❌ No | iOS | [Core AI](https://developer.apple.com/core-ai/) | Run app-provided Core AI model bundles |
| [Llama](#llama) | ❌ No | iOS, Android | [llama.rn](https://github.com/mybigday/llama.rn) | Run GGUF models via llama.rn |
| [MLC](#mlc) | ❌ No | iOS, Android | [MLC LLM](https://github.com/mlc-ai/mlc-llm) | Run open-source LLMs via MLC runtime |

---

Expand Down Expand Up @@ -119,6 +120,47 @@ See the [Apple documentation](https://react-native-ai.dev/docs/apple/getting-sta

---

### Core AI

Run app-provided Apple Core AI model bundles exported from [`apple/coreai-models`](https://github.com/apple/coreai-models). This provider is for custom `.aimodel` assets, not Apple system models.

- **Language Sessions** - Core AI language models with persistent native sessions
- **AI SDK Adapters** - Language, embeddings, image generation, and transcription where AI SDK has matching interfaces
- **Native-only APIs** - Segmentation, object detection, depth, super-resolution, model inspection, specialization, and raw `.aimodel` functions
- **Catalog Metadata** - Starter model metadata from Apple’s registry, including iOS/macOS support

#### Installation

```bash
npm install @react-native-ai/core-ai
```

Core AI also requires adding `https://github.com/apple/coreai-models` as a Swift Package dependency in the host iOS app and linking the products you use, such as `CoreAILM`, `CoreAIDiffusion`, `CoreAISegmentation`, and `CoreAIObjectDetection`.

React Native installs the package through CocoaPods, but the Apple helpers are Swift Package products. Expo apps should use a config plugin to patch the generated Xcode project; bare React Native apps should add the package in Xcode or through a predictable project patch.

#### Usage

```typescript
import { coreAI } from '@react-native-ai/core-ai'
import { generateText } from 'ai'

const model = coreAI.languageModel({
id: 'qwen3-0.6b',
source: { type: 'file', uri: qwen3ModelDirectory },
variant: 'iOS',
})

const { text } = await generateText({
model,
prompt: 'Explain Core AI',
})
```

See the [Core AI documentation](https://react-native-ai.dev/docs/core-ai/getting-started) for setup, starter models, and direct native APIs.

---

### Llama

Run any GGUF model on-device using [llama.rn](https://github.com/mybigday/llama.rn). **Requires download** - models are downloaded from HuggingFace.
Expand Down
18 changes: 18 additions & 0 deletions bun.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

21 changes: 21 additions & 0 deletions packages/core-ai/CoreAI.podspec
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
require "json"

package = JSON.parse(File.read(File.join(__dir__, "package.json")))

Pod::Spec.new do |s|
s.name = "ReactNativeAICoreAI"

s.version = package["version"]
s.summary = package["description"]
s.homepage = package["homepage"]
s.license = package["license"]
s.authors = package["author"]

s.platforms = { :ios => min_ios_version_supported }
s.source = { :git => "https://github.com/callstackincubator/ai.git", :tag => "#{s.version}" }

s.source_files = "ios/**/*.{h,m,mm,swift}"
s.exclude_files = "ios/Tests/**/*"

install_modules_dependencies(s)
end
116 changes: 116 additions & 0 deletions packages/core-ai/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
# @react-native-ai/core-ai

Apple Core AI provider for React Native AI.

This package is for app-provided Core AI model bundles exported from
[`apple/coreai-models`](https://github.com/apple/coreai-models). It is separate
from `@react-native-ai/apple`, which wraps Apple system capabilities such as
Foundation Models, NLContextualEmbedding, SpeechAnalyzer, and AVSpeechSynthesizer.

## API Layers

Core AI exposes two layers:

- AI SDK adapters where AI SDK has a matching model interface: language models,
image generation, text embeddings, and transcription.
- Direct native APIs for Core AI-specific capabilities: model loading,
persistent language sessions, model inspection, specialization, segmentation,
object detection, depth, super-resolution, and raw `.aimodel` functions.

The first native wrappers cover language sessions, diffusion image generation,
segmentation, and object detection. Embeddings, transcription, depth,
super-resolution, classification, reranking, speech generation, video
generation, and arbitrary raw tensor calls stay explicit: embeddings and
transcription keep AI SDK-shaped adapters because Apple has starter export
recipes, while reranking, speech, and video have no provider helpers until a
real Core AI model/runtime exists.

## Native Setup

Add `https://github.com/apple/coreai-models` as a Swift Package dependency in
the host iOS app and link the products used by your wrappers:

This is the intended integration path, but it still needs full verification in
an Xcode 27 / iOS 27 host app with Apple Core AI products linked into the app
target.

- `CoreAILM` for language models and sessions
- `CoreAIDiffusion` for image generation
- `CoreAISegmentation` for segmentation
- `CoreAIObjectDetection` for object detection

React Native installs this package through CocoaPods, but Apple's helpers are
Swift Package products. The app target still needs those SPM products linked.
For Expo, add the config plugin. It links all known Apple Core AI Swift Package
products by default:

```json
{
"expo": {
"plugins": ["@react-native-ai/core-ai"]
}
}
```

Pass `products` only when you want to link a smaller set:

```json
{
"expo": {
"plugins": [
["@react-native-ai/core-ai", { "products": ["CoreAILM"] }]
]
}
}
```

For bare React Native, add the Swift Package in Xcode or with a project patching
script.

## Usage

```ts
import { coreAI } from '@react-native-ai/core-ai'
import { streamText } from 'ai'

const model = coreAI.languageModel({
id: 'qwen3-0.6b',
source: { type: 'file', uri: qwen3ModelDirectory },
variant: 'iOS',
})

const result = streamText({
model,
prompt: 'Explain Core AI in one sentence.',
})
```

For persistent native sessions:

```ts
await model.prepare()

const session = await model.createSession({
instructions: 'Answer tersely.',
})

const response = await session.respond('What is Qwen3?')
```

For lower-level calls, import `CoreAI` and call the native TurboModule directly:

```ts
import { CoreAI, toNativeModelConfig } from '@react-native-ai/core-ai'

await CoreAI.inspectModel(toNativeModelConfig(modelConfig))
```

## Model Management TBD

Remote Core AI assets still need a first-class download path. Apple’s native
API loads and specializes local `.aimodel` or `.aimodelc` URLs, so this package
should add helpers that download model bundles into app storage, return a local
file source, optionally specialize the model before first use, and persist
bookmark data for cached specialized assets. Where possible, the downloader and
storage pieces should be shared with the other React Native AI runtimes instead
of duplicating provider-specific plumbing.
1 change: 1 addition & 0 deletions packages/core-ai/app.plugin.js
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
module.exports = require('./lib/commonjs/expo-plugin')
Loading
Loading