GitHub - ondeinference/onde-swift: Onde Inference Swift SDK

Run LLMs on-device from Swift with Onde Inference, with first-class support for Apple silicon.

Rust SDK · Kotlin Multiplatform SDK · Flutter SDK · React Native SDK · Website

Compatibility

Installation

In Xcode, open File → Add Package Dependencies and enter:

https://github.com/ondeinference/onde-swift

Or add it to your Package.swift:

dependencies: [
    .package(url: "https://github.com/ondeinference/onde-swift", from: "0.1.0")
],
targets: [
    .target(name: "MyApp", dependencies: [
        .product(name: "Onde", package: "onde-swift")
    ])
]

Platforms

Platform	Minimum	GPU Backend
iOS	15.0	Metal
macOS	11.0	Metal
tvOS	15.0	Metal

Quick Start

import Onde

let engine = OndeChatEngine()

// Load the default model for the current device:
//   iOS / tvOS → Qwen 2.5 Coder 1.5B (~941 MB)
//   macOS      → Qwen 2.5 Coder 3B (~1.93 GB)
let elapsed = try await engine.loadDefaultModel(
    systemPrompt: "You are a helpful assistant.",
    sampling: nil
)

let result = try await engine.sendMessage(message: "Hello!")
print(result.text)

Streaming

import Onde

class StreamHandler: StreamChunkListener {
    func onChunk(chunk: StreamChunk) -> Bool {
        print(chunk.delta, terminator: "")
        return !chunk.done
    }
}

try await streamChatMessage(
    engine: engine,
    message: "Tell me a story.",
    listener: StreamHandler()
)

One-Shot Generation

This runs inference without changing conversation history:

let result = try await engine.generate(
    messages: [userMessage(content: "Expand: a cat in space")],
    sampling: deterministicSamplingConfig()
)

Sandboxed App Setup

On iOS and App Store macOS, the default ~/.cache/huggingface path sits outside the app sandbox. This helper points HF_HOME at the App Group shared container (group.com.ondeinference.apps) so Onde-powered apps can share downloaded models. Call it once at app launch before creating an OndeChatEngine.

import Foundation

func setupInferenceEnvironment() {
    guard let container = FileManager.default.containerURL(
        forSecurityApplicationGroupIdentifier: "group.com.ondeinference.apps"
    ) else {
        // Fall back to the app's private Application Support directory if the App Group is unavailable.
        let appSupport = FileManager.default
            .urls(for: .applicationSupportDirectory, in: .userDomainMask)[0]
        setupHfCache(at: appSupport)
        return
    }
    setupHfCache(at: container)
}

private func setupHfCache(at base: URL) {
    let hfHome = base.appendingPathComponent("models")
    let hfHub  = hfHome.appendingPathComponent("hub")
    try? FileManager.default.createDirectory(at: hfHub, withIntermediateDirectories: true)

    setenv("HF_HOME",      hfHome.path, 1)
    setenv("HF_HUB_CACHE", hfHub.path,  1)

    let tmp = base.appendingPathComponent("tmp")
    try? FileManager.default.createDirectory(at: tmp, withIntermediateDirectories: true)
    setenv("TMPDIR", tmp.path, 1)
}

Local Development

For local SDK work, keep onde-swift next to the Rust repo:

Repositories/
├── onde/
└── onde-swift/

Then from the onde-swift root:

That builds the local Rust core from ../onde, regenerates Sources/Onde/onde.swift, and writes OndeFramework.xcframework into the package root so Xcode and SwiftPM can link it.

To work on the example iOS app:

cd Examples/OndeExample
xcodegen generate
open OndeExample.xcodeproj

Documentation

on-device AI inference for iOS and macOS
Developer Guide
Onde Inference Rust Crate
Onde CLI, for testing model downloads and GGUF export before wiring them into a Swift app

License

Onde is dual-licensed under MIT and Apache 2.0. You can use either one.