nobodywho-ooo/nobodywho

On-device AI for any device.
NobodyWho is an inference engine that lets you run LLMs locally and efficiently.

---

Features

• Run locally, offline — no API keys needed or hidden fees
• Run any chat LLM — Gemma, Qwen, Mistral and more
• Fast, type-safe tool calling — automatically generates structured grammars from your function signatures, no schema writing needed
• Multimodal input — provide image and audio information to your LLM
• Text-to-speech — synthesize local WAV audio with Kokoro, Pocket TTS and Supertonic backends
• Speech-to-text — transcribe audio into text with Whisper
• Voice Activity Detection — know when to stop listening and start transcribing with Silero
• Model downloading — load models directly from Hugging Face or any URL

Under the Hood

flowchart TD
    FL["Flutter"]:::lang
    PY["Python"]:::lang
    GO["Godot"]:::lang
    K["Kotlin"]:::lang
    S["Swift"]:::lang
    RN["React Native"]:::lang
FRB["flutter_rust_bridge"]:::glue
    P3["PyO3"]:::glue
    GX["gdext"]:::glue
    U["UniFFI"]:::glue
FL --> FRB
    PY --> P3
    GO --> GX
    K --> U
    S --> U
    RN --> U
CORE["NobodyWho · Rust
chat · templates · grammars · sampling · context shifting"]:::core
FRB --> CORE
    P3 --> CORE
    GX --> CORE
    U --> CORE
CORE --> LCPP["llama.cpp
text · vision · embeddings · reranking"]:::engine
    CORE --> ORT["ONNX Runtime
speech-to-text · text-to-speech · VAD"]:::engine
LCPP --> HW["Hardware
Vulkan · Metal · CUDA · GPU · CPU"]:::hw
    ORT --> HW
classDef lang fill:#e8eefc,stroke:#5b7bd5,color:#11204a
    classDef glue fill:#f3f0fb,stroke:#8b7bd5,color:#2a1f4a
    classDef core fill:#eaf6ee,stroke:#4fa46a,color:#0f3b1f
    classDef engine fill:#fdf0e3,stroke:#d58f3b,color:#4a2d0b
    classDef hw fill:#f0eeea,stroke:#a09a90,color:#3a352e

• GPU-accelerated inference via Vulkan or Metal — runs fast on any OS
• Compatible with thousands of pre-trained LLMs — use any LLM in the GGUF format
• Powered by the wonderful llama.cpp

---

Platforms

| Binding | Install | Runs on | Documentation | |---------|---------|---------|---------------| | Kotlin | Maven Central | Desktop, Android | docs.nobodywho.ooo/kotlin | | Swift | SPM | macOS, iOS, visionOS, watchOS | docs.nobodywho.ooo/swift | | React Native / Expo | npm | Desktop, Android, iOS | docs.nobodywho.ooo/react-native | | Flutter | pub.dev | Desktop, Android, iOS | docs.nobodywho.ooo/flutter | | Python | PyPI | Desktop | docs.nobodywho.ooo/python | | Godot | AssetLib | Desktop, Android | docs.nobodywho.ooo/godot |

Desktop means Linux, macOS and Windows throughout. Three gaps worth knowing before you start:

• Godot has no iOS export. Use the Flutter, React Native or Swift binding on iOS.
• Windows ARM64 is not supported yet.
• There is no web export. It is tracked in issue #111.

Requirements

Inference uses the GPU where available and CPU where not. The real constraint is memory.

Desktop

• Hardware — any 64-bit Linux, macOS or Windows machine. Windows is x86_64 only, there are no

• Memory — roughly 1.5× the model file in free RAM, or 2× on a machine that's already busy.

• Discrete GPUs — NobodyWho offloads as many layers as fit in free VRAM and runs the rest on

Mobile

• iOS — iPhone 11 or newer, 4 GB RAM or more.
• Android — Snapdragon 855 / Adreno 640 / 6 GB RAM or better.
• Rule of thumb — the device needs roughly twice the model file size in available RAM. iOS

Models

Any model in the GGUF format works. Pass a hf:owner/repo:QUANT reference, an HTTPS URL, or a local path anywhere a model path is expected; remote models are downloaded and cached on first use. Pass "auto" to fit one to available memory.

Start with Qwen3 0.6B: at ~330 MB it is small enough for any phone and good enough to tell whether the integration works.

---

Quick Start

pip install nobodywho

from nobodywho import Chat
chat = Chat('hf:NobodyWho/Qwen_Qwen3-0.6B-GGUF:Q4_K_M')
response = chat.ask('What is the capital of Denmark?')
print(response.completed()) # The capital of Denmark is Copenhagen.

// Android
implementation("ai.nobodywho:nobodywho-android:")
// Desktop JVM (Linux, macOS, Windows)
implementation("ai.nobodywho:nobodywho:")

import ai.nobodywho.Chat
val chat = Chat.fromPath(
    modelPath = "hf:NobodyWho/Qwen_Qwen3-0.6B-GGUF:Q4_K_M"
)
val response = chat.ask("What is the capital of Denmark?").completed()
println(response) // The capital of Denmark is Copenhagen.

https://github.com/nobodywho-ooo/nobodywho-swift.git

import NobodyWho
let chat = try await Chat.fromPath(
    modelPath: "hf:NobodyWho/Qwen_Qwen3-0.6B-GGUF:Q4_K_M"
)
let response = try await chat.ask("What is the capital of Denmark?").completed()
print(response) // The capital of Denmark is Copenhagen.

# React Native
npm install react-native-nobodywho
Expo
npx expo install react-native-nobodywho

import { Chat } from "react-native-nobodywho";
const chat = await Chat.fromPath({
  modelPath: "hf:NobodyWho/Qwen_Qwen3-0.6B-GGUF:Q4_K_M",
});
const msg = await chat.ask("What is the capital of Denmark?").completed();
console.log(msg); // The capital of Denmark is Copenhagen.

flutter pub add nobodywho

import 'package:nobodywho/nobodywho.dart' as nobodywho;
void main() async {
  await nobodywho.NobodyWho.init();
final chat = await nobodywho.Chat.fromPath(
    modelPath: 'hf:NobodyWho/Qwen_Qwen3-0.6B-GGUF:Q4_K_M',
  );
final msg = await chat.ask('What is the capital of Denmark?').completed();
  print(msg); // The capital of Denmark is Copenhagen.
}

uvx --from 'git+https://github.com/nobodywho-ooo/nobodywho.git#subdirectory=nobodywho/server' nobodywho-server --model hf:NobodyWho/Qwen_Qwen3-0.6B-GGUF:Q4_K_M --name qwen

---

Documentation

The documentation has everything you might want to know: https://docs.nobodywho.ooo/

Working with a coding agent? Point it at llms.txt or llms-full.txt, or install the NobodyWho skill so your agent can look up the current APIs and documentation:

npx skills add https://github.com/nobodywho-ooo/nobodywho --skill nobodywho

Community

• Discord & Matrix — ask us anything
• Issues and Discussions — bugs and feature requests
• CONTRIBUTING.md — set up the repo and send a PR
• CHANGELOG.md · SECURITY.md · CODE_OF_CONDUCT.md

License

NobodyWho is licensed under the EUPL-1.2. You may use it in proprietary and commercial projects, free of charge. There has been some confusion about this, so to be precise:

Linking two programs or linking an existing software with your own work does not – at least under European law – produce a derivative or extend the coverage of the linked software licence to your own work. [[1]](https://interoperable-europe.ec.europa.eu/collection/eupl/licence-compatibility-permissivity-reciprocity-and-interoperability)

If you distribute modified versions of the code in this repo, you must open source those changes.

Feel free to make proprietary projects using NobodyWho, but don't make a proprietary fork of NobodyWho.