GitHub - NiaExperience/PearlOS: Your Interface to Intelligence

PearlOS is an intelligent environment that runs in your browser. The AI isn't an app you open, it IS the interface. Talk to it, tap it, let it handle things. Phone, tablet, laptop, whatever. Pearl has memory, awareness, and control of your visual and audio experience, spins up swarms of intelligence for complex tasks, and gets better the more you use it.

Why PearlOS?

Most AI lives inside a chat window. You type, it responds, you close the tab, it forgets you existed. That is not a relationship, it is a vending machine.

PearlOS is what happens when AI has a home: persistent memory, a real desktop, voice conversations that pick up where you left off, and teams of agents that keep working while you do not. The code is open because trust requires transparency. Your data stays yours because that was never negotiable. We are building the AI experience people actually deserve.

Features

PearlOS brings a lot to the table. Here's what you can explore:

Conversation & Memory

• 🎤 Voice Companion - Full voice conversations via PearlOS, like talking to a friend who actually helps
• 🤝 Conversation Etiquette - Knows when to talk and when to shut up. Reactions over noise
• 🧠 Persistent Memory - Remembers across sessions. Daily logs, long-term memory, cross-session sync
• ✨ Actual Personality - Opinions, humor, warmth, and zero "I'd be happy to help!" energy
• 💬 Multi-Channel Chat - Discord, Telegram, Signal, WhatsApp, iMessage, Slack, IRC, all from one brain
• 🔄 Cross-Session Awareness - Discord me knows what voice me just did. One entity, many surfaces
• 🔍 Memory Search - Semantic search across all memory files for instant context recall

Automation & Tools

• 🤖 Sub-Agent Swarms - Spawn background workers for heavy tasks, get pinged when they finish
• 🔗 Node Control - Camera snaps, screen recording, location, and remote commands on paired devices
• ⏰ Cron & Reminders - Schedule anything: one-shot reminders, recurring jobs, timed agent runs
• 🌐 Research & Act - Search the web, fetch content, then do something with it: build, email, summarize, deploy
• 🖥️ Browser Automation - Full browser control for scraping, testing, or navigating complex sites
• 💻 Shell Access - Run commands, manage processes, deploy services, debug live systems
• 📁 File Operations - Read, write, edit files across the workspace by Voice. Git commits included
• 🪄 Frontend Design - Build production-grade web UIs from descriptions, mockups, screenshots, or vibes
• 📺 Real-Time Desktop Content - Weather, news, visuals, and interactive scenes rendered live on the PearlOS desktop
• 📧 Email - Send emails directly, no app switching required

Intelligence & Flexibility

• ⚙️ Skill System - Skills for weather, image gen, transcription, coding agents, and more. She uses them and builds new ones herself
• 🎨 Image Generation - Create images from prompts (e.g. via local ComfyUI or nano banana pro), built into the environment
• 🖼️ Image Analysis - Vision model for analyzing screenshots, photos, diagrams, whatever
• 📊 Model Flexibility - Sonnet for quick stuff, Opus for heavy lifting, swap on the fly
• ❤️ Proactive Heartbeats - Checks on its own: emails, services, calendar, anything worth flagging
• 🛡️ Service Health Monitoring - Auto-detect downed PearlOS services and restart them

Your Stack

• 🏠 Runs Locally - Full stack on your machine with your own API keys. No cloud lock-in.

Screenshots

Desktop Interface PearlOS desktop with application icons, chat interface, and pixel-art fantasy forest background

Task Management Real-time task tracking with job status, progress indicators, and active work monitoring

Dashboard & Analytics Interactive dashboard showing stock performance analysis with Pearl's AI-powered insights

Introducing Your Control Panel

PearlOS Settings at a glance: configure providers, models, launch mode, and channel assignments from one place.

Quick Start

PearlOS ships with setup scripts that handle everything: dependencies, environment files, database seeding, and bot configuration.

Option 1: Interactive Setup Wizard (recommended)

git clone https://github.com/NiaExperience/PearlOS.git
cd PearlOS
bash scripts/new-setup.sh

The wizard walks you through preset selection (full, minimal, or custom), installs all dependencies (Node, Python, Poetry, uv), creates .env files with API key placeholders, seeds the database, and configures the voice bot. It works on Linux, macOS, and Windows (via Git Bash/WSL).

You can also run it non-interactively:

bash scripts/new-setup.sh --preset full --non-interactive

Option 2: Classic Setup Script

git clone https://github.com/NiaExperience/PearlOS.git
cd PearlOS
bash scripts/setup.sh

This runs the full setup in one shot. After it finishes, add your API keys to .env.local and run:

Option 3: Manual Setup

If you prefer doing things by hand, PearlOS is a monorepo. You will need Node.js 20+, Python 3.11+, and npm (this repo uses npm workspaces, not pnpm).

git clone https://github.com/NiaExperience/PearlOS.git
cd PearlOS
npm install

# Set up environment variables
cp apps/interface/.env.example apps/interface/.env.local
cp apps/pipecat-daily-bot/.env.example apps/pipecat-daily-bot/.env
cp apps/mesh/.env.example apps/mesh/.env

# Set up the Python voice bot
cd apps/pipecat-daily-bot
python -m venv venv
source venv/bin/activate   # Windows: venv\Scripts\activate
pip install -r requirements.txt
cd ../..

npm run dev

Once running, open http://localhost:3000. Click the microphone icon (or just start talking) and say hello to Pearl.

Note: You will need API keys for Deepgram, Daily.co, an LLM provider, and PocketTTS before the voice pipeline will work. See Configuration.

Architecture Overview

PearlOS is an AI-native desktop: the browser is the shell, the voice pipeline is the assistant, and the GraphQL mesh is the shared brain. Three services do the heavy lifting: the interface (what you see and click), the voice bot (what you talk to), and the mesh (where state, config, and tools meet). The UI and the bot both talk to the mesh over HTTP; real-time voice runs over Daily.co WebRTC so latency stays low.

What this means in practice: Pearl can open apps, run tools, and remember context because the interface and the bot share the same GraphQL API and event system. Change a setting in the dashboard or over voice, and both sides stay in sync.

  ┌─────────────────────────────────────┐
  │   apps/interface         :3000       │
  │   apps/dashboard         :4000       │
  ├─────────────────────────────────────┤
  │   apps/pipecat-daily-bot :4444       │
  │   apps/mesh              :2000       │
  ├─────────────────────────────────────┤
  │   packages/prism  · events           │
  │   packages/features · redis          │
  ├─────────────────────────────────────┤
  │   PostgreSQL · Redis · Cloudflare    │
  └─────────────────────────────────────┘

Project structure (monorepo)

The repo is a single npm workspace. All runnable apps and shared packages live under apps/ and packages/. Scripts, config, and docs sit at the root.

Rule: packages/* must not import from apps/*. The interface and dashboard consume Prism, events, and features; the mesh and bot use them on the server side.

Port Map

Voice Pipeline

Pearl's voice pipeline is the core of the real-time conversation experience. Audio flows left to right through seven stages, from your microphone back to your speakers.

Your mic Browser captures audio via Daily.co WebRTC (low-latency upstream)	Pipecat Orchestrates the full pipeline: VAD, turn-taking, interrupt handling	Deepgram STT Real-time streaming speech-to-text conversion	LLM (Brain) Claude, OpenAI, or any compatible model generates Pearl's response	50+ Tools Open apps, edit notes, control canvas, change settings, search the web	PocketTTS Text-to-speech via the Azelma voice model	Audio out Daily.co WebRTC streams Pearl's voice back to your speakers

Pipecat ties the whole chain together so the conversation feels immediate and natural. Tool calls execute on the mesh/interface side; results feed back into the pipeline before TTS.

Desktop Apps

PearlOS ships with a set of built-in apps that Pearl can open, control, and interact with on your behalf:

Apps are rendered as draggable, resizable windows within the desktop shell. Pearl can open any app, bring it to focus, resize it, or close it using the tool system.

Tools

Pearl has access to 50+ tools organized into functional categories. These tools are what give Pearl genuine agency over the desktop environment rather than just answering questions.

Categories

• 🖥️ Desktop and Window Management Open, close, focus, resize, and arrange app windows. Control the taskbar and desktop layout.
• 🪄 Wonder Canvas Generate and display visual content. Set canvas modes, push images, control transitions.
• 📝 Notes Create notes, read notes, edit content, search across notes, delete notes.
• 📺 YouTube Search YouTube, queue videos, play, pause, skip, seek, control volume.
• 🎵 Soundtrack Select ambient tracks, adjust volume, fade in/out, pause, resume playback.
• 🎭 Sprite System Set sprite poses, expressions, and animations. Show or hide the overlay character.
• 🚩 Feature Flags Enable or disable feature flags at runtime without redeployment.
• ⚡ System and UI Send notifications, update status displays, navigate between views, trigger UI events.
• 💬 Conversation and Memory Read session context, set reminders, recall prior topics, manage conversation state.
• 🌐 External Integrations Fetch web content, query external APIs, retrieve structured data.

Configuration

Each service uses its own .env file. Copy the examples and fill in your keys.

apps/interface/.env.local

# API endpoint for the voice bot
NEXT_PUBLIC_BOT_URL=http://localhost:4444

# GraphQL mesh endpoint
NEXT_PUBLIC_MESH_URL=http://localhost:2000/graphql

# Optional: YouTube Data API key for search
YOUTUBE_API_KEY=your_youtube_api_key

# Optional: feature flag defaults
FEATURE_WONDER_CANVAS=true
FEATURE_SPRITES=true
FEATURE_SOUNDTRACK=true

apps/pipecat-daily-bot/.env

# Daily.co (WebRTC rooms)
DAILY_API_KEY=your_daily_api_key

# Deepgram (speech-to-text)
DEEPGRAM_API_KEY=your_deepgram_api_key

# LLM provider (choose one or configure multiple)
OPENAI_API_KEY=your_openai_api_key
ANTHROPIC_API_KEY=your_anthropic_api_key

# PocketTTS (text-to-speech, voice: Azelma)
POCKET_TTS_URL=http://your-pockettts-server:port
POCKET_TTS_VOICE=Azelma

# Interface callback URL (for tool results)
INTERFACE_URL=http://localhost:3000

apps/mesh/.env

# Database connection
DATABASE_URL=postgresql://user:password@localhost:5432/pearos

# JWT secret for session validation
JWT_SECRET=your_jwt_secret

# Port (default 2000)
PORT=2000

Project Structure

PearlOS/
├── apps/
│   ├── interface/              # Next.js desktop UI (port 3000)
│   │   ├── app/                # App router pages
│   │   ├── components/         # Desktop shell, windows, apps
│   │   │   ├── desktop/        # Taskbar, window manager
│   │   │   ├── apps/           # Notes, YouTube, Canvas, etc.
│   │   │   └── sprites/        # Animated overlay system
│   │   ├── lib/                # Shared utilities, hooks
│   │   ├── public/             # Static assets
│   │   └── .env.local.example
│   │
│   ├── pipecat-daily-bot/      # Voice pipeline (port 4444)
│   │   ├── bot.py              # Main bot entrypoint
│   │   ├── tools/              # 50+ Pearl tool definitions
│   │   ├── pipeline/           # Pipecat pipeline config
│   │   ├── tts/                # PocketTTS integration
│   │   ├── requirements.txt
│   │   └── .env.example
│   │
│   └── mesh/                   # GraphQL API (port 2000)
│       ├── schema/             # GraphQL schema definitions
│       ├── resolvers/          # Query and mutation resolvers
│       ├── prisma/             # Database schema and migrations
│       └── .env.example
│
├── packages/                   # Shared packages (types, utils)
├── package.json                # npm workspaces root
├── .github/CONTRIBUTING.md
└── README.md

Community

PearlOS is open source and built in the open. Contributions, issues, and ideas are welcome.

• Contributing: Read CONTRIBUTING.md before opening a PR.
• Issues: Use GitHub Issues for bugs and feature requests.
• Discussion: Join the community on Discord.

Built by Nia Holdings and the PearlOS community.

A Quick Note from the Dev Team

PearlOS is a self-evolving operating system. You talk to Pearl in PearlOS, she builds, creates, informs, learns, evolves. This is an open source browser-based, voice and touch desktop environment powered by an intelligent learning companion named Pearl. Pearl has full awareness and control of your visual and audio experience inside PearlOS in real time, so you can think out loud, ask for anything, and watch your desktop respond. She is not an assistant embedded in a UI. She is the UI. Over time she improves understanding how you work and adapts.

We're a small team (NiaXP) and this is a very early access. We want you to help us grow Pearl together. She still has rough edges everywhere. But she is full of possibility. Plus something we believe is a first: Whenever you encounter a bug in our OS, you tell her what you wanted her to do instead. Immediately Pearl dispatches the swarm intelligence to investigate, improve, and evolve the OS code to adapt to you. Our hope is that by making it incredibly easy to build on this open source experience, anyone and everyone can become contributors to a code base run by people all over the world.

Our stack is fully open source: Next.js frontend, OpenClaw agent framework, Pipecat voice pipeline (Deepgram STT, PocketTTS running locally), Daily.co WebRTC. 70+ tools. Everything runs local first with your own keys. The reason this matters: we're at a moment where AI is already being used against the public to benefit a small group of powerful individuals. PearlOS is a complete reimagining of how the same technology which can be used to exploit humanity can instead be used to liberate us. This isn't a product demo or a research paper; it's a working canvas for people all over the world to build just by using their voice.

The goal isn't just transparency. We intend to grow a community of developers, designers, creators, scientists, artists, and innovators who take this foundation and make the power of good intelligence accessible to all.

More info: https://pearlos.org

License

PearlOS is released under the PearlOS Source-Available License (PSAL-NC).

Copyright © 2026 Nia Holdings, Inc. All rights reserved.

You can: Use it, modify it, fork it, evolve it, experiment with it, research with it, build with it, learn from it, create culture with it.

You cannot monetize it, sell it, host it commercially, package it into products, build SaaS on it, or extract value from it commercially — without a commercial license.

PearlOS is distributed under a dual-license model:

• Non-Commercial License (PSAL-NC) — free for personal, research, and non-commercial use
• Commercial License — separate contract required for commercial use

See LICENSE for full terms.