jofizcd/Soul-of-Waifu

🌌 ONE SOUL - MULTIPLE DIMENSIONS 🌌

Your character is no longer locked in a text chat - they live wherever you are.
Welcome to the world where your AI characters chats, remembers, adventures in tabletop RPGs, and lives directly on your desktop with one continuous, evolving identity.

---

📚 Documentation  •  🗨️ Discord  •  🌐 Website

---

---

🌟 What is Soul of Waifu?

Soul of Waifu (SoW) is a free, open-source desktop app where AI characters can remember, evolve, and exist across multiple dimensions of interaction simultaneously. The core idea: a character you create once exists across multiple interconnected modes at the same time - text and voice conversation, a full tabletop RPG engine, a desktop agent with real OS and tool access, and a persistent memory architecture that builds genuine continuity across all your sessions.

Create or import a character once, and seamlessly interact with them across interconnected dimensions: 1. Interactive Text & Voice Chat with customizable UI, Live2D/VRM avatars, and real-time voice calls. 2. Soul Memory Architecture featuring long-term emotional evolution, a personal diary, and self-healing memory logs. 3. Soul Stage RPG Engine running a full tabletop text-RPG driven by a dedicated AI Game Master, complete with dice rolls and hidden story arcs. 4. Soul Companion - an autonomous agent living on your screen, powered by neurohormonal simulation. Out of the box it can see your screen, control your mouse and keyboard, browse the web on its own, run code, and organize your files - no extra setup needed. Want more? Write your own Python plugins or connect any external MCP server to give it entirely new abilities. Every risky action is gated behind an on-screen approval system, so nothing runs without your say-so.

Everything runs 100% locally on your PC by default - your chats with characters can live entirely offline, with no subscriptions and no filters. Cloud AI providers (OpenAI, Claude, Gemini, and others) are available as an optional plug-in for low-spec hardware or heavier reasoning, and browser/web tools naturally reach the internet when a task calls for it - but the core experience never requires an account or sends your data anywhere you haven't asked it to.

---

🆕 What's New Since Soul Trinity (v2.4.0)

Soul of Waifu has been moving fast since the original Soul Trinity (v2.4.0) release. The highlights:

• Soul Companion is now a full autonomous OS agent. It can move your mouse, click and type into any window, browse the web on its own through a real headless browser, run sandboxed Python/PowerShell scripts, organize your files, and chain multi-step tasks together - all gated behind a Human-in-the-Loop approval banner with a 25-second fail-safe countdown, so nothing risky ever runs without your say-so.
• Soul Stage grew into a genuine tabletop RPG simulator. Deterministic dice rolls and skill checks, hidden Story Arcs your party has to actually uncover, a live Campaign Board with pressure clocks, structural relationship tracking that never forgets a grudge or a bond, and Scene Folders to organize whole campaigns.
• Tool calling isn't companion-exclusive anymore. Web search, clipboard access, and app control now work in regular 1-on-1 text and voice chats too.
• Discord Gateway got smarter. Full computer vision for images/GIFs, plus per-member awareness so it never mixes up who said what in a server.

> The full, detailed changelog for every version lives on the Releases page.

---

🚀 Key Modules & Capabilities

1. One Soul, Multiple Dimensions

Every character you build is a single continuous identity that steps through several connected modes - the same personality and memories follow them everywhere.

| Mode | Description | |---|---| | 🧠 Soul Memory | An independent digital brain for every character - tracking mood, relationships, and lived history far beyond basic keyword fact-searching. | | 💬 Text & Voice Chat | Full-duplex conversation with Live2D & VRM avatars, a customizable state variable HUD, lorebooks, and real-time voice calls. | | 🎲 Soul Stage | A full tabletop RPG under a virtual Game Master - dice rolls, generating options for your answers, WorldState tracking, inventory, and dynamic NPCs with its own memory. | | 🖥️ Soul Companion | An autonomous desktop agent powered by neurohormones - reads your screen, controls your mouse/keyboard, browses the web on its own, runs code, and organizes your files, extensible with custom plugins and MCP servers. |

---

🧠 Soul Memory — Long-Term Cognitive Architecture

Standard vector-RAG approaches search old chat logs for keyword matches. Soul Memory functions as an autonomous background agent that continuously rewrites four isolated cognitive files in real time:

• Psychology Layer: Tracks current mood, internal tension, hidden motives, and character development.
• Relationship Profile: Remembers user habits, preferences, shared promises, and trust progression.
• Episodic Topic Archive: Categorized knowledge base indexing locations, events, and key facts, matched against your queries using local semantic embeddings - with strong bilingual accuracy.
• Personal Diary: First-person reflective entries written by the character after meaningful conversations.

[!TIP]

Cognitive Mechanics:

- Emotional Decay: Grudges and negative mood spikes naturally cool down over time if not revisited.

- Memory Self-Healing: The system detects logical contradictions between new responses and past facts, automatically overwriting outdated information and logging corrections.

- Duplicate Detection: Near-identical memories are automatically merged so the archive stays clean over long-term use.

- Crash-Safe Writes: Memory updates save atomically, so an unexpected error mid-session can no longer cost you your character's memories.

Choose between Full Sync (maximum depth), Soul Link (balanced), Mind Spark (lightweight), or Reflection Flow (diary only) to fit your VRAM/RAM constraints.

---

🎲 Soul Stage — Tabletop RPG & Game Master Engine

Soul Stage transitions your chat into a structured tabletop roleplaying campaign managed by a virtual Game Master. The GM is a separate AI layer running independently of your character: it maintains the world state, directs events, and enforces structure for group roleplay sessions.

• WorldState Engine: Rigorous state tracking for in-game time of day, weather, active location, and key facts.
• Dice Rolls & Skill Checks: Deterministic mechanics resolve actions, instead of leaving outcomes to the AI's whim.
• Story Arcs: Hidden plot twists and investigations stay concealed from NPCs and the party until concrete clues are uncovered during play.
• Campaign Board: A live objective tracker with "Pressure Clocks" - guard alert levels, trust meters, ritual countdowns - that shift dynamically with your choices.
• Turn Control Bar: Switch between Say, Do, Think, and Director input modes, pick the next speaker manually, or send Private Whispers only one party member can see.
• Structural Relationship Tracker: Trust levels and social roles are serialized and fed into every turn, so companions and NPCs never "forget" their dynamic with you, even dozens of turns later. Roles and titles can mutate mid-story and behavior updates immediately.
• Interactive Party & NPCs: Main party members sync memories via Soul Memory, while dynamic temporary NPCs spawn (with their own persistent memory) for specific encounters.
• Inventory & Status HUD: Track equipment, health, energy, stress, conditions (poisoned, exhausted, inspired), and gold in real time.
• Scene Folders: Organize your scenario library into campaigns, genres, or settings.
• Atmospheric Automation: Scene backgrounds and ambient audio loops switch automatically as the story moves to new locations.
• Event Cards: Random encounter cards keep campaigns unpredictable.

---

🖥️ Soul Companion — Autonomous Desktop Agent

Soul Companion turns your character into an autonomous inhabitant of your Windows desktop - and, as of the latest release, a genuine computer-use agent. Existing as a transparent overlay, the companion stays with you while you work, game, or browse. Its behavior is driven by a neurohormone system (dopamine, cortisol, oxytocin, fatigue) rather than simple reactive prompting: it has moods that shift over time based on activity levels, time of day, and your interaction patterns, gets tired after long sessions, gets bored during extended inactivity, and initiates conversation on its own when something notable happens on your screen.

Computer Use & OS Control:

| Tool | What it does | |---|---| | 🖱️ GUI Action Tool | Moves the cursor, clicks (single/double/right-click), scrolls, dispatches keyboard shortcuts, and types text natively into any active window (Discord, Notepad, VS Code, browsers, etc.) | | 🌐 Browser Agent | Autonomously navigates URLs, extracts clean article text, clicks buttons, fills out forms, and downloads files through a real headless browser (Playwright) | | 🐍 Code Execution | Writes and runs sandboxed Python/PowerShell scripts on the fly for calculations, data parsing, or batch tasks, with strict timeouts | | 📁 File Organizer | Inspects your desktop and folders, auto-sorts loose files into categories, and runs fast filename search | | 🧭 Task Planner | Converts a complex spoken goal into a sequential, multi-step tool chain and executes it end-to-end | | ❤️‍🩹 System Vitals Watchdog | Proactively warns you about low battery, GPU overheating, or finished downloads | | 🔎 Screen Reader | Sees your screen and comments on, analyzes, or assists with what's visible | | 📋 Clipboard Reader | Reads and processes copied text on request | | 🔍 Web Search | Looks up information in real time | | 🎵 Music Control | Manages audio playback | | 🔌 Custom MCP Server | Connect any external MCP server for expanded integrations |

Safety by design: every potentially risky action - moving files, running code, controlling your mouse or keyboard - is intercepted by an on-screen approval banner with a 25-second fail-safe countdown. Nothing executes without your explicit confirmation, and the app automatically restores focus to your active window afterward so no keystrokes get lost.

---

📊 Custom State Variables & Adaptive HUD

Build no-code Dating Sims, RPG stat systems, or Tamagotchi mechanics directly inside character cards without writing code.

• Adaptive HUD Bar: Displays animated progress bars (int), YES/NO badges (bool), item lists (list), or text labels (str) at the top of the chat.
• Resilient AI Interceptor: Backend interceptor parses state updates robustly, making updates immune to LLM JSON syntax typos.
• 11 Built-In Presets: Ready-to-use templates for Romance, RPG, Survival, Interrogation, Horror, and Visual Novels.

---

🌐 Ecosystem & Extended Integrations

📱 Local Web Client (Mobile / Tablet Access)

Run Soul of Waifu on your main PC and connect from your phone or tablet over home Wi-Fi via http://:8000. Features real-time bidirectional WebSocket syncing, custom chat backgrounds, avatar rendering, and voice input routed directly through your PC's local Faster Whisper model.

---

🤖 Discord Gateway

Connect your Soul of Waifu characters directly to Discord bots. Chat with your characters inside server channels or private messages, preserving their full personality, system prompt settings, and long-term memories.

• Multi-User Server Awareness: Distinguishes between individual server members and keeps a distinct conversation context per speaker.
• Computer Vision: Sees, inspects, and reacts to images and GIFs sent in direct messages or server channels.

---

🎨 AI Image Generation

Generate illustrations directly within chat messages via context menu actions.

• Local Engines: Automatic1111 WebUI, ComfyUI API.
• Cloud Engines: NovelAI Diffusion, DALL-E 3, FLUX.
• Automatically builds contextual visual prompts based on the character's appearance, current pose, emotional state, and scene environment.

---

🛠️ Deep Tech & Fine-Tuning

🎙️ Voice, Speech-to-Text & Avatars

• 6 Text-to-Speech Engines:
• Qwen3 TTS (Local): Ultra-low latency, voice cloning from a 3-second audio sample, prompt-based voice design, 9 built-in speakers, 1.7B and 0.6B models.
• XTTSv2 & Kokoro 82M (Local): High-fidelity voice synthesis and zero-shot voice cloning.
• Silero TTS (Local): Fast Russian voice engine.
• EdgeTTS & ElevenLabs (Cloud): High-quality cloud neural voices.
• RVC Support: Integrated Voice Conversion parameter tuning (pitch shift, index rate, protection).
• Sentence-by-Sentence Streaming: Audio playback starts on the first generated sentence, eliminating response lag.
• Full-Duplex Calls (SoW System): Silero VAD (Voice Activity Detection) + Faster Whisper transcription with seamless user-interruption support.
• Avatars & 28 Emotion Classifier: Live2D (with motion animation links), VRM 3D models, GIFs, and static images. Automatic 28-state emotional sentiment analysis with LipSync audio matching.

🎛️ Advanced Text Sampling (Loop Prevention)

• Dynamic Temperature: Dynamically shifts temperature based on confidence to balance creativity and logic.
• DRY Sampler (Don't Repeat Yourself): Prevents multi-word phrase loops without degrading grammatical quality.
• XTC (Exclude Top Candidates): Filters out overused AI clichés and repetitive vocabulary.
• Min-P: Trims low-probability tokens relative to the top candidate for coherent output.

📖 Updated Lorebooks Engine

• Multi-Binding: Link multiple lorebooks to a single character simultaneously.
• Semantic Situation Matching: Local vector search matches lore entries based on situational context rather than strict keywords.
• Scene Tension Accumulator: Dynamically builds story energy to trigger random events.
• Chain Dependencies: Create multi-stage questlines with delayed entry unlocks.
• Injection Modes: Choose between Passive (Background Knowledge) and Active (System Directive) injection.

💻 Local & Cloud LLM Backends

• Reasoning Model Support: Dedicated Thinking Token Budget setting and automatic `` block parsing for reasoning models like DeepSeek.
• Models Hub: Integrated Hugging Face repository search, GGUF downloader with progress bar and instant cancel, and automated Llama.cpp binary updater.
• Local Acceleration: Llama.cpp HTTP server supporting CUDA (NVIDIA), HIP (AMD), SYCL (Intel), and Vulkan.
• Performance Options: Flash Attention, MLock, KV-Cache quantization (q8_0, q4_0), Thinking Mode, and CPU MoE layer offloading.
• 10 Cloud Providers Supported: OpenRouter, OpenAI, Anthropic Claude, DeepSeek, Grok (xAI), Google Gemini, Qwen, Mistral AI, Z.AI, Player2 and custom OpenAI-compatible endpoints.

---

📥 Installation

No Python knowledge, Node.js, or command-line experience required.

1. Download the latest release archive (currently Soul-of-Waifu-v2.4.7.rar) from Releases.
2. Extract it with 7-Zip or WinRAR to a path without spaces or Cyrillic characters (e.g. C:\AI\Soul-Of-Waifu\).
3. Run installer.bat (Do NOT run as Administrator).
4. Launch via start.bat.

[!WARNING]

Do not run installer.bat as Administrator! Windows changes the working directory for Administrator processes to C:\Windows\System32, causing installation path errors. Use a standard double-click.

System Requirements

| | Minimum | Recommended | |---|---|---| | OS | Windows 10/11 (64-bit) | Windows 10/11 (64-bit) | | RAM | 8 GB | 16 GB+ | | GPU | Any (for Cloud APIs) | NVIDIA / AMD with 6+ GB VRAM (for local models) | | Storage | 10 GB free space | Fast NVMe SSD |

---

⚡ Quick Start

• Option A — Cloud AI (No GPU Required):

Go to Options -> Configuration, input your OpenRouter or OpenAI API key. Import a character card from the Characters Gateway or drag-and-drop a .png / .json file into the app window.

• Option B — Local AI:

Navigate to Models Hub, search for a model (e.g. Gemma or Qwen), and download a Q4_K_M GGUF quantization. In Options -> LLM Settings, select your GPU backend (CUDA/HIP/SYCL/Vulkan), enable Flash Attention, and launch the server via Models Hub.

Once you're set up, web search, clipboard access, app control and other tools are available straight away in regular text and voice chats - no need to switch to Soul Companion mode to use them. 📖 Read full documentation & guides: jofizcd.github.io/soul-of-waifu-site/docs ---

💖 Support & Community

Soul of Waifu is a solo-developer open-source project. All features are 100% free with no paywalls or premium tiers. Financial Support:

• Boosty
• Donation Alerts

Community & Help:

• Star this repository - helps boost visibility on GitHub!
• Join our Discord Server - share custom character cards, lorebooks, RPG scenes, and get help from the community.

---

📊 Repository Stats

---

👤 Developer