GitHub - clay-good/worldify: Worldify is a persistent fact store for AI world simulations.

10 min read Original article ↗

Worldify

A persistent fact store for AI world simulations.

Worldify provides a structured way to store and query facts about entities with temporal validity and causal tracking. It uses SQLite for persistence and has zero external dependencies.


When to Use Worldify

Worldify is designed for scenarios where you need to:

  • Track state that changes over time: Game characters, simulation agents, or any system where you need to know "what was true when"
  • Maintain persistent memory for AI agents: Chatbots, assistants, or autonomous agents that need to remember facts across sessions
  • Build knowledge graphs: Store relationships between entities and query them efficiently
  • Create reproducible simulations: Save checkpoints and restore to previous states
  • Audit state changes: Understand why something changed by tracing causal chains

Installation

# From source
git clone https://github.com/clay-good/worldify.git
cd worldify
pip install -e .

Requirements:

  • Python 3.9 or higher
  • No external dependencies (uses only Python standard library)

Quick Start

from worldify import World

with World("my_world.db") as world:
    # Create entities (things in your world)
    alice = world.create_entity("person", "Alice")
    office = world.create_entity("place", "Main Office")

    # Assert facts about entities
    world.assert_fact(alice.id, "occupation", value="Engineer")
    world.assert_fact(alice.id, "works_at", entity_ref=office.id)

    # Query current facts
    for fact in world.get_current_facts(alice.id):
        print(f"{fact.predicate}: {fact.object_value or fact.object_entity_id}")

Integration Examples

Example 1: Persistent Memory for a Chatbot

Use case: You're building a chatbot that should remember user preferences and information across conversations.

from worldify import World

class ChatbotMemory:
    def __init__(self, db_path="chatbot_memory.db"):
        self.world = World(db_path)
        self.world.__enter__()

    def remember_user(self, user_id, key, value):
        """Store a fact about a user."""
        # Find or create the user entity
        users = self.world.find_entities(name_pattern=user_id)
        if users:
            user = users[0]
        else:
            user = self.world.create_entity("user", user_id)

        # Store the fact with auto_supersede to update existing values
        self.world.assert_fact(
            user.id,
            key,
            value=value,
            source="conversation",
            multi_value=(key in ["interest", "topic"])  # Allow multiple interests
        )

    def recall(self, user_id, key=None):
        """Retrieve facts about a user."""
        users = self.world.find_entities(name_pattern=user_id)
        if not users:
            return None

        facts = self.world.get_current_facts(users[0].id)
        if key:
            return next((f.object_value for f in facts if f.predicate == key), None)
        return {f.predicate: f.object_value for f in facts}

    def close(self):
        self.world.__exit__(None, None, None)

# Usage
memory = ChatbotMemory()
memory.remember_user("user_123", "name", "Alice")
memory.remember_user("user_123", "preferred_language", "Python")
memory.remember_user("user_123", "interest", "machine learning")
memory.remember_user("user_123", "interest", "web development")

print(memory.recall("user_123"))
# {'name': 'Alice', 'preferred_language': 'Python', 'interest': 'machine learning', ...}

memory.close()

Why this works: Facts persist in the SQLite database, so when the chatbot restarts, it still remembers everything. The multi_value=True parameter allows storing multiple interests without conflicts.


Example 2: Game State with Save/Load

Use case: You're building a game and need to save player progress, then restore it later.

from worldify import World

class GameState:
    def __init__(self, save_file="game.db"):
        self.world = World(save_file)
        self.world.__enter__()
        self.player = None

    def new_game(self, player_name):
        """Start a new game."""
        self.world.clear(confirm=True)
        self.player = self.world.create_entity("player", player_name)
        self.world.assert_typed_fact(self.player.id, "health", 100)
        self.world.assert_typed_fact(self.player.id, "gold", 0)
        self.world.assert_typed_fact(self.player.id, "level", 1)

    def get_stat(self, stat_name):
        """Get a player stat."""
        facts = self.world.get_current_facts(self.player.id)
        for f in facts:
            if f.predicate == stat_name:
                return f.get_typed_value()  # Returns int, not string
        return None

    def update_stat(self, stat_name, new_value, reason=None):
        """Update a player stat, keeping history."""
        # Find current fact
        facts = self.world.query_facts() \
            .for_subject(self.player.id) \
            .with_predicate(stat_name) \
            .execute(self.world.db)

        if facts:
            # Supersede the old value (keeps history)
            self.world.supersede_fact(
                facts[0].id,
                new_value=f"int:{new_value}",  # Typed encoding
                reason=reason
            )
        else:
            self.world.assert_typed_fact(self.player.id, stat_name, new_value)

    def save_checkpoint(self, name):
        """Save current state."""
        return self.world.save_snapshot(name, f"Checkpoint at level {self.get_stat('level')}")

    def load_checkpoint(self, snapshot_id):
        """Restore to a previous state."""
        self.world.restore_snapshot(snapshot_id)
        # Re-find player after restore
        players = self.world.find_entities(entity_type="player")
        if players:
            self.player = players[0]

    def close(self):
        self.world.__exit__(None, None, None)

# Usage
game = GameState()
game.new_game("Hero")

game.update_stat("gold", 50, reason="Found treasure chest")
game.update_stat("health", 80, reason="Took damage from goblin")

checkpoint = game.save_checkpoint("before_boss")

# Attempt boss fight (fails)
game.update_stat("health", 0, reason="Defeated by dragon")

# Player wants to retry - restore checkpoint
game.load_checkpoint(checkpoint.id)
print(f"Health after restore: {game.get_stat('health')}")  # 80

game.close()

Why this works: The snapshot system captures the entire world state. When you restore, it's like nothing ever happened after the checkpoint. The supersede_fact method keeps a complete history, so you could even replay what happened.


Example 3: Knowledge Base for an AI Agent

Use case: You're building an AI agent that needs to maintain a knowledge base about the world it operates in.

from worldify import World

class AgentKnowledge:
    def __init__(self, db_path=":memory:"):
        self.world = World(db_path)
        self.world.__enter__()

    def learn(self, subject, predicate, obj, confidence=1.0, source="observation"):
        """Learn a new fact."""
        # Get or create subject entity
        subjects = self.world.find_entities(name_pattern=subject)
        if subjects:
            subj_entity = subjects[0]
        else:
            subj_entity = self.world.create_entity("concept", subject)

        # Check if object is another entity
        objects = self.world.find_entities(name_pattern=obj)

        if objects:
            # Relationship between entities
            self.world.assert_fact(
                subj_entity.id, predicate,
                entity_ref=objects[0].id,
                confidence=confidence,
                source=source
            )
        else:
            # Just a value
            self.world.assert_fact(
                subj_entity.id, predicate,
                value=obj,
                confidence=confidence,
                source=source
            )

    def query(self, subject, predicate=None):
        """Query what we know about a subject."""
        subjects = self.world.find_entities(name_pattern=subject)
        if not subjects:
            return []

        q = self.world.query_facts().for_subject(subjects[0].id)
        if predicate:
            q = q.with_predicate(predicate)

        results = []
        for fact in q.execute(self.world.db):
            if fact.object_entity_id:
                obj_entity = self.world.get_entity(fact.object_entity_id)
                obj = obj_entity.name if obj_entity else fact.object_entity_id
            else:
                obj = fact.object_value

            results.append({
                "predicate": fact.predicate,
                "object": obj,
                "confidence": fact.confidence,
                "source": fact.source
            })
        return results

    def close(self):
        self.world.__exit__(None, None, None)

# Usage
kb = AgentKnowledge()

# Agent learns facts from different sources
kb.learn("Python", "is_a", "programming language", source="wikipedia")
kb.learn("Python", "created_by", "Guido van Rossum", source="wikipedia")
kb.learn("Python", "good_for", "data science", confidence=0.9, source="observation")
kb.learn("Python", "good_for", "web development", confidence=0.85, source="observation")

# Query what we know
print(kb.query("Python"))
# [{'predicate': 'is_a', 'object': 'programming language', ...}, ...]

print(kb.query("Python", "good_for"))
# [{'predicate': 'good_for', 'object': 'data science', 'confidence': 0.9}, ...]

kb.close()

Why this works: The confidence and source tracking lets the agent weight its beliefs. The query builder makes it easy to find specific facts or get everything about a topic.


Core Concepts

Entities

Entities are things in your world: people, places, objects, concepts. Each entity has:

  • A unique ID (auto-generated UUID)
  • A type (e.g., "person", "item", "location")
  • A name (human-readable identifier)
  • Optional metadata (JSON dictionary for custom data)
entity = world.create_entity("person", "Alice", metadata={"age": 28})

Facts

Facts are statements about entities. Each fact has:

  • A subject (the entity the fact is about)
  • A predicate (the relationship type)
  • An object (either a string value OR a reference to another entity)
  • Temporal bounds (when the fact is/was valid)
  • Optional confidence score (0.0 to 1.0)
  • Optional source attribution
# Value fact
world.assert_fact(person.id, "mood", value="happy")

# Relationship fact (linking two entities)
world.assert_fact(person.id, "works_at", entity_ref=company.id)

# Fact with confidence
world.assert_fact(person.id, "location", value="office", confidence=0.7)

Temporal Validity

Every fact has a valid_from timestamp (defaults to now) and an optional valid_until timestamp. This lets you query "what was true at time X":

# Query facts valid at a specific time
past_facts = world.query_facts() \
    .for_subject(person.id) \
    .valid_at("2024-01-15T00:00:00Z") \
    .execute(world.db)

Causal Links

You can explain why facts exist or changed:

# Fact caused by an event
world.assert_fact(
    person.id, "mood", value="happy",
    cause_event="Received promotion"
)

# Trace why something is true
print(world.explain_fact(fact.id))

API Reference

World Class (Main Entry Point)

Method Description
create_entity(type, name, metadata) Create a new entity
get_entity(id) Get entity by ID
find_entities(type, name_pattern) Search entities (pattern uses SQL LIKE)
delete_entity(id, cascade) Delete entity (cascade=True removes related facts)
assert_fact(subject_id, predicate, ...) Create a fact
assert_typed_fact(subject_id, predicate, typed_value) Create fact with Python types (int, float, bool, dict, list)
retract_fact(id, reason) End a fact's validity
supersede_fact(id, ...) Replace a fact with a new one (keeps history)
get_fact(id) Get a fact by ID
get_current_facts(entity_id) Get all currently valid facts for an entity
query_facts() Start a query builder chain
explain_fact(id) Get human-readable causal explanation
trace_causes(id) Get list of causal links leading to a fact
trace_effects(id) Get list of effects caused by a fact
save_snapshot(name, description) Capture current world state
list_snapshots() List all snapshots
export_snapshot(id, path) Export snapshot to JSON file
import_snapshot(path, mode) Import from JSON (modes: merge, replace, error)
restore_snapshot(id) Restore world to snapshot state
summary() Get world statistics
clear(confirm=True) Delete all data (requires confirm=True)
start_session(agent_id, mode) Start agent session (mode: read or write)
end_session(session_id) End an agent session
get_active_sessions() List active sessions

Query Builder

results = world.query_facts() \
    .for_subject(entity_id) \        # Filter by subject entity
    .for_subject_type("person") \    # Filter by subject entity type
    .with_predicate("works_at") \    # Exact predicate match
    .with_predicate_like("skill_%") \ # Pattern match (SQL LIKE)
    .valid_at(timestamp) \           # Valid at specific time
    .valid_between(start, end) \     # Valid during time range
    .with_min_confidence(0.8) \      # Minimum confidence threshold
    .from_source("observation") \    # Filter by source
    .include_history() \             # Include superseded facts
    .limit(10) \                     # Limit results
    .offset(5) \                     # Skip first N results
    .execute(world.db)               # Execute and get QueryResult

# QueryResult methods
results.as_dict()           # Convert to list of dictionaries
results.as_json()           # Convert to JSON string
results.group_by_predicate() # Group facts by predicate
results.group_by_subject()  # Group facts by subject
results.count(world.db)     # Get count without fetching
results.first(world.db)     # Get first result or None

CLI Reference

# Initialize a new database
worldify init myworld.db

# Entity commands
worldify entity create myworld.db --type person --name "Alice"
worldify entity get myworld.db <entity_id>
worldify entity find myworld.db --type person
worldify entity delete myworld.db <entity_id> --cascade

# Fact commands
worldify fact assert myworld.db --subject <id> --predicate occupation --value Engineer
worldify fact get myworld.db <fact_id>
worldify fact query myworld.db --subject <id> --predicate occupation
worldify fact retract myworld.db <fact_id> --reason "No longer valid"

# Query (shorthand)
worldify query myworld.db subject=<id> predicate=occupation

# Snapshot commands
worldify snapshot create myworld.db --name "checkpoint_1"
worldify snapshot list myworld.db
worldify snapshot export myworld.db <snapshot_id> backup.json
worldify snapshot import myworld.db backup.json --mode merge

# Other
worldify summary myworld.db
worldify explain myworld.db <fact_id>

# Output formats
worldify entity find myworld.db --format json  # Also: table, csv

Limitations

Design Limitations (By Choice)

These are intentional constraints that keep Worldify simple and focused:

  1. No Network Access: Worldify uses local SQLite files only. It's not a distributed database.

  2. No Authentication: Anyone with file access has full read/write access. Handle security at the application level.

  3. No Visualization: Worldify is data storage only. Export to JSON and use external tools for visualization.

  4. Fixed Schema: You cannot add columns to built-in tables. Use the entity metadata field for custom data:

    entity = world.create_entity("person", "Alice", metadata={"custom_field": "value"})
  5. No Real-Time Updates: There's no event system or push notifications. Poll for changes if needed.

Technical Limitations (SQLite)

  1. Single Writer: Only one process can write at a time. Concurrent reads are fine.

  2. No Remote Access: The database file must be on the local filesystem.

  3. Scale Limits: Performance degrades with very large datasets:

    • Comfortable: Up to 100K entities, 1M facts
    • Workable: Up to 1M entities, 10M facts
    • Beyond that: Consider a different database
  4. Long Transactions Block: Long-running writes block other writers. Keep transactions short.

Behavioral Constraints

  1. Single-Value by Default: By default, you cannot have two facts with the same subject and predicate overlapping in time. Use multi_value=True for lists (skills, tags, etc.) or auto_supersede=True to automatically end old facts.

  2. Timestamp Precision: Timestamps use ISO 8601 format with microsecond precision. Comparisons are lexicographic (string-based), which works correctly for ISO format.

  3. No Partial Updates: You cannot update a fact's value in place. Use supersede_fact() to create a new fact with the updated value (this preserves history).

  4. Agent Sessions Are Advisory: The session system tracks who is accessing the database, but doesn't actually lock anything. It's for coordination, not enforcement.


Project Structure

worldify/
    worldify/           # Main package
        __init__.py     # Public API exports
        world.py        # World class (main interface)
        entity.py       # Entity management
        fact.py         # Fact management
        query.py        # Query builder
        causal.py       # Causal chain tracking
        snapshot.py     # Export/import functionality
        session.py      # Agent session tracking
        database.py     # SQLite connection management
        schema.py       # Database schema definition
        exceptions.py   # Custom exception classes
        utils.py        # Utility functions
        aggregates.py   # Aggregate query functions
        cli.py          # Command-line interface
    tests/              # Test suite (84 tests)
    docs/               # Detailed documentation
    examples/           # Working example scripts

Examples

The examples/ directory contains complete, runnable scripts:

Example Description
01_basic_usage.py Core operations: entities, facts, queries
02_game_state.py Game state management with health, inventory, locations
03_chatbot_memory.py Persistent memory for a chatbot
04_knowledge_graph.py Building and querying a knowledge graph
05_simulation_checkpoints.py Save/restore simulation state

Run any example:

python examples/01_basic_usage.py

Documentation


Testing

# Run all tests
python -m unittest discover tests

# Run with coverage
python scripts/run_tests.py --coverage

# Run specific test file
python -m unittest tests.test_core