debaite

Debaite: Multi-Model Document Debate & Refinement System

Debaite orchestrates multiple LLM models to collaboratively refine documents through iterative critique and debate.

What It Does

Given a brief summary, debaite:

1. Has each model expand it into a comprehensive document
2. Has all models critique each document (parallel execution)
3. Refines documents based on collective feedback
4. Repeats until quality threshold or max rounds

Each model's document evolves independently based on all models' feedback.

Installation

pip install llm

Or install from source:

cd /path/to/debaite
pip install -e .

Configuration

Set your OpenRouter API key:

export OPENROUTER_API_KEY="your-api-key"

Usage

debaite -m "model1,model2,model3" -i summary.txt

Options

Flag	Description	Default
-m, --models	Comma-separated model identifiers	Required
-i, --input	Path to input summary file	Required
-r, --rounds	Maximum refinement rounds	10
--min-rounds	Minimum rounds before early stopping	3
-t, --threshold	Quality threshold (0-10)	8.0
-o, --output-dir	Output directory	Current
-v, --verbose	Enable verbose output	False

Example

debaite \
    -m "openrouter/meta/llama-3-8b,openrouter/anthropic/claude-3-5-sonnet" \
    -i project_summary.txt \
    -r 10 \
    --min-rounds 3 \
    -t 8.0 \
    -v

Output Files

Files are named {base}-{model_slug}-{doc_id}-{round}.md:

project_summary-openrouter-meta-llama-3-8b-a7x2-1.md      # Initial expansion
project_summary-openrouter-meta-llama-3-8b-a7x2-2-b3k9.md # Comment from another model
project_summary-openrouter-meta-llama-3-8b-a7x2-3.md      # Refined version
...

Comments use the same format with an extra {comment_id}:

{base}-{model_slug}-{doc_id}-{round}-{comment_id}.md

How It Works

1. Initialization

Each model gets a unique 4-character ID and creates its initial document (round 1).

2. Debate Loop

For each pending document:

• All models judge the document in parallel
• Comments are saved with scores (0-10)
• Average score is calculated

3. Stopping Conditions

A document stops when:

• round >= max_rounds, OR
• avg_score >= threshold AND round >= min_rounds

The min-rounds requirement ensures even high-scoring documents receive adequate critique.

4. Refinement

The original model reviews all feedback and creates an improved version. It can accept, modify, or reject critiques as it sees fit.

Output Format

Judge responses must use:

<comments>Your critique here</comments>
<complete>8</complete>

Requirements

• Python 3.9+
• llm package
• OpenRouter API key (for OpenRouter models)

License

MIT