GitHub - davidesantangelo/cton: CTON provides a JSON-compatible, token-efficient text representation optimized for LLM prompts.

{
  "task": "planning",
  "urgent": true,
  "id": 123
}

task=planning,urgent=true,id=123

{
  "user": {
    "name": "Davide",
    "settings": {
      "theme": "dark"
    }
  }
}

user(name=Davide,settings(theme=dark))

{
  "tags": ["ruby", "gem", "llm"],
  "files": [
    { "name": "README.md", "size": 1024 },
    { "name": "lib/cton.rb", "size": 2048 }
  ]
}

tags[3]=ruby,gem,llm
files[2]{name,size}=README.md,1024;lib/cton.rb,2048

require "cton"

payload = {
  "context" => {
    "task" => "Our favorite hikes together",
    "location" => "Boulder",
    "season" => "spring_2025"
  },
  "friends" => %w[ana luis sam],
  "hikes" => [
    { "id" => 1, "name" => "Blue Lake Trail", "distanceKm" => 7.5, "elevationGain" => 320, "companion" => "ana", "wasSunny" => true },
    { "id" => 2, "name" => "Ridge Overlook", "distanceKm" => 9.2, "elevationGain" => 540, "companion" => "luis", "wasSunny" => false },
    { "id" => 3, "name" => "Wildflower Loop", "distanceKm" => 5.1, "elevationGain" => 180, "companion" => "sam", "wasSunny" => true }
  ]
}

# Encode to CTON
cton = Cton.dump(payload)
# => "context(... )\nfriends[3]=ana,luis,sam\nhikes[3]{...}"

# Decode back to Hash
round_tripped = Cton.load(cton)
# => original hash

# Need symbols?
symbolized = Cton.load(cton, symbolize_names: true)

# Want a truly inline document? Opt in explicitly (decoding becomes unsafe for ambiguous cases).
inline = Cton.dump(payload, separator: "")

# Pretty print for human readability
pretty = Cton.dump(payload, pretty: true)

# Stream to an IO object (file, socket, etc.)
File.open("data.cton", "w") do |f|
  Cton.dump(payload, f)
end

# Toggle float normalization strategies
fast  = Cton.dump(payload) # default :fast mode
strict = Cton.dump(payload, decimal_mode: :precise)

# Convert JSON to CTON
echo '{"hello": "world"}' | cton
# => hello=world

# Convert CTON to JSON
echo 'hello=world' | cton --to-json
# => {"hello":"world"}

# Pretty print
cton --pretty input.json

# Minify (fully inline, no separators)
cton --minify input.json

# Validate CTON syntax
cton --validate input.cton
# => ✓ Valid CTON

# Show token savings statistics
echo '{"name": "test", "items": [1,2,3]}' | cton --stats
# => JSON:  33 chars / 33 bytes (~9 tokens)
# => CTON:  26 chars / 26 bytes (~7 tokens)
# => Saved: 21.2% (7 chars, ~2 tokens)

cton_with_comments = <<~CTON
  # User configuration
  user(
    name=Alice,
    # Age is optional
    age=30
  )
CTON

Cton.load(cton_with_comments)
# => {"user" => {"name" => "Alice", "age" => 30}}

# Add comments when encoding
Cton.dump(data, comments: { "user" => "User configuration" })

# Quick validity check
Cton.valid?("key=value")  # => true
Cton.valid?("key=(broken")  # => false

# Detailed validation with error info
result = Cton.validate("key=(broken")
result.valid?  # => false
result.errors.first.message  # => "Expected '=' in object"
result.errors.first.line     # => 1
result.errors.first.column   # => 5

stats = Cton.stats(data)
puts stats.savings_percent    # => 45.5
puts stats.estimated_token_savings  # => 12

# Full comparison
puts stats.to_s
# => JSON:  100 chars / 100 bytes (~25 tokens)
# => CTON:   55 chars /  55 bytes (~14 tokens)
# => Saved: 45.0% (45 chars, ~11 tokens)

# Compare all format variants
Cton::Stats.compare(data)
# => { cton: {...}, cton_inline: {...}, json: {...}, ... }

class Money
  attr_reader :cents, :currency
  def initialize(cents, currency)
    @cents = cents
    @currency = currency
  end
end

# Register as object
Cton.register_type(Money) do |money|
  { amount: money.cents, currency: money.currency }
end

Cton.dump("price" => Money.new(1999, "USD"))
# => "price(amount=1999,currency=USD)"

# Register as scalar
Cton.register_type(UUID, as: :scalar) { |uuid| uuid.to_s }

# Unregister when done
Cton.unregister_type(Money)

begin
  Cton.load("user(name=Alice,invalid")
rescue Cton::ParseError => e
  puts e.message       # => "Unterminated object at line 1, column 20"
  puts e.line          # => 1
  puts e.column        # => 20
  puts e.source_excerpt  # => "...name=Alice,invalid"
  puts e.suggestions   # => ["Did you forget a closing ')'?"]
end

bundle exec ruby bench/encode_decode_bench.rb
# customize input size / iterations
ITERATIONS=2000 STREAM_SIZE=400 bundle exec ruby bench/encode_decode_bench.rb

You are an expert in data serialization and specifically in CTON (Compact Token-Oriented Notation). CTON is a token-efficient data format optimized for LLMs that serves as a compact alternative to JSON.

Your task is to interpret CTON input and convert it to JSON, or convert JSON input into valid CTON format, following the specification below.

### CTON Specification

CTON minimizes syntax characters (braces, quotes) while preserving structure and type safety.

**1. Basic Structure (Key-Value)**
- **Rule:** Do not use outer curly braces `{}` for the root object.
- **Rule:** Use `=` to separate keys and values.
- **Rule:** Use `,` to separate fields.
- **Rule:** Do not use quotes around "safe" strings (alphanumeric, simple text).
- **Example:** - JSON: `{"task": "planning", "urgent": true}`
  - CTON: `task=planning,urgent=true`

**2. Nested Objects**
- **Rule:** Use parentheses `()` to denote a nested object instead of `{}`.
- **Example:**
  - JSON: `{"context": {"user": "Davide", "theme": "dark"}}`
  - CTON: `context(user=Davide,theme=dark)`

**3. Arrays of Objects (Table Compression)**
- **Rule:** Use the syntax `key[count]{columns}=values` for arrays of objects to avoid repeating keys.
- **Structure:** `key[Length]{col1,col2}=val1,val2;val1,val2`
- **Details:** - `[N]` denotes the number of items in the array.
  - `{col1,col2}` defines the schema headers.
  - `;` separates distinct objects (rows).
  - `,` separates values within an object.
- **Example:**

JSON:
```json
{
  "files": [
    { "name": "README.md", "size": 1024 },
    { "name": "lib.rb", "size": 2048 }
  ]
}
```

CTON: `files[2]{name,size}=README.md,1024;lib.rb,2048`

**4. Type Safety & Literals**
- **Booleans/Null:** `true`, `false`, and `null` are preserved as literals (unquoted).
- **Numbers:** Integers and floats are written as is (e.g., `1024`, `3.14`).
- **Escaping:** If a string value looks like a boolean, number, or contains reserved characters (like `,`, `;`, `=`, `(`, `)`), it must be wrapped in double quotes (e.g., `"true"`).

### Examples for Training

**Input (JSON):**
```json
{
  "id": 123,
  "active": true,
  "metadata": {
    "created_at": "2023-01-01",
    "tags": "admin"
  }
}
```

bin/setup        # install dependencies
bundle exec rake # run tests and rubocop
bin/console      # interactive playground
bundle exec ruby bench/encode_decode_bench.rb # performance smoke test