Odigos · Dynamic telemetry for AI agents

Odigos

5 min read Original article ↗

Dynamic telemetry, decided at the moment you ask

Let your agentsinterrogate production.

Ask production a question nobody set it up to answer. Any function in any running service, answered in seconds, with no code change.

function values

code "BLACK50"
rule nil
returned 0.00

When your agent hits missing data

Both agents need the same value. Only one can go and get it.

without Odigosit opens a pull request

>why is checkout revenue down 12%

Traces look clean. 200s across POST /orders, p99 flat. The only thing I cannot see into is the discount path.

Read(promo.go)Read 84 lines

applyDiscount returns early when the rule lookup misses, and nothing logs what it returned. I cannot confirm this from here.

Update(promo.go)Updated promo.go with 1 addition

41 rule := rules.For(code)
42 if rule == nil {
43+ log.Info("discount", "returned", 0.0)44 return 0

Opened PR #4471. Once it is reviewed and deployed I will know after the next occurrence.

hours to days before it can answer

with Odigosit queries production

>why is checkout revenue down 12%

Traces look clean. 200s across POST /orders, p99 flat. The only thing I cannot see into is the discount path.

odigos - find_functions("checkout", "discount")84 functions, applyDiscount at promo.go:41

odigos - capture("applyDiscount", args, returns)attached in 1.2s, nothing redeployed

odigos - read("applyDiscount")code "BLACK50", cart 49.00, returned 0.00

Found it. BLACK50 is missing from the rules table, so applyDiscount returns 0 and every order using it paid full price.

seconds and it can ask the next one

The part that took years

Everyone else’s eBPF stops at the syscall.

Our own eBPF runtime reads what is actually happening inside a running process. Whatever you ask for, it captures on demand and exports as OpenTelemetry.

01 / 03

eBPF was built for the kernel. We made it see inside the application.

Off-the-shelf eBPF sees syscalls and network traffic. Ours reads inside the process: the functions that ran, the queries they made, the arguments they carried. Including a stripped, statically linked Go binary, which is the case every other approach gives up on. Nothing loads into your application to do it.

our own eBPF runtimeevery languagefunction-level depth

A different kind of eBPF

everyone else stops herethe syscall

POST /orders214ms · 200 OK

GET /cart31ms · 200 OK

that is the whole story they can tell

the edge of your service

odigos reads hereinside the code

applyDiscount("BLACK50", $49.00)

returned $0.00on every call

the value that explains the drop, out of a running service

02 / 03

Decide what to capture while the incident is still open.

Point at a function nobody ever set up to be watched, and Odigos starts capturing it inside running production: its arguments, what it returned, the calls underneath it. Seconds, not a deploy cycle. This is the whole difference between an agent that can investigate and a model that has to guess.

no redeployscaptured on demandsafe in production

Captured on demand

asks ❯ capture goroutine stack for checkout

stack + 14 spans returnedlive

03 / 03

Out of process. Out of your blast radius.

Every other way of getting this depth runs inside your process, one bad agent release away from taking the app down with it. Ours never touches your process. Under 1% CPU, safe to leave on across the whole estate, with RBAC and policy controls over what may be captured and by whom.

out of processno agent in your appRBAC & governance

Safe on all of production

out-of-processkernel-enforcedno in-process agent

What a capture actually is

Every capture is one line you can read.

A capture names a workload and a function. It is scoped per cluster, reversible, and governed by RBAC, so reading a value out of production is an action somebody authorised rather than a side effect of an agent.

The Odigos console: detected sources, in-flight actions and export destinations for a production cluster

How it works

The first thing you ask it is something your stack has never answered.

One command. Odigos maps the environment, attaches its own eBPF runtime, and starts answering questions nobody set your stack up to answer. It produces OpenTelemetry from every service with no code change and sends it to any backend you already run, replacing nothing.

One command, any environment

Run one command on Kubernetes or Linux. Odigos finds every service, works out what language it runs and what it talks to, and starts there. You do not tell it anything.

odigos install

Inside the process, not at the syscall

Our own eBPF runtime reads what is actually happening inside each process: the functions running, the queries they make, the arguments they carry. Nothing is wired into your apps.

eBPF probe attachedargumentsreturn valuesstacks

Ask, and keep asking

What did that function return. Which arguments did the failing requests carry. Which query did the slow path actually run. Anything missing is captured live, then exported as OpenTelemetry into whatever you already run.

DatadogGrafanaSplunkyour AI

Measured in production

Cheaper than the agent you are already running.

eBPF runs outside your process, so depth stops costing you throughput. One Fortune 500 customer benchmarked us against their legacy bytecode agent on the same traces, across 1.04 million cores. Another, one of the largest retailers in the world, built its own regression agent in-house on this data.

< 1%CPU overhead, out of process. Safe to leave on across all of production.

27.6%less CPU than their legacy bytecode agent at the top of the range they measured, on identical traces, on their own hardware.

1.04Mcores under measurement when they ran it. Zero code changes to capture something new.

// 11 enterprises in production, SOC 2 audited

One command. Any Kubernetes cluster.

Bring us the question your stack can’t answer.

Our own eBPF. Nothing enters your process.