As AI agents move out of the realm of experimentation and into core enterprise workflows, the question of how to govern them effectively becomes critical. Companies are deploying agents across teams and runtime environments, making software delivery, customer operations, risk management, and other workflows more productive and efficient. But as deployments expand, the concern shifts from how to build capable agents to how to govern them consistently at scale.
The primary challenge is fragmentation. Teams are constantly evaluating new solutions, deciding what to adopt and what to replace, and this tends to make the underlying solution stacks transient and heterogeneous, with agents operating under different governance policies and monitoring practices. A cobbled-together approach of this sort can work with a dozen agents, but becomes harder to sustain with a hundred or more. As agents gain access to enterprise workflows, sensitive data, and operational decision-making, regulatory scrutiny compounds, making the issue even more urgent.
In our work developing Agentic AI Mesh architecture, we have found a way to address this challenge using a distributed governance model that equips each agent with an identity and compliance framework. Using a standard open specification that represents the agentic harness, we created smart contracts that embed governance within each agent. At runtime, these contracts can power an agentic service mesh that runs alongside the main agent, monitoring and evaluating its behavior, and enforcing policy and guardrails. This is one approach among several: centralized control planes and framework-native governance are others, and each has different tradeoffs. What follows is the path we took and what we learned from it.
The governance gaps that prevent agentic AI from scaling
Unless explicit steps are taken to prevent it, fragmentation tends to set in as enterprises scale up agent deployment. This manifests in several ways:
- At the configuration layer, different frameworks are applied, each with its own proprietary schemas, runtime abstractions, and orchestration patterns. This leads to a lack of standardization.
- At the operational layer, guardrails are developed on an ad-hoc basis, often embedded directly into individual applications rather than managed through reusable governance layers. This limits runtime control.
- At the validation layer, evaluation and quality assurance controls are often implemented late in the process, frequently in reaction to observed misbehaviors in production. This relegates validation to a reactive process, rather than a predictive, pre-deployment one.
- Combined, these gaps make frameworks and orchestration models difficult to transfer efficiently, leading to lock-in that can slow enterprise-wide scaling efforts.
Press enter or click to view image in full size
In our experience, building governance in rather than bolting it on has held up better as deployments grow. When built into the agentic architecture, governance can actually accelerate scaling and innovation by:
- preventing operational inconsistencies
- making agents more modular and reusable
- reducing the cost of migrating to better solutions as technology evolves
Towards built-in agentic governance using smart contracts
In modern microservices environments, a service mesh separates operational concerns (security, observability, resiliency, policy enforcement) from application logic itself. We applied a similar architectural principle to our AI agents, introducing a sidecar proxy that operates alongside each agent at runtime. The proxy was responsible for enforcing governance policies, validating runtime behavior on sample traces, monitoring execution, maintaining auditability, and metering guardrails independently of the orchestration framework.
Under this model, the smart contract and sidecar create a built-in distributed governance layer that can operate across different environments. This approach offers several advantages over traditional centralized control models:
- governance becomes attached to the agent, making it self-auditable, rather than tied exclusively to the platform
- runtime validation and policy enforcement remain consistent even when orchestration frameworks change
- operational controls became built-in across the lifecycle especially as part of the CI/CD pipelines as well as probed within live environments
Press enter or click to view image in full size
The practical shift is from an external control plane that governs agents from the outside to an embedded control plane that travels with each agent. That is what contributed to make our agents more self-standing, reusable, and auditable across use cases, building trust at the level of the individual agent.
The smart contract defines. The sidecar enforces.
The smart contract represents the agentic harness manifest, carrying the agent’s identity, model choices, tools, memory, guardrails, evaluation rules, observability controls, compliance expectations, and runtime requirements.
Press enter or click to view image in full size
Based on this, the agent sidecar proxy:
- continuously enforces or verifies agent behavior against the harness manifest.
- operationalizes these controls during execution by validating approved models and tools’ use, alerting or enforcing permissions and policies, monitoring runtime behavior, and triggering human-in-the-loop requests.
- runs evaluations on a sample of production traces, manages authentication, exports audit evidence, and eventually detects or escalates contract violations.
- operates as a deterministic enforcement layer, evaluating runtime behavior against the declared contract and executing predefined actions such as allow, block, throttle, log, or escalate
We also used the sidecar in an agentic mode, in which it selectively invoked evaluators to assess behaviors that are difficult to capture through static rules alone. For example, a local evaluator or lightweight LLM-as-a-judge reviewed a configurable sample of interactions (e.g., five percent of traffic) and helped flag cases where observed behavior appeared to deviate from the declared contract.
Case Example: A leading technology provider has deployed reliable, reusable agents to large government entities, intending them to scale rapidly across administrative workloads, citizen facing workflows, risk reporting, and software development. As part of this effort, the team decides to direct some of the traffic to a local LLM to ensure data residency without needing a full platform to monitor and control the agent.
Applying a Story Grooming agent, they move several capabilities previously embedded in the application — including model configuration, prompts, tool definitions, and operational controls — into a declarative smart contract. The team can then isolate and tailor the agent, optimizing it on a smaller self-hosted LLM focused on specialized tasks that don’t require hyperscaler models. With roughly 15% of traffic routed locally, the team can restrict data residency on some part of the SDLC flow within built-in metrics and evaluations to ensure that the agent behaves as expected.
Press enter or click to view image in full size
Five principles for operating at scale
This architecture builds on principles introduced in our earlier work on the Agentic AI Mesh, extending them through Smart Contracts for AI Agents and portable runtime enforcement. Five core principles inherited from the original Agentic AI Mesh hold this architecture together — and each one directly addresses a scaling bottleneck:
- Composability means that any agent, regardless of framework or runtime, can be introspected and discovered in a trust network without requiring system-wide rework.
- Distributed intelligence lets tasks be decomposed and resolved by networks of cooperating agents, each with compliance baked in. Instead of a single monolithic orchestrator, we use a web of capable, governed agents collaborating at the task level, with each sidecar ensuring its agent stays within declared boundaries.
- Layered decoupling separates an agent’s logic, memory, orchestration, and interface functions into declared layers, maximizing both modularity and portability. When one layer needs to change because of a new model, a different memory backend, or an updated set of tools, the rest stays untouched.
- Vendor neutrality means every agent can be re-targeted to any framework, updated or replaced as the technology evolves, without locking you into a single vendor’s ecosystem. The same manifest that defines an agent for a framework can generate runnable code to another one.
- Governed autonomy keeps agent behavior proactively controlled through embedded policies, evaluations suite and idempotent escalation mechanisms. Agents operate with significant freedom, but within boundaries that are declared in the manifest, enforced by the sidecar policy engine, and auditable — even across heterogeneous runtimes.
Case Example: A Telecom player is embarking on a full agentic transformation journey where agentic workloads are expected to span multiple cloud providers and runtimes. The organization opts for a distributed governance architecture, with the objective of increasing the governance coverage from 10% to +80%, where each agent will be attached with a side agent mesh proxy to enforce required policies and report to a central agentic governance plane.
The SCO framework
This approach does carry overhead: organizations take on sidecar infrastructure, policy consistency, and governance ownership across teams. To support that, we used a Shape-Control-Operate (SCO) framework:
- Shape: During the shaping phase, organizations define the operational characteristics of the agent: model selection, tool configuration, prompts, memory architecture, evaluation criteria, and API interfaces. This is also the phase when security and governance controls are introduced, including policy definitions, access restrictions, PII safeguards, and prompt injection protections.
- Control: The control phase focuses on validation and auditability. Schema validation, runtime health checks, evaluation scoring, and compliance testing help ensure that agents meet operational and governance requirements before deployment. This phase can also support alignment against emerging regulatory and security standards, including internal governance baselines and external frameworks.
- Operate: Once deployed, the operating phase focuses on runtime reconciliation and continuous monitoring. The system evaluates whether deployed agents continue to align with their declared specifications, policies, and operational constraints. Drift detection, runtime monitoring, audit logging, and policy enforcement remain active throughout execution.
Press enter or click to view image in full size
Model choices, memory boundaries, tool permissions, evaluation standards, runtime policies and escalation rules are declared during the design stage, rather than discovered after deployment. Together, these lifecycle stages establish a continuous governance loop capable of evolving alongside the agent ecosystem itself.
Summary
The enterprise agentic AI landscape is still evolving rapidly. Frameworks, orchestration models, and deployment patterns are likely to change significantly over the coming years. The governance challenge, though, looks structural rather than temporary. Centralized platform models give strong integrated controls but introduce ecosystem dependency. Many frameworks maximize flexibility but leave governance responsibilities fragmented across teams and implementations.
We observed that built-in governance provides a practical tradeoff that also fits, when necessary, with a central platform governance. The broader implication is that enterprise AI governance starts to look like modern distributed systems of governance, with portable governance layers that enforce policies consistently across changing runtime environments.