Run agents in multiple namespaces

Prepare additional namespaces to run agents, restrict the controller to the namespaces that you choose, and share configuration across them.

By default, kagent runs agents only in the kagent namespace. The controller watches every namespace in the cluster. When several teams run agents, give each team a namespace of its own. This guide prepares a namespace to run agents, limits the controller to the namespaces that you list, and points to the settings that share a model or an MCP server across namespaces.

Default namespace layout

A kagent installation places its components and permissions in a few namespaces. The layout depends on how you installed kagent. Each default links to the section of this guide that changes it.

When you install kagent with kagent install, the installation sets up the following defaults:

The kagent namespace holds the controller, the UI, the tool server, the development database, and the kagent-default WorkerPool. It is the only namespace that can run agents. To run agents in another namespace, see Prepare a namespace to run agents.
The ate-system namespace holds Agent Substrate. The podcertificate-controller-system namespace holds its Pod Certificate controller.
The controller watches every namespace. The chart grants the controller cluster-scoped role-based access control (RBAC). To narrow both scopes, see Restrict kagent to specific namespaces.
Agent Substrate has one credential policy. The policy lets agents in kagent read Secrets in kagent only. Each namespace that you prepare gets a policy of its own.
The default-model-config ModelConfig and the kagent-tool-server RemoteMCPServer admit references from the kagent namespace only. To use them from another namespace, see Share configuration across namespaces.

When you install the charts directly with Helm, the installation sets up the following defaults:

The kagent namespace holds the controller, the UI, and the tool server. The namespace has no WorkerPool, because the kagent chart sets substrateWorkerPool.create to false. No namespace can run agents, including kagent, until you prepare it as described in Prepare a namespace to run agents.
The ate-system namespace holds Agent Substrate.
The controller watches every namespace. The chart grants the controller cluster-scoped RBAC. To narrow both scopes, see Restrict kagent to specific namespaces.
Agent Substrate has no credential policy, because the Agent Substrate chart sets credentialProvider.namespacePolicies to an empty list. Each namespace that you prepare gets a policy of its own.
Agent Substrate has no identity pools. Only kagent install creates the certificate authority pools and the JWT authority pool. For more information, see The Agent Substrate plane.
The default-model-config ModelConfig and the kagent-tool-server RemoteMCPServer admit references from the kagent namespace only. The chart creates default-model-config for the provider that providers.default names, which defaults to openAI. To use either resource from another namespace, see Share configuration across namespaces.

Before you begin

Install kagent.
Install yq version 4. The steps use yq to edit the Agent Substrate Helm values.

Prepare a namespace to run agents

An Agent needs a WorkerPool and an Agent Substrate credential policy in its own namespace. A Harness can select only a WorkerPool in the Harness’s namespace. Agent Substrate denies every Secret read from a namespace that has no credential policy. The denial includes reads of that namespace’s own Secrets. Without the policy, the Agent becomes ready. However, every model call fails with a 403 Forbidden error.

The following steps prepare the team-a namespace.

Create a ServiceAccount and a WorkerPool in the namespace. The pool must set both placement labels to its own name and namespace. For more information about the labels, see Keep placement scoped to the pool’s namespace.

kubectl create namespace team-a
kubectl create serviceaccount team-a-worker -n team-a
kubectl apply -f - <<EOF
apiVersion: ate.dev/v1
kind: WorkerPool
metadata:
  name: team-a-pool
  namespace: team-a
  labels:
    kagent.dev/worker-pool-name: team-a-pool
    kagent.dev/worker-pool-namespace: team-a
spec:
  replicas: 1
  sandboxClasses:
    - name: gvisor
      configRef:
        name: gvisor-default
  workerImage: ghcr.io/kagent-dev/substrate/ateworker-gvisor:v0.6.0-alpha3
  template:
    serviceAccountName: team-a-worker
EOF

Add an Agent Substrate credential policy for the namespace.

Save the current values of the Agent Substrate release. The credentialProvider.namespacePolicies list that you set replaces the existing list. The next substep builds the new list from these values.
```
helm get values substrate -n ate-system --all -o yaml > substrate-current-values.yaml
```

Write a values file that keeps the other namespaces’ policies and sets the policy for this namespace. The command replaces any existing policy for the namespace. The atespace value is the namespace of the Agent. The allowedNamespaces value lists the namespaces whose Secrets its agents can read.

yq '{"credentialProvider": {"namespacePolicies": ((.credentialProvider.namespacePolicies // []) | map(select(.atespace != "team-a"))) + [{"atespace": "team-a", "allowedNamespaces": ["team-a"]}]}}' \
  substrate-current-values.yaml > substrate-namespace-policies.yaml
cat substrate-namespace-policies.yaml

Example output:

credentialProvider:
  namespacePolicies:
    - allowedNamespaces:
        - kagent
      atespace: kagent
    - atespace: team-a
      allowedNamespaces:
        - team-a

Upgrade the Agent Substrate release with the values file.

helm upgrade substrate \
  oci://ghcr.io/kagent-dev/substrate/helm/substrate \
  --version 0.4.0-alpha1 \
  --namespace ate-system \
  --reuse-values \
  -f substrate-namespace-policies.yaml

Confirm that the WorkerPool has a ready Worker.

kubectl get workerpools -n team-a

The READY column for the team-a-pool lists 1:

NAME          DESIRED   REPLICAS   READY   SANDBOXCONFIG    RESOLVED   AGE
team-a-pool   1         1          1       gvisor-default   True       6s

Create a Harness in the namespace that selects the pool. The spec.substrate.workerPoolRef field names the pool. For the other Harness fields, see Agent harness.

kubectl apply -f - <<EOF
apiVersion: api.kagent.dev/v1alpha3
kind: Harness
metadata:
  name: team-a-harness
  namespace: team-a
spec:
  kagent: {}
  substrate:
    workerPoolRef:
      name: team-a-pool
    snapshotPolicy:
      location: s3://ate-snapshots/team-a/
EOF

Create a ModelConfig and its API key Secret in the namespace. Agents in team-a cannot use default-model-config, because it admits only the kagent namespace. For the commands, see the page for your provider, such as Create the ModelConfig for OpenAI, and replace kagent with team-a. To use a ModelConfig from another namespace instead, see Share configuration across namespaces.

Restrict kagent to specific namespaces

By default, the controller watches every namespace. The chart creates ClusterRoles and ClusterRoleBindings for cluster-scoped RBAC. To limit the controller, restrict it to the namespaces that run agents. Then, verify that an unwatched namespace stays unmanaged.

Set the watched namespaces

global.watchNamespaces sets two scopes from one value: the namespaces where the chart grants the controller RBAC, and the namespaces that the controller watches. For each listed namespace, the chart replaces the controller’s getter and writer ClusterRoles and the tool server’s ClusterRole with a Role and RoleBinding. The ClusterRole for service account authentication and the metrics ClusterRoles stay cluster-scoped. The chart also adds each listed namespace to the watch. The chart appends the install namespace to the list, so the release’s own resources remain managed. Because one value sets both scopes, the two scopes cannot disagree.

Set global.watchNamespaces when you install kagent, or later with a helm upgrade of the kagent release. Create every listed namespace before upgrading, because the chart creates a Role and RoleBinding in each one.

helm upgrade kagent \
  oci://ghcr.io/kagent-dev/kagent/helm/kagent \
  --version 1.0.0-alpha17 \
  --namespace kagent \
  --reuse-values \
  --set 'global.watchNamespaces={prod-agents,staging-agents}'

The change restarts the controller with the new watch scope. To return to watching every namespace, set the value to an empty list, such as --set-json 'global.watchNamespaces=[]'.

Set the RBAC and watch scopes separately

The chart also has a separate key for each scope. The rbac.namespaces key sets the RBAC scope, and the config.kubernetes.watchNamespaces key sets the watch scope. Use these keys only when the two scopes must differ. Otherwise, use global.watchNamespaces. The following rules apply to the separate keys:

rbac.namespaces takes precedence over global.watchNamespaces whenever the key is present. An explicit rbac.namespaces: [] restores cluster-scoped RBAC. When rbac.namespaces is set, it must include the install namespace. Otherwise, the render fails.
config.kubernetes.watchNamespaces is an explicit watch override. When rbac.namespaces is set, a watch entry that references a namespace outside the RBAC scope fails the render, because the controller would loop on Forbidden errors at runtime. When global.watchNamespaces is set, the chart adds the entries to the watch scope without failing the render. In both cases, the watch list replaces the default list. The chart does not add the install namespace. To keep watching the install namespace, list it explicitly.

Grant Namespace access for selector-based sharing

A ModelConfig or RemoteMCPServer whose allowedNamespaces.from field is set to Selector admits namespaces by their labels. To read those labels, the controller must list and watch Namespaces. A restricted installation grants only namespaced Roles, so the controller cannot list Namespaces. The Agent’s ResolvedRefs condition is then False, with a message that contains requires Namespace list/watch access.

To share configuration by selector on a restricted installation, grant the controller’s ServiceAccount list and watch on Namespaces.

kubectl apply -f - <<EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: kagent-namespace-reader
rules:
- apiGroups: [""]
  resources: ["namespaces"]
  verbs: ["list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kagent-namespace-reader
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kagent-namespace-reader
subjects:
- kind: ServiceAccount
  name: kagent-controller
  namespace: kagent
EOF

Verify the restricted installation

Verify a namespaced installation by checking that an unwatched namespace stays unmanaged. The kubectl get agents command and conditions checks from the guide to create your first agent apply to each watched namespace.

Create one Agent in each watched namespace. An Agent outside the kagent namespace needs a WorkerPool and an Agent Substrate credential policy in its namespace before it can run. For the steps, see Prepare a namespace to run agents.
```
kubectl apply -n prod-agents -f my-agent.yaml
kubectl apply -n staging-agents -f my-agent.yaml
```
Create one Agent in a namespace that the list does not cover.
```
kubectl apply -n dev-agents -f my-agent.yaml
```

Confirm that the controller reconciles the watched Agents.

kubectl get agents -n prod-agents
kubectl get agents -n staging-agents

Example output:

NAME       READY   AGE
my-agent   True    30s
NAME       READY   AGE
my-agent   True    30s

Confirm that the Agent in the unwatched namespace stays unmanaged. The controller does not have a watch entry for that namespace. The Agent’s READY value therefore stays empty, and the conditions command prints nothing.
```
kubectl get agents -n dev-agents
kubectl get agent my-agent -n dev-agents -o jsonpath='{.status.conditions}'
```
Example output:
```
NAME       READY   AGE
my-agent           30s
```

Share configuration across namespaces

An Agent in one namespace can use a ModelConfig or a RemoteMCPServer in another namespace when the target resource admits the Agent’s namespace. A platform team can then keep one model configuration and its credentials in a shared namespace. For the conditions and the steps, see the following sections:

Namespace isolation

A namespace separates the agents of one team from another team’s agents in the following places:

Kubernetes RBAC governs who can create Agents, Harnesses, AgentTemplates, and EgressPolicies in the namespace.
The Agent Substrate credential policy limits which namespaces’ Secrets the namespace’s agents can read.
An Agent or SandboxTemplate can select only the EgressPolicies in its own namespace. For more information, see Access control.
A ModelConfig or RemoteMCPServer admits references only from the namespaces that its allowedNamespaces field selects.

A namespace does not separate the callers of the kagent API. The open source authorizer permits every API check. A caller that reaches the controller can start Sessions with an Agent in any watched namespace. The caller can also write AgentTemplates and Harnesses through the API without Kubernetes RBAC. Creator scoping keeps one caller’s Sessions out of another caller’s list. However, creator scoping is not a security boundary. For how kagent identifies and authorizes callers, see Identity.

Next steps

IdentityLearn how kagent identifies each caller and scopes a Session to its creator. Egress policiesGrant an agent access to more destinations, and inject credentials into its requests. SandboxingLearn how Agent Substrate isolates each Actor in its own sandbox.

Tune Agent Substrate Debug