ExoExo
§04 — Resources & CRDs

MCP & LLMGateway

Two infrastructure CRDs the operator turns into Deployments + Services: an MCP proxy that fronts upstream tool servers, and an LLM gateway that fronts model providers.

5 min read·Set by Exo Editorial·v0.3.0 Beta

Where Agent and Tool describe work, MCP and LLMGateway describe shared infrastructure. Each reconciles into a Deployment plus a Service that agents in the namespace point at.

MCP

An MCP runs a proxy that fronts one or more upstream MCP servers and exposes a filtered tool surface. Agents bind it through spec.mcpRefs. Upstreams can be http, another mcpRef, or stdio, each with its own auth and tool filter.

An MCP proxy

●mcp.yaml· yaml
1apiVersion: v1
2kind: Secret
3metadata: { name: github-upstream-pat, namespace: team-a }
4type: Opaque
5stringData: { token: "ghp_REPLACE_ME" }
6---
7apiVersion: agentkube.io/v1alpha1
8kind: MCP
9metadata:
10  name: github-tools
11  namespace: team-a
12spec:
13  port: 8080
14  replicas: 2
15  upstreams:
16    - name: github
17      kind: http                      # http | mcpRef | stdio
18      endpoint: https://mcp.github.com
19      auth:
20        mode: bearer
21        secretRef: { name: github-upstream-pat, key: token }
22      tools:
23        mode: allow
24        allow: [create_issue, list_issues, add_comment]

LLMGateway

An LLMGateway is a provider gateway: it terminates model traffic, routes to upstreams (OpenAI, Anthropic, Bedrock, Vertex, Azure OpenAI, Ollama, vLLM), and applies rate limits, guardrails, and caching. Agents reach it instead of calling a provider directly, so credentials and policy live in one place.

A gateway

●llmgateway.yaml· yaml
1apiVersion: agentkube.io/v1alpha1
2kind: LLMGateway
3metadata:
4  name: shared-gateway
5  namespace: platform
6spec:
7  upstreams:
8    - name: anthropic
9      provider: anthropic
10      credentialsRef: { name: anthropic-key, key: api-key }
11      models: [claude-opus-4-7, claude-sonnet-4-6, claude-haiku-4-5]
12      priority: 1
13  routing:
14    defaultModel: claude-opus-4-7
15    modelMap:
16      - { from: "opus",   to: "claude-opus-4-7",   upstream: anthropic }
17      - { from: "sonnet", to: "claude-sonnet-4-6", upstream: anthropic }
18  rateLimit:
19    requestsPerMinute: 600
20    scope: perClient            # global | perClient | perModel
21  guardrails:
22    piiRedaction: true
23    inputFilters: [prompt-injection]
24  auth:
25    mode: apikey
Previous · Resources & CRDsToolNext · Resources & CRDs Claw