Apigee — AI Integration & Best Practices

The same gateway that governs your REST APIs is ideal for governing LLM traffic. As AI usage spreads across teams, Apigee becomes the control plane: one place for auth, cost, caching, routing, and safety in front of model providers.

Apigee as an AI gateway

A proxy fronting model providers gives you:

• Centralised credentials — provider keys live in the gateway (KVM/Secret Manager), not in every app. Apps authenticate to Apigee.
• Per-consumer auth & quotas — VerifyAPIKey/OAuth + Quota so each team/app has identity and limits.
• Token-based cost control — extract token usage from responses and meter/alert on it; enforce budgets per app.
• Caching — ResponseCache for identical requests; semantic caching via a callout for similar ones.
• Routing & fallback — RouteRules to send tasks to the right model or fail over to a backup provider.
• Safety & governance — payload/threat protection, PII handling, and full logging/analytics for audit.
• Streaming passthrough — proxy SSE/streamed tokens so UX isn't broken.

Sketch: a model proxy

<!-- PreFlow: identify caller, enforce quota, inject the provider key -->
<Step><Name>VerifyAPIKey</Name></Step>
<Step><Name>QuotaPerApp</Name></Step>
<Step><Name>AM-SetProviderAuth</Name></Step>   <!-- AssignMessage: add Authorization from KVM -->

Keep the provider key in an encrypted KVM and attach it via AssignMessage — consumers never see it.

Consolidated best practices

• ✅ Thin proxy: the gateway enforces cross-cutting policy; business logic stays in backends.
• ✅ Native policies over code; JavaScript for light variable glue only.
• ✅ Externalise config & secrets to target servers + encrypted KVM / Secret Manager.
• ✅ Always protect with SpikeArrest + Quota + threat protection in PreFlow.
• ✅ Shared flows for common auth/logging — DRY governance.
• ✅ CI/CD: proxies in git, apigeelint + tests, build once / promote the same revision, gated prod.
• ✅ Observability: structured async logging + analytics; for AI, meter tokens and cost.

Consolidated anti-patterns

• ❌ JavaScript ServiceCallout for backend/HTTP I/O — use the ServiceCallout policy or route rules; JS callouts are slow, fragile, and unobservable.
• ❌ Heavy logic in JS/Java policies when a native policy exists.
• ❌ Synchronous logging to a slow sink in the request path.
• ❌ Secrets in bundles or policy XML; per-environment hardcoded URLs.
• ❌ Editing proxies in the prod UI; rebuilding per environment.
• ❌ Unprotected proxies (no spike arrest/quota); for AI, no token budgets → runaway spend.

You've completed the course 🎉

You can now architect, build, secure, deploy, and automate Apigee APIs — and use Apigee as an enterprise AI gateway. Take the practice quiz below and revisit any page to go deeper.

Related reading

• AI Gateways: Managing LLM Traffic in the Enterprise
• Microservices Best Practices
• AI System Resilience