이미지: AWS ML Blog
Summary
- AWS has added rate limiting to the Amazon Bedrock AgentCore gateway
- Traffic can be controlled using three metrics: requests (RPS/RPM), tokens (TPM), and connections (CPS)
- Granular rules can be set per user or group using JWT claims or IAM identities
- 발표 매체
- AWS ML Blog (작성자 Anagh Agrawal)
- 발행일
- 2026-08-06
- 대상 서비스
- Amazon Bedrock AgentCore gateway
- 지원 대상 타깃
- MCP 타깃, 추론(inference) 타깃, HTTP 패스스루 타깃
- 제한 지표
- 요청(RPS·RPM), 토큰(TPM), 연결(CPS)
- 인증 방식
- OAuth 또는 IAM 기반, JWT 클레임/IAM 아이디로 스코프 설정
AWS announced it has added rate limiting to Amazon Bedrock AgentCore gateway, its fully managed, serverless AI gateway. The AgentCore gateway serves as a single entry point for AI traffic heading to managed web search, knowledge bases, MCP servers, LLM inference models, agents (including A2A), and HTTP endpoints.
Three rate-limiting metrics
With this update, administrators can now control traffic using three metrics: request count (RPS/RPM), token throughput (TPM), and concurrent connections (CPS). Request-based limits apply to all target types, with each request counted as a single unit regardless of processing time. Token-based limits apply only to inference targets and cover both input and output tokens — the gateway deducts a preliminary estimate using a general-purpose tokenizer, then reconciles it afterward against the actual model response. Connection-based limits target long-lived sessions such as streaming, where a 100-second request occupies a connection slot for the entire duration.
Granular control by user and group
As an example, AWS presented a configuration with three user groups — Basic, Advanced, and Beta — combining JWT authentication via Microsoft Entra ID with AgentCore Identity and policy-based role access control (RBAC). In this setup, Basic users are given stricter limits while Beta users receive relatively relaxed limits for specific models, allowing an organization to validate a new model's performance and fit before rolling it out broadly. Rate limiting rules consist of "dimension keys," which group requests, and "entries," which define the allowed throughput for each group.
