Product Introduction
Overview
Tencent Cloud EdgeOne AI Gateway ensures security, visibility, and request behavior control management for accessing Large Language Model (LLM) service providers.
It currently supports capabilities like cache configuration and is developing features such as rate limiting, request retry, LLM model fallback, and virtual key. The combination of these features can effectively ensure the security and stability of accessing LLM service providers while reducing access costs.
Scenarios
Enterprise Office: Suitable for enterprise administrators to build an AI Gateway between employees and LLM service providers, controlling the secure access of employees to LLM service providers and cost control.
Personal Development: Suitable for AIGC individual developers to set up an AI Gateway between consumer users and LLM service providers to control the request behavior of consumer users.
Strengths
Cost Reduction: Utilize caching technology to respond to repeated Prompt Requests directly from the cache, without invoking LLM service providers again, effectively avoiding unnecessary duplicate fees, thus significantly reducing your operational costs.
Flexible Configuration: Handle various exceptions and complex scenarios by configuring request retries, rate limiting, LLM model fallback capabilities, and more to ensure service availability.
Data Monitoring: Through the Data Dashboard, you can obtain detailed statistical information on AI Gateway requests. This data will help you gain insights into traffic patterns, optimize business processes, and make more accurate business decisions.
High Security: Adopt virtual key technology to provide an extra layer of security. This mechanism ensures that the access keys to your LLM service providers are not exposed, protecting your data security and business privacy.
Warning:
The above capabilities are not fully available yet. If you are particularly interested in any part of these capabilities, please provide feedback to the product team.
LLM Service Providers
Currently, it supports Open AI, Minimax, Moonshot AI, Gemini AI, Tencent Hunyuan, Baidu Qianfan, Alibaba Tongyi Qianwen, and ByteDouban.