API provider Kong is introducing its open-source AI Gateway today, expanding on its existing API gateway to enable developers and operations teams to connect their applications with one or more large language models (LLMs) and access them via a unified API. Additionally, Kong is unveiling various AI-specific functionalities such as prompt engineering and credential management.
According to Kong co-founder and CTO Marco Palladino, AI represents another facet of API use cases alongside mobile and microservices. Palladino emphasizes the intertwined nature of APIs and AI, highlighting their role in both consuming and refining AI technologies. He envisions a future where organizations will prioritize running AI models locally to maintain data privacy, with cloud utilization as a secondary option. In the interim, companies must address challenges like credential management, traffic monitoring, and quota regulation for cloud-based models.
The API gateway by Kong aims to streamline AI development by enabling developers to seamlessly integrate LLM providers without the need for extensive code modifications. Currently, supports Anthropic, Azure, Cohere, Meta’s LLaMA models, Mistral, and OpenAI.
While many API providers treat AI APIs similarly to other APIs, Kong’s approach involves enhancing APIs with AI-specific features to unlock new possibilities and simplify implementation. By leveraging AI response and request transformers within the new AI Gateway, developers can dynamically adjust prompts and results, facilitating tasks like automatic translation and data anonymization.
The gateway also prioritizes prompt engineering, empowering businesses to enforce guidelines on LLMs effectively. This centralized management approach ensures consistency in guideline nforcement and prompt customization.
Since the inception of the Kong API Management Platform nearly nine years ago, the company has evolved significantly from its earlier identity as Mashape. Kong’s CEO Augusto Marietti reflects on the company’s transformation, highlighting Kong’s emergence as a leading API product on GitHub and its current financial stability. Kong’s Gateway serves as the foundation of its platform and powers the innovative AI Gateway seamlessly accessible through a simple upgrade for existing Kong users.
The newly introduced AI features are currently offered at no cost, with plans for premium paid features in the future. Kong’s primary focus with this release is to enhance user experience and functionality, with monetization as a secondary consideration down the line.
