🐦 What is Glide?
Glide is a cloud-native open-source (opens in a new tab) LLM gateway, delivering high-performance LLMOps in one single binary.
The idea of Glide is to take away common functionality needed during communication with LLMs and building GenAI applications on a level of a separate service. That way you don't have to put that pile of logic into your applications and just seamlessly utilize it via Glide. So you can focus on building your core business logic and solve common concerns via configuring Glide.
Glide is currently under active development. Feel free to give us a star and watch in Github (opens in a new tab) not to miss any updates.
Why Glide?
Focus on your business logic and application while letting Glide handle common LLMOps problems (like fallbacks, cost control, routing, caching, etc.)
High performance and scalability are our priority, so Glide won't slow you down and scale out as you grow.
Glide is designed around resiliency and failures, so your application can achieve high availability working with external models and survive the toughest real-world situations.
A Glide cluster can serve multiple applications with totally different needs.
Switch your model providers seamlessly without a need to modify your applications.
We welcome all kinds of feedback and contributions in Glide. Let's make it work in the best possible way for you!
Features
- Unified API: Change your model providers without a need to modify your applications
- High Performance: Written in Golang (opens in a new tab) with the best practices of software development in a cloud-native environments in mind
- Flexible routing across several models and various strategies like the least latency, weighted round-robin (opens in a new tab), round-robin (opens in a new tab), priority-based routing. All strategies include low-latency fallback mechanisms to ensure resiliency
- Support popular LLM providers
- Centralized configuration and API key management
- Production-ready Observability: Always know what's going on with the heart of your system via logs, metrics, and traces. OpenTelemetry (opens in a new tab) is natively supported (coming soon).
- and much more is coming (opens in a new tab)
Installation
Get started with Glide in a few minutes with Docker
Install Glide locally into your developer machine or bare metal setup via CLI (MacOS, Linux, Windows)
Install Glide into a Kubernetes cluster
License
Glide is provided under Apache 2.0 (opens in a new tab) open source license.
Developed with ❤️ by Roma (opens in a new tab), Max (opens in a new tab), and the rest of the EinStack Community (opens in a new tab).