Glide
Docs
Configuration

Configuration

Below details how to configure Glide.

In order to start, start Glide gateway, we need to pass a config file:

glide --config /path/to/config.yaml

Configuration Structure

Glide's configuration is a declarative YAML file (opens in a new tab) with the following high level sections:

  • routers - defines Glide's routers per model modality (e.g. routers.language, routers.speech, etc). For more information on router types see the router page.
  • api - defines Glide's server configuration for specific protocol. Read more about this below
  • telemetry - defines configuration for Glide's observability. Read more about this below.

Basic Configuration

At its simplest the gateway can be configured with a single router and two models.

This example shows a failover router leveraging OpenAI and Azure OpenAI. For OpenAI, Cohere, and OctoML, only an API key needs to be provided. All other parameters, including the model, are provided by default. For Azure OpenAI, a model and base_url must be specified.

  • language - This is the language API endpoint which support a /chat endpoints from model providers
  • id - A unique id for your router
  • strategy - The type of router can be round-robin, weighted-round-robin, least-latency, priority
  • models/id - A unique ID for the model configuration
routers:
  language:
    - id: my-chat-router
      strategy: priority
      models:
        - id: primary
          openai:
            api_key: "${env:OPENAI_API_KEY}"
        - id: secondary
          azureopenai:
            api_key: "${env:AZURE_OPENAI_API_KEY}"
            model: "glide-GPT-35" # the Azure OpenAI deployment name
            base_url: "https://mydeployment.openai.azure.com/" # the name of your Azure OpenAI Resource

See an example of a more advanced configuration

Secrets

Glide supports two ways of passing sensitive information like API keys:

  • via environment variables:
routers:
  language:
    - id: default
      strategy: priority
      models:
        - id: primary
          openai:
            api_key: "${env:OPENAI_API_KEY}"
  • via separate files (useful in cloud setups):
routers:
  language:
    - id: default
      strategy: priority
      models:
        - id: primary
          openai:
            api_key: "${file:/paht/to/secret}" # the file should be a plain text file with the content of the secret

Dotenv files

To make it easier for you to manage secrets locally, Glide automatically attempts to load environment variables from .env files located in the directory where it was called.

You can use a separate --env flag to provide a specific path:

glide --env .env.dev --config config.yaml

API Configuration

HTTP is the only supported protocol for Glide's API right now. The following configurations are exposed for you to tweak:

  • api.http.host (default: localhost) - defines address to which Glide binds its server. Usually, it's localhost or 0.0.0.0
  • api.http.read_timeout (default: 3m) - the timeout of data reading from clients
  • api.http.write_timeout (default: 3m) - the timeout of data writing to clients
  • api.http.idle_timeout (default: 1m) - the free timeout of the request link for persistent keep-alive connections
  • api.http.max_request_body_size (default: 4Mb) - max body size of a request

A sample of the configuration:

api:
  http:
    host: 0.0.0.0 # default: localhost
    port: 9099 # default 9099
    read_timeout: 3s
    write_timeout: 3s
    idle_timeout: 1s
    max_request_body_size: 2Mi
 
# everything else
# ...

Telemetry

Learn more about observability in this section.

ConceptsChat