LiteLLM: one interface for many language models

LiteLLM is an open-source toolkit that lets you call 100+ language-model providers through one OpenAI-compatible interface. Use it as a Python library in your app, or as a proxy so any OpenAI client can reach many backends.

Overview
The whole picture
LiteLLM — one interface for many models imports sends request uses uses selects backend forwards records usage Your App chatbot, agent, or script Python SDK library in your process Proxy Server shared AI gateway Unified API OpenAI format in and out Router retry, fallback, balance Spend and Logs cost and traces Model Providers OpenAI, Anthropic, Bedrock, and more
Actors, inputs, and outputs
Processes and services
Stores
Infrastructure
100%

How the pieces fit

  • Your app sends one OpenAI-style request.
  • The SDK or the proxy hands that request to LiteLLM's unified API.
  • The Router forwards the call to a model provider and LiteLLM can record the spend.

What the neighbors do

  • The Python SDK is a library one app imports. The Proxy Server is a gateway many clients call.
  • The Router chooses among deployments and can retry or fall back.
  • Model providers are the vendors that actually run the models.

Why this exists

  • Stop writing a new client for every vendor API.
  • Change models without starting a new application project.
  • Keep working if one provider fails, and see what each call cost.