llm-connectorRust LLM Connector

Provider abstraction for OpenAI/Anthropic/Gemini and more — streaming, tools, multimodal, and proxy-ready.

Get Started

View on GitHub

Crates.io

Responses API Ready

Native `/responses` + streaming support with automatic fallback to `/chat/completions`.

Unified Streaming

One streaming model across providers with a consistent response type.

Tools / Function Calling

OpenAI-compatible tools with streaming support across all major providers.

Multi-modal

Text + images in a single message block, automatically adapted per provider.

Provider Matrix

12+ providers with clean Protocol/Provider separation and per-request overrides.

Proxy / Middleware Ready

All protocol request types support both Serialize and Deserialize for intercepting wire-format traffic.

Mock Client

Built-in mock provider for testing — no real API calls needed.

Install

bash

cargo add llm-connector

Or in Cargo.toml:

toml

[dependencies]
llm-connector = "1.1.12"
tokio = { version = "1", features = ["full"] }

Quick Example

rust

use llm_connector::{LlmClient, types::{ChatRequest, Message}};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = LlmClient::openai("sk-...", "https://api.openai.com/v1")?;

    let request = ChatRequest::new("gpt-4o")
        .add_message(Message::user("Hello!"));

    let response = client.chat(&request).await?;
    println!("{}", response.content);

    Ok(())
}