Providers

Providers are the AI backends that power ClawDesk's agent responses. ClawDesk supports five providers out of the box through a unified trait-based abstraction, with automatic fallback and capability negotiation.

Supported Providers

Provider	Models	Streaming	Tool Use	Vision	Embeddings
Anthropic	Claude 4 Opus, Sonnet, Haiku; Claude 3.5	✅	✅	✅	❌
OpenAI	GPT-4o, GPT-4 Turbo, GPT-3.5, o1, o3	✅	✅	✅	✅
Ollama	Llama 3, Mistral, Phi, Qwen, any GGUF	✅	✅	✅	✅
Gemini	Gemini 2.0, 1.5 Pro, 1.5 Flash	✅	✅	✅	✅
Bedrock	Claude, Llama, Mistral, Titan (via AWS)	✅	✅	✅	✅

Provider Trait

All providers implement the core Provider trait:

#[async_trait]
pub trait Provider: Send + Sync + 'static {
    /// Unique identifier for this provider
    fn id(&self) -> &str;

    /// List available models
    async fn list_models(&self) -> Result<Vec<ModelInfo>>;

    /// Send a request and get a complete response
    async fn send(&self, request: Request) -> Result<Response>;

    /// Send a request and get a streaming response
    async fn send_stream(
        &self,
        request: Request,
    ) -> Result<Pin<Box<dyn Stream<Item = Result<StreamChunk>> + Send>>>;

    /// Query provider capabilities for a specific model
    fn capabilities(&self, model: &str) -> ModelCapabilities;
}

pub struct ModelCapabilities {
    pub max_tokens: u32,
    pub supports_tools: bool,
    pub supports_vision: bool,
    pub supports_streaming: bool,
    pub supports_system_prompt: bool,
    pub context_window: u32,
}

Configuration

Anthropic

[providers.anthropic]
api_key = "${CLAWDESK_ANTHROPIC_API_KEY}"
default_model = "claude-sonnet-4-20250514"
max_retries = 3
timeout_secs = 60
base_url = "https://api.anthropic.com"  # optional, for proxies

# Model-specific overrides
[providers.anthropic.models.claude-sonnet-4-20250514]
max_tokens = 8192
temperature = 0.7

[providers.anthropic.models.claude-3-5-haiku-20241022]
max_tokens = 4096
temperature = 0.5

tip

Use environment variable interpolation for API keys. Never store keys directly in config files that might be committed to version control:

api_key = "${CLAWDESK_ANTHROPIC_API_KEY}"

OpenAI

[providers.openai]
api_key = "${CLAWDESK_OPENAI_API_KEY}"
default_model = "gpt-4o"
organization = "org-..."  # optional
max_retries = 3
timeout_secs = 60
base_url = "https://api.openai.com/v1"  # optional, for Azure or proxies

[providers.openai.models.gpt-4o]
max_tokens = 4096
temperature = 0.7

Azure OpenAI

[providers.openai]
api_key = "${AZURE_OPENAI_API_KEY}"
base_url = "https://my-resource.openai.azure.com/openai/deployments/gpt-4o"
api_version = "2024-02-15-preview"
azure = true

Ollama

[providers.ollama]
base_url = "http://localhost:11434"  # default Ollama address
default_model = "llama3.1:70b"
timeout_secs = 120  # local models may be slower

# GPU configuration
[providers.ollama.options]
num_gpu = 1
num_ctx = 8192

info

Ollama must be running separately. Install it from ollama.com and pull models before use:

ollama pull llama3.1:70b
ollama pull nomic-embed-text  # for embeddings

Gemini

[providers.gemini]
api_key = "${CLAWDESK_GEMINI_API_KEY}"
default_model = "gemini-2.0-flash"
max_retries = 3
timeout_secs = 60

[providers.gemini.safety_settings]
harassment = "BLOCK_MEDIUM_AND_ABOVE"
hate_speech = "BLOCK_MEDIUM_AND_ABOVE"
sexually_explicit = "BLOCK_ONLY_HIGH"
dangerous_content = "BLOCK_MEDIUM_AND_ABOVE"

Bedrock

[providers.bedrock]
region = "${AWS_REGION}"  # or "us-east-1"
default_model = "anthropic.claude-sonnet-4-20250514-v2:0"
# Uses AWS credential chain: env vars → config file → IAM role
# Optionally specify a profile:
profile = "clawdesk"

[providers.bedrock.models."anthropic.claude-sonnet-4-20250514-v2:0"]
max_tokens = 4096

warning

Bedrock requires appropriate IAM permissions. Ensure your role/user has bedrock:InvokeModel and bedrock:InvokeModelWithResponseStream permissions.

Provider Registry

The ProviderRegistry manages all configured providers and handles model resolution:

pub struct ProviderRegistry {
    providers: HashMap<String, Arc<dyn Provider>>,
    default_provider: String,
}

impl ProviderRegistry {
    /// Get a provider by name
    pub fn get(&self, name: &str) -> Option<&Arc<dyn Provider>>;

    /// Get the default provider
    pub fn default(&self) -> &Arc<dyn Provider>;

    /// List all registered providers
    pub fn list(&self) -> Vec<&str>;

    /// Find a provider that supports a specific model
    pub fn find_for_model(&self, model: &str) -> Option<&Arc<dyn Provider>>;

    /// Register a custom provider
    pub fn register(&mut self, name: &str, provider: Arc<dyn Provider>);
}

Capability Negotiation

When the agent pipeline sends a request, the Negotiator ensures the request is compatible with the target model's capabilities:

pub struct Negotiator;

impl Negotiator {
    pub fn negotiate(
        request: &mut Request,
        capabilities: &ModelCapabilities,
    ) -> NegotiationResult {
        let mut warnings = Vec::new();

        if !capabilities.supports_tools && request.has_tools() {
            request.strip_tools();
            warnings.push("Tools stripped: model does not support tool use");
        }

        if !capabilities.supports_vision && request.has_images() {
            request.convert_images_to_text();
            warnings.push("Images converted: model does not support vision");
        }

        if request.token_count() > capabilities.context_window {
            request.truncate_history(capabilities.context_window);
            warnings.push("History truncated to fit context window");
        }

        NegotiationResult { warnings }
    }
}

Fallback System

ClawDesk implements a Finite State Machine (FSM) for provider fallback when errors occur:

Fallback States

State	Description
Idle	No fallback in progress
Selecting	Choosing the next provider from the fallback order
Attempting	Sending the request to the selected provider
Succeeded	Provider responded successfully
Retrying	Retrying the same provider after a transient error
Exhausted	Max retries reached for the current provider
Aborted	All providers failed, returning an error to the user

Error Classification

The fallback system classifies errors to decide whether to retry or fail over:

Error Type	Action	Examples
Transient	Retry same provider	Rate limit (429), server error (500-503), timeout
Permanent	Fail over to next	Auth error (401), invalid model (404), content filter
Fatal	Abort immediately	Network unreachable, invalid request format

Configuration

[agent.failover]
enabled = true

# Provider priority order
fallback_order = ["anthropic", "openai", "gemini", "ollama"]

# Per-provider retry settings
max_retries_per_provider = 3
retry_delay_ms = 1000
backoff_factor = 2.0
max_retry_delay_ms = 30000

# Model mapping for fallbacks
[agent.failover.model_map]
"claude-sonnet-4-20250514" = { openai = "gpt-4o", gemini = "gemini-2.0-flash", ollama = "llama3.1:70b" }
"claude-3-5-haiku-20241022" = { openai = "gpt-4o-mini", gemini = "gemini-2.0-flash", ollama = "llama3.1:8b" }

info

The model_map ensures that when falling back to a different provider, ClawDesk selects an equivalent model. Without a mapping, the fallback provider's default model is used.

Model Selection

Via CLI

# Specify provider and model
clawdesk message -P anthropic -m claude-sonnet-4-20250514 "Hello"

# Use default provider with a specific model
clawdesk message -m gpt-4o "Hello"

# List available models
clawdesk config get providers

Via API

# List all available models
curl http://localhost:1420/api/v1/models

# OpenAI-compatible model listing
curl http://localhost:1420/v1/models

Per-Channel Model Selection

[agent.channel_overrides.tg_support]
provider = "anthropic"
model = "claude-sonnet-4-20250514"

[agent.channel_overrides.dc_community]
provider = "ollama"
model = "llama3.1:70b"

Custom Provider

Implement the Provider trait to add a custom provider:

use clawdesk_agents::providers::{Provider, Request, Response, ModelInfo};

pub struct MyProvider {
    api_key: String,
    client: reqwest::Client,
}

#[async_trait]
impl Provider for MyProvider {
    fn id(&self) -> &str {
        "my_provider"
    }

    async fn list_models(&self) -> Result<Vec<ModelInfo>> {
        Ok(vec![
            ModelInfo {
                id: "my-model-v1".into(),
                name: "My Model v1".into(),
                context_window: 32_768,
                max_output_tokens: 4_096,
            },
        ])
    }

    async fn send(&self, request: Request) -> Result<Response> {
        // Transform request to your API format
        let api_request = transform_request(&request);

        // Call your API
        let api_response = self.client
            .post("https://api.my-provider.com/v1/chat")
            .bearer_auth(&self.api_key)
            .json(&api_request)
            .send()
            .await?;

        // Transform response back
        Ok(transform_response(api_response.json().await?))
    }

    async fn send_stream(
        &self,
        request: Request,
    ) -> Result<Pin<Box<dyn Stream<Item = Result<StreamChunk>> + Send>>> {
        // Implement streaming...
        todo!()
    }

    fn capabilities(&self, _model: &str) -> ModelCapabilities {
        ModelCapabilities {
            max_tokens: 4096,
            supports_tools: true,
            supports_vision: false,
            supports_streaming: true,
            supports_system_prompt: true,
            context_window: 32_768,
        }
    }
}

let mut registry = ProviderRegistry::new();
registry.register("my_provider", Arc::new(MyProvider::new(api_key)));

Monitoring

Provider Metrics

Metric	Type	Description
`clawdesk_provider_requests_total`	Counter	Total requests by provider and model
`clawdesk_provider_errors_total`	Counter	Errors by provider, model, and error type
`clawdesk_provider_latency_seconds`	Histogram	Request latency by provider
`clawdesk_provider_tokens_total`	Counter	Tokens consumed (input + output)
`clawdesk_provider_fallbacks_total`	Counter	Fallback events
`clawdesk_provider_cost_dollars`	Counter	Estimated cost in USD

Cost Tracking

[providers.cost_tracking]
enabled = true
warn_threshold_daily = 50.00   # USD
hard_limit_daily = 100.00      # USD — stops requests
alert_channel = "tg_admin"     # send alerts to this channel

# View cost summary
clawdesk config get providers.cost_tracking

# Example output:
# Today: $12.34 (anthropic: $8.21, openai: $4.13)
# This month: $234.56
# Budget remaining: $765.44

Troubleshooting

Problem	Solution
"Provider not found"	Check provider name in config, run `clawdesk doctor`
"Model not available"	Verify model ID, check `clawdesk config get providers.<name>`
"Rate limited"	Reduce request frequency or upgrade API tier
"Context window exceeded"	Reduce `history.max_tokens` or use a model with larger context
Fallback not working	Verify `failover.enabled = true` and `fallback_order` is set
Ollama timeout	Increase `timeout_secs`, check GPU availability
Bedrock auth failure	Verify AWS credentials and IAM permissions

Supported Providers​

Provider Trait​

Configuration​

Anthropic​

OpenAI​

Azure OpenAI​

Ollama​

Gemini​

Bedrock​

Provider Registry​

Capability Negotiation​

Fallback System​

Fallback States​

Error Classification​

Configuration​

Model Selection​

Via CLI​

Via API​

Per-Channel Model Selection​

Custom Provider​

Monitoring​

Provider Metrics​

Cost Tracking​

Troubleshooting​

Supported Providers

Provider Trait

Configuration

Anthropic

OpenAI

Azure OpenAI

Ollama

Gemini

Bedrock

Provider Registry

Capability Negotiation

Fallback System

Fallback States

Error Classification

Configuration

Model Selection

Via CLI

Via API

Per-Channel Model Selection

Custom Provider

Monitoring

Provider Metrics

Cost Tracking

Troubleshooting