clawdesk-observability

OpenTelemetry integration providing distributed tracing, metrics, and logging for the ClawDesk gateway. Instruments the agent pipeline, provider calls, and channel operations.

Dependencies

Internal: clawdesk-types

External: opentelemetry, opentelemetry-otlp, tracing, tracing-opentelemetry, tracing-subscriber, tokio

Modules

Module	Description
tracing	Distributed tracing setup and span management
metrics	Metrics registry (counters, histograms, gauges)
logging	Structured logging integration with tracing
middleware	Axum middleware for request tracing

Key Types

/// Initialize the observability stack
pub fn init(config: &ObservabilityConfig) -> Result<(), ObsError> {
    // Set up tracing subscriber with OpenTelemetry layer
    // Register metrics exporters
    // Configure structured logging
}

/// Configuration for observability
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ObservabilityConfig {
    pub tracing_enabled: bool,
    pub metrics_enabled: bool,
    pub otlp_endpoint: Option<String>,
    pub service_name: String,
    pub log_format: LogFormat,
    pub sample_rate: f64,
}

#[derive(Debug, Clone, Serialize, Deserialize)]
pub enum LogFormat {
    Json,
    Pretty,
    Compact,
}

/// Metrics registry
pub struct MetricsRegistry {
    pub request_counter: Counter<u64>,
    pub request_duration: Histogram<f64>,
    pub active_sessions: UpDownCounter<i64>,
    pub provider_latency: Histogram<f64>,
    pub token_usage: Counter<u64>,
}

/// Tracing middleware for Axum
pub fn tracing_layer() -> TraceLayer<...> {
    TraceLayer::new_for_http()
        .make_span_with(|req: &Request| {
            tracing::info_span!("http_request",
                method = %req.method(),
                path = %req.uri().path(),
                request_id = %uuid::Uuid::new_v4(),
            )
        })
}

Traced Operations

Operation	Span Name	Key Attributes
HTTP request	http_request	method, path, status
Agent pipeline	agent.pipeline	session_id, model
Provider call	provider.chat	provider, model, tokens
Channel send	channel.send	channel_id
Storage read	storage.get	store_type, key
Tool execution	agent.tool	tool_name, duration

Example Usage

use clawdesk_observability::{init, ObservabilityConfig, LogFormat};

let config = ObservabilityConfig {
    tracing_enabled: true,
    metrics_enabled: true,
    otlp_endpoint: Some("http://localhost:4317".into()),
    service_name: "clawdesk-gateway".into(),
    log_format: LogFormat::Json,
    sample_rate: 1.0,
};

init(&config)?;

// Spans are automatically created by instrumented code
#[tracing::instrument(skip(provider))]
async fn chat_with_provider(provider: &dyn Provider, messages: &[Message]) {
    tracing::info!("Sending chat request");
    // ...
}

tip

Use sample_rate < 1.0 in production to reduce tracing overhead. A rate of 0.1 captures 10% of traces.