Add framework-level OpenTelemetry tracing#310
Open
cpsievert wants to merge 4 commits into
Open
Conversation
Adds a new chatlas/_otel.py module that emits OpenTelemetry spans for the chat lifecycle: invoke_agent (top-level), chat (per model call), and execute_tool (per tool invocation). Spans follow the GenAI semantic conventions with attributes like gen_ai.usage.input_tokens, gen_ai.response.model, and optional message content capture controlled by the OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT env var. Also adds an `otel` optional dependency extra (`pip install chatlas[otel]`).
Wires the _otel span functions into the six core Chat methods: _chat_impl/_chat_impl_async (agent spans), _submit_turns/ _submit_turns_async (chat spans), and _invoke_tool/ _invoke_tool_async (tool spans). Parent context is passed explicitly via _otel_parent to avoid async context hazards.
Adds 7 tests covering span hierarchy, token usage, content capture (on/off), tool error recording, streaming lifecycle, and no-op behavior. Includes VCR cassettes for replay without live API keys. Updates docs/get-started/monitor.qmd with a new framework-level tracing section (console quickstart, Logfire production path, config-module pattern) before the existing provider-specific content.
The OTel API is ~212KB with no heavy transitive deps, and its default ProxyTracer already no-ops when no SDK is configured. Making it a hard dep lets us drop the lazy initialization (cache_tracer/initialized/is_tracing guards) and always create spans, relying on the no-op machinery for zero overhead when nobody is collecting. Removes the `chatlas[otel]` extra — the API is now always available. Users still opt in to collection by installing opentelemetry-sdk and configuring a TracerProvider.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Chatlas now emits OpenTelemetry spans that capture the full structure of multi-turn conversations and tool execution — without requiring any provider-specific instrumentor libraries. When a TracerProvider is configured, every
chat()/stream()call automatically produces a 3-level span hierarchy:Users opt in with
pip install "chatlas[otel]"and a standard TracerProvider setup (console exporter, Logfire, or any OTLP-compatible backend). The approach is consistent with Shiny for Python's OTel story — same[otel]extra pattern, same recommended tools, same config-module pattern.Spans follow the GenAI semantic conventions and record token usage, response model/ID, and optionally full message content (gated by
OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT). These framework spans complement (not replace) provider-specific SDK instrumentors likeopentelemetry-instrumentation-openai-v2.chatlas/_otel.pymodule with span lifecycle functionsChatmethods (sync + async for agent/chat/tool spans)docs/get-started/monitor.qmdwith framework-level tracing docsTest plan
pytest tests/test_otel.py— 7/7 passingpyright chatlas/_otel.py— 0 errorsruff checkandruff format— clean