mirror of
https://github.com/PrefectHQ/fastmcp.git
synced 2026-08-22 21:44:18 +02:00
Add telemetry off-switch and mcp.protocol.version span attribute (#4481)
* Turn OpenTelemetry instrumentation on by default with explicit off-switch Add FASTMCP_ENABLE_TELEMETRY setting (default true) and mcp.protocol.version span attribute for SDK parity. * Make disabled telemetry a transparent pass-through, not a NoOpTracer The stock NoOpTracer.start_as_current_span attaches a NonRecordingSpan, hijacking the current OTel context from any enclosing application span. When telemetry is disabled, get_tracer() now returns a non-attaching pass-through tracer so trace.get_current_span() inside handlers still resolves to the caller's span.
This commit is contained in:
Jeremiah Lowin
GitHub
parent
266c129b62
commit
a04f6fd911
7 changed files with 301 additions and 4 deletions
|
|
@ -152,6 +152,12 @@ SDK v2 seeds an `OpenTelemetryMiddleware` into every lowlevel `Server`, so each
|
|||
|
||||
*Verify:* `fastmcp_slim/fastmcp/server/low_level.py` (the `OpenTelemetryMiddleware` filter); `tests/server/telemetry/test_server_tracing.py::TestSingleServerSpan`.
|
||||
|
||||
### Telemetry on by default, with an explicit off-switch — Absorbed
|
||||
|
||||
FastMCP's OpenTelemetry instrumentation is on by default. Because FastMCP uses only the OpenTelemetry API, span creation is a no-op with negligible overhead (the API's `NonRecordingSpan`) unless the user configures an SDK and exporter — so being always-on costs nothing until you opt into collection. The new `FASTMCP_ENABLE_TELEMETRY` setting (`fastmcp.settings.enable_telemetry`, default `true`) is the explicit off-switch: set it to `false` and `get_tracer()` returns a genuine no-op tracer, so no FastMCP spans are created even when an SDK is configured. The off-switch governs FastMCP's own spans (all SERVER spans, plus FastMCP's high-level CLIENT span); the SDK's low-level `mcp-python-sdk` `MCP send <method>` CLIENT spans are governed by the user's OpenTelemetry SDK, not this setting. FastMCP's SERVER span now also carries `mcp.protocol.version` — the attribute the dropped SDK `OpenTelemetryMiddleware` set — restoring parity with the SDK's semantic conventions.
|
||||
|
||||
*Verify:* `fastmcp_slim/fastmcp/settings.py` (`enable_telemetry`); `fastmcp_slim/fastmcp/telemetry.py` (`get_tracer` off-switch); `fastmcp_slim/fastmcp/server/telemetry.py` (`get_protocol_span_attributes`); `tests/server/telemetry/test_server_tracing.py::TestTelemetryEnabledByDefault`, `::TestProtocolVersionAttribute`.
|
||||
|
||||
### Spec-correct error codes via a central translator — Breaking (wire error code)
|
||||
|
||||
Resource-not-found responses from the core `resources/read` handler previously used `-32002`. SEP-2164 (and the SDK's own mcpserver, which maps `ResourceNotFoundError` → `INVALID_PARAMS`) makes this `-32602`. The per-adapter `MCPError(code=..., ...)` literals in `server/mixins/mcp_operations.py` are replaced by a single `fastmcp.exceptions.to_mcp_error()` translator that maps FastMCP's public exceptions to the `mcp_types` code constants (`NotFoundError`/`DisabledError`/`ValidationError` → `INVALID_PARAMS`, else `INTERNAL_ERROR`). Clients that string-matched on the old `-32002` for resource-not-found must switch to `-32602`; the human-readable message ("Resource not found: ...") is unchanged. The opt-in `ErrorHandlingMiddleware`, which has its own documented per-method-prefix code mapping, is intentionally left as-is.
|
||||
|
|
|
|||
|
|
@ -72,6 +72,12 @@ These control how the server listens when running with an HTTP transport.
|
|||
| `FASTMCP_SHOW_SERVER_BANNER` | `bool` | `true` | Show the server banner on startup. Also controllable via `--no-banner` or `server.run(show_banner=False)`. |
|
||||
| `FASTMCP_CHECK_FOR_UPDATES` | `Literal["stable", "prerelease", "off"]` | `stable` | Update checking on CLI startup. `stable` checks stable releases only, `prerelease` includes pre-releases, `off` disables checking. |
|
||||
|
||||
## Telemetry
|
||||
|
||||
| Environment Variable | Type | Default | Description |
|
||||
|---|---|---|---|
|
||||
| `FASTMCP_ENABLE_TELEMETRY` | `bool` | `true` | Whether FastMCP's native [OpenTelemetry instrumentation](/servers/telemetry) is active. Enabled by default; FastMCP uses only the OpenTelemetry API, so span creation is a no-op with negligible overhead unless an OpenTelemetry SDK and exporter are configured. Set to `false` to turn instrumentation off entirely, in which case no FastMCP spans are created even when an SDK is configured. |
|
||||
|
||||
## Tasks (Docket)
|
||||
|
||||
These configure the [Docket](https://github.com/prefecthq/docket) task queue used by [server tasks](/servers/tasks). All use the `FASTMCP_DOCKET_` prefix.
|
||||
|
|
|
|||
|
|
@ -12,11 +12,17 @@ FastMCP includes native OpenTelemetry instrumentation for observability. Traces
|
|||
|
||||
FastMCP uses the OpenTelemetry API for instrumentation. This means:
|
||||
|
||||
- **Zero configuration required** - Instrumentation is always active
|
||||
- **On by default** - Instrumentation is active out of the box, no opt-in required
|
||||
- **No overhead when unused** - Without an SDK, all operations are no-ops
|
||||
- **Bring your own SDK** - You control collection, export, and sampling
|
||||
- **Works with any OTEL backend** - Jaeger, Zipkin, Datadog, New Relic, etc.
|
||||
|
||||
Because FastMCP only depends on the OpenTelemetry API, span creation is a no-op until you configure an SDK and exporter — so being on by default costs nothing until you opt into collection.
|
||||
|
||||
### Turning Telemetry Off
|
||||
|
||||
To disable FastMCP's instrumentation entirely, set `FASTMCP_ENABLE_TELEMETRY=false` (or `fastmcp.settings.enable_telemetry = False`). When disabled, FastMCP creates no spans even if an SDK is configured.
|
||||
|
||||
## Enabling Telemetry
|
||||
|
||||
The easiest way to export traces is using `opentelemetry-instrument`, which configures the SDK automatically:
|
||||
|
|
@ -271,6 +277,7 @@ FastMCP implements the [OpenTelemetry MCP semantic conventions](https://opentele
|
|||
| Attribute | Description |
|
||||
|-----------|-------------|
|
||||
| `mcp.method.name` | The MCP method being called (`tools/call`, `resources/read`, `prompts/get`) |
|
||||
| `mcp.protocol.version` | The negotiated MCP protocol version for the request |
|
||||
| `mcp.session.id` | Session identifier for the MCP connection |
|
||||
| `mcp.resource.uri` | The resource URI (for resource operations) |
|
||||
| `gen_ai.tool.name` | Tool name (on `tools/call` spans) |
|
||||
|
|
|
|||
|
|
@ -58,6 +58,21 @@ def get_session_span_attributes() -> dict[str, str]:
|
|||
return attrs
|
||||
|
||||
|
||||
def get_protocol_span_attributes() -> dict[str, str]:
|
||||
"""Get the negotiated MCP protocol version for the current request.
|
||||
|
||||
Mirrors the `mcp.protocol.version` attribute the SDK's own
|
||||
`OpenTelemetryMiddleware` sets — FastMCP drops that middleware to avoid a
|
||||
duplicate SERVER span, so this restores the attribute on FastMCP's span.
|
||||
"""
|
||||
from fastmcp.server.dependencies import fastmcp_request_ctx
|
||||
|
||||
req_ctx = fastmcp_request_ctx.get()
|
||||
if req_ctx is not None and req_ctx.protocol_version:
|
||||
return {"mcp.protocol.version": req_ctx.protocol_version}
|
||||
return {}
|
||||
|
||||
|
||||
def _get_parent_trace_context() -> Context | None:
|
||||
"""Get parent trace context from request meta for distributed tracing."""
|
||||
from fastmcp.server.dependencies import fastmcp_request_ctx
|
||||
|
|
@ -84,6 +99,7 @@ def _build_server_span_attrs(
|
|||
"fastmcp.server.name": server_name,
|
||||
"fastmcp.component.type": component_type,
|
||||
"fastmcp.component.key": component_key,
|
||||
**get_protocol_span_attributes(),
|
||||
**get_auth_span_attributes(),
|
||||
**get_session_span_attributes(),
|
||||
}
|
||||
|
|
@ -131,6 +147,7 @@ def seam_span(method: str, server_name: str) -> Generator[Span, None, None]:
|
|||
{
|
||||
"mcp.method.name": method,
|
||||
"fastmcp.server.name": server_name,
|
||||
**get_protocol_span_attributes(),
|
||||
**get_auth_span_attributes(),
|
||||
**get_session_span_attributes(),
|
||||
}
|
||||
|
|
@ -244,6 +261,7 @@ __all__ = [
|
|||
"SEAM_SPAN_MARKER",
|
||||
"delegate_span",
|
||||
"get_auth_span_attributes",
|
||||
"get_protocol_span_attributes",
|
||||
"get_session_span_attributes",
|
||||
"record_span_exception",
|
||||
"seam_span",
|
||||
|
|
|
|||
|
|
@ -210,6 +210,24 @@ class Settings(BaseSettings):
|
|||
),
|
||||
] = True
|
||||
|
||||
enable_telemetry: Annotated[
|
||||
bool,
|
||||
Field(
|
||||
description=inspect.cleandoc(
|
||||
"""
|
||||
Whether FastMCP's native OpenTelemetry instrumentation is active.
|
||||
Enabled by default: FastMCP uses only the OpenTelemetry API, so
|
||||
span creation is a no-op with negligible overhead unless an
|
||||
OpenTelemetry SDK and exporter are configured. Set to False to
|
||||
turn instrumentation off entirely, in which case FastMCP's span
|
||||
helpers become a transparent pass-through: no FastMCP spans are
|
||||
created even when an SDK is configured, and the surrounding OTel
|
||||
trace context is left untouched.
|
||||
"""
|
||||
)
|
||||
),
|
||||
] = True
|
||||
|
||||
deprecation_warnings: Annotated[
|
||||
bool,
|
||||
Field(
|
||||
|
|
|
|||
|
|
@ -21,13 +21,24 @@ Example usage with SDK:
|
|||
```
|
||||
"""
|
||||
|
||||
from collections.abc import Iterator
|
||||
from contextlib import contextmanager
|
||||
from typing import Any
|
||||
|
||||
from opentelemetry import context as otel_context
|
||||
from opentelemetry import propagate, trace
|
||||
from opentelemetry.context import Context
|
||||
from opentelemetry.trace import Span, Status, StatusCode, Tracer
|
||||
from opentelemetry.trace import (
|
||||
INVALID_SPAN,
|
||||
NoOpTracer,
|
||||
Span,
|
||||
SpanKind,
|
||||
Status,
|
||||
StatusCode,
|
||||
Tracer,
|
||||
)
|
||||
from opentelemetry.trace import get_tracer as otel_get_tracer
|
||||
from opentelemetry.util import types as otel_types
|
||||
|
||||
INSTRUMENTATION_NAME = "fastmcp"
|
||||
|
||||
|
|
@ -35,15 +46,60 @@ TRACE_PARENT_KEY = "traceparent"
|
|||
TRACE_STATE_KEY = "tracestate"
|
||||
|
||||
|
||||
class _DisabledTracer(NoOpTracer):
|
||||
"""A tracer that neither records spans nor touches the OTel context.
|
||||
|
||||
When telemetry is disabled FastMCP must be fully transparent. The stock
|
||||
`NoOpTracer.start_as_current_span` still *attaches* a `NonRecordingSpan` to
|
||||
the current OTel context, so an enclosing application span (from ASGI/HTTP
|
||||
instrumentation or a user-created span) is hidden while a FastMCP span
|
||||
helper is active — `trace.get_current_span()` inside a handler would then
|
||||
return that non-recording span instead of the caller's span. This tracer
|
||||
yields the invalid span *without* entering it as current, leaving the
|
||||
surrounding trace context untouched.
|
||||
"""
|
||||
|
||||
@contextmanager
|
||||
def start_as_current_span(
|
||||
self,
|
||||
name: str,
|
||||
context: Context | None = None,
|
||||
kind: SpanKind = SpanKind.INTERNAL,
|
||||
attributes: otel_types.Attributes = None,
|
||||
links: Any = None,
|
||||
start_time: int | None = None,
|
||||
record_exception: bool = True,
|
||||
set_status_on_exception: bool = True,
|
||||
end_on_exit: bool = True,
|
||||
) -> Iterator[Span]:
|
||||
yield INVALID_SPAN
|
||||
|
||||
|
||||
_DISABLED_TRACER = _DisabledTracer()
|
||||
|
||||
|
||||
def get_tracer(version: str | None = None) -> Tracer:
|
||||
"""Get the FastMCP tracer for creating spans.
|
||||
|
||||
Instrumentation is on by default. FastMCP uses only the OpenTelemetry API,
|
||||
so span creation is a no-op with negligible overhead unless an OpenTelemetry
|
||||
SDK and exporter are configured. Set `fastmcp.settings.enable_telemetry` to
|
||||
False (env `FASTMCP_ENABLE_TELEMETRY=false`) to turn instrumentation off
|
||||
entirely, in which case this returns a pass-through tracer that leaves the
|
||||
current OTel context untouched even when an SDK is configured.
|
||||
|
||||
Args:
|
||||
version: Optional version string for the instrumentation
|
||||
|
||||
Returns:
|
||||
A tracer instance. Returns a no-op tracer if no SDK is configured.
|
||||
A tracer instance. Returns a non-attaching pass-through tracer if
|
||||
telemetry is disabled; span creation is otherwise a no-op unless an SDK
|
||||
is configured.
|
||||
"""
|
||||
import fastmcp
|
||||
|
||||
if not fastmcp.settings.enable_telemetry:
|
||||
return _DISABLED_TRACER
|
||||
return otel_get_tracer(INSTRUMENTATION_NAME, version)
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -5,9 +5,11 @@ from __future__ import annotations
|
|||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
from opentelemetry import trace
|
||||
from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter
|
||||
from opentelemetry.trace import SpanKind, StatusCode
|
||||
from opentelemetry.trace import Span, SpanKind, StatusCode
|
||||
|
||||
import fastmcp
|
||||
from fastmcp import Client, FastMCP
|
||||
from fastmcp.exceptions import NotFoundError, ToolError
|
||||
from fastmcp.server.auth import AccessToken
|
||||
|
|
@ -575,3 +577,187 @@ class TestFailurePathServerSpan:
|
|||
tool_call_server_spans[0].attributes is not None
|
||||
and tool_call_server_spans[0].attributes["mcp.method.name"] == "tools/call"
|
||||
)
|
||||
|
||||
|
||||
class TestTelemetryEnabledByDefault:
|
||||
"""Instrumentation is on by default and controllable via the off-switch.
|
||||
|
||||
FastMCP uses only the OpenTelemetry API, so spans are created unconditionally
|
||||
and light up when an SDK is configured. `FASTMCP_ENABLE_TELEMETRY=false`
|
||||
(`fastmcp.settings.enable_telemetry`) turns span creation off entirely, so no
|
||||
FastMCP spans are exported even with an SDK configured.
|
||||
"""
|
||||
|
||||
async def test_spans_fire_by_default(self, trace_exporter: InMemorySpanExporter):
|
||||
"""No opt-in required: a tool call produces a span out of the box."""
|
||||
assert fastmcp.settings.enable_telemetry is True
|
||||
|
||||
mcp = FastMCP("test-server")
|
||||
|
||||
@mcp.tool()
|
||||
def greet(name: str) -> str:
|
||||
return f"Hello, {name}!"
|
||||
|
||||
await mcp.call_tool("greet", {"name": "World"})
|
||||
|
||||
spans = trace_exporter.get_finished_spans()
|
||||
assert len(spans) == 1
|
||||
assert spans[0].name == "tools/call greet"
|
||||
|
||||
async def test_off_switch_suppresses_spans(
|
||||
self,
|
||||
trace_exporter: InMemorySpanExporter,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
):
|
||||
"""With telemetry disabled, no spans are created even with an SDK
|
||||
configured (the exporter fixture installs one)."""
|
||||
monkeypatch.setattr(fastmcp.settings, "enable_telemetry", False)
|
||||
|
||||
mcp = FastMCP("test-server")
|
||||
|
||||
@mcp.tool()
|
||||
def greet(name: str) -> str:
|
||||
return f"Hello, {name}!"
|
||||
|
||||
result = await mcp.call_tool("greet", {"name": "World"})
|
||||
assert "Hello, World!" in str(result)
|
||||
|
||||
spans = trace_exporter.get_finished_spans()
|
||||
assert len(spans) == 0
|
||||
|
||||
async def test_off_switch_suppresses_spans_via_client(
|
||||
self,
|
||||
trace_exporter: InMemorySpanExporter,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
):
|
||||
"""The off-switch suppresses every FastMCP span on the full request path
|
||||
(seam SERVER span and FastMCP CLIENT span alike).
|
||||
|
||||
The SDK's own low-level `mcp-python-sdk` CLIENT spans ("MCP send ...")
|
||||
are governed by the user's OpenTelemetry SDK, not FastMCP's off-switch,
|
||||
so they may still appear — the assertion filters them out.
|
||||
"""
|
||||
monkeypatch.setattr(fastmcp.settings, "enable_telemetry", False)
|
||||
|
||||
mcp = FastMCP("test-server")
|
||||
|
||||
@mcp.tool()
|
||||
def greet(name: str) -> str:
|
||||
return f"Hello, {name}!"
|
||||
|
||||
async with Client(mcp) as client:
|
||||
await client.call_tool("greet", {"name": "World"})
|
||||
|
||||
spans = trace_exporter.get_finished_spans()
|
||||
fastmcp_spans = [
|
||||
s
|
||||
for s in spans
|
||||
if s.instrumentation_scope is not None
|
||||
and s.instrumentation_scope.name == "fastmcp"
|
||||
]
|
||||
assert fastmcp_spans == []
|
||||
# In particular, no FastMCP SERVER span (FastMCP owns all SERVER spans).
|
||||
assert [s for s in spans if s.kind == SpanKind.SERVER] == []
|
||||
|
||||
async def test_off_switch_leaves_enclosing_span_current(
|
||||
self,
|
||||
trace_exporter: InMemorySpanExporter,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
):
|
||||
"""Disabling telemetry must be a transparent pass-through.
|
||||
|
||||
The stock OpenTelemetry `NoOpTracer.start_as_current_span` attaches a
|
||||
`NonRecordingSpan` as the current span, which would hijack the trace
|
||||
context from an enclosing application span. With FastMCP's off-switch,
|
||||
`trace.get_current_span()` inside a handler must still return the
|
||||
caller's enclosing span, and attributes written there must land on it.
|
||||
"""
|
||||
monkeypatch.setattr(fastmcp.settings, "enable_telemetry", False)
|
||||
|
||||
tracer = trace.get_tracer("test-enclosing")
|
||||
captured: dict[str, Span] = {}
|
||||
|
||||
mcp = FastMCP("test-server")
|
||||
|
||||
@mcp.tool()
|
||||
def annotate() -> str:
|
||||
current = trace.get_current_span()
|
||||
captured["current"] = current
|
||||
current.set_attribute("tool.touched", True)
|
||||
return "ok"
|
||||
|
||||
with tracer.start_as_current_span("enclosing-app-span") as enclosing:
|
||||
await mcp.call_tool("annotate", {})
|
||||
# The tool ran with the enclosing span still current — FastMCP did
|
||||
# not attach a replacement (non-recording) span.
|
||||
assert captured["current"] is enclosing
|
||||
assert enclosing.is_recording()
|
||||
|
||||
spans = trace_exporter.get_finished_spans()
|
||||
app_spans = [s for s in spans if s.name == "enclosing-app-span"]
|
||||
assert len(app_spans) == 1
|
||||
assert app_spans[0].attributes is not None
|
||||
assert app_spans[0].attributes["tool.touched"] is True
|
||||
# No FastMCP spans were exported.
|
||||
fastmcp_spans = [
|
||||
s
|
||||
for s in spans
|
||||
if s.instrumentation_scope is not None
|
||||
and s.instrumentation_scope.name == "fastmcp"
|
||||
]
|
||||
assert fastmcp_spans == []
|
||||
|
||||
|
||||
class TestProtocolVersionAttribute:
|
||||
"""The SERVER span carries `mcp.protocol.version`, matching the SDK.
|
||||
|
||||
FastMCP drops the SDK's `OpenTelemetryMiddleware` to avoid a duplicate SERVER
|
||||
span, so it re-emits the SDK's `mcp.protocol.version` attribute on its own
|
||||
span for parity.
|
||||
"""
|
||||
|
||||
async def test_tool_call_span_has_protocol_version(
|
||||
self, trace_exporter: InMemorySpanExporter
|
||||
):
|
||||
mcp = FastMCP("test-server")
|
||||
|
||||
@mcp.tool()
|
||||
def greet(name: str) -> str:
|
||||
return f"Hello, {name}!"
|
||||
|
||||
async with Client(mcp) as client:
|
||||
await client.call_tool("greet", {"name": "World"})
|
||||
|
||||
spans = trace_exporter.get_finished_spans()
|
||||
server_span = next(
|
||||
s
|
||||
for s in spans
|
||||
if s.kind == SpanKind.SERVER
|
||||
and s.attributes is not None
|
||||
and s.attributes.get("mcp.method.name") == "tools/call"
|
||||
)
|
||||
assert server_span.attributes is not None
|
||||
version = server_span.attributes.get("mcp.protocol.version")
|
||||
assert isinstance(version, str)
|
||||
assert version
|
||||
|
||||
async def test_seam_span_has_protocol_version(
|
||||
self, trace_exporter: InMemorySpanExporter
|
||||
):
|
||||
"""Seam-only methods (never reaching the high-level path) also carry the
|
||||
protocol version."""
|
||||
mcp = FastMCP("test-server")
|
||||
|
||||
async with Client(mcp) as client:
|
||||
await client.set_logging_level("info")
|
||||
|
||||
spans = trace_exporter.get_finished_spans()
|
||||
seam_span = next(
|
||||
s
|
||||
for s in spans
|
||||
if s.kind == SpanKind.SERVER and s.name == "logging/setLevel"
|
||||
)
|
||||
assert seam_span.attributes is not None
|
||||
version = seam_span.attributes.get("mcp.protocol.version")
|
||||
assert isinstance(version, str)
|
||||
assert version
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue