mirror of
https://github.com/PrefectHQ/fastmcp.git
synced 2026-08-09 15:19:10 +02:00
Editorial pass on the sampling and roots docs
This commit is contained in:
Jeremiah Lowin
parent
d5ff831602
commit
7fe3c1e8bd
10 changed files with 94 additions and 133 deletions
|
|
@ -185,16 +185,11 @@ Set `mode="legacy"` to force the initialize handshake. This behaves identically
|
|||
client = Client("https://example.com/mcp", mode="legacy")
|
||||
```
|
||||
|
||||
Legacy mode is also what you need for the capabilities that depend on a live session between client and server. The handshake opens a persistent back-channel the server can push requests down, and the modern era removed it. Pin `mode="legacy"` when your code relies on any of these:
|
||||
|
||||
- **[Sampling](/clients/sampling)** — server-initiated LLM completion requests
|
||||
- **[Roots](/clients/roots)** — server-initiated requests for the client's roots
|
||||
- **[Elicitation](/clients/elicitation)** — server-initiated requests for user input, which modern connections replace with [input-required rounds](/clients/elicitation#input-required-rounds)
|
||||
- `client.ping()` and `transport.get_session_id()`
|
||||
Legacy mode is also what carries the *pushed* form of a server's requests. The handshake opens a persistent back-channel down which a server can send a sampling, roots, or elicitation request mid-call, and the modern era removed it. Your handlers are unaffected by that: a [sampling](/clients/sampling), [roots](/clients/roots), or [elicitation](/clients/elicitation) handler you register answers a modern server's [input-required rounds](/clients/elicitation#input-required-rounds) from the same registration. Pin `mode="legacy"` when you connect to a server that pushes, or when your code calls `client.ping()` or `transport.get_session_id()`, which need the session the modern era does not open.
|
||||
|
||||
Conversely, [background tasks](/clients/tasks) are **modern-only**: the tasks capability is negotiated over `2026-07-28` connections, so `mode="legacy"` never triggers one and a task-enabled tool just runs synchronously.
|
||||
|
||||
A FastMCP server serves both eras, so a default client negotiates the modern one and these raise an era-specific error. Pinning the handshake restores them.
|
||||
A FastMCP server serves both eras, so a default client negotiates the modern one and the session-dependent calls raise an era-specific error there. Pinning the handshake restores them.
|
||||
|
||||
You can also pin a specific modern protocol version to adopt it directly, without a discovery probe:
|
||||
|
||||
|
|
@ -342,7 +337,7 @@ See [Prompts](/clients/prompts) for detailed documentation including argument se
|
|||
|
||||
The client supports callback handlers for advanced server interactions. These let you respond to server-initiated requests and receive notifications.
|
||||
|
||||
Sampling, elicitation, and roots are the requests a server makes of the client, and a server reaches your handlers by two routes depending on the era it speaks — see [protocol negotiation](#protocol-negotiation). On the handshake era it pushes the request down the open session mid-call. On the modern era there is no back-channel, so the server returns an input-required result naming what it needs and the client answers with a fresh call. Both routes dispatch to the same handler, so registering one covers both; the example below pins `mode="legacy"` only because it demonstrates the push route. Logging and progress arrive as notifications on the response stream and work in either era.
|
||||
Sampling, elicitation, and roots are the requests a server makes of the client. A server reaches your handler by whichever route its [era](#protocol-negotiation) allows — pushed down the open session on the handshake, returned as an input-required result on the modern protocol — and both routes dispatch to the same handler, so one registration covers both. Logging and progress arrive as notifications on the response stream and work in either era.
|
||||
|
||||
```python
|
||||
from fastmcp import Client
|
||||
|
|
@ -360,7 +355,6 @@ async def sampling_handler(messages, params, context):
|
|||
|
||||
client = Client(
|
||||
"my_mcp_server.py",
|
||||
mode="legacy",
|
||||
log_handler=log_handler,
|
||||
progress_handler=progress_handler,
|
||||
sampling_handler=sampling_handler,
|
||||
|
|
|
|||
|
|
@ -11,15 +11,13 @@ import { VersionBadge } from '/snippets/version-badge.mdx'
|
|||
|
||||
Use this when you need to tell servers what local resources the client has access to.
|
||||
|
||||
Roots inform servers about resources the client can provide. Servers can use this information to adjust behavior or provide more relevant responses.
|
||||
A root is a path your client is willing to expose — a project directory, a workspace, a document store. Servers read them to scope their work, so a tool that searches files searches where you pointed it, and a server that gets no roots has to ask the user for paths instead. Roots describe where the client can reach; the server takes them as its working boundary.
|
||||
|
||||
You register roots once, with `roots=`, and the client answers however the server asks for them. A **handshake-era** server pushes a `roots/list` request down the open session and reads the reply mid-call. A **modern** (`2026-07-28`) server has no such channel, so it returns an input-required result naming a roots request instead; `fastmcp.Client` fulfils it from the same `roots=` you registered and re-issues the call with the answer attached. One registration covers both routes, and the examples below work on either — the default `mode="auto"` negotiates whichever era the server speaks. See [protocol negotiation](/clients/client#protocol-negotiation) for how that choice is made.
|
||||
|
||||
Roots is a good fit for the modern route rather than merely a workable one. A server asks for roots once and then has what it needs, so the single extra round trip buys the whole answer — unlike generation, where a loop of guard rounds would spend the round-trip budget over and over, which is why [server-side sampling](/servers/sampling) recommends calling an LLM directly instead. On the server side the request travels in the `input_requests` map of an `InputRequiredResult`, described under [the guard pattern](/servers/elicitation#sampling-and-roots).
|
||||
Register them once with `roots=`, and the client answers however the server asks. A handshake-era server pushes a `roots/list` request down the open session and reads the reply mid-call; a modern (`2026-07-28`) server has no such channel, so it returns a roots request and `fastmcp.Client` fulfils it from the same registration and re-issues the call with the answer attached. The default `mode="auto"` negotiates whichever era the server speaks, so the examples below work on either — see [protocol negotiation](/clients/client#protocol-negotiation) for how that choice is made, and [the guard pattern](/servers/elicitation#sampling-and-roots) for how a server issues the modern form.
|
||||
|
||||
## Static Roots
|
||||
|
||||
Provide a list of roots when creating the client:
|
||||
When the paths are known up front, pass them as a list. The client holds them for the life of the connection and hands back the same set every time a server asks.
|
||||
|
||||
```python
|
||||
from fastmcp import Client
|
||||
|
|
@ -32,7 +30,7 @@ client = Client(
|
|||
|
||||
## Dynamic Roots
|
||||
|
||||
Use a callback to compute roots when the server asks for them. The callback runs on both routes — a pushed `roots/list` on a handshake connection, and a returned roots request on a modern one:
|
||||
Pass a callback instead when the roots depend on something the client learns at runtime, such as the workspace the user has open. It runs at the moment a server asks, on either route, and receives the request context so you can see which request it is answering:
|
||||
|
||||
```python
|
||||
from fastmcp import Client
|
||||
|
|
|
|||
|
|
@ -11,57 +11,44 @@ import { VersionBadge } from "/snippets/version-badge.mdx";
|
|||
|
||||
Use this when a server asks your client to run an LLM completion on its behalf.
|
||||
|
||||
A sampling handler is the client's answer to that request: the server describes the messages it wants completed, your handler runs them against whatever model you control, and the result goes back. Servers reach your handler by two different routes, and a handler you register serves both.
|
||||
Sampling is how a server borrows your model. Rather than hold an API key of its own, the server describes the messages it wants completed and asks you to run them — you pick the model, and you pay for the tokens. Your side of that arrangement is one function, a **sampling handler**, registered when you create the client.
|
||||
|
||||
On a **handshake-era** connection the server pushes a `sampling/createMessage` request down the open session mid-tool-call. On a **modern** (`2026-07-28`) connection there is no such channel, so a server instead *returns* an input-required result naming what it needs, and your client answers it and re-calls the tool. Both paths dispatch to the same `sampling_handler`, so register one and it works either way — this page's examples pin `mode="legacy"` only because they demonstrate the push route with a legacy server.
|
||||
|
||||
<Note>
|
||||
This page is the client side. Writing a FastMCP **server** is the other direction, and there is no `ctx.sample()` there — see [Sampling](/servers/sampling) under Servers for how a server returns a sampling request instead, and why calling an LLM directly is usually the better choice for generation.
|
||||
</Note>
|
||||
The handler receives the conversation the server wants completed, the parameters it asked for, and a request context carrying metadata about the call. Return the generated text as a string and FastMCP wraps it in the protocol's result for you; return a `CreateMessageResult` yourself when you want to report the real model name or hand back content that isn't text. If the handler raises, the client sends the error back in place of a completion and the server's tool decides what to do about it.
|
||||
|
||||
## Handler Template
|
||||
|
||||
```python
|
||||
from fastmcp import Client
|
||||
from fastmcp.client.sampling import SamplingMessage, SamplingParams, RequestContext
|
||||
from mcp_types import TextContent
|
||||
|
||||
|
||||
async def sampling_handler(
|
||||
messages: list[SamplingMessage],
|
||||
params: SamplingParams,
|
||||
context: RequestContext
|
||||
context: RequestContext,
|
||||
) -> str:
|
||||
"""
|
||||
Handle server requests for LLM completions.
|
||||
|
||||
Args:
|
||||
messages: Conversation messages to send to the LLM
|
||||
params: Sampling parameters (temperature, max_tokens, etc.)
|
||||
context: Request context with metadata
|
||||
|
||||
Returns:
|
||||
Generated text response from your LLM
|
||||
"""
|
||||
# Extract message content
|
||||
conversation = []
|
||||
for message in messages:
|
||||
content = message.content.text if hasattr(message.content, 'text') else str(message.content)
|
||||
conversation.append(f"{message.role}: {content}")
|
||||
|
||||
# Use the system prompt if provided
|
||||
"""Run the server's messages against your LLM and return the completion."""
|
||||
conversation = [
|
||||
f"{message.role}: {message.content.text}"
|
||||
for message in messages
|
||||
if isinstance(message.content, TextContent)
|
||||
]
|
||||
system_prompt = params.system_prompt or "You are a helpful assistant."
|
||||
|
||||
# Integrate with your LLM service here
|
||||
# Call your LLM here with `conversation` and `system_prompt`.
|
||||
return "Generated response based on the messages"
|
||||
|
||||
client = Client(
|
||||
"my_mcp_server.py",
|
||||
mode="legacy",
|
||||
sampling_handler=sampling_handler,
|
||||
)
|
||||
|
||||
client = Client("my_mcp_server.py", sampling_handler=sampling_handler)
|
||||
```
|
||||
|
||||
The client answers with this handler however the server asks for a completion. The default `mode="auto"` negotiates whichever protocol era the server speaks, and one handler covers both of the routes an era can use — see [Request Routes](#request-routes).
|
||||
|
||||
## Handler Parameters
|
||||
|
||||
Everything the server sends arrives in the first two arguments. The messages are the conversation to complete; the parameters are how the server would like it completed. You decide how much of that to honor, since the client owns the model — a preference your provider cannot express is yours to ignore.
|
||||
|
||||
<Card icon="code" title="SamplingMessage">
|
||||
<ResponseField name="role" type='Literal["user", "assistant"]'>
|
||||
The role of the message
|
||||
|
|
@ -73,11 +60,11 @@ client = Client(
|
|||
</Card>
|
||||
|
||||
<Card icon="code" title="SamplingParams">
|
||||
<ResponseField name="systemPrompt" type="str | None">
|
||||
<ResponseField name="system_prompt" type="str | None">
|
||||
Optional system prompt the server wants to use
|
||||
</ResponseField>
|
||||
|
||||
<ResponseField name="modelPreferences" type="ModelPreferences | None">
|
||||
<ResponseField name="model_preferences" type="ModelPreferences | None">
|
||||
Server preferences for model selection (hints, cost/speed/intelligence priorities)
|
||||
</ResponseField>
|
||||
|
||||
|
|
@ -85,11 +72,11 @@ client = Client(
|
|||
Sampling temperature
|
||||
</ResponseField>
|
||||
|
||||
<ResponseField name="maxTokens" type="int">
|
||||
<ResponseField name="max_tokens" type="int">
|
||||
Maximum tokens to generate
|
||||
</ResponseField>
|
||||
|
||||
<ResponseField name="stopSequences" type="list[str] | None">
|
||||
<ResponseField name="stop_sequences" type="list[str] | None">
|
||||
Stop sequences for sampling
|
||||
</ResponseField>
|
||||
|
||||
|
|
@ -97,14 +84,14 @@ client = Client(
|
|||
Tools the LLM can use during sampling
|
||||
</ResponseField>
|
||||
|
||||
<ResponseField name="toolChoice" type="ToolChoice | None">
|
||||
<ResponseField name="tool_choice" type="ToolChoice | None">
|
||||
Tool usage behavior (`auto`, `required`, or `none`)
|
||||
</ResponseField>
|
||||
</Card>
|
||||
|
||||
## Built-in Handlers
|
||||
|
||||
FastMCP provides built-in handlers for OpenAI, Anthropic, and Google Gemini APIs that support the full sampling API including tool use.
|
||||
Writing the provider call yourself is rarely worth it. FastMCP ships handlers for OpenAI, Anthropic, and Google Gemini that implement the full sampling API, tool use included, and translate the protocol's parameters into each provider's own. Give one a default model and pass it where your own handler would go. Write a custom handler when you need routing across providers, caching, or a provider FastMCP does not cover.
|
||||
|
||||
### OpenAI Handler
|
||||
|
||||
|
|
@ -116,19 +103,19 @@ from fastmcp.client.sampling.handlers.openai import OpenAISamplingHandler
|
|||
|
||||
client = Client(
|
||||
"my_mcp_server.py",
|
||||
mode="legacy",
|
||||
sampling_handler=OpenAISamplingHandler(default_model="gpt-4o"),
|
||||
)
|
||||
```
|
||||
|
||||
For OpenAI-compatible APIs (like local models):
|
||||
Point the handler at any OpenAI-compatible API, including a local model server, by passing your own provider client:
|
||||
|
||||
```python
|
||||
from fastmcp import Client
|
||||
from fastmcp.client.sampling.handlers.openai import OpenAISamplingHandler
|
||||
from openai import AsyncOpenAI
|
||||
|
||||
client = Client(
|
||||
"my_mcp_server.py",
|
||||
mode="legacy",
|
||||
sampling_handler=OpenAISamplingHandler(
|
||||
default_model="llama-3.1-70b",
|
||||
client=AsyncOpenAI(base_url="http://localhost:8000/v1"),
|
||||
|
|
@ -150,7 +137,6 @@ from fastmcp.client.sampling.handlers.anthropic import AnthropicSamplingHandler
|
|||
|
||||
client = Client(
|
||||
"my_mcp_server.py",
|
||||
mode="legacy",
|
||||
sampling_handler=AnthropicSamplingHandler(default_model="claude-sonnet-4-5"),
|
||||
)
|
||||
```
|
||||
|
|
@ -169,7 +155,6 @@ from fastmcp.client.sampling.handlers.google_genai import GoogleGenaiSamplingHan
|
|||
|
||||
client = Client(
|
||||
"my_mcp_server.py",
|
||||
mode="legacy",
|
||||
sampling_handler=GoogleGenaiSamplingHandler(default_model="gemini-2.0-flash"),
|
||||
)
|
||||
```
|
||||
|
|
@ -178,25 +163,32 @@ client = Client(
|
|||
Install the Google Gemini handler with `pip install 'fastmcp[gemini]'`.
|
||||
</Note>
|
||||
|
||||
## Sampling Capabilities
|
||||
The [source of these handlers](https://github.com/PrefectHQ/fastmcp/tree/main/fastmcp_slim/fastmcp/client/sampling/handlers) is the best reference for writing your own.
|
||||
|
||||
When you provide a `sampling_handler`, FastMCP automatically advertises full sampling capabilities to the server, including tool support. To disable tool support for simpler handlers:
|
||||
## Tool Use
|
||||
|
||||
A sampling request can carry tools. When it does, your handler passes them to the model and returns whatever comes back, tool calls included — the server executes the tools itself and sends a follow-up sampling request with the results if it needs another turn. Your handler never runs a tool.
|
||||
|
||||
Registering any `sampling_handler` advertises full sampling support, tools included. A handler that only generates text should say so, so servers know not to send tools it will drop:
|
||||
|
||||
```python
|
||||
from fastmcp import Client
|
||||
from mcp_types import SamplingCapability
|
||||
|
||||
|
||||
async def text_only_handler(messages, params, context) -> str:
|
||||
return "Generated response based on the messages"
|
||||
|
||||
|
||||
client = Client(
|
||||
"my_mcp_server.py",
|
||||
mode="legacy",
|
||||
sampling_handler=basic_handler,
|
||||
sampling_capabilities=SamplingCapability(), # No tool support
|
||||
sampling_handler=text_only_handler,
|
||||
sampling_capabilities=SamplingCapability(),
|
||||
)
|
||||
```
|
||||
|
||||
## Tool Execution
|
||||
## Request Routes
|
||||
|
||||
Tool execution happens on the server side. The client's role is to pass tools to the LLM and return the LLM's response (which may include tool use requests). The server then executes the tools and may send follow-up sampling requests with tool results.
|
||||
Servers reach your handler by two routes, and which one applies depends on the protocol era the connection negotiated. A handshake-era server pushes a `sampling/createMessage` request down the open session while a tool is running and waits for the reply. A modern (`2026-07-28`) connection has no such channel, so the tool ends its round by returning a request for a completion instead; the client answers from your handler and calls the tool again with the result attached.
|
||||
|
||||
<Tip>
|
||||
To implement a custom sampling handler, see the [handler source code](https://github.com/PrefectHQ/fastmcp/tree/main/fastmcp_slim/fastmcp/client/sampling/handlers) as a reference.
|
||||
</Tip>
|
||||
One registration covers both, so this is rarely something you configure — it matters only when you pin an era, since `mode="legacy"` is the sole route that carries a pushed request. See [protocol negotiation](/clients/client#protocol-negotiation) for how the era is chosen, and [Sampling](/servers/sampling) under Servers for how a server issues these requests.
|
||||
|
|
|
|||
|
|
@ -698,7 +698,7 @@ When deploying FastMCP behind a load balancer or running multiple server instanc
|
|||
|
||||
#### Understanding Sessions
|
||||
|
||||
By default, FastMCP's Streamable HTTP transport maintains server-side sessions. Sessions carry the handshake-era back-channel that server-initiated requests like [elicitation](/servers/elicitation) depend on, where the server needs to maintain context across multiple requests from the same client.
|
||||
By default, FastMCP's Streamable HTTP transport maintains server-side sessions. A session holds the context a server keeps across multiple requests from the same client, and it carries the handshake-era back-channel that server-initiated requests like [elicitation](/servers/elicitation) push down.
|
||||
|
||||
This works perfectly for single-instance deployments. However, sessions are stored in memory on each server instance, which creates challenges when scaling horizontally.
|
||||
|
||||
|
|
|
|||
|
|
@ -239,11 +239,11 @@ The warning comes from the MCP SDK, not from FastMCP, and it is benign. `ctx.inf
|
|||
|
||||
FastMCP servers built on the SDK v2 serve multiple protocol eras from the same server. The SDK negotiates the era each client speaks: the sessionless `2026-07-28` era (which discovers capabilities through `server/discover`) and earlier session-based handshake versions are all handled simultaneously. This formally supersedes FastMCP's earlier "latest protocol only" stance — a single server now works with clients across the protocol transition.
|
||||
|
||||
FastMCP 4 is a modern MCP toolkit, so the server API is the modern protocol's API. Where a capability survived the transition in a different shape, FastMCP carries it across in that shape and drops the old spelling rather than shipping a method that only works on old connections: **`ctx.sample()`, `ctx.sample_step()`, and `ctx.list_roots()` are gone from `Context` entirely**, along with the `sampling_handler=` and `sampling_handler_behavior=` arguments to `FastMCP()`. Calling them raises `AttributeError` on every era, not an era-specific runtime error, and `FastMCP(sampling_handler=...)` raises `TypeError` naming the migration.
|
||||
**`ctx.sample()`, `ctx.sample_step()`, and `ctx.list_roots()` are gone from `Context`**, along with the `sampling_handler=` and `sampling_handler_behavior=` arguments to `FastMCP()`. Touching a removed method raises `AttributeError` on every era, and `FastMCP(sampling_handler=...)` raises a `TypeError` naming the migration, so the break surfaces when you upgrade rather than in production against whichever client happens to negotiate the modern era.
|
||||
|
||||
This is a deliberate stance rather than an unfinished port. `ctx.sample()` and `ctx.list_roots()` were *pushes*: the server sent a request down a live back-channel and blocked for the answer, and the sessionless protocol has no such channel. Since `fastmcp.Client` now negotiates the modern protocol by default, keeping those methods would mean shipping an API whose default outcome is a runtime failure. What the protocol removed is the pushing, not the asking — sampling, elicitation, and roots all still reach the client through the [guard pattern](/servers/elicitation#elicitation-on-the-modern-protocol), where a tool *returns* an `InputRequiredResult` describing what it needs, the client answers, and it calls again with the answer attached.
|
||||
All three *pushed*: the server sent a request down a live back-channel and blocked for the answer, and the sessionless protocol has no such channel. Since `fastmcp.Client` now negotiates the modern protocol by default, a method like that would fail against a default client. What the protocol removed is the pushing, not the asking — sampling, elicitation, and roots all still reach the client through the [guard pattern](/servers/elicitation#elicitation-on-the-modern-protocol), where a tool *returns* an `InputRequiredResult` describing what it needs, the client answers, and it calls again with the answer attached.
|
||||
|
||||
Migrating differs by capability. For **roots**, the guard pattern is the direct replacement and a good one: a server asks once and has what it needs, so returning a roots request costs a single round trip. Accepting the paths as tool arguments remains simpler when the caller can just supply them. For **sampling**, the guard route works the same way and conforms on `2026-07-28`, but generation usually belongs in your server — every round is a full request-response cycle, so a generation loop pays that cost repeatedly. [Call an LLM from your server](/servers/sampling) with your own API key and your tool behaves the same for every client, including the many that never implemented sampling; reach for the guard route when the point is specifically to use the caller's model.
|
||||
Migrating differs by capability. For **roots**, the guard pattern is the direct replacement: a server asks once and has what it needs, so the extra round buys the whole answer, and taking the paths as tool arguments is simpler still when the caller can just supply them. For **sampling**, the guard route works the same way, but generation usually belongs in your server, because every round is a full request-response cycle and a generation loop pays that cost repeatedly. [Call an LLM from your server](/servers/sampling) with your own API key and your tool behaves the same for every client, including the many that never implemented sampling; reach for the guard route when the point is specifically to use the caller's model.
|
||||
|
||||
| Context feature | Earlier eras (session-based) | `2026-07-28` (sessionless) |
|
||||
| --- | --- | --- |
|
||||
|
|
@ -259,7 +259,7 @@ Migrating differs by capability. For **roots**, the guard pattern is the direct
|
|||
|
||||
Two of these bite by default now, because **`fastmcp.Client` defaults to `mode="auto"`** in v4 — an ordinary `Client(server)` negotiates the newest protocol both sides share, which against a FastMCP server is the sessionless `2026-07-28` era. On that era there is no `initialize` handshake, so a `Middleware.on_initialize` hook never runs; and each request is a fresh connection, so state written with `ctx.set_state` in one call is not visible in the next. A server that gates access in `on_initialize` or relies on per-session state must keep its clients on the session-based era. The narrow escape is per-client: `Client(server, mode="legacy")`. The durable, server-side answer is to declare the versions the server actually serves so a modern client is refused at connect time rather than silently losing those features — see the server's protocol-version restriction (added alongside this change).
|
||||
|
||||
The client side is unaffected by any of this. `sampling_handler=` and `roots=` mean what they always did — see [client sampling](/clients/sampling) and [client roots](/clients/roots) — and one registration serves both routes: a handshake-era server's pushed request and a modern server's returned one dispatch to the same handler, so a client needs no era-specific wiring either way.
|
||||
The client side is unaffected. `sampling_handler=` and `roots=` mean what they always did — see [client sampling](/clients/sampling) and [client roots](/clients/roots) — and one registration serves both routes, since a handshake-era server's pushed request and a modern server's returned one dispatch to the same handler.
|
||||
|
||||
## Upgrade checklist
|
||||
|
||||
|
|
|
|||
|
|
@ -25,7 +25,9 @@ A FastMCP 4 server answers clients across the protocol transition from one deplo
|
|||
|
||||
The same negotiation runs from the client, and its default flipped. A plain `Client(url)` now probes for the modern protocol and adopts it when the server offers it, falling back to the handshake otherwise — where every earlier FastMCP version pinned the handshake outright. That flip is what brings the modern capabilities within reach of ordinary client code: a task-enabled tool hands back a handle to poll, and multi-round-trip elicitation resolves across successive requests, neither requiring the caller to opt in. Set `mode="legacy"` to pin the handshake when you need the session-based back-channel or the classic `initialize` result. See [Protocol negotiation](/clients/client#protocol-negotiation).
|
||||
|
||||
The modern protocol is sessionless, so it drops the server's ability to call back into the client mid-request (SEP-2577), and FastMCP 4's server API reflects that rather than papering over it. `ctx.elicit` moves to a request-shaped pattern that works on modern connections. `ctx.sample`, `ctx.sample_step`, and `ctx.list_roots` are not in the API at all — they pushed a request into a live connection, and shipping methods that only work against old clients would be shipping a trap. The capabilities behind them survive through the same request-shaped pattern: a tool returns a sampling or roots request and reads the answer on the next round. For roots that is the natural replacement, since one round trip buys the whole answer. For generation, [call an LLM from your server](/servers/sampling) — a loop of guard rounds spends the round-trip budget several times over. Logging is untouched: `ctx.info` and its siblings are notifications, which ride the response stream and reach the client on every era. Everything else about writing a server is unchanged.
|
||||
The modern protocol is sessionless, so it drops the server's ability to call back into the client mid-request (SEP-2577), and FastMCP 4's server API reflects that. `ctx.elicit` moves to a request-shaped pattern that works on modern connections: the tool returns a description of the input it needs, and the client answers with a fresh call. `ctx.sample`, `ctx.sample_step`, and `ctx.list_roots` are gone from the API, because each of them pushed a request down a live connection and a method that only works against old clients is a trap.
|
||||
|
||||
Both capabilities survive in the same request-shaped form. Asking for roots that way is the natural replacement, since one round trip buys the whole answer. Generation usually belongs in the server instead, because a loop of asking rounds spends the round-trip budget over and over — [call an LLM from your server](/servers/sampling). Logging is untouched: `ctx.info` and its siblings are notifications, and notifications ride the response stream on every era. Everything else about writing a server is unchanged.
|
||||
|
||||
## State without a session
|
||||
|
||||
|
|
|
|||
|
|
@ -151,14 +151,9 @@ if result.action == "accept":
|
|||
|
||||
See [User Elicitation](/servers/elicitation) for detailed examples and supported response types.
|
||||
|
||||
### Server-initiated requests
|
||||
|
||||
<Note>
|
||||
`Context` has no `sample()` or `list_roots()`. Both *pushed* a request into a live client connection, and the modern MCP protocol has no channel to carry that ([SEP-2577](/servers/sampling)). Both capabilities remain reachable through the [guard pattern](/servers/elicitation#sampling-and-roots), where a tool returns a sampling or roots request in `input_requests` and reads the answer from `ctx.input_responses` on the next round. That is the natural route for roots. For generation, [call an LLM directly from your server](/servers/sampling) instead — each guard round costs a full round trip.
|
||||
|
||||
Logging is unaffected — `ctx.info()` and friends are *notifications*, which ride the response stream and work on every protocol era.
|
||||
</Note>
|
||||
### Sampling and Roots
|
||||
|
||||
Neither capability has a `Context` method. Both used to *push* a request into a live client connection, which the modern MCP protocol has no channel to carry, so a tool now asks for them by returning the request and reading the answer on the next round — the same [guard pattern](/servers/elicitation#sampling-and-roots) elicitation uses on modern connections. That route is the natural one for roots; for generation, [call an LLM directly from your server](/servers/sampling).
|
||||
|
||||
### Progress Reporting
|
||||
|
||||
|
|
|
|||
|
|
@ -596,9 +596,9 @@ The same protocol requirement applies: returning an `InputRequiredResult` from a
|
|||
|
||||
### Sampling and roots
|
||||
|
||||
Elicitation is the most common request to carry this way, and **roots** and **sampling** requests work identically — the `input_requests` map holds a `ListRootsRequest` or a `CreateMessageRequest` the same way it holds an `ElicitRequest`, and each answer comes back in `ctx.input_responses` under its key as a `ListRootsResult` or `CreateMessageResult`. A single map can mix all three. `fastmcp.Client` answers each one from the handlers you already configured — `elicitation_handler=`, `roots=`, `sampling_handler=` — so a guard tool that mixes them needs no extra client wiring. See [Client Roots](/clients/roots) for what a roots request contains.
|
||||
Elicitation is the most common request to carry this way, and the map carries the others just as well. A `ListRootsRequest` or a `CreateMessageRequest` sits in `input_requests` exactly as an `ElicitRequest` does, and its answer arrives in `ctx.input_responses` under the same key as a `ListRootsResult` or a `CreateMessageResult`. One map can mix all three, and `fastmcp.Client` answers each from the handlers it already has — `elicitation_handler=`, `roots=`, and `sampling_handler=` — so a tool that asks for a mixture needs no extra client wiring. [Client Roots](/clients/roots) covers what a roots request contains.
|
||||
|
||||
The two differ in when you should reach for them. Roots suits this pattern well: a server asks once and has what it needs, so one extra round trip buys the whole answer. Generation usually does not, because every round is a full request-response cycle and a loop pays that cost each time — [call an LLM directly from your server](/servers/sampling) unless the point is specifically to use the caller's model.
|
||||
Roots and sampling differ in how well they suit the round trip. A server asks for roots once and then has what it needs, so the extra round buys the whole answer. Generation rarely works out that way, because every round is a full request-response cycle and a tool that generates in a loop pays that cost each time — [call an LLM directly from your server](/servers/sampling) unless the point is specifically to use the caller's model.
|
||||
|
||||
### Middleware
|
||||
|
||||
|
|
|
|||
|
|
@ -5,37 +5,43 @@ description: Generate text from a FastMCP server — by calling an LLM directly,
|
|||
icon: robot
|
||||
---
|
||||
|
||||
FastMCP 4 targets the modern MCP protocol, and that protocol has no channel for a server to push a request into a live client connection. A tool cannot pause mid-execution to borrow the caller's model and resume when the completion arrives, so the imperative methods that did exactly that are gone: there is no `ctx.sample()` and no `ctx.sample_step()`, and `FastMCP()` no longer accepts `sampling_handler=`. Calling them raises `AttributeError` on every protocol era rather than failing at runtime only against modern clients.
|
||||
A tool that needs text generated calls a model to get it, and in FastMCP 4 that call is ordinary Python: your server holds an API key, creates a provider client, and awaits a completion inside the tool. No protocol is involved, so the tool behaves the same for every client — including the many that never implemented sampling at all.
|
||||
|
||||
The *capability* survives in a different shape. A tool that genuinely needs the caller's model asks for a completion by **returning** a request for one, the same [guard pattern](/servers/elicitation#elicitation-on-the-modern-protocol) elicitation uses: the round ends as an ordinary response, the client runs the completion against its own model, and it calls the tool again with the answer attached. FastMCP's own conformance suite exercises this on `2026-07-28`.
|
||||
The alternative is to ask the caller. Sampling borrows *the caller's* model — their provider, their credentials, their bill — by returning a request for a completion that the client fulfils and hands back. Every ask costs a full round trip, so it earns its keep when using the caller's model is the point, and rarely otherwise.
|
||||
|
||||
For most generation you should skip that round trip and call an LLM directly from your server. Each guard round is a full request-response cycle, so a tool that generates in a loop spends the round-trip budget several times over, while a direct call is an ordinary async function call inside a single tool invocation. Ask the client to sample when the point is to use *the caller's* model — their credentials, their choice of provider, their bill. Call an LLM yourself when the point is to generate text.
|
||||
`Context` carries no sampling methods in FastMCP 4. If you came here looking for `ctx.sample()`, [the removed methods](#the-removed-methods) covers what happened and where the capability went.
|
||||
|
||||
## Requests and notifications
|
||||
## Calling an LLM directly
|
||||
|
||||
The distinction that makes this make sense is between *asking* and *telling*.
|
||||
|
||||
A notification is fire-and-forget. Your server emits it and moves on, and it travels down the response stream the caller already opened for the request in flight. Nothing has to be held open on the server's behalf, so notifications survive the move to a stateless protocol untouched. This is why [logging](/servers/logging) still works exactly as it always has: `ctx.info()`, `ctx.debug()`, and the rest reach the client mid-call on every protocol era.
|
||||
Hold a provider API key in your server's environment, create the client once at module scope so connections are reused across calls, and generate inside the tool. You choose the model, control the prompt, see the token usage, and can test the tool with no client attached.
|
||||
|
||||
```python
|
||||
from fastmcp import Context, FastMCP
|
||||
import anthropic
|
||||
from fastmcp import FastMCP
|
||||
|
||||
mcp = FastMCP("Reports")
|
||||
mcp = FastMCP("Summarizer")
|
||||
llm = anthropic.AsyncAnthropic()
|
||||
|
||||
|
||||
@mcp.tool
|
||||
async def build_report(rows: int, ctx: Context) -> str:
|
||||
await ctx.info(f"Processing {rows} rows")
|
||||
return "done"
|
||||
async def summarize(text: str) -> str:
|
||||
"""Summarize a document in two sentences."""
|
||||
response = await llm.messages.create(
|
||||
model="claude-sonnet-4-5",
|
||||
max_tokens=512,
|
||||
system="Summarize the user's text in exactly two sentences.",
|
||||
messages=[{"role": "user", "content": text}],
|
||||
)
|
||||
return response.content[0].text
|
||||
```
|
||||
|
||||
Sampling is the other kind. It is a *request* — `sampling/createMessage` goes out and the caller must answer before the tool can continue. Pushing that request needs a live, addressable connection the server can reach into, which is precisely what a stateless protocol does not have, and MCP removed server-initiated requests in the `2026-07-28` revision ([SEP-2577](https://modelcontextprotocol.io/community/sep-guidelines)) for that reason. What the protocol removed is the *pushing*, not the asking. Asking survives by inverting who initiates the next message: instead of the server reaching down mid-call, the tool returns a description of what it needs and the client comes back.
|
||||
Any provider SDK works the same way — swap the client and the call, and the tool signature is unchanged. Because generation is ordinary application code, the concerns around it are ordinary too: retries, timeouts, caching, and cost accounting go wherever you want them rather than being negotiated across a protocol boundary. A tool that chains several generations pays nothing extra for the second and third, where asking the caller would pay a full round trip for each.
|
||||
|
||||
## Asking the client to sample
|
||||
## Asking the caller's model
|
||||
|
||||
A tool asks for a completion by returning an `InputRequiredResult` whose `input_requests` map holds a `CreateMessageRequest` under a key you choose. That result completes the round normally. The client runs the completion, then re-issues the same `call_tool` with the answer attached, and your tool reads it from `ctx.input_responses` under the same key — a `CreateMessageResult`. Because the tool runs from the top on every round, the presence of `ctx.input_responses` is what tells the two rounds apart: `None` on the first call, populated on the continuation.
|
||||
|
||||
`fastmcp.Client` drives this loop for you and answers from the [`sampling_handler`](/clients/sampling) you already registered, so a client configured for a handshake-era server needs no extra wiring to satisfy a modern guard tool.
|
||||
`fastmcp.Client` drives that loop for you and answers from the [`sampling_handler`](/clients/sampling) it already has, so a client written for a handshake-era server needs no extra wiring to satisfy a modern tool that asks this way.
|
||||
|
||||
```python
|
||||
from fastmcp import Context, FastMCP
|
||||
|
|
@ -82,39 +88,15 @@ async def ask_the_caller(question: str, ctx: Context) -> str | InputRequiredResu
|
|||
return "The client returned no completion."
|
||||
```
|
||||
|
||||
Returning an `InputRequiredResult` needs a `2026-07-28` connection; FastMCP names the era mismatch if an older client reaches the tool. The map can carry several requests at once and mix kinds — a sampling request beside an elicitation or a roots request — and each answer comes back under its own key. [Elicitation](/servers/elicitation#sampling-and-roots) covers the mechanics of the pattern in full, including how to carry state across rounds.
|
||||
Returning an `InputRequiredResult` needs a `2026-07-28` connection, and FastMCP names the era mismatch if an older client reaches the tool; the conformance suite exercises this route on that version. The map can carry several requests at once and mix kinds — a sampling request beside an elicitation or a roots request — with each answer coming back under its own key. [Elicitation](/servers/elicitation#sampling-and-roots) covers the mechanics of the pattern in full, including how to carry state across rounds.
|
||||
|
||||
## Calling an LLM directly
|
||||
## The removed methods
|
||||
|
||||
For generation, your server calls the model. Hold a provider API key in your server's environment, create the client once at module scope so connections are reused across calls, and generate inside the tool. The result is a plain async function call with no protocol involvement, which also means you choose the model, control the prompt, see the token usage, and can test the tool without a client attached.
|
||||
`Context` has no `sample()` and no `sample_step()`; touching either raises `AttributeError` on every protocol era, rather than failing at runtime only against modern clients. `FastMCP()` accepts neither `sampling_handler=` nor `sampling_handler_behavior=`, and naming one raises a `TypeError` that points at the migration.
|
||||
|
||||
```python
|
||||
import anthropic
|
||||
from fastmcp import FastMCP
|
||||
The reason is the distinction MCP draws between telling and asking. A notification is fire-and-forget: the server emits it and moves on, and it travels down the response stream the caller already opened, so nothing has to be held open on the server's behalf. That is why [logging](/servers/logging) is untouched by any of this — `ctx.info()` and its siblings reach the client mid-call on every era. Sampling is the other kind. `sampling/createMessage` goes out and the caller must answer before the tool can continue, which needs a live, addressable connection the server can reach into, and the `2026-07-28` revision removed server-initiated requests ([SEP-2577](https://modelcontextprotocol.io/community/sep-guidelines)) precisely because a stateless protocol has no such thing.
|
||||
|
||||
mcp = FastMCP("Summarizer")
|
||||
llm = anthropic.AsyncAnthropic()
|
||||
|
||||
|
||||
@mcp.tool
|
||||
async def summarize(text: str) -> str:
|
||||
"""Summarize a document in two sentences."""
|
||||
response = await llm.messages.create(
|
||||
model="claude-sonnet-4-5",
|
||||
max_tokens=512,
|
||||
system="Summarize the user's text in exactly two sentences.",
|
||||
messages=[{"role": "user", "content": text}],
|
||||
)
|
||||
return response.content[0].text
|
||||
```
|
||||
|
||||
Any provider SDK works the same way — swap the client and the call, and the tool signature is unchanged. Because generation is now ordinary application code, the surrounding concerns become ordinary too: retries, timeouts, caching, and cost accounting are yours to place where you want them rather than negotiated across a protocol boundary. A tool that chains several generations pays nothing extra for the second and third, where the guard route would pay a full round trip for each.
|
||||
|
||||
The trade this makes is explicit. Sampling let a server borrow the caller's model and the caller's bill; calling directly means you supply the key and pay for the tokens. In exchange your tool behaves identically for every client, including the many that never implemented sampling at all.
|
||||
|
||||
## Clients answering servers
|
||||
|
||||
Both routes land in the same place on the client. A `fastmcp.Client` passes `sampling_handler=` once, and that handler answers a handshake-era server's pushed `sampling/createMessage` request and a modern server's returned sampling request alike — see [Sampling](/clients/sampling) under Clients.
|
||||
What the protocol removed is the pushing, not the asking, so the capability survives in the shape described above. Keeping `ctx.sample()` alongside it would mean shipping a method whose outcome against a default client — one that negotiates the modern era — is a runtime failure.
|
||||
|
||||
<Note>
|
||||
Servers on FastMCP 3 still have `ctx.sample()` and `ctx.sample_step()`, documented in the [FastMCP 3 sampling guide](/v3/servers/sampling). Nothing changes for them until they upgrade.
|
||||
|
|
|
|||
|
|
@ -1056,16 +1056,14 @@ mcp = FastMCP(name="ContextDemo")
|
|||
async def process_data(data_uri: str, ctx: Context) -> dict:
|
||||
"""Process data from a resource with progress reporting."""
|
||||
await ctx.info(f"Processing data from {data_uri}")
|
||||
|
||||
# Read a resource
|
||||
resource = await ctx.read_resource(data_uri)
|
||||
data = resource[0].content if resource else ""
|
||||
|
||||
# Report progress
|
||||
|
||||
result = await ctx.read_resource(data_uri)
|
||||
data = result.contents[0].content if result.contents else ""
|
||||
await ctx.report_progress(progress=50, total=100)
|
||||
|
||||
summary = str(data)[:200]
|
||||
await ctx.report_progress(progress=100, total=100)
|
||||
return {"length": len(data)}
|
||||
return {"length": len(data), "summary": summary}
|
||||
```
|
||||
|
||||
The Context object provides access to:
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue