mirror of
https://github.com/PrefectHQ/fastmcp.git
synced 2026-08-09 15:19:10 +02:00
Docs: sampling and roots work on modern via the guard pattern
The imperative ctx.sample()/ctx.list_roots() stay removed, but both capabilities survive as input-required requests, as tests/conformance exercises on 2026-07-28. Direct LLM calls remain the recommendation for generation; roots has no round-trip-budget objection.
This commit is contained in:
Jeremiah Lowin
parent
dec25ba6be
commit
cf7edc895c
8 changed files with 80 additions and 27 deletions
|
|
@ -342,7 +342,7 @@ See [Prompts](/clients/prompts) for detailed documentation including argument se
|
|||
|
||||
The client supports callback handlers for advanced server interactions. These let you respond to server-initiated requests and receive notifications.
|
||||
|
||||
Sampling, elicitation, and roots are all server-initiated, so they belong to the handshake era described under [protocol negotiation](#protocol-negotiation). A default client negotiates the newest era both peers share, where the server has no back-channel to push those requests down, so an example that exercises them pins `mode="legacy"`. Logging and progress arrive as notifications on the response stream and work in either era.
|
||||
Sampling, elicitation, and roots are the requests a server makes of the client, and a server reaches your handlers by two routes depending on the era it speaks — see [protocol negotiation](#protocol-negotiation). On the handshake era it pushes the request down the open session mid-call. On the modern era there is no back-channel, so the server returns an input-required result naming what it needs and the client answers with a fresh call. Both routes dispatch to the same handler, so registering one covers both; the example below pins `mode="legacy"` only because it demonstrates the push route. Logging and progress arrive as notifications on the response stream and work in either era.
|
||||
|
||||
```python
|
||||
from fastmcp import Client
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
---
|
||||
title: Client Roots
|
||||
sidebarTitle: Roots
|
||||
description: Provide local context and resource boundaries to MCP servers.
|
||||
description: Tell servers which local paths your client can reach.
|
||||
icon: folder-tree
|
||||
---
|
||||
|
||||
|
|
@ -13,9 +13,9 @@ Use this when you need to tell servers what local resources the client has acces
|
|||
|
||||
Roots inform servers about resources the client can provide. Servers can use this information to adjust behavior or provide more relevant responses.
|
||||
|
||||
<Note>
|
||||
**Roots require the older MCP protocol.** A server reads roots by sending a request down to the client, and protocol version `2026-07-28` removed the server's ability to do that. Clients default to `mode="auto"`, which negotiates the newest version both sides support, so the examples below pass `mode="legacy"`. See [protocol negotiation](/clients/client#protocol-negotiation).
|
||||
</Note>
|
||||
You register roots once, with `roots=`, and the client answers however the server asks for them. A **handshake-era** server pushes a `roots/list` request down the open session and reads the reply mid-call. A **modern** (`2026-07-28`) server has no such channel, so it returns an input-required result naming a roots request instead; `fastmcp.Client` fulfils it from the same `roots=` you registered and re-issues the call with the answer attached. One registration covers both routes, and the examples below work on either — the default `mode="auto"` negotiates whichever era the server speaks. See [protocol negotiation](/clients/client#protocol-negotiation) for how that choice is made.
|
||||
|
||||
Roots is a good fit for the modern route rather than merely a workable one. A server asks for roots once and then has what it needs, so the single extra round trip buys the whole answer — unlike generation, where a loop of guard rounds would spend the round-trip budget over and over, which is why [server-side sampling](/servers/sampling) recommends calling an LLM directly instead. On the server side the request travels in the `input_requests` map of an `InputRequiredResult`, described under [the guard pattern](/servers/elicitation#sampling-and-roots).
|
||||
|
||||
## Static Roots
|
||||
|
||||
|
|
@ -26,14 +26,13 @@ from fastmcp import Client
|
|||
|
||||
client = Client(
|
||||
"my_mcp_server.py",
|
||||
mode="legacy",
|
||||
roots=["/path/to/root1", "/path/to/root2"]
|
||||
)
|
||||
```
|
||||
|
||||
## Dynamic Roots
|
||||
|
||||
Use a callback to compute roots dynamically when the server requests them:
|
||||
Use a callback to compute roots when the server asks for them. The callback runs on both routes — a pushed `roots/list` on a handshake connection, and a returned roots request on a modern one:
|
||||
|
||||
```python
|
||||
from fastmcp import Client
|
||||
|
|
@ -45,7 +44,6 @@ async def roots_callback(context: RequestContext) -> list[str]:
|
|||
|
||||
client = Client(
|
||||
"my_mcp_server.py",
|
||||
mode="legacy",
|
||||
roots=roots_callback
|
||||
)
|
||||
```
|
||||
|
|
|
|||
|
|
@ -16,7 +16,7 @@ A sampling handler is the client's answer to that request: the server describes
|
|||
On a **handshake-era** connection the server pushes a `sampling/createMessage` request down the open session mid-tool-call. On a **modern** (`2026-07-28`) connection there is no such channel, so a server instead *returns* an input-required result naming what it needs, and your client answers it and re-calls the tool. Both paths dispatch to the same `sampling_handler`, so register one and it works either way — this page's examples pin `mode="legacy"` only because they demonstrate the push route with a legacy server.
|
||||
|
||||
<Note>
|
||||
This page is the client side. Writing a FastMCP **server** is the other direction, and there is no `ctx.sample()` there — see [Sampling](/servers/sampling) under Servers for why, and for calling an LLM from your own server instead.
|
||||
This page is the client side. Writing a FastMCP **server** is the other direction, and there is no `ctx.sample()` there — see [Sampling](/servers/sampling) under Servers for how a server returns a sampling request instead, and why calling an LLM directly is usually the better choice for generation.
|
||||
</Note>
|
||||
|
||||
## Handler Template
|
||||
|
|
|
|||
|
|
@ -239,19 +239,19 @@ The warning comes from the MCP SDK, not from FastMCP, and it is benign. `ctx.inf
|
|||
|
||||
FastMCP servers built on the SDK v2 serve multiple protocol eras from the same server. The SDK negotiates the era each client speaks: the sessionless `2026-07-28` era (which discovers capabilities through `server/discover`) and earlier session-based handshake versions are all handled simultaneously. This formally supersedes FastMCP's earlier "latest protocol only" stance — a single server now works with clients across the protocol transition.
|
||||
|
||||
FastMCP 4 is a modern MCP toolkit, so the server API is the modern protocol's API. Where a capability survived the transition in a different shape, FastMCP carries it across in that shape. Where the protocol removed a capability outright, FastMCP 4 does not carry a version of it that only works on old connections: **`ctx.sample()`, `ctx.sample_step()`, and `ctx.list_roots()` are gone from `Context` entirely**, along with the `sampling_handler=` and `sampling_handler_behavior=` arguments to `FastMCP()`. Calling them raises `AttributeError` on every era, not an era-specific runtime error, and `FastMCP(sampling_handler=...)` raises `TypeError` naming the migration.
|
||||
FastMCP 4 is a modern MCP toolkit, so the server API is the modern protocol's API. Where a capability survived the transition in a different shape, FastMCP carries it across in that shape and drops the old spelling rather than shipping a method that only works on old connections: **`ctx.sample()`, `ctx.sample_step()`, and `ctx.list_roots()` are gone from `Context` entirely**, along with the `sampling_handler=` and `sampling_handler_behavior=` arguments to `FastMCP()`. Calling them raises `AttributeError` on every era, not an era-specific runtime error, and `FastMCP(sampling_handler=...)` raises `TypeError` naming the migration.
|
||||
|
||||
This is a deliberate stance rather than an unfinished port. Server-initiated sampling and roots are *requests*: the server sends one and blocks for an answer, which needs a live back-channel the sessionless protocol does not have. Since `fastmcp.Client` now negotiates the modern protocol by default, keeping these methods would mean shipping an API whose default outcome is a runtime failure. Elicitation is the one server-initiated capability that survives, because the modern protocol carries it in a new shape: a tool *returns* an `InputRequiredResult` describing what it needs, and the client answers with a fresh call (see [Elicitation on the modern protocol](/servers/elicitation#elicitation-on-the-modern-protocol)).
|
||||
This is a deliberate stance rather than an unfinished port. `ctx.sample()` and `ctx.list_roots()` were *pushes*: the server sent a request down a live back-channel and blocked for the answer, and the sessionless protocol has no such channel. Since `fastmcp.Client` now negotiates the modern protocol by default, keeping those methods would mean shipping an API whose default outcome is a runtime failure. What the protocol removed is the pushing, not the asking — sampling, elicitation, and roots all still reach the client through the [guard pattern](/servers/elicitation#elicitation-on-the-modern-protocol), where a tool *returns* an `InputRequiredResult` describing what it needs, the client answers, and it calls again with the answer attached.
|
||||
|
||||
Migrating is direct in both cases. For sampling, [call an LLM from your server](/servers/sampling) with your own API key — your tool then behaves the same for every client, including the many that never implemented sampling. For roots, accept the paths you need as tool arguments, or ask for them through the same guard pattern, whose `input_requests` map carries a roots request alongside elicitation.
|
||||
Migrating differs by capability. For **roots**, the guard pattern is the direct replacement and a good one: a server asks once and has what it needs, so returning a roots request costs a single round trip. Accepting the paths as tool arguments remains simpler when the caller can just supply them. For **sampling**, the guard route works the same way and conforms on `2026-07-28`, but generation usually belongs in your server — every round is a full request-response cycle, so a generation loop pays that cost repeatedly. [Call an LLM from your server](/servers/sampling) with your own API key and your tool behaves the same for every client, including the many that never implemented sampling; reach for the guard route when the point is specifically to use the caller's model.
|
||||
|
||||
| Context feature | Earlier eras (session-based) | `2026-07-28` (sessionless) |
|
||||
| --- | --- | --- |
|
||||
| `ctx.info` / logging notifications | Supported | Supported |
|
||||
| Tools, resources, prompts, completions | Supported | Supported |
|
||||
| `ctx.elicit` | Supported | Use the guard pattern (return `InputRequiredResult`) |
|
||||
| `ctx.sample` / `ctx.sample_step` | Removed from the API — call an LLM server-side | Removed from the API — call an LLM server-side |
|
||||
| `ctx.list_roots` | Removed from the API — take paths as arguments, or use the guard pattern | Removed from the API — take paths as arguments, or use the guard pattern |
|
||||
| `ctx.sample` / `ctx.sample_step` | Method removed — call an LLM server-side | Method removed — call an LLM server-side, or ask via the guard pattern |
|
||||
| `ctx.list_roots` | Method removed — take paths as tool arguments | Method removed — ask via the guard pattern, or take paths as tool arguments |
|
||||
| `client.set_logging_level()` | Supported | Raises — `logging/setLevel` needs session state the era lacks |
|
||||
| `Middleware.on_initialize` | Runs on connect | Never runs — there is no `initialize` handshake |
|
||||
| Session state (`ctx.set_state` across calls) | Persists for the session | Does not persist — every request is a fresh connection |
|
||||
|
|
@ -259,7 +259,7 @@ Migrating is direct in both cases. For sampling, [call an LLM from your server](
|
|||
|
||||
Two of these bite by default now, because **`fastmcp.Client` defaults to `mode="auto"`** in v4 — an ordinary `Client(server)` negotiates the newest protocol both sides share, which against a FastMCP server is the sessionless `2026-07-28` era. On that era there is no `initialize` handshake, so a `Middleware.on_initialize` hook never runs; and each request is a fresh connection, so state written with `ctx.set_state` in one call is not visible in the next. A server that gates access in `on_initialize` or relies on per-session state must keep its clients on the session-based era. The narrow escape is per-client: `Client(server, mode="legacy")`. The durable, server-side answer is to declare the versions the server actually serves so a modern client is refused at connect time rather than silently losing those features — see the server's protocol-version restriction (added alongside this change).
|
||||
|
||||
The client side is unaffected by any of this. A `fastmcp.Client` still answers a legacy server's sampling and roots requests through `sampling_handler=` and `roots=` — see [client sampling](/clients/sampling) and [client roots](/clients/roots) — because a modern client still has to interoperate with servers built before the protocol changed.
|
||||
The client side is unaffected by any of this. `sampling_handler=` and `roots=` mean what they always did — see [client sampling](/clients/sampling) and [client roots](/clients/roots) — and one registration serves both routes: a handshake-era server's pushed request and a modern server's returned one dispatch to the same handler, so a client needs no era-specific wiring either way.
|
||||
|
||||
## Upgrade checklist
|
||||
|
||||
|
|
@ -268,7 +268,7 @@ Most servers upgrade untouched. Work down this list to find the ones that don't:
|
|||
1. **Bump your environment.** Raise any pin below `pydantic>=2.12`; upgrade FastAPI if your resolver complains about Starlette `<1.0.1`.
|
||||
2. **Fix imports that moved out.** Replace `from mcp.types import X` with `from mcp_types import X`, and update any import from the [removed modules](#moved-imports) (`fastmcp.server.proxy`, `fastmcp.server.openapi`, `fastmcp.server.apps`, the `fastmcp.tools.tool` / `resources.resource` / `prompts.prompt` component shims).
|
||||
3. **Update removed server APIs.** Swap `as_proxy` → `create_proxy`, `import_server` → `mount`, `mount(prefix=)` → `mount(namespace=)`, and the [other removed methods and keywords](#removed-server-methods-and-mount-keywords).
|
||||
4. **Replace `ctx.sample` and `ctx.list_roots`.** Both are gone from `Context`, as are `FastMCP(sampling_handler=...)` and `sampling_handler_behavior=`. Call an LLM directly from your server for generation; take file paths as tool arguments for roots.
|
||||
4. **Replace `ctx.sample` and `ctx.list_roots`.** Both are gone from `Context`, as are `FastMCP(sampling_handler=...)` and `sampling_handler_behavior=`. Call an LLM directly from your server for generation; ask for roots through the guard pattern, or take file paths as tool arguments.
|
||||
5. **Update removed tool parameters.** Replace tool `serializer=` (return a `ToolResult`), `exclude_args=` (use `Depends()`), and `StreamableHttpTransport(sse_read_timeout=)`.
|
||||
6. **Fix `McpError` construction.** Positional `McpError(ErrorData(...))` becomes keyword `McpError(code=..., message=...)`. Catching is unchanged.
|
||||
7. **Move httpx to httpx2.** Grep for `except httpx.` and for custom `httpx_client_factory` / `httpx.Auth` objects handed to FastMCP, and swap the import to `httpx2`.
|
||||
|
|
|
|||
|
|
@ -25,7 +25,7 @@ A FastMCP 4 server answers clients across the protocol transition from one deplo
|
|||
|
||||
The same negotiation runs from the client, and its default flipped. A plain `Client(url)` now probes for the modern protocol and adopts it when the server offers it, falling back to the handshake otherwise — where every earlier FastMCP version pinned the handshake outright. That flip is what brings the modern capabilities within reach of ordinary client code: a task-enabled tool hands back a handle to poll, and multi-round-trip elicitation resolves across successive requests, neither requiring the caller to opt in. Set `mode="legacy"` to pin the handshake when you need the session-based back-channel or the classic `initialize` result. See [Protocol negotiation](/clients/client#protocol-negotiation).
|
||||
|
||||
The modern protocol is sessionless, so it drops the server's ability to call back into the client mid-request (SEP-2577), and FastMCP 4's server API reflects that rather than papering over it. `ctx.elicit` moves to a request-shaped pattern that works on modern connections. `ctx.sample`, `ctx.sample_step`, and `ctx.list_roots` are not in the API at all — the capabilities they wrapped no longer exist in the protocol FastMCP 4 targets, so shipping methods that only work against old clients would be shipping a trap. [Call an LLM from your server](/servers/sampling) for generation, and take file paths as tool arguments for roots. Logging is untouched: `ctx.info` and its siblings are notifications, which ride the response stream and reach the client on every era. Everything else about writing a server is unchanged.
|
||||
The modern protocol is sessionless, so it drops the server's ability to call back into the client mid-request (SEP-2577), and FastMCP 4's server API reflects that rather than papering over it. `ctx.elicit` moves to a request-shaped pattern that works on modern connections. `ctx.sample`, `ctx.sample_step`, and `ctx.list_roots` are not in the API at all — they pushed a request into a live connection, and shipping methods that only work against old clients would be shipping a trap. The capabilities behind them survive through the same request-shaped pattern: a tool returns a sampling or roots request and reads the answer on the next round. For roots that is the natural replacement, since one round trip buys the whole answer. For generation, [call an LLM from your server](/servers/sampling) — a loop of guard rounds spends the round-trip budget several times over. Logging is untouched: `ctx.info` and its siblings are notifications, which ride the response stream and reach the client on every era. Everything else about writing a server is unchanged.
|
||||
|
||||
## State without a session
|
||||
|
||||
|
|
|
|||
|
|
@ -154,7 +154,7 @@ See [User Elicitation](/servers/elicitation) for detailed examples and supported
|
|||
### Server-initiated requests
|
||||
|
||||
<Note>
|
||||
`Context` has no `sample()` or `list_roots()`. Both were server→client *requests*, and the modern MCP protocol has no channel to carry them ([SEP-2577](/servers/sampling)). For generation, [call an LLM directly from your server](/servers/sampling). For roots, accept paths as tool arguments, or ask for them through the [guard pattern](/servers/elicitation#elicitation-on-the-modern-protocol), which carries a roots request in `input_requests`.
|
||||
`Context` has no `sample()` or `list_roots()`. Both *pushed* a request into a live client connection, and the modern MCP protocol has no channel to carry that ([SEP-2577](/servers/sampling)). Both capabilities remain reachable through the [guard pattern](/servers/elicitation#sampling-and-roots), where a tool returns a sampling or roots request in `input_requests` and reads the answer from `ctx.input_responses` on the next round. That is the natural route for roots. For generation, [call an LLM directly from your server](/servers/sampling) instead — each guard round costs a full round trip.
|
||||
|
||||
Logging is unaffected — `ctx.info()` and friends are *notifications*, which ride the response stream and work on every protocol era.
|
||||
</Note>
|
||||
|
|
|
|||
|
|
@ -596,9 +596,9 @@ The same protocol requirement applies: returning an `InputRequiredResult` from a
|
|||
|
||||
### Sampling and roots
|
||||
|
||||
Elicitation is the most common request to carry this way, and **roots** requests work identically — the `input_requests` map holds them the same way, and each answer comes back in `ctx.input_responses` under its key (an `ElicitResult` or `ListRootsResult`). See [Client Roots](/clients/roots) for what a roots request contains. `fastmcp.Client` answers both from the handlers you already configured, so a guard tool that mixes them needs no extra client wiring.
|
||||
Elicitation is the most common request to carry this way, and **roots** and **sampling** requests work identically — the `input_requests` map holds a `ListRootsRequest` or a `CreateMessageRequest` the same way it holds an `ElicitRequest`, and each answer comes back in `ctx.input_responses` under its key as a `ListRootsResult` or `CreateMessageResult`. A single map can mix all three. `fastmcp.Client` answers each one from the handlers you already configured — `elicitation_handler=`, `roots=`, `sampling_handler=` — so a guard tool that mixes them needs no extra client wiring. See [Client Roots](/clients/roots) for what a roots request contains.
|
||||
|
||||
The map can structurally hold a **sampling** request too (its answer would be a `CreateMessageResult`), but MCP removed server-initiated sampling as a pattern rather than only one spelling of it, so generation belongs in your server. See [Sampling](/servers/sampling) for how to call an LLM directly.
|
||||
The two differ in when you should reach for them. Roots suits this pattern well: a server asks once and has what it needs, so one extra round trip buys the whole answer. Generation usually does not, because every round is a full request-response cycle and a loop pays that cost each time — [call an LLM directly from your server](/servers/sampling) unless the point is specifically to use the caller's model.
|
||||
|
||||
### Middleware
|
||||
|
||||
|
|
|
|||
|
|
@ -1,13 +1,15 @@
|
|||
---
|
||||
title: Sampling
|
||||
sidebarTitle: Sampling
|
||||
description: Server-initiated sampling is not part of FastMCP 4 — here is why, and what to build instead.
|
||||
description: Generate text from a FastMCP server — by calling an LLM directly, or by asking the client to sample.
|
||||
icon: robot
|
||||
---
|
||||
|
||||
FastMCP 4 targets the modern MCP protocol, and that protocol has no channel for a server to send a request to a client. A tool cannot pause mid-execution to borrow the caller's model and wait for a completion, so server-initiated sampling is not part of the FastMCP 4 server API. There is no `ctx.sample()` and no server-side sampling handler. Generation belongs to your server now: you call an LLM with your own credentials, the same way you would call any other service.
|
||||
FastMCP 4 targets the modern MCP protocol, and that protocol has no channel for a server to push a request into a live client connection. A tool cannot pause mid-execution to borrow the caller's model and resume when the completion arrives, so the imperative methods that did exactly that are gone: there is no `ctx.sample()` and no `ctx.sample_step()`, and `FastMCP()` no longer accepts `sampling_handler=`. Calling them raises `AttributeError` on every protocol era rather than failing at runtime only against modern clients.
|
||||
|
||||
This follows the protocol rather than getting ahead of it. MCP removed server-initiated requests in the `2026-07-28` revision ([SEP-2577](https://modelcontextprotocol.io/community/sep-guidelines)), and FastMCP 4's client negotiates that revision by default. Keeping `ctx.sample()` around would mean shipping a method whose ordinary, default outcome is a runtime error.
|
||||
The *capability* survives in a different shape. A tool that genuinely needs the caller's model asks for a completion by **returning** a request for one, the same [guard pattern](/servers/elicitation#elicitation-on-the-modern-protocol) elicitation uses: the round ends as an ordinary response, the client runs the completion against its own model, and it calls the tool again with the answer attached. FastMCP's own conformance suite exercises this on `2026-07-28`.
|
||||
|
||||
For most generation you should skip that round trip and call an LLM directly from your server. Each guard round is a full request-response cycle, so a tool that generates in a loop spends the round-trip budget several times over, while a direct call is an ordinary async function call inside a single tool invocation. Ask the client to sample when the point is to use *the caller's* model — their credentials, their choice of provider, their bill. Call an LLM yourself when the point is to generate text.
|
||||
|
||||
## Requests and notifications
|
||||
|
||||
|
|
@ -27,11 +29,64 @@ async def build_report(rows: int, ctx: Context) -> str:
|
|||
return "done"
|
||||
```
|
||||
|
||||
Sampling is the other kind. It is a *request* — the server sends `sampling/createMessage` and then blocks until an answer comes back the other way. That requires a live, addressable connection the server can reach into, which is precisely the thing a stateless protocol does not have. There is no version of sampling that fits, which is why it has no replacement in the way elicitation does. Elicitation moved to the [guard pattern](/servers/elicitation#elicitation-on-the-modern-protocol), where a tool *returns* a description of what it needs and the client answers with a fresh call; generation does not decompose into rounds that way, because an agentic loop would spend the round-trip budget several times over.
|
||||
Sampling is the other kind. It is a *request* — `sampling/createMessage` goes out and the caller must answer before the tool can continue. Pushing that request needs a live, addressable connection the server can reach into, which is precisely what a stateless protocol does not have, and MCP removed server-initiated requests in the `2026-07-28` revision ([SEP-2577](https://modelcontextprotocol.io/community/sep-guidelines)) for that reason. What the protocol removed is the *pushing*, not the asking. Asking survives by inverting who initiates the next message: instead of the server reaching down mid-call, the tool returns a description of what it needs and the client comes back.
|
||||
|
||||
## Asking the client to sample
|
||||
|
||||
A tool asks for a completion by returning an `InputRequiredResult` whose `input_requests` map holds a `CreateMessageRequest` under a key you choose. That result completes the round normally. The client runs the completion, then re-issues the same `call_tool` with the answer attached, and your tool reads it from `ctx.input_responses` under the same key — a `CreateMessageResult`. Because the tool runs from the top on every round, the presence of `ctx.input_responses` is what tells the two rounds apart: `None` on the first call, populated on the continuation.
|
||||
|
||||
`fastmcp.Client` drives this loop for you and answers from the [`sampling_handler`](/clients/sampling) you already registered, so a client configured for a handshake-era server needs no extra wiring to satisfy a modern guard tool.
|
||||
|
||||
```python
|
||||
from fastmcp import Context, FastMCP
|
||||
from mcp_types import (
|
||||
CreateMessageRequest,
|
||||
CreateMessageRequestParams,
|
||||
CreateMessageResult,
|
||||
InputRequiredResult,
|
||||
SamplingMessage,
|
||||
TextContent,
|
||||
)
|
||||
|
||||
mcp = FastMCP("Research")
|
||||
|
||||
|
||||
@mcp.tool
|
||||
async def ask_the_caller(question: str, ctx: Context) -> str | InputRequiredResult:
|
||||
"""Put a question to the caller's model and report what it answered."""
|
||||
responses = ctx.input_responses
|
||||
if responses is None:
|
||||
return InputRequiredResult(
|
||||
result_type="input_required",
|
||||
input_requests={
|
||||
"answer": CreateMessageRequest(
|
||||
method="sampling/createMessage",
|
||||
params=CreateMessageRequestParams(
|
||||
messages=[
|
||||
SamplingMessage(
|
||||
role="user",
|
||||
content=TextContent(type="text", text=question),
|
||||
)
|
||||
],
|
||||
max_tokens=100,
|
||||
),
|
||||
)
|
||||
},
|
||||
)
|
||||
|
||||
answer = responses["answer"]
|
||||
if isinstance(answer, CreateMessageResult) and isinstance(
|
||||
answer.content, TextContent
|
||||
):
|
||||
return answer.content.text
|
||||
return "The client returned no completion."
|
||||
```
|
||||
|
||||
Returning an `InputRequiredResult` needs a `2026-07-28` connection; FastMCP names the era mismatch if an older client reaches the tool. The map can carry several requests at once and mix kinds — a sampling request beside an elicitation or a roots request — and each answer comes back under its own key. [Elicitation](/servers/elicitation#sampling-and-roots) covers the mechanics of the pattern in full, including how to carry state across rounds.
|
||||
|
||||
## Calling an LLM directly
|
||||
|
||||
Your server calls the model. Hold a provider API key in your server's environment, create the client once at module scope so connections are reused across calls, and generate inside the tool. The result is a plain async function call with no protocol involvement, which also means you choose the model, control the prompt, see the token usage, and can test the tool without a client attached.
|
||||
For generation, your server calls the model. Hold a provider API key in your server's environment, create the client once at module scope so connections are reused across calls, and generate inside the tool. The result is a plain async function call with no protocol involvement, which also means you choose the model, control the prompt, see the token usage, and can test the tool without a client attached.
|
||||
|
||||
```python
|
||||
import anthropic
|
||||
|
|
@ -53,13 +108,13 @@ async def summarize(text: str) -> str:
|
|||
return response.content[0].text
|
||||
```
|
||||
|
||||
Any provider SDK works the same way — swap the client and the call, and the tool signature is unchanged. Because generation is now ordinary application code, the surrounding concerns become ordinary too: retries, timeouts, caching, and cost accounting are yours to place where you want them rather than negotiated across a protocol boundary.
|
||||
Any provider SDK works the same way — swap the client and the call, and the tool signature is unchanged. Because generation is now ordinary application code, the surrounding concerns become ordinary too: retries, timeouts, caching, and cost accounting are yours to place where you want them rather than negotiated across a protocol boundary. A tool that chains several generations pays nothing extra for the second and third, where the guard route would pay a full round trip for each.
|
||||
|
||||
The trade this makes is explicit. Sampling let a server borrow the caller's model and the caller's bill; calling directly means you supply the key and pay for the tokens. In exchange your tool behaves identically for every client, including the many that never implemented sampling at all.
|
||||
|
||||
## Clients answering servers
|
||||
|
||||
The client half of sampling is unaffected. A `fastmcp.Client` connecting to a handshake-era server may still receive `sampling/createMessage` requests from it, and passing `sampling_handler=` is how you answer them — see [Sampling](/clients/sampling) under Clients. That path exists for interoperating with older servers and has nothing to do with authoring one.
|
||||
Both routes land in the same place on the client. A `fastmcp.Client` passes `sampling_handler=` once, and that handler answers a handshake-era server's pushed `sampling/createMessage` request and a modern server's returned sampling request alike — see [Sampling](/clients/sampling) under Clients.
|
||||
|
||||
<Note>
|
||||
Servers on FastMCP 3 still have `ctx.sample()` and `ctx.sample_step()`, documented in the [FastMCP 3 sampling guide](/v3/servers/sampling). Nothing changes for them until they upgrade.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue