---
title: LLM Sampling
sidebarTitle: Sampling
description: Answer a server's request for an LLM completion.
icon: robot
---

import { VersionBadge } from "/snippets/version-badge.mdx";

<VersionBadge version="2.0.0" />

Use this when a server asks your client to run an LLM completion on its behalf.

Sampling is how a server borrows your model. Rather than hold an API key of its own, the server describes the messages it wants completed and asks you to run them — you pick the model, and you pay for the tokens. Your side of that arrangement is one function, a **sampling handler**, registered when you create the client.

The handler receives the conversation the server wants completed, the parameters it asked for, and a request context carrying metadata about the call. Return the generated text as a string and FastMCP wraps it in the protocol's result for you; return a `CreateMessageResult` yourself when you want to report the real model name or hand back content that isn't text. If the handler raises, the client sends the error back in place of a completion and the server's tool decides what to do about it.

## Handler Template

```python
from fastmcp import Client
from fastmcp.client.sampling import SamplingMessage, SamplingParams, RequestContext
from mcp.types import TextContent


async def sampling_handler(
    messages: list[SamplingMessage],
    params: SamplingParams,
    context: RequestContext,
) -> str:
    """Run the server's messages against your LLM and return the completion."""
    conversation = [
        f"{message.role}: {message.content.text}"
        for message in messages
        if isinstance(message.content, TextContent)
    ]
    system_prompt = params.system_prompt or "You are a helpful assistant."

    # Call your LLM here with `conversation` and `system_prompt`.
    return "Generated response based on the messages"


client = Client("my_mcp_server.py", sampling_handler=sampling_handler)
```

The client answers with this handler however the server asks for a completion. The default `mode="auto"` negotiates whichever protocol era the server speaks, and one handler covers both of the routes an era can use — see [Request Routes](#request-routes).

## Handler Parameters

Everything the server sends arrives in the first two arguments. The messages are the conversation to complete; the parameters are how the server would like it completed. You decide how much of that to honor, since the client owns the model — a preference your provider cannot express is yours to ignore.

<Card icon="code" title="SamplingMessage">
<ResponseField name="role" type='Literal["user", "assistant"]'>
  The role of the message
</ResponseField>

<ResponseField name="content" type="TextContent | ImageContent | AudioContent">
  The content of the message. TextContent has a `.text` attribute.
</ResponseField>
</Card>

<Card icon="code" title="SamplingParams">
<ResponseField name="system_prompt" type="str | None">
  Optional system prompt the server wants to use
</ResponseField>

<ResponseField name="model_preferences" type="ModelPreferences | None">
  Server preferences for model selection (hints, cost/speed/intelligence priorities)
</ResponseField>

<ResponseField name="temperature" type="float | None">
  Sampling temperature
</ResponseField>

<ResponseField name="max_tokens" type="int">
  Maximum tokens to generate
</ResponseField>

<ResponseField name="stop_sequences" type="list[str] | None">
  Stop sequences for sampling
</ResponseField>

<ResponseField name="tools" type="list[Tool] | None">
  Tools the LLM can use during sampling
</ResponseField>

<ResponseField name="tool_choice" type="ToolChoice | None">
  Tool usage behavior (`auto`, `required`, or `none`)
</ResponseField>
</Card>

## Built-in Handlers

Writing the provider call yourself is rarely worth it. FastMCP ships handlers for OpenAI, Anthropic, and Google Gemini that implement the full sampling API, tool use included, and translate the protocol's parameters into each provider's own. Give one a default model and pass it where your own handler would go. Write a custom handler when you need routing across providers, caching, or a provider FastMCP does not cover.

### OpenAI Handler

<VersionBadge version="2.11.0" />

```python
from fastmcp import Client
from fastmcp.client.sampling.handlers.openai import OpenAISamplingHandler

client = Client(
    "my_mcp_server.py",
    sampling_handler=OpenAISamplingHandler(default_model="gpt-4o"),
)
```

Point the handler at any OpenAI-compatible API, including a local model server, by passing your own provider client:

```python
from fastmcp import Client
from fastmcp.client.sampling.handlers.openai import OpenAISamplingHandler
from openai import AsyncOpenAI

client = Client(
    "my_mcp_server.py",
    sampling_handler=OpenAISamplingHandler(
        default_model="llama-3.1-70b",
        client=AsyncOpenAI(base_url="http://localhost:8000/v1"),
    ),
)
```

<Note>
Install the OpenAI handler with `pip install 'fastmcp[openai]'`.
</Note>

### Anthropic Handler

<VersionBadge version="2.14.1" />

```python
from fastmcp import Client
from fastmcp.client.sampling.handlers.anthropic import AnthropicSamplingHandler

client = Client(
    "my_mcp_server.py",
    sampling_handler=AnthropicSamplingHandler(default_model="claude-sonnet-4-5"),
)
```

<Note>
Install the Anthropic handler with `pip install 'fastmcp[anthropic]'`.
</Note>

### Google Gemini Handler

<VersionBadge version="3.1.0" />

```python
from fastmcp import Client
from fastmcp.client.sampling.handlers.google_genai import GoogleGenaiSamplingHandler

client = Client(
    "my_mcp_server.py",
    sampling_handler=GoogleGenaiSamplingHandler(default_model="gemini-2.0-flash"),
)
```

<Note>
Install the Google Gemini handler with `pip install 'fastmcp[gemini]'`.
</Note>

The [source of these handlers](https://github.com/PrefectHQ/fastmcp/tree/main/fastmcp_slim/fastmcp/client/sampling/handlers) is the best reference for writing your own.

## Tool Use

A sampling request can carry tools. When it does, your handler passes them to the model and returns whatever comes back, tool calls included — the server executes the tools itself and sends a follow-up sampling request with the results if it needs another turn. Your handler never runs a tool.

Registering any `sampling_handler` advertises full sampling support, tools included. A handler that only generates text should say so, so servers know not to send tools it will drop:

```python
from fastmcp import Client
from mcp.types import SamplingCapability


async def text_only_handler(messages, params, context) -> str:
    return "Generated response based on the messages"


client = Client(
    "my_mcp_server.py",
    sampling_handler=text_only_handler,
    sampling_capabilities=SamplingCapability(),
)
```

## Request Routes

Servers reach your handler by two routes, and which one applies depends on the protocol era the connection negotiated. A handshake-era server pushes a `sampling/createMessage` request down the open session while a tool is running and waits for the reply. A modern (`2026-07-28`) connection has no such channel, so the tool ends its round by returning a request for a completion instead; the client answers from your handler and calls the tool again with the result attached.

One registration covers both, so this is rarely something you configure — it matters only when you pin an era, since `mode="legacy"` is the sole route that carries a pushed request. See [protocol negotiation](/clients/client#protocol-negotiation) for how the era is chosen, and [Sampling](/servers/sampling) under Servers for how a server issues these requests.