mirror of
https://github.com/PrefectHQ/fastmcp.git
synced 2026-08-22 05:24:18 +02:00
Document sampling handlers as client feature, reference from server docs
- Add Pre-built Handlers section to clients/sampling.mdx with Anthropic and OpenAI - Simplify servers/sampling.mdx to reference client docs for handler details - Add examples for Azure OpenAI and local model providers
This commit is contained in:
Jeremiah Lowin
parent
d34f065417
commit
179874628e
2 changed files with 99 additions and 95 deletions
|
|
@ -269,4 +269,93 @@ async def sampling_handler_with_tools(
|
|||
|
||||
<Note>
|
||||
Tool execution happens on the server side. The client's role is to pass tools to the LLM and return the LLM's response (which may include tool use requests). The server then executes the tools and may send follow-up sampling requests with tool results.
|
||||
</Note>
|
||||
</Note>
|
||||
|
||||
## Pre-built Handlers
|
||||
|
||||
<VersionBadge version="2.14.0" />
|
||||
|
||||
FastMCP provides ready-to-use sampling handlers for Anthropic and OpenAI that handle all the complexity of message conversion, tool formatting, and response parsing.
|
||||
|
||||
### Anthropic Handler
|
||||
|
||||
The Anthropic handler uses the Claude API:
|
||||
|
||||
```python
|
||||
from anthropic import Anthropic
|
||||
from fastmcp import Client
|
||||
from fastmcp.server.sampling.anthropic import AnthropicSamplingHandler
|
||||
|
||||
handler = AnthropicSamplingHandler(
|
||||
default_model="claude-sonnet-4-20250514",
|
||||
client=Anthropic(), # Uses ANTHROPIC_API_KEY env var
|
||||
)
|
||||
|
||||
async with Client("server.py", sampling_handler=handler) as client:
|
||||
result = await client.call_tool("summarize", {"text": "..."})
|
||||
```
|
||||
|
||||
The handler automatically:
|
||||
- Converts MCP messages to Anthropic's format
|
||||
- Translates tool definitions to Anthropic's tool format
|
||||
- Maps stop reasons (`tool_use` → `toolUse`, `end_turn` → `endTurn`)
|
||||
- Selects models based on server preferences (any model starting with `claude` is accepted)
|
||||
|
||||
<Note>
|
||||
Requires the `anthropic` package. Install with `pip install fastmcp[anthropic]` or `pip install anthropic`.
|
||||
</Note>
|
||||
|
||||
### OpenAI Handler
|
||||
|
||||
The OpenAI handler works with the OpenAI API and compatible providers:
|
||||
|
||||
```python
|
||||
from openai import OpenAI
|
||||
from fastmcp import Client
|
||||
from fastmcp.server.sampling.openai import OpenAISamplingHandler
|
||||
|
||||
handler = OpenAISamplingHandler(
|
||||
default_model="gpt-4o-mini",
|
||||
client=OpenAI(), # Uses OPENAI_API_KEY env var
|
||||
)
|
||||
|
||||
async with Client("server.py", sampling_handler=handler) as client:
|
||||
result = await client.call_tool("analyze", {"data": "..."})
|
||||
```
|
||||
|
||||
The handler automatically:
|
||||
- Converts MCP messages to OpenAI's chat format
|
||||
- Translates tool definitions to OpenAI's function calling format
|
||||
- Maps stop reasons (`tool_calls` → `toolUse`, `stop` → `endTurn`)
|
||||
- Selects models based on server preferences
|
||||
|
||||
<Note>
|
||||
Requires the `openai` package. Install with `pip install fastmcp[openai]` or `pip install openai`.
|
||||
</Note>
|
||||
|
||||
### Using with Compatible Providers
|
||||
|
||||
The OpenAI handler works with any OpenAI-compatible API:
|
||||
|
||||
```python
|
||||
from openai import OpenAI
|
||||
from fastmcp.server.sampling.openai import OpenAISamplingHandler
|
||||
|
||||
# Azure OpenAI
|
||||
handler = OpenAISamplingHandler(
|
||||
default_model="gpt-4o",
|
||||
client=OpenAI(
|
||||
api_key=os.getenv("AZURE_OPENAI_API_KEY"),
|
||||
base_url="https://your-resource.openai.azure.com/openai/deployments/your-deployment",
|
||||
),
|
||||
)
|
||||
|
||||
# Local models (Ollama, vLLM, etc.)
|
||||
handler = OpenAISamplingHandler(
|
||||
default_model="llama3",
|
||||
client=OpenAI(
|
||||
api_key="not-needed",
|
||||
base_url="http://localhost:11434/v1",
|
||||
),
|
||||
)
|
||||
```
|
||||
|
|
@ -455,12 +455,24 @@ However, you can provide a `sampling_handler` to the FastMCP server, which sends
|
|||
- **`"fallback"`** (default): Uses the handler only when the client doesn't support sampling. Requests go to the client first, falling back to the handler if needed.
|
||||
- **`"always"`**: Always uses the handler, bypassing the client entirely. Useful when you want full control over the LLM used for sampling.
|
||||
|
||||
### OpenAI Handler
|
||||
### Using Pre-built Handlers
|
||||
|
||||
FastMCP provides an OpenAI-compatible sampling handler that supports both basic sampling and tool use:
|
||||
The server fallback handler uses the same signature as client sampling handlers. FastMCP provides pre-built handlers for [Anthropic and OpenAI](/clients/sampling#pre-built-handlers) that work in both contexts:
|
||||
|
||||
```python
|
||||
from fastmcp import FastMCP
|
||||
from fastmcp.server.sampling.anthropic import AnthropicSamplingHandler
|
||||
|
||||
server = FastMCP(
|
||||
name="Sampling Server",
|
||||
sampling_handler=AnthropicSamplingHandler(
|
||||
default_model="claude-sonnet-4-20250514",
|
||||
),
|
||||
sampling_handler_behavior="fallback",
|
||||
)
|
||||
```
|
||||
|
||||
```python
|
||||
import os
|
||||
from openai import OpenAI
|
||||
from fastmcp import FastMCP
|
||||
from fastmcp.server.sampling.openai import OpenAISamplingHandler
|
||||
|
|
@ -469,109 +481,12 @@ server = FastMCP(
|
|||
name="Sampling Server",
|
||||
sampling_handler=OpenAISamplingHandler(
|
||||
default_model="gpt-4o-mini",
|
||||
client=OpenAI(api_key=os.getenv("OPENAI_API_KEY")),
|
||||
client=OpenAI(),
|
||||
),
|
||||
sampling_handler_behavior="fallback",
|
||||
sampling_handler_behavior="always", # Always use server's LLM
|
||||
)
|
||||
```
|
||||
|
||||
The OpenAI handler automatically handles tool conversion - when you pass tools to `ctx.sample()`, they're converted to OpenAI's function calling format.
|
||||
|
||||
### Fallback Mode (Default)
|
||||
|
||||
Uses the handler only when the client doesn't support sampling:
|
||||
|
||||
```python
|
||||
import asyncio
|
||||
import os
|
||||
from openai import OpenAI
|
||||
|
||||
from fastmcp import FastMCP, Context
|
||||
from fastmcp.server.sampling.openai import OpenAISamplingHandler
|
||||
|
||||
|
||||
async def async_main():
|
||||
server = FastMCP(
|
||||
name="OpenAI Sampling Fallback Example",
|
||||
sampling_handler=OpenAISamplingHandler(
|
||||
default_model="gpt-4o-mini",
|
||||
client=OpenAI(api_key=os.getenv("OPENAI_API_KEY")),
|
||||
),
|
||||
sampling_handler_behavior="fallback",
|
||||
)
|
||||
|
||||
@server.tool
|
||||
async def summarize(text: str, ctx: Context) -> str:
|
||||
# Uses client's LLM if available, falls back to OpenAI
|
||||
result = await ctx.sample(
|
||||
messages=f"Summarize: {text}",
|
||||
)
|
||||
return result.text
|
||||
|
||||
await server.run_http_async()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(async_main())
|
||||
```
|
||||
|
||||
### Always Mode
|
||||
|
||||
Always uses the handler, bypassing the client:
|
||||
|
||||
```python
|
||||
server = FastMCP(
|
||||
name="Server-Controlled Sampling",
|
||||
sampling_handler=OpenAISamplingHandler(
|
||||
default_model="gpt-4o-mini",
|
||||
client=OpenAI(api_key=os.getenv("API_KEY")),
|
||||
),
|
||||
sampling_handler_behavior="always",
|
||||
)
|
||||
|
||||
@server.tool
|
||||
async def analyze_data(data: str, ctx: Context) -> str:
|
||||
# Always uses the server's configured LLM
|
||||
result = await ctx.sample(
|
||||
messages=f"Analyze this data: {data}",
|
||||
system_prompt="You are a data analyst.",
|
||||
)
|
||||
return result.text
|
||||
```
|
||||
|
||||
### Fallback with Tools
|
||||
|
||||
The fallback handler fully supports sampling with tools:
|
||||
|
||||
```python
|
||||
from fastmcp import FastMCP, Context
|
||||
from fastmcp.server.sampling.openai import OpenAISamplingHandler
|
||||
from openai import OpenAI
|
||||
import os
|
||||
|
||||
def search(query: str) -> str:
|
||||
"""Search for information."""
|
||||
return f"Results for: {query}"
|
||||
|
||||
server = FastMCP(
|
||||
name="Agentic Server",
|
||||
sampling_handler=OpenAISamplingHandler(
|
||||
default_model="gpt-4o-mini",
|
||||
client=OpenAI(api_key=os.getenv("OPENAI_API_KEY")),
|
||||
),
|
||||
sampling_handler_behavior="always",
|
||||
)
|
||||
|
||||
@server.tool
|
||||
async def research(question: str, ctx: Context) -> str:
|
||||
"""Research a topic using tools."""
|
||||
result = await ctx.sample(
|
||||
messages=question,
|
||||
tools=[search],
|
||||
)
|
||||
return result.text
|
||||
```
|
||||
|
||||
## Client Requirements
|
||||
|
||||
By default, LLM sampling requires client support:
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue