Merge branch 'main' into claude/issue-2055-20251010-2356

This commit is contained in:
William Easton 2025-10-12 09:34:50 -04:00 committed by GitHub
commit 2611b7c353
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
4 changed files with 153 additions and 307 deletions

View file

@ -1,246 +0,0 @@
# FastMCP OpenAPI Integration
This document explains how FastMCP's OpenAPI integration works, what features are supported, and how to extend it. The OpenAPI functionality is split across two main files:
- `server/openapi.py` - High-level FastMCP server implementation and MCP component creation
- `utilities/openapi.py` - Low-level OpenAPI parsing and intermediate representation
## Architecture Overview
```
OpenAPI Spec → Parse → HTTPRoute IR → Create MCP Components → FastMCP Server
```
### 1. Parsing Phase (`utilities/openapi.py`)
OpenAPI specifications are parsed into an intermediate representation (IR) that normalizes differences between OpenAPI 3.0 and 3.1:
- **Input**: Raw OpenAPI spec (dict)
- **Output**: List of `HTTPRoute` objects with normalized parameter information
- **Key Classes**:
- `HTTPRoute` - Represents a single operation
- `ParameterInfo` - Represents a parameter with location, style, explode, etc.
- `RequestBodyInfo` - Represents request body information
- `ResponseInfo` - Represents response information
### 2. Component Creation Phase (`server/openapi.py`)
HTTPRoute objects are converted into FastMCP components based on route mapping rules:
- **Tools** (`OpenAPITool`) - HTTP operations that can be called
- **Resources** (`OpenAPIResource`) - HTTP endpoints that return data
- **Resource Templates** (`OpenAPIResourceTemplate`) - Parameterized resources
## Parameter Handling
FastMCP supports various OpenAPI parameter serialization styles and formats:
### Supported Parameter Locations
- `query` - Query string parameters
- `path` - Path parameters
- `header` - HTTP headers
- `cookie` - Cookie parameters (parsed but not used in requests)
### Supported Parameter Styles
#### Query Parameters
- **`form`** (default) - Standard query parameter format
- `explode=true` (default): `?tags=red&tags=blue`
- `explode=false`: `?tags=red,blue`
- **`deepObject`** - Object parameters with bracket notation
- `explode=true`: `?filter[name]=John&filter[age]=30`
- `explode=false`: Falls back to JSON string (non-standard, logs warning)
#### Path Parameters
- **`simple`** (default) - Comma-separated for arrays: `/users/1,2,3`
#### Header Parameters
- **`simple`** (default) - Standard header format
### Parameter Type Support
#### Arrays
- String arrays with `explode=true/false`
- Number arrays with `explode=true/false`
- Boolean arrays with `explode=true/false`
- Complex object arrays (basic support, may not handle all cases)
#### Objects
- Objects with `deepObject` style and `explode=true`
- Objects with other styles fall back to JSON serialization
#### Primitives
- Strings, numbers, booleans
- Enums
- Default values
## Request Body Handling
### Supported Content Types
- `application/json` - JSON request bodies
### Schema Support
- Object schemas with properties
- Array schemas
- Primitive schemas
- Schema references (`$ref` to local schemas only)
- Required properties
- Default values
## Response Handling
### Content Type Detection
- `application/json` - Parsed as JSON
- `text/*` - Returned as text
- `application/xml` - Returned as text
- Other types - Returned as binary
### Output Schema Generation
- Success response schemas (200, 201, 202, 204)
- Object response wrapping for MCP compliance
- Schema compression (removes unused `$defs`)
## Route Mapping
Routes are mapped to MCP component types using `RouteMap` configurations:
```python
RouteMap(
methods=["GET", "POST"], # HTTP methods to match
pattern=r"/api/users/.*", # Regex pattern for path
mcp_type=MCPType.RESOURCE_TEMPLATE, # Target component type
tags={"user"}, # OpenAPI tags to match (AND condition)
mcp_tags={"fastmcp-user"} # Tags to add to created components
)
```
### Default Behavior
- All routes become **Tools** by default
- Use route maps to override specific patterns
### Component Types
- `MCPType.TOOL` - Callable operations
- `MCPType.RESOURCE` - Static data endpoints
- `MCPType.RESOURCE_TEMPLATE` - Parameterized data endpoints
- `MCPType.EXCLUDE` - Skip route entirely
## Known Limitations & Edge Cases
### Parameter Edge Cases
1. **Parameter Name Collisions** - When path/query parameters have same names as request body properties, non-body parameters get `__location` suffixes
2. **Complex Array Serialization** - Limited support for arrays containing objects
3. **Cookie Parameters** - Parsed but not used in requests
4. **Non-standard Combinations** - e.g., `deepObject` with `explode=false`
### Request Body Edge Cases
1. **Content Type Priority** - Only first available content type is used
2. **Nested Objects** - Deep nesting may not serialize correctly
3. **Binary Content** - No support for file uploads or binary data
### Response Edge Cases
1. **Multiple Content Types** - Only JSON-compatible types are used for output schemas
2. **Error Responses** - Not used for MCP output schema generation
3. **Response Headers** - Not captured or exposed
### Schema Edge Cases
1. **External References** - `$ref` to external files not supported
2. **Circular References** - May cause issues in schema processing
3. **Polymorphism** - `oneOf`/`anyOf`/`allOf` limited support
## Debugging Tips
### Common Issues
1. **"Unknown tool/resource"** - Check route mapping configuration
2. **Parameter not found** - Check for name collisions or incorrect style/explode
3. **Invalid request format** - Check parameter serialization and content types
4. **Schema validation errors** - Check for external refs or complex schemas
### Debugging Tools
```python
# Parse routes to inspect intermediate representation
routes = parse_openapi_to_http_routes(openapi_spec)
for route in routes:
print(f"{route.method} {route.path}")
for param in route.parameters:
print(f" {param.name} ({param.location}): style={param.style}, explode={param.explode}")
# Check component creation
server = FastMCP.from_openapi(openapi_spec, client)
tools = await server.get_tools()
print(f"Created {len(tools)} tools: {list(tools.keys())}")
```
### Logging
- Set `FASTMCP_LOG_LEVEL=DEBUG` to see detailed parameter processing
- Look for warnings about non-standard parameter combinations
- Check for schema parsing errors in logs
## Extension Points
### Adding New Parameter Styles
1. Add style handling in `utilities/openapi.py` - `ParameterInfo` class
2. Implement serialization logic in `server/openapi.py` - `OpenAPITool.run()`
3. Add tests for parsing and serialization
### Adding New Content Types
1. Extend request body handling in `OpenAPITool.run()`
2. Add response parsing logic for new types
3. Update content type priority in utilities
### Custom Route Mapping
Use `route_map_fn` for complex routing logic:
```python
def custom_mapper(route: HTTPRoute, current_type: MCPType) -> MCPType:
if route.path.startswith("/admin"):
return MCPType.EXCLUDE
return current_type
server = FastMCP.from_openapi(spec, client, route_map_fn=custom_mapper)
```
## Testing Patterns
### Unit Tests
- Test parameter parsing with various styles/explode combinations
- Test route mapping with different patterns and tags
- Test schema generation and compression
### Integration Tests
- Mock HTTP client to verify actual request parameters
- Test end-to-end component creation and execution
- Test error handling and edge cases
### Example Test Pattern
```python
async def test_parameter_style():
# 1. Create OpenAPI spec with specific parameter configuration
spec = {"openapi": "3.1.0", ...}
# 2. Parse and create components
routes = parse_openapi_to_http_routes(spec)
tool = OpenAPITool(mock_client, routes[0], ...)
# 3. Execute and verify request parameters
await tool.run({"param": "value"})
actual_params = mock_client.request.call_args.kwargs["params"]
assert actual_params == expected_params
```
## Testing
OpenAPI functionality is tested across multiple files in `tests/server/openapi/`:
- `test_basic_functionality.py` - Core component creation and execution
- `test_explode_integration.py` - Parameter explode behavior
- `test_deepobject_style.py` - DeepObject style parameter encoding
- `test_parameter_collisions.py` - Parameter name collision handling
- `test_openapi_path_parameters.py` - Path parameter serialization
- `test_configuration.py` - Route mapping and MCP names
- `test_description_propagation.py` - Schema and description handling
When adding new OpenAPI features, create focused test files rather than adding to existing monolithic files.
---
*This document should be updated when new OpenAPI features are added or when edge cases are discovered and addressed.*

View file

@ -1,58 +0,0 @@
# Getting your development environment set up properly
To get your environment up and running properly, you'll need a slightly different set of commands that are windows specific:
```bash
uv venv
.venv\Scripts\activate
uv pip install -e ".[dev]"
```
This will install the package in editable mode, and install the development dependencies.
# Fixing `AttributeError: module 'collections' has no attribute 'Callable'`
- open `.venv\Lib\site-packages\pyreadline\py3k_compat.py`
- change `return isinstance(x, collections.Callable)` to
```
from collections.abc import Callable
return isinstance(x, Callable)
```
# Helpful notes
For developing FastMCP
## Install local development version of FastMCP into a local FastMCP project server
- ensure
- change directories to your FastMCP Server location so you can install it in your .venv
- run `.venv\Scripts\activate` to activate your virtual environment
- Then run a series of commands to uninstall the old version and install the new
```bash
# First uninstall
uv pip uninstall fastmcp
# Clean any build artifacts in your fastmcp directory
cd C:\path\to\fastmcp
del /s /q *.egg-info
# Then reinstall in your weather project
cd C:\path\to\new\fastmcp_server
uv pip install --no-cache-dir -e C:\Users\justj\PycharmProjects\fastmcp
# Check that it installed properly and has the correct git hash
pip show fastmcp
```
## Running the FastMCP server with Inspector
MCP comes with a node.js application called Inspector that can be used to inspect the FastMCP server. To run the inspector, you'll need to install node.js and npm. Then you can run the following commands:
```bash
fastmcp dev server.py
```
This will launch a web app on http://localhost:5173/ that you can use to inspect the FastMCP server.
## If you start development before creating a fork - your get out of jail free card
- Add your fork as a new remote to your local repository `git remote add fork git@github.com:YOUR-USERNAME/REPOSITORY-NAME.git`
- This will add your repo, short named 'fork', as a remote to your local repository
- Verify that it was added correctly by running `git remote -v`
- Commit your changes
- Push your changes to your fork `git push fork <branch>`
- Create your pull request on GitHub

View file

@ -8,7 +8,7 @@ import sys
import warnings
from collections.abc import AsyncIterator
from pathlib import Path
from typing import Any, Literal, TypeVar, cast, overload
from typing import Any, Literal, TextIO, TypeVar, cast, overload
import anyio
import httpx
@ -313,6 +313,7 @@ class StdioTransport(ClientTransport):
env: dict[str, str] | None = None,
cwd: str | None = None,
keep_alive: bool | None = None,
log_file: Path | TextIO | None = None,
):
"""
Initialize a Stdio transport.
@ -326,6 +327,11 @@ class StdioTransport(ClientTransport):
Defaults to True. When True, the subprocess remains active
after the connection context exits, allowing reuse in
subsequent connections.
log_file: Optional path or file-like object where subprocess stderr will
be written. Can be a Path or TextIO object. Defaults to sys.stderr
if not provided. When a Path is provided, the file will be created
if it doesn't exist, or appended to if it does. When set, server
errors will be written to this file instead of appearing in the console.
"""
self.command = command
self.args = args
@ -334,6 +340,7 @@ class StdioTransport(ClientTransport):
if keep_alive is None:
keep_alive = True
self.keep_alive = keep_alive
self.log_file = log_file
self._session: ClientSession | None = None
self._connect_task: asyncio.Task | None = None
@ -368,6 +375,7 @@ class StdioTransport(ClientTransport):
args=self.args,
env=self.env,
cwd=self.cwd,
log_file=self.log_file,
session_kwargs=session_kwargs,
ready_event=self._ready_event,
stop_event=self._stop_event,
@ -421,6 +429,7 @@ async def _stdio_transport_connect_task(
args: list[str],
env: dict[str, str] | None,
cwd: str | None,
log_file: Path | TextIO | None,
session_kwargs: SessionKwargs,
ready_event: anyio.Event,
stop_event: anyio.Event,
@ -438,7 +447,19 @@ async def _stdio_transport_connect_task(
env=env,
cwd=cwd,
)
transport = await stack.enter_async_context(stdio_client(server_params))
# Handle log_file: Path needs to be opened, TextIO used as-is
if log_file is None:
log_file_handle = sys.stderr
elif isinstance(log_file, Path):
log_file_handle = open(log_file, "a")
stack.callback(log_file_handle.close)
else:
# Must be TextIO - use it directly
log_file_handle = log_file
transport = await stack.enter_async_context(
stdio_client(server_params, errlog=log_file_handle)
)
read_stream, write_stream = transport
session_future.set_result(
await stack.enter_async_context(
@ -471,6 +492,7 @@ class PythonStdioTransport(StdioTransport):
cwd: str | None = None,
python_cmd: str = sys.executable,
keep_alive: bool | None = None,
log_file: Path | TextIO | None = None,
):
"""
Initialize a Python transport.
@ -485,6 +507,11 @@ class PythonStdioTransport(StdioTransport):
Defaults to True. When True, the subprocess remains active
after the connection context exits, allowing reuse in
subsequent connections.
log_file: Optional path or file-like object where subprocess stderr will
be written. Can be a Path or TextIO object. Defaults to sys.stderr
if not provided. When a Path is provided, the file will be created
if it doesn't exist, or appended to if it does. When set, server
errors will be written to this file instead of appearing in the console.
"""
script_path = Path(script_path).resolve()
if not script_path.is_file():
@ -502,6 +529,7 @@ class PythonStdioTransport(StdioTransport):
env=env,
cwd=cwd,
keep_alive=keep_alive,
log_file=log_file,
)
self.script_path = script_path
@ -516,6 +544,7 @@ class FastMCPStdioTransport(StdioTransport):
env: dict[str, str] | None = None,
cwd: str | None = None,
keep_alive: bool | None = None,
log_file: Path | TextIO | None = None,
):
script_path = Path(script_path).resolve()
if not script_path.is_file():
@ -529,6 +558,7 @@ class FastMCPStdioTransport(StdioTransport):
env=env,
cwd=cwd,
keep_alive=keep_alive,
log_file=log_file,
)
self.script_path = script_path
@ -544,6 +574,7 @@ class NodeStdioTransport(StdioTransport):
cwd: str | None = None,
node_cmd: str = "node",
keep_alive: bool | None = None,
log_file: Path | TextIO | None = None,
):
"""
Initialize a Node transport.
@ -558,6 +589,11 @@ class NodeStdioTransport(StdioTransport):
Defaults to True. When True, the subprocess remains active
after the connection context exits, allowing reuse in
subsequent connections.
log_file: Optional path or file-like object where subprocess stderr will
be written. Can be a Path or TextIO object. Defaults to sys.stderr
if not provided. When a Path is provided, the file will be created
if it doesn't exist, or appended to if it does. When set, server
errors will be written to this file instead of appearing in the console.
"""
script_path = Path(script_path).resolve()
if not script_path.is_file():
@ -570,7 +606,12 @@ class NodeStdioTransport(StdioTransport):
full_args.extend(args)
super().__init__(
command=node_cmd, args=full_args, env=env, cwd=cwd, keep_alive=keep_alive
command=node_cmd,
args=full_args,
env=env,
cwd=cwd,
keep_alive=keep_alive,
log_file=log_file,
)
self.script_path = script_path

View file

@ -253,3 +253,112 @@ class TestKeepAlive:
with pytest.raises(RuntimeError, match="Client failed to connect"):
async with client:
pass
class TestLogFile:
@pytest.fixture
def stdio_script_with_stderr(self, tmp_path):
script = inspect.cleandoc('''
import sys
from fastmcp import FastMCP
mcp = FastMCP()
@mcp.tool
def write_error(message: str) -> str:
"""Writes a message to stderr and returns it"""
print(message, file=sys.stderr, flush=True)
return message
if __name__ == "__main__":
mcp.run()
''')
script_file = tmp_path / "stderr_script.py"
script_file.write_text(script)
return script_file
async def test_log_file_parameter_accepted_by_stdio_transport(self, tmp_path):
"""Test that log_file parameter can be set on StdioTransport"""
log_file_path = tmp_path / "errors.log"
transport = StdioTransport(
command="python", args=["script.py"], log_file=log_file_path
)
assert transport.log_file == log_file_path
async def test_log_file_parameter_accepted_by_python_stdio_transport(
self, tmp_path, stdio_script_with_stderr
):
"""Test that log_file parameter can be set on PythonStdioTransport"""
log_file_path = tmp_path / "errors.log"
transport = PythonStdioTransport(
script_path=stdio_script_with_stderr, log_file=log_file_path
)
assert transport.log_file == log_file_path
async def test_log_file_parameter_accepts_textio(self, tmp_path):
"""Test that log_file parameter can accept a TextIO object"""
log_file_path = tmp_path / "errors.log"
with open(log_file_path, "w") as log_file:
transport = StdioTransport(
command="python", args=["script.py"], log_file=log_file
)
assert transport.log_file == log_file
async def test_log_file_captures_stderr_output_with_path(
self, tmp_path, stdio_script_with_stderr
):
"""Test that stderr output is written to the log_file when using Path"""
log_file_path = tmp_path / "errors.log"
transport = PythonStdioTransport(
script_path=stdio_script_with_stderr, log_file=log_file_path
)
client = Client(transport=transport)
async with client:
await client.call_tool("write_error", {"message": "Test error message"})
# Need to wait a bit for stderr to flush
await asyncio.sleep(0.1)
content = log_file_path.read_text()
assert "Test error message" in content
async def test_log_file_captures_stderr_output_with_textio(
self, tmp_path, stdio_script_with_stderr
):
"""Test that stderr output is written to the log_file when using TextIO"""
log_file_path = tmp_path / "errors.log"
with open(log_file_path, "w") as log_file:
transport = PythonStdioTransport(
script_path=stdio_script_with_stderr, log_file=log_file
)
client = Client(transport=transport)
async with client:
await client.call_tool(
"write_error", {"message": "Test error with TextIO"}
)
# Need to wait a bit for stderr to flush
await asyncio.sleep(0.1)
content = log_file_path.read_text()
assert "Test error with TextIO" in content
async def test_log_file_none_uses_default_behavior(
self, tmp_path, stdio_script_with_stderr
):
"""Test that log_file=None uses default stderr handling"""
transport = PythonStdioTransport(
script_path=stdio_script_with_stderr, log_file=None
)
client = Client(transport=transport)
async with client:
# Should work without error even without explicit log_file
result = await client.call_tool(
"write_error", {"message": "Default stderr"}
)
assert result.data == "Default stderr"