diff --git a/README_OPENAPI.md b/README_OPENAPI.md new file mode 100644 index 000000000..cb0d5f9c0 --- /dev/null +++ b/README_OPENAPI.md @@ -0,0 +1,246 @@ +# FastMCP OpenAPI Integration + +This document explains how FastMCP's OpenAPI integration works, what features are supported, and how to extend it. The OpenAPI functionality is split across two main files: + +- `server/openapi.py` - High-level FastMCP server implementation and MCP component creation +- `utilities/openapi.py` - Low-level OpenAPI parsing and intermediate representation + +## Architecture Overview + +``` +OpenAPI Spec → Parse → HTTPRoute IR → Create MCP Components → FastMCP Server +``` + +### 1. Parsing Phase (`utilities/openapi.py`) + +OpenAPI specifications are parsed into an intermediate representation (IR) that normalizes differences between OpenAPI 3.0 and 3.1: + +- **Input**: Raw OpenAPI spec (dict) +- **Output**: List of `HTTPRoute` objects with normalized parameter information +- **Key Classes**: + - `HTTPRoute` - Represents a single operation + - `ParameterInfo` - Represents a parameter with location, style, explode, etc. + - `RequestBodyInfo` - Represents request body information + - `ResponseInfo` - Represents response information + +### 2. Component Creation Phase (`server/openapi.py`) + +HTTPRoute objects are converted into FastMCP components based on route mapping rules: + +- **Tools** (`OpenAPITool`) - HTTP operations that can be called +- **Resources** (`OpenAPIResource`) - HTTP endpoints that return data +- **Resource Templates** (`OpenAPIResourceTemplate`) - Parameterized resources + +## Parameter Handling + +FastMCP supports various OpenAPI parameter serialization styles and formats: + +### Supported Parameter Locations +- `query` - Query string parameters +- `path` - Path parameters +- `header` - HTTP headers +- `cookie` - Cookie parameters (parsed but not used in requests) + +### Supported Parameter Styles + +#### Query Parameters +- **`form`** (default) - Standard query parameter format + - `explode=true` (default): `?tags=red&tags=blue` + - `explode=false`: `?tags=red,blue` +- **`deepObject`** - Object parameters with bracket notation + - `explode=true`: `?filter[name]=John&filter[age]=30` + - `explode=false`: Falls back to JSON string (non-standard, logs warning) + +#### Path Parameters +- **`simple`** (default) - Comma-separated for arrays: `/users/1,2,3` + +#### Header Parameters +- **`simple`** (default) - Standard header format + +### Parameter Type Support + +#### Arrays +- String arrays with `explode=true/false` +- Number arrays with `explode=true/false` +- Boolean arrays with `explode=true/false` +- Complex object arrays (basic support, may not handle all cases) + +#### Objects +- Objects with `deepObject` style and `explode=true` +- Objects with other styles fall back to JSON serialization + +#### Primitives +- Strings, numbers, booleans +- Enums +- Default values + +## Request Body Handling + +### Supported Content Types +- `application/json` - JSON request bodies + +### Schema Support +- Object schemas with properties +- Array schemas +- Primitive schemas +- Schema references (`$ref` to local schemas only) +- Required properties +- Default values + +## Response Handling + +### Content Type Detection +- `application/json` - Parsed as JSON +- `text/*` - Returned as text +- `application/xml` - Returned as text +- Other types - Returned as binary + +### Output Schema Generation +- Success response schemas (200, 201, 202, 204) +- Object response wrapping for MCP compliance +- Schema compression (removes unused `$defs`) + +## Route Mapping + +Routes are mapped to MCP component types using `RouteMap` configurations: + +```python +RouteMap( + methods=["GET", "POST"], # HTTP methods to match + pattern=r"/api/users/.*", # Regex pattern for path + mcp_type=MCPType.RESOURCE_TEMPLATE, # Target component type + tags={"user"}, # OpenAPI tags to match (AND condition) + mcp_tags={"fastmcp-user"} # Tags to add to created components +) +``` + +### Default Behavior +- All routes become **Tools** by default +- Use route maps to override specific patterns + +### Component Types +- `MCPType.TOOL` - Callable operations +- `MCPType.RESOURCE` - Static data endpoints +- `MCPType.RESOURCE_TEMPLATE` - Parameterized data endpoints +- `MCPType.EXCLUDE` - Skip route entirely + +## Known Limitations & Edge Cases + +### Parameter Edge Cases +1. **Parameter Name Collisions** - When path/query parameters have same names as request body properties, non-body parameters get `__location` suffixes +2. **Complex Array Serialization** - Limited support for arrays containing objects +3. **Cookie Parameters** - Parsed but not used in requests +4. **Non-standard Combinations** - e.g., `deepObject` with `explode=false` + +### Request Body Edge Cases +1. **Content Type Priority** - Only first available content type is used +2. **Nested Objects** - Deep nesting may not serialize correctly +3. **Binary Content** - No support for file uploads or binary data + +### Response Edge Cases +1. **Multiple Content Types** - Only JSON-compatible types are used for output schemas +2. **Error Responses** - Not used for MCP output schema generation +3. **Response Headers** - Not captured or exposed + +### Schema Edge Cases +1. **External References** - `$ref` to external files not supported +2. **Circular References** - May cause issues in schema processing +3. **Polymorphism** - `oneOf`/`anyOf`/`allOf` limited support + +## Debugging Tips + +### Common Issues +1. **"Unknown tool/resource"** - Check route mapping configuration +2. **Parameter not found** - Check for name collisions or incorrect style/explode +3. **Invalid request format** - Check parameter serialization and content types +4. **Schema validation errors** - Check for external refs or complex schemas + +### Debugging Tools +```python +# Parse routes to inspect intermediate representation +routes = parse_openapi_to_http_routes(openapi_spec) +for route in routes: + print(f"{route.method} {route.path}") + for param in route.parameters: + print(f" {param.name} ({param.location}): style={param.style}, explode={param.explode}") + +# Check component creation +server = FastMCP.from_openapi(openapi_spec, client) +tools = await server.get_tools() +print(f"Created {len(tools)} tools: {list(tools.keys())}") +``` + +### Logging +- Set `FASTMCP_LOG_LEVEL=DEBUG` to see detailed parameter processing +- Look for warnings about non-standard parameter combinations +- Check for schema parsing errors in logs + +## Extension Points + +### Adding New Parameter Styles +1. Add style handling in `utilities/openapi.py` - `ParameterInfo` class +2. Implement serialization logic in `server/openapi.py` - `OpenAPITool.run()` +3. Add tests for parsing and serialization + +### Adding New Content Types +1. Extend request body handling in `OpenAPITool.run()` +2. Add response parsing logic for new types +3. Update content type priority in utilities + +### Custom Route Mapping +Use `route_map_fn` for complex routing logic: + +```python +def custom_mapper(route: HTTPRoute, current_type: MCPType) -> MCPType: + if route.path.startswith("/admin"): + return MCPType.EXCLUDE + return current_type + +server = FastMCP.from_openapi(spec, client, route_map_fn=custom_mapper) +``` + +## Testing Patterns + +### Unit Tests +- Test parameter parsing with various styles/explode combinations +- Test route mapping with different patterns and tags +- Test schema generation and compression + +### Integration Tests +- Mock HTTP client to verify actual request parameters +- Test end-to-end component creation and execution +- Test error handling and edge cases + +### Example Test Pattern +```python +async def test_parameter_style(): + # 1. Create OpenAPI spec with specific parameter configuration + spec = {"openapi": "3.1.0", ...} + + # 2. Parse and create components + routes = parse_openapi_to_http_routes(spec) + tool = OpenAPITool(mock_client, routes[0], ...) + + # 3. Execute and verify request parameters + await tool.run({"param": "value"}) + actual_params = mock_client.request.call_args.kwargs["params"] + assert actual_params == expected_params +``` + +## Testing + +OpenAPI functionality is tested across multiple files in `tests/server/openapi/`: + +- `test_basic_functionality.py` - Core component creation and execution +- `test_explode_integration.py` - Parameter explode behavior +- `test_deepobject_style.py` - DeepObject style parameter encoding +- `test_parameter_collisions.py` - Parameter name collision handling +- `test_openapi_path_parameters.py` - Path parameter serialization +- `test_configuration.py` - Route mapping and MCP names +- `test_description_propagation.py` - Schema and description handling + +When adding new OpenAPI features, create focused test files rather than adding to existing monolithic files. + +--- + +*This document should be updated when new OpenAPI features are added or when edge cases are discovered and addressed.* \ No newline at end of file diff --git a/src/fastmcp/server/openapi.py b/src/fastmcp/server/openapi.py index 35e5d996e..924762485 100644 --- a/src/fastmcp/server/openapi.py +++ b/src/fastmcp/server/openapi.py @@ -29,6 +29,7 @@ from fastmcp.utilities.openapi import ( _combine_schemas, extract_output_schema_from_responses, format_array_parameter, + format_deep_object_parameter, format_description_with_responses, ) @@ -357,18 +358,36 @@ class OpenAPITool(Tool): param_value = arguments[p.name] if param_value is not None: - # Format array query parameters as comma-separated strings - # following OpenAPI form style (default for query parameters) - if ( + # Handle different parameter styles and types + param_style = ( + p.style or "form" + ) # Default style for query parameters is "form" + param_explode = ( + p.explode if p.explode is not None else True + ) # Default explode for query is True + + # Handle deepObject style for object parameters + if param_style == "deepObject" and isinstance(param_value, dict): + if param_explode: + # deepObject with explode=true: object properties become separate parameters + # e.g., target[id]=123&target[type]=user + deep_obj_params = format_deep_object_parameter( + param_value, p.name + ) + query_params.update(deep_obj_params) + else: + # deepObject with explode=false is not commonly used, fallback to JSON + logger.warning( + f"deepObject style with explode=false for parameter '{p.name}' is not standard. " + f"Using JSON serialization fallback." + ) + query_params[p.name] = json.dumps(param_value) + # Handle array parameters with form style (default) + elif ( isinstance(param_value, list) and p.schema_.get("type") == "array" ): - # Get explode parameter from the parameter info, default is True for query parameters - # If explode is True, the array is serialized as separate parameters - # If explode is False, the array is serialized as a comma-separated string - explode = p.explode if p.explode is not None else True - - if explode: + if param_explode: # When explode=True, we pass the array directly, which HTTPX will serialize # as multiple parameters with the same name query_params[p.name] = param_value @@ -379,7 +398,7 @@ class OpenAPITool(Tool): ) query_params[p.name] = formatted_value else: - # Non-array parameters are passed as is + # Non-array, non-deepObject parameters are passed as is query_params[p.name] = param_value # Prepare headers - fix typing by ensuring all values are strings diff --git a/src/fastmcp/utilities/openapi.py b/src/fastmcp/utilities/openapi.py index 97f0e89fb..3681bb51d 100644 --- a/src/fastmcp/utilities/openapi.py +++ b/src/fastmcp/utilities/openapi.py @@ -93,6 +93,40 @@ def format_array_parameter( return str_value +def format_deep_object_parameter( + param_value: dict, parameter_name: str +) -> dict[str, str]: + """ + Format a dictionary parameter for deepObject style serialization. + + According to OpenAPI 3.0 spec, deepObject style with explode=true serializes + object properties as separate query parameters with bracket notation. + + For example: {"id": "123", "type": "user"} becomes: + param[id]=123¶m[type]=user + + Args: + param_value: Dictionary value to format + parameter_name: Name of the parameter + + Returns: + Dictionary with bracketed parameter names as keys + """ + if not isinstance(param_value, dict): + logger.warning( + f"deepObject style parameter '{parameter_name}' expected dict, got {type(param_value)}" + ) + return {} + + result = {} + for key, value in param_value.items(): + # Format as param[key]=value + bracketed_key = f"{parameter_name}[{key}]" + result[bracketed_key] = str(value) + + return result + + class ParameterInfo(FastMCPBaseModel): """Represents a single parameter for an HTTP operation in our IR.""" @@ -102,6 +136,7 @@ class ParameterInfo(FastMCPBaseModel): schema_: JsonSchema = Field(..., alias="schema") # Target name in IR description: str | None = None explode: bool | None = None # OpenAPI explode property for array parameters + style: str | None = None # OpenAPI style property for parameter serialization class RequestBodyInfo(FastMCPBaseModel): @@ -153,6 +188,7 @@ __all__ = [ "JsonSchema", "parse_openapi_to_http_routes", "extract_output_schema_from_responses", + "format_deep_object_parameter", ] # Type variables for generic parser @@ -415,8 +451,9 @@ class OpenAPIParser( ): param_schema_dict["default"] = resolved_media_schema.default - # Extract explode property if present + # Extract explode and style properties if present explode = getattr(parameter, "explode", None) + style = getattr(parameter, "style", None) # Create parameter info object param_info = ParameterInfo( @@ -426,6 +463,7 @@ class OpenAPIParser( schema=param_schema_dict, description=parameter.description, explode=explode, + style=style, ) extracted_params.append(param_info) except Exception as e: diff --git a/tests/server/openapi/test_deepobject_style.py b/tests/server/openapi/test_deepobject_style.py new file mode 100644 index 000000000..b066c37b8 --- /dev/null +++ b/tests/server/openapi/test_deepobject_style.py @@ -0,0 +1,281 @@ +"""Integration test for OpenAPI deepObject style parameter handling. + +This test verifies that the deepObject style and explode properties are correctly +parsed from OpenAPI specifications and properly applied during HTTP request serialization. +""" + +from unittest.mock import AsyncMock, MagicMock + +import httpx + +from fastmcp.server.openapi import OpenAPITool +from fastmcp.utilities.openapi import parse_openapi_to_http_routes + + +class TestDeepObjectStyle: + """Test the complete pipeline from OpenAPI spec to HTTP request parameters for deepObject style.""" + + def test_deepobject_style_parsing_from_openapi_spec(self): + """Test that deepObject style is correctly parsed from OpenAPI specification.""" + # Real OpenAPI spec with style: deepObject and explode: true + openapi_spec = { + "openapi": "3.1.0", + "info": {"title": "Test API", "version": "1.0.0"}, + "paths": { + "/api/surveys": { + "get": { + "operationId": "getSurveys", + "parameters": [ + { + "name": "target", + "in": "query", + "required": False, + "style": "deepObject", + "explode": True, + "schema": { + "type": "object", + "properties": { + "id": { + "type": "string", + "description": "Valid ID for an object", + }, + "type": { + "type": "string", + "enum": ["location", "organisation"], + "description": "The type of object for given id", + }, + }, + "required": ["type", "id"], + }, + } + ], + "responses": { + "200": { + "description": "Success", + "content": { + "application/json": {"schema": {"type": "integer"}} + }, + } + }, + } + } + }, + } + + # Parse the spec + routes = parse_openapi_to_http_routes(openapi_spec) + route = routes[0] + parameter = route.parameters[0] + + # Verify style and explode properties were captured correctly + assert parameter.name == "target" + assert parameter.location == "query" + assert parameter.style == "deepObject", ( + f"Expected style='deepObject', got {parameter.style}" + ) + assert parameter.explode is True, ( + f"Expected explode=True, got {parameter.explode}" + ) + + async def test_deepobject_style_request_serialization(self): + """Test that deepObject style results in bracketed query parameters in HTTP requests. + + This is the critical integration test that reproduces the GitHub issue. + """ + # OpenAPI spec matching the GitHub issue example + openapi_spec = { + "openapi": "3.1.0", + "info": {"title": "Test API", "version": "1.0.0"}, + "paths": { + "/api/surveys": { + "get": { + "operationId": "getSurveys", + "parameters": [ + { + "name": "target", + "in": "query", + "required": False, + "style": "deepObject", + "explode": True, + "schema": { + "type": "object", + "properties": { + "id": {"type": "string"}, + "type": {"type": "string"}, + }, + "required": ["type", "id"], + }, + } + ], + "responses": {"200": {"description": "Success"}}, + } + } + }, + } + + # Parse and create tool + routes = parse_openapi_to_http_routes(openapi_spec) + route = routes[0] + + # Mock HTTP client + mock_client = AsyncMock(spec=httpx.AsyncClient) + mock_response = MagicMock() + mock_response.status_code = 200 + mock_response.json.return_value = {} + mock_response.raise_for_status.return_value = None + mock_client.request.return_value = mock_response + + # Create tool + tool = OpenAPITool( + client=mock_client, + route=route, + name="getSurveys", + description="Get surveys", + parameters={}, + ) + + # Execute tool with object parameter (as it would come from user input) + await tool.run( + {"target": {"id": "57dc372a81b610496e8b465e", "type": "organisation"}} + ) + + # Verify the HTTP request was made with deepObject-style parameters + mock_client.request.assert_called_once() + call_kwargs = mock_client.request.call_args.kwargs + + # Check that params contains bracketed parameters, not JSON string + params = call_kwargs.get("params", {}) + + # Should have target[id] and target[type] parameters + assert "target[id]" in params, "target[id] parameter should be present" + assert "target[type]" in params, "target[type] parameter should be present" + + # Values should be correctly set + assert params["target[id]"] == "57dc372a81b610496e8b465e", ( + f"Expected target[id]=57dc372a81b610496e8b465e, got {params.get('target[id]')}" + ) + assert params["target[type]"] == "organisation", ( + f"Expected target[type]=organisation, got {params.get('target[type]')}" + ) + + # Should NOT have the original parameter name as JSON + assert "target" not in params, ( + "Original 'target' parameter should not be present when using deepObject style" + ) + + async def test_deepobject_style_with_explode_false(self): + """Test that deepObject style with explode=false falls back to JSON serialization.""" + openapi_spec = { + "openapi": "3.1.0", + "info": {"title": "Test API", "version": "1.0.0"}, + "paths": { + "/api/surveys": { + "get": { + "operationId": "getSurveys", + "parameters": [ + { + "name": "target", + "in": "query", + "style": "deepObject", + "explode": False, # Non-standard combination + "schema": { + "type": "object", + "properties": { + "id": {"type": "string"}, + "type": {"type": "string"}, + }, + }, + } + ], + "responses": {"200": {"description": "Success"}}, + } + } + }, + } + + routes = parse_openapi_to_http_routes(openapi_spec) + route = routes[0] + + mock_client = AsyncMock(spec=httpx.AsyncClient) + mock_response = MagicMock() + mock_response.status_code = 200 + mock_response.json.return_value = {} + mock_response.raise_for_status.return_value = None + mock_client.request.return_value = mock_response + + tool = OpenAPITool( + client=mock_client, + route=route, + name="getSurveys", + description="Get surveys", + parameters={}, + ) + + await tool.run({"target": {"id": "123", "type": "test"}}) + + mock_client.request.assert_called_once() + call_kwargs = mock_client.request.call_args.kwargs + + params = call_kwargs.get("params", {}) + + # Should fall back to JSON serialization + assert "target" in params, "target parameter should be present" + assert params["target"] == '{"id": "123", "type": "test"}', ( + f"Expected JSON string fallback, got {params.get('target')}" + ) + + async def test_non_object_with_deepobject_style(self): + """Test that non-object parameters with deepObject style are handled gracefully.""" + openapi_spec = { + "openapi": "3.1.0", + "info": {"title": "Test API", "version": "1.0.0"}, + "paths": { + "/api/test": { + "get": { + "operationId": "testEndpoint", + "parameters": [ + { + "name": "param", + "in": "query", + "style": "deepObject", + "explode": True, + "schema": {"type": "string"}, # Not an object + } + ], + "responses": {"200": {"description": "Success"}}, + } + } + }, + } + + routes = parse_openapi_to_http_routes(openapi_spec) + route = routes[0] + + mock_client = AsyncMock(spec=httpx.AsyncClient) + mock_response = MagicMock() + mock_response.status_code = 200 + mock_response.json.return_value = {} + mock_response.raise_for_status.return_value = None + mock_client.request.return_value = mock_response + + tool = OpenAPITool( + client=mock_client, + route=route, + name="testEndpoint", + description="Test endpoint", + parameters={}, + ) + + # Pass a string value instead of an object + await tool.run({"param": "test_value"}) + + mock_client.request.assert_called_once() + call_kwargs = mock_client.request.call_args.kwargs + + params = call_kwargs.get("params", {}) + + # Should use the parameter as-is since it's not an object + assert "param" in params, "param parameter should be present" + assert params["param"] == "test_value", ( + f"Expected 'test_value', got {params.get('param')}" + )