Commit graph

12 commits

Author SHA1 Message Date
Jeremiah Lowin
610551c7b6
Split large test files to comply with loq line limit (#3328) 2026-02-28 11:21:11 -05:00
Claude
7690e995b8 Add tests for default sampling capability (issue #3329)
🤖 Generated with Claude Code

https://claude.ai/code/session_01BsxYNsUhJPx14QiJ4FHqx6
2026-02-28 11:06:21 -05:00
Bill Easton
5bab188106
Add concurrent tool execution with sequential flag (#3022)
Co-authored-by: Bill Easton <strawgate@users.noreply.github.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Jeremiah Lowin <153965+jlowin@users.noreply.github.com>
2026-02-09 20:43:53 -05:00
Jeremiah Lowin
049bd22827
Replace type: ignore[attr-defined] with isinstance assertions in tests (#2665)
* Replace type: ignore[attr-defined] with isinstance assertions in tests

* Fix isinstance assertions in failing tests

- Fix enum test to check for ResponseEnum instead of str
- Fix binary resource test to check for BlobResourceContents instead of TextResourceContents
- Fix Root type tests to check attributes directly instead of isinstance checks

* Fix type errors without using type: ignore

- Remove execution methods from TransformingProvider (only handles transformations)
- Add execution methods to base Provider class with default implementations
- Fix type narrowing in tests using cast() instead of type: ignore
- Fix PromptResult type handling in prompt render tests
- Fix type narrowing in middleware test for arguments and structured_content
2025-12-21 16:37:20 -05:00
Jeremiah Lowin
41ec7ee06d
SEP-1577: Sampling with tools (#2551)
* MCP → SDK (vocab change only)

* WIP: Sampling API with SamplingResult[T] and result_type

* SEP-1577: Sampling with tools

- Add tools and result_type parameters to ctx.sample()
- Update OpenAI handler for tool content types
- Client advertises sampling.tools capability by default
- Collect tool results into single message with list content

* Fix tool result content handling in OpenAI handler

* Remove @sampling_tool decorator - pass functions directly to sample()

Functions passed to ctx.sample(tools=[...]) are now auto-converted
via SamplingTool.from_function(). Users can still use that method
directly for custom name/description overrides.

* Remove auto-conversion of MCP tools to sampling tools

Users want MCP tools passed to ctx.sample() to go through the full MCP
machinery (middleware, native responses) rather than being auto-converted
to direct function calls. Now only SamplingTool and plain callables are
accepted - passing a FastMCP Tool raises a clear TypeError.

Also bumps mcp dependency to >=1.24.0 for required sampling features.

* Refactor sampling API: replace sample_iter() with sample_step()

Replace the mutable SampleRun/sample_iter() pattern with a simpler stateless
sample_step() function. sample_step() makes a single LLM call and returns a
SampleStep with the response and history. sample() now loops sample_step()
internally.

Key changes:
- Add sample_step() for fine-grained control over the sampling loop
- Remove SampleRun class and sample_iter() method
- Structured output uses tool description only (no prompt modification)
- execute_tools parameter controls automatic vs manual tool execution

* Address CodeRabbit nitpicks

* Address CodeRabbit review feedback for sampling tools

- Fix temperature=0.0 being dropped due to falsy evaluation
- Add ToolChoice.name support for forcing specific tools
- Replace assert statements with explicit RuntimeError checks
- Add mask_error_details parameter to sample()/sample_step() with ToolError escape hatch
- Fix hasattr patterns with proper isinstance checks
- Document mask_error_details and add OpenAI prerequisites to docs

* Address additional CodeRabbit review feedback

- Catch ValidationError specifically instead of bare Exception
- Update result_type docs to mention dataclasses and basic types
- Raise ValueError for unknown tool_choice modes
- Validate sampling_handler_behavior to catch typos
- Remove ToolChoice.name handling (not part of MCP spec)
- Validate tool_choice string in sample_step()

* Review fixes for sampling tools PR

- Remove internal functions from sampling __init__.py exports
- Remove fragile is_text property, use not is_tool_use instead
- Inline call_client into context.py, remove from run.py
- Fix SamplingMessage docs to use TextContent
- Handle result.text being None in doc examples
- Simplify client sampling docs to recommend OpenAISamplingHandler
- Add sampling_capabilities override documentation
- Raise iteration limit from 50 to 100
- Remove _parse_model_preferences duplication
- Use AsyncOpenAI in OpenAISamplingHandler
- Fix tool_choice docstring

* Fix OpenAI handler tests to use AsyncOpenAI

* Address remaining CodeRabbit review comments

- Fix message ordering in OpenAI handler: tool results now correctly
  follow assistant message with tool_calls
- sample_step() now always includes assistant message in history
- Raise ValueError on JSON parse errors instead of silent {}
- Add has_sampling capability check when behavior is None
- Raise RuntimeError when structured output receives text response
- Wrap primitive result_type schemas in object wrapper
- Fix docs example using invalid SamplingMessage construction
- Add comprehensive client_sampling_test.py example

* Add return type annotation to OpenAISamplingHandler.__init__

* Use explicit 'is not None' check for sampling_capabilities defaulting

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Bill Easton <strawgate@users.noreply.github.com>
2025-12-14 13:51:05 -05:00
Chris Guidry
654442bfd3 Update FastMCP for MCP SDK 1.23.1 auth changes
- Bump mcp SDK to >=1.23.1
- Add `client_secret_basic` authentication support (SDK PR #1334)
- TokenHandler now wraps SDK's handle() to transform `unauthorized_client`
  to `invalid_client` on 401 responses per OAuth 2.1 spec
- Update `sample()` return type to use SDK's SamplingMessageContentBlock
- Update test expectations for new SDK fields (`task`, `_meta`)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-03 11:50:08 -05:00
William Easton
d32a2b953e
Fallback to a Completions API when Sampling is not available (#1145) 2025-08-21 07:49:29 -04:00
Jeremiah Lowin
23f7873314
Add unit test for sampling with image messages (#1329) 2025-08-01 15:55:05 -04:00
Jeremiah Lowin
d8dc28b9b2 Add structured content & update almost all tests 2025-06-27 15:40:52 -04:00
Jeremiah Lowin
d1549c3d35 Remove empty parens 2025-06-04 15:37:55 -04:00
Jeremiah Lowin
ea13dc114f Improve client return types 2025-04-12 12:16:07 -04:00
Jeremiah Lowin
69751a1786 Add sampling and roots functionality and tests 2025-04-11 11:11:53 -04:00