mirror of
https://github.com/PrefectHQ/fastmcp.git
synced 2026-08-09 15:19:10 +02:00
Add KeyValueResponseCacheStore for distributed client response caching (#4479)
* Add KeyValueResponseCacheStore adapter for client response cache * Test KeyValueResponseCacheStore round-trip, isolation, and distributed sharing * Document distributed response cache store
This commit is contained in:
parent
1d932cc778
commit
836ceac30e
4 changed files with 538 additions and 0 deletions
|
|
@ -234,6 +234,27 @@ async with client:
|
|||
fresh = await client.list_tools_mcp(cache_mode="refresh")
|
||||
```
|
||||
|
||||
### Sharing a cache across clients
|
||||
|
||||
The default cache lives in each client's process. To share cached responses across a fleet — a set of proxy replicas backed by one Redis, for example — pass a `KeyValueResponseCacheStore`, FastMCP's adapter over the same `AsyncKeyValue` key-value abstraction the event store and OAuth proxy use. It accepts any compatible backend (memory, Redis, and more).
|
||||
|
||||
A shared store mingles responses from different principals, so it requires an explicit `partition` that isolates them. Derive the partition from a verified credential — never from request data or the server URL — and construct a new client when the principal changes. Only responses the server marks `"public"` are ever served across partitions.
|
||||
|
||||
```python
|
||||
from fastmcp import Client
|
||||
from fastmcp.client.caching import KeyValueResponseCacheStore
|
||||
from mcp.client.caching import CacheConfig
|
||||
from key_value.aio.stores.redis import RedisStore
|
||||
|
||||
backend = RedisStore(url="redis://localhost")
|
||||
store = KeyValueResponseCacheStore(storage=backend)
|
||||
|
||||
config = CacheConfig(store=store, partition="tenant-a", target_id="weather-api")
|
||||
client = Client("https://example.com/mcp", mode="auto", cache=config)
|
||||
```
|
||||
|
||||
The adapter serializes each result through a type-tagged envelope validated against an allowlist of cacheable result models, so a value naming an unknown type is treated as a cache miss rather than deserialized blindly. Each store instance owns its own collection namespace; `clear()` affects only that namespace, never another tenant's entries.
|
||||
|
||||
## Operations
|
||||
|
||||
FastMCP clients interact with three types of server components.
|
||||
|
|
|
|||
|
|
@ -219,6 +219,21 @@ Proxy forwarding handlers stash the request context so a backend that issues a s
|
|||
|
||||
*Verify:* commit `1ac166bd`, `server/providers/proxy.py`.
|
||||
|
||||
### Shared response cache via `KeyValueResponseCacheStore` — New
|
||||
|
||||
The SDK's client response cache (SEP-2549) reads and writes through a pluggable `ResponseCacheStore`; the default is a per-client in-memory LRU. FastMCP adds `KeyValueResponseCacheStore`, an adapter over the same `AsyncKeyValue` key-value abstraction the event store and OAuth proxy already use, so a fleet of clients (e.g. proxy replicas) can share one Redis-backed response cache. Pass it via `CacheConfig(store=...)`; a custom store requires an explicit `partition` (SDK) and `target_id` (FastMCP). Results serialize through a type-tagged envelope validated against an allowlist of cacheable result models — an unknown tag is a cache miss, never an import-by-name — and each adapter owns its own collection so `clear()` never touches another tenant.
|
||||
|
||||
```python
|
||||
from fastmcp.client.caching import KeyValueResponseCacheStore
|
||||
from mcp.client.caching import CacheConfig
|
||||
from key_value.aio.stores.redis import RedisStore
|
||||
|
||||
store = KeyValueResponseCacheStore(storage=RedisStore(url="redis://localhost"))
|
||||
config = CacheConfig(store=store, partition="tenant-a", target_id="weather-api")
|
||||
```
|
||||
|
||||
*Verify:* `fastmcp_slim/fastmcp/client/caching.py`, `tests/client/client/test_kv_response_cache.py`.
|
||||
|
||||
## HTTP
|
||||
|
||||
The maintainer asked whether FastMCP can now delete its custom HTTP app and let the SDK's `Server.streamable_http_app()` handle everything. The answer for this PR is **no** — every override earns its keep. Convergence is a v4 project gated on three upstream additions (see [Feature Program](/development/v4-notes/feature-program)).
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue