Add KeyValueResponseCacheStore for distributed client response caching (#4479)

* Add KeyValueResponseCacheStore adapter for client response cache

* Test KeyValueResponseCacheStore round-trip, isolation, and distributed sharing

* Document distributed response cache store
This commit is contained in:
Jeremiah Lowin 2026-07-17 16:41:35 -04:00 committed by GitHub
commit 836ceac30e
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
4 changed files with 538 additions and 0 deletions

View file

@ -234,6 +234,27 @@ async with client:
fresh = await client.list_tools_mcp(cache_mode="refresh")
```
### Sharing a cache across clients
The default cache lives in each client's process. To share cached responses across a fleet — a set of proxy replicas backed by one Redis, for example — pass a `KeyValueResponseCacheStore`, FastMCP's adapter over the same `AsyncKeyValue` key-value abstraction the event store and OAuth proxy use. It accepts any compatible backend (memory, Redis, and more).
A shared store mingles responses from different principals, so it requires an explicit `partition` that isolates them. Derive the partition from a verified credential — never from request data or the server URL — and construct a new client when the principal changes. Only responses the server marks `"public"` are ever served across partitions.
```python
from fastmcp import Client
from fastmcp.client.caching import KeyValueResponseCacheStore
from mcp.client.caching import CacheConfig
from key_value.aio.stores.redis import RedisStore
backend = RedisStore(url="redis://localhost")
store = KeyValueResponseCacheStore(storage=backend)
config = CacheConfig(store=store, partition="tenant-a", target_id="weather-api")
client = Client("https://example.com/mcp", mode="auto", cache=config)
```
The adapter serializes each result through a type-tagged envelope validated against an allowlist of cacheable result models, so a value naming an unknown type is treated as a cache miss rather than deserialized blindly. Each store instance owns its own collection namespace; `clear()` affects only that namespace, never another tenant's entries.
## Operations
FastMCP clients interact with three types of server components.

View file

@ -219,6 +219,21 @@ Proxy forwarding handlers stash the request context so a backend that issues a s
*Verify:* commit `1ac166bd`, `server/providers/proxy.py`.
### Shared response cache via `KeyValueResponseCacheStore` — New
The SDK's client response cache (SEP-2549) reads and writes through a pluggable `ResponseCacheStore`; the default is a per-client in-memory LRU. FastMCP adds `KeyValueResponseCacheStore`, an adapter over the same `AsyncKeyValue` key-value abstraction the event store and OAuth proxy already use, so a fleet of clients (e.g. proxy replicas) can share one Redis-backed response cache. Pass it via `CacheConfig(store=...)`; a custom store requires an explicit `partition` (SDK) and `target_id` (FastMCP). Results serialize through a type-tagged envelope validated against an allowlist of cacheable result models — an unknown tag is a cache miss, never an import-by-name — and each adapter owns its own collection so `clear()` never touches another tenant.
```python
from fastmcp.client.caching import KeyValueResponseCacheStore
from mcp.client.caching import CacheConfig
from key_value.aio.stores.redis import RedisStore
store = KeyValueResponseCacheStore(storage=RedisStore(url="redis://localhost"))
config = CacheConfig(store=store, partition="tenant-a", target_id="weather-api")
```
*Verify:* `fastmcp_slim/fastmcp/client/caching.py`, `tests/client/client/test_kv_response_cache.py`.
## HTTP
The maintainer asked whether FastMCP can now delete its custom HTTP app and let the SDK's `Server.streamable_http_app()` handle everything. The answer for this PR is **no** — every override earns its keep. Convergence is a v4 project gated on three upstream additions (see [Feature Program](/development/v4-notes/feature-program)).