fastmcp/examples/smart_home/docs/usage.md
nate nowack b67c66eb52
examples: add interactive media picker MCP app (#5281)
* examples: add interactive media picker MCP app

* examples: route media picker playback through MCP

* examples: constrain media picker to actuator capabilities

* examples: clarify smart home setup and device boundaries

* examples: refine media picker with restrained glass styling

* auth: add ATProtoProvider for AT Protocol sign-in

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017uN3zXKrzsKxYKNmkNK9Dz

* examples: media picker verifies model-found links and supports AT Protocol sign-in

Drop the static catalog: the model searches, show_media_picker takes URLs,
and each link is checked with YouTube oEmbed before it renders. Setting
MEDIA_PICKER_BASE_URL requires sign-in through ATProtoProvider.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017uN3zXKrzsKxYKNmkNK9Dz

* auth: move ATProtoProvider to fastmcp.experimental.auth.atproto

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017uN3zXKrzsKxYKNmkNK9Dz

* examples: import ATProtoProvider from fastmcp.experimental

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017uN3zXKrzsKxYKNmkNK9Dz

* examples: add a home view with Hue room controls to the media picker

show_home renders every Hue room with its live color, an on/off switch,
brightness presets and saved scenes, next to the verified TV picks. Light
changes go through app-only tools to the smart-home Hue server over MCP.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017uN3zXKrzsKxYKNmkNK9Dz

* auth: skip the ATProto handle page when exactly one DID is allowed

With a single allowed DID the server already knows who is signing in, so
the login step goes straight to that account's PDS. The handle page still
renders when there is an error to show.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017uN3zXKrzsKxYKNmkNK9Dz

* examples: remember consent in the media picker's AT Protocol sign-in

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017uN3zXKrzsKxYKNmkNK9Dz

* apps: accept a csp on FastMCPApp.ui

FastMCPApp.ui built its AppConfig without a CSP, so an app UI could not load
images or other resources from outside the renderer's defaults, unlike tools
registered with PrefabAppConfig(csp=...).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017uN3zXKrzsKxYKNmkNK9Dz

* examples: redesign the home view as compact rows lit by each room's color

Room rows take their tint, lamp glow, switch and active-scene chip from the
room's live Hue color; scene chips show each scene's palette color. Watch
rows use YouTube thumbnails, which the UI's CSP now allows. Tokens and row
treatment follow plyr.fm, scene swatches follow after-hours.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017uN3zXKrzsKxYKNmkNK9Dz

* examples: keep home view room state on the client so taps update it

Level, scene, power and color highlights were rendered from server data,
so they stayed on the old values after a tap. Each room now holds its
state client-side; taps update it before the command is sent, and the
glow, readout and header count follow it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017uN3zXKrzsKxYKNmkNK9Dz

* auth: resolve ATProto handles through DNS and re-verify the DID after sign-in

Handles now resolve from their own _atproto TXT record or well-known file
instead of a Bluesky AppView. After the token exchange the provider resolves
the DID, PDS and authorization server again and requires the same issuer,
and the handle claim is set only when the handle resolves back to the DID.
The docs describe handles, DIDs and hosting as separate layers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017uN3zXKrzsKxYKNmkNK9Dz

* auth: build ATProtoProvider on atproto-oauth and OAuthProxy callback hooks

The provider no longer carries its own AT Protocol client: the new
`atproto` extra installs atproto-oauth, which handles resolution, PAR,
DPoP, token exchange, re-verification and revocation. OAuthProxy's
upstream callback now calls two overridable steps, the callback's
transaction ID and the code exchange, so the provider plugs into them
instead of replacing the callback.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017uN3zXKrzsKxYKNmkNK9Dz

* examples: reduce the media picker to the picker

The home view, Hue controls and AT Protocol sign-in moved to a separate
deployment; thumbnails need FastMCPApp.ui(csp=), which lands separately.
Changes outside examples/ go back to main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017uN3zXKrzsKxYKNmkNK9Dz

* examples/media_picker: drop MEDIA_PICKER_ACTUATOR_SOURCES

YouTube is the only source the picker verifies, so a required setting whose one legal value is youtube only added configuration. A device that can't play an item now reports it through the actuator's error, which the picker surfaces as a playback failure; a test covers that path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0185U3LZpcxFQQJnb6ABuxr1

* examples/smart_home: connect to the Fire TV on first use

The lifespan opened the ADB connection at startup and raised when the TV was unavailable, so a sleeping TV stopped the whole server, lights included. FireTVConnection now connects on the first tool call, reconnects on later calls, and raises a ToolError while the TV is unreachable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0185U3LZpcxFQQJnb6ABuxr1

* examples/smart_home: explain "No route to host" as macOS Local Network privacy

Restarting the ADB daemon only appeared to fix it because the restarted daemon inherited a different launching app's permission. Also document that a sleeping TV no longer blocks startup.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0185U3LZpcxFQQJnb6ABuxr1

* examples/media_picker: name unsupported links as non-YouTube, drop client-specific copy

Links the picker can't parse are reported as "aren't YouTube videos" instead of "can't play on this device", which was wrong without an actuator; state carries unsupported_count. The empty state and "more like this" no longer mention Claude or a home view the example doesn't have.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0185U3LZpcxFQQJnb6ABuxr1

* examples/smart_home: describe the picker and connection lifetimes as they are

The README still called the picker's input a sample catalog, and both docs described every device connection as pooled at startup; the Fire TV now connects on first use.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0185U3LZpcxFQQJnb6ABuxr1

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 23:14:58 -05:00

178 lines
8.1 KiB
Markdown

# smart home setup and operation
Control Philips Hue lights and an Amazon Fire TV through FastMCP. Hue uses the
`phue2` 1.0 alpha's local V2 API. Fire TV control reuses the `androidtv` backend
shipped by Home Assistant's Android Debug Bridge integration; raw ADB shell access
is not exposed to agents.
## Design
This example is a local device-control server. An MCP client can discover devices,
issue a command, and read back the result. It does not require Pi, a graphical
picker, or a workflow engine.
- `src/smart_home/hub.py` mounts the independent Hue and Fire TV servers under
`hue` and `fire_tv` namespaces.
- Each device package owns its connection lifespan, typed results, and tools.
- The optional [media picker](../../apps/media_picker/README.md) presents choices and
calls the same playback tools over MCP.
Media search and recommendation belong to the calling agent or a discovery
service. Schedules, presence policies, and durable desired state belong to the
calling application or workflow engine. Device tools do not infer those policies.
## Run
From the repository root, enter the example and install its dependencies (Python
3.12 or newer and `uv` are required):
```bash
cd examples/smart_home
uv sync
```
Create `.env` in this directory with your existing bridge credentials:
```dotenv
HUE_BRIDGE_IP=<bridge IP>
HUE_BRIDGE_USERNAME=<bridge application key>
HUE_BRIDGE_CERTIFICATE=/absolute/path/to/trusted-bridge.pem
FIRE_TV_HOST=<Fire TV IP>
FIRE_TV_ADB_SERVER_IP=127.0.0.1
```
HTTPS verification is enabled. The optional certificate file is an explicitly
trusted certificate obtained and verified for your bridge; its identity replaces
hostname matching when connecting by IP. Without it, normal system trust and
hostname verification apply. Credentials remain local and are not saved by the SDK.
`FIRE_TV_HOST` is optional. When omitted, Hue tools continue to work and Fire TV
tools return a configuration error. The example supports either an existing ADB
server via `FIRE_TV_ADB_SERVER_IP` or direct Python ADB authentication via
`FIRE_TV_ADB_KEY`. Enable ADB debugging and approve the host on the TV first.
```bash
uv run smart-home
```
This starts a stdio MCP server; configure your MCP client to run this command with
this directory as its working directory. The combined hub requires Hue settings.
For TV-only use, run the Fire TV server independently:
```bash
uv run fastmcp run src/smart_home/fire_tv/server.py:fire_tv_mcp --transport http --host 127.0.0.1 --port 8764
```
Standalone tools are named `read_status` and `play_media`; through the hub they
are `fire_tv_read_status` and `fire_tv_play_media`. Use the standalone HTTP endpoint
`http://127.0.0.1:8764/mcp` with the media picker's `play_media` configuration.
Each device server's lifespan owns its connection. Hue opens a pooled
connection at startup; the Fire TV connects on the first tool call and reconnects
after the TV drops off. Tools receive the connection through dependency
injection. Settings load at
startup, so importing the example does not require live credentials.
## Agent workflow
### TV and ad hoc media selection
Use `fire_tv_read_status` before and after `fire_tv_press_home`,
`fire_tv_launch_app`, or `fire_tv_play_youtube_video`. App launch requires an exact
installed package ID; YouTube playback requires an 11-character video ID. Commands
return acceptance receipts, not proof that navigation completed. The constrained
surface deliberately exposes neither arbitrary URLs nor raw ADB shell commands.
For a request such as "play a live bird camera where it is daylight": find a
current feed, check the camera's location and local time, verify that the video
is live and playable, then call `fire_tv_play_youtube_video` with its video ID.
Use the broadcaster's current live page: old livestream links can return
"recording is not available." This workflow does not require editing a catalog
or adding a new tool for every video.
Status verifies the foreground app and broad playback state, not the video's
identity, daylight, or audible content. Inspect the actual picture or obtain
device media metadata when those details matter. A command receipt alone is not
evidence that the requested stream is playing.
### Hue rooms, scenes, and effects
Start with `hue_read_rooms` and `hue_read_lights`. Rooms include member light UUIDs;
lights include state, device connectivity and supported effects. Names must match
exactly and be unique. V2 UUIDs replace the old numeric light and group IDs.
To turn on candle flicker, check each room member's `supported_effects` and call
`hue_set_light` for each supported bulb:
```json
{
"target": "<light UUID>",
"state": {"on": true, "effect": "candle", "effect_speed": 0.5}
}
```
This preserves brightness. Read `state.effect` and `state.effect_parameters` afterward to verify the active
effect; use `effect: "no_effect"` to stop it. Effect names and support come from the
bulb, not a fixed list. Speed is between zero and one; color or temperature supplied
with an active effect changes its parameters.
For ordinary room-wide lighting, use `hue_set_room` with brightness percent,
`temperature_kelvin`, and optional `transition_seconds`. Color can instead use CIE
`xy` coordinates. Native effects target individual bulbs. Brightness, color and
effects do not implicitly turn lights on; include `on: true` when desired.
Use `hue_read_scenes` to inspect room associations, palettes and per-light actions.
That distinguishes a scene with warm static colors from one with candle or fire
effects. `hue_activate_scene` resolves names within the chosen room and recalls the
saved actions. Its optional `dynamic_palette` action requests palette cycling where
supported by Hue.
A write acknowledgement does not prove the resulting state. Read lights after a
transition. Hue may partially apply a command before reporting an error; failures
are exposed as MCP tool errors. This example does not implement scheduling,
custom animation loops, or entertainment streaming.
## Test with an agent
```bash
uv run pytest
uv run scripts/pi_harness.py --json
```
The tests exercise actual MCP calls with a simulated bridge. The Pi harness
requires Pi and `pi-mcp-adapter`, exposes only this MCP, disables built-in tools,
and defaults to read-only discovery. Pass `--env-file /path/to/existing.env` if
credentials live elsewhere; `HUE_BRIDGE_CERTIFICATE` can also be exported in the
launching environment. A quoted prompt may request real changes to lights.
`--json` records tool calls and results for verification.
## Discovery and result metadata
`hue_read_lights(room="living room")` and `hue_read_scenes(room="living room")`
limit discovery to a room, using its exact unique name or UUID. Light results put
observed state, supported effects and capabilities first. `details=true` includes
the complete Hue light resource when needed. Unknown observations remain null;
color temperature is reported only when Hue marks it valid.
`hue_set_room` exposes ordinary lighting controls only. Apply native effects with
`hue_set_light` to each supported bulb. All writes return an accepted receipt with
`state_verified=false`; read the affected room after a transition to verify state.
Repeating an effect or scene command may restart its animation or transition, so
write tools do not promise idempotence. Tools interact with the external bridge.
## Connection troubleshooting
With the ADB-server transport, first check `adb devices` and connect with
`adb connect <Fire TV IP>:5555`. If the TV is reachable but ADB reports a missing
device, reconnect it.
On macOS, "No route to host" for a LAN address that ping and TCP reach from a
terminal is Local Network privacy, not the network. A process inherits the
permission of the app that launched it, so a server or ADB daemon started from an
app without the grant (a tmux server, an editor) fails where the same command in
a permitted terminal succeeds. Grant Local Network access to that app, or run the
server on a Linux host on the same LAN.
The Fire TV connection opens on the first tool call, not at startup, so a TV that
is asleep or off does not stop the server or the lights. A call made while the
TV is unreachable fails with an error, and the next call reconnects.