mirror of
https://github.com/pewdiepie-archdaemon/odysseus.git
synced 2026-08-09 10:39:11 +02:00
c8a012d4d2* fix(memory): don't let an unreadable store get overwritten with an empty one
load_all() answered a failed read the same way it answered an empty store:
with []. Every mutation path is a read-modify-write (load the whole file,
change it, save it back), so a failed read became
load_all() -> [] -> [].append(new) -> save([new])
and save() is atomic, so the replacement stuck.
The case that actually destroys data is a store that is READABLE but not
parseable - a truncated file, or one holding {} instead of []. Nothing
obstructs the write, so adding a memory returns HTTP 200 and every memory
already stored is gone. Verified end-to-end against a running instance: on the
current code a truncated memory.json plus one add leaves the file holding only
the new entry. Truncation is reachable - core/database.py rewrites memory.json
during migration with a plain open(.., "w") + json.dump, which is not atomic.
A live exclusive lock is not the dangerous case: it blocks the read and the
os.replace alike, so the save fails too and the store survives. That path
currently 500s and loses nothing.
_read_entries() now returns [] only when the file genuinely does not exist and
raises MemoryStoreUnreadable for every other failure, including a store that
parses but is not a JSON array. load_all() keeps the old lenient behaviour so
display, search and context injection still degrade quietly instead of
breaking chat. The read-modify-write callers switch to load_all_for_update(),
which propagates the error: the memory routes turn it into a 503 and change
nothing, backup import refuses rather than saving only the incoming rows, and
auto-extraction and the audit merge skip the write. The audit merge mattered
most - it rebuilds the whole file from one owner's slice plus everyone else's
rows, so an empty read there dropped every other tenant's memories.
The corrupt-JSON path still gets its one shot at the legacy memory.txt
migration before raising, so that recovery is unchanged.
The two updated fakes gained load_all_for_update because the real class has it;
MagicMock would otherwise hand the import path a Mock instead of the seeded list.
Fixes #5673
* fix(memory): fail closed on the remaining read-modify-write add paths
The strict loader landed with the routes, the backup import and the extractor
converted, but three read-modify-write sinks still called load_all(), which
degrades an unreadable store to []. Two of them are the paths users actually
reach, so the data loss in #5673 stayed reproducible:
- src/ai_interaction.py do_manage_memory, action "add" — reached from ordinary
chat via src/tool_execution.py:793 -> dispatch_ai_tool. "Remember that I
prefer X" against an unreadable store wrote a one-entry file over it and
reported success.
- mcp_servers/memory_server.py, action "add" — the same shape through
_scope_entries(), registered as a built-in in src/builtin_mcp.py.
- src/memory_provider.py NativeMemoryProvider.remember and .delete — wired
into app state in src/app_initializer.py but not consumed outside tests yet,
converted here so the pattern is uniform before it goes live.
The MCP server takes _scope_entries(for_update=True) so list keeps the lenient
read. The edit and delete branches on both tool paths were already fail-closed
by accident — an empty view matches nothing and returns before the save — so
they are left alone.
The three new tests drive the real entry points rather than replaying the
shape, and use a truncated store, which is the case that reads back fine so
nothing stops the save. Each asserts memory.json is byte-identical afterwards;
all three fail on the previous commit with the store overwritten.
|
||
|---|---|---|
| .. | ||
| admin_wipe | ||
| cleanup | ||
| compare | ||
| contacts | ||
| document | ||
| gallery | ||
| history | ||
| memory | ||
| note | ||
| research | ||
| search | ||
| vault | ||
| webhook | ||
| __init__.py | ||
| _validators.py | ||
| admin_wipe_routes.py | ||
| api_token_routes.py | ||
| assistant_routes.py | ||
| auth_routes.py | ||
| backup_routes.py | ||
| calendar_routes.py | ||
| chat_helpers.py | ||
| chat_routes.py | ||
| chatgpt_subscription_routes.py | ||
| cleanup_routes.py | ||
| codex_routes.py | ||
| compare_routes.py | ||
| contacts_routes.py | ||
| cookbook_helpers.py | ||
| cookbook_output.py | ||
| cookbook_routes.py | ||
| copilot_routes.py | ||
| device_flow.py | ||
| diagnostics_routes.py | ||
| document_helpers.py | ||
| document_routes.py | ||
| editor_draft_routes.py | ||
| email_helpers.py | ||
| email_pollers.py | ||
| email_routes.py | ||
| embedding_routes.py | ||
| emoji_routes.py | ||
| font_routes.py | ||
| gallery_helpers.py | ||
| gallery_routes.py | ||
| history_routes.py | ||
| hwfit_routes.py | ||
| mcp_routes.py | ||
| memory_routes.py | ||
| model_routes.py | ||
| note_routes.py | ||
| personal_routes.py | ||
| prefs_routes.py | ||
| preset_routes.py | ||
| research_routes.py | ||
| search_routes.py | ||
| session_routes.py | ||
| shell_routes.py | ||
| signature_routes.py | ||
| skills_routes.py | ||
| stt_routes.py | ||
| task_routes.py | ||
| tts_routes.py | ||
| upload_routes.py | ||
| vault_routes.py | ||
| webhook_routes.py | ||
| workspace_routes.py | ||