diff --git a/docs/design/quark-architecture-review.md b/docs/design/quark-architecture-review.md deleted file mode 100644 index 28607534de..0000000000 --- a/docs/design/quark-architecture-review.md +++ /dev/null @@ -1,306 +0,0 @@ -# Quark Timeline Architecture — and How It Can Be Faster - -An architecture review of the `opencode-quark-timeline` experiment: what the layers are, why the design is sound, and the _mechanical_ reason it can beat the Solid Store timeline — with the actual code. - -## The Stack in One Picture - -Four small layers, each with one job. The reusable core remains about 200 lines. - -```mermaid -flowchart TD - E[Server events] --> D[DataProvider
Solid Store, unchanged] - D --> R[createSessionRows
row reduction policy] - R -->|"set / update / insert / remove"| K[Keyed<SessionRow, string>
packages/quark/src/keyed.ts] - K -->|"slots — structure channel"| FOR["Solid <For>"] - K -->|"slot — per-row value channel"| UV[useValue adapter
one Solid signal per row] - K -->|"slots + immutable identity"| BM[boundaries memo] - UV --> SRV[SessionRowView] - FOR --> SRV -``` - -```tree -packages/quark/src -├── reactivity.ts # 59 lines — Readable/Writable/Computed/Transaction over alien-signals -├── keyed.ts # Keyed collection (the whole idea lives here) -├── solid.ts # 8 lines — useValue: one quark Readable → one Solid signal -└── index.ts # 2 lines — exports -``` - -The experimental boundary is disciplined: `DataProvider`, the event protocol, -rendered row policy, and `SessionRowView` are unchanged. Row ownership and the -incremental mutation mechanics changed; batch and live grouping now share one -policy helper. The boundary remains narrow enough to attribute regressions or -improvements. - -## The Core Idea: Identity and Equivalence Are Inputs, Not Discoveries - -This is the entire architectural bet. Solid Store must _discover_ what changed; Quark is _told_ what identity and sameness mean, once, at construction: - -```typescript title="packages/tui/src/routes/session/rows.ts" caption="The complete reconciliation policy is two functions" -const state = Keyed.make({ key: rowKey, equivalent: sameRow }) -``` - -`rowKey` answers _"which slot is this?"_ — including the subtle case where a group's key is the ref that **created** it, so refs moving to `pending` can never change identity. Keys are precomputed once at row construction with length-prefixed segments, so reconciliation pays zero key-extraction work: - -```typescript title="packages/tui/src/routes/session/rows.ts" caption="Keys are built once, at construction — rowKey is a field read" -export type SessionRow = { readonly id: string } & ( /* ...variants... */ ) - -function partRow(ref: PartRef): SessionRow { - return { id: `p${segment(ref.messageID)}${segment(ref.partID)}`, type: "part", ref } -} - -function segment(value: string) { - return `${value.length}:${value}` // unambiguous without trusting a delimiter -} - -function rowKey(row: SessionRow) { - return row.id -} -``` - -`sameRow` answers a different question: _"can any consumer tell these two values apart?"_ It compares only render-relevant fields. If it says yes-they're-the-same, **nothing downstream runs at all**. - -## Two Channels Instead of One - -A Solid Store exposes one reactive graph; every consumer subscribes into the same proxy web. `Keyed` splits change into two independent surfaces: - -```typescript title="packages/quark/src/keyed.ts" start=4 -export interface Keyed { - readonly slots: Readable[]> // fires ONLY on insert/remove/reorder - readonly values: Readable // fires on any current value change - has(key: Key): boolean - get(key: Key): Readable | undefined - set(values: readonly A[]): void - update(value: A): boolean - insert(value: A, position?: "end" | { before: Key } | { after: Key }): Readable - remove(key: Key): boolean - move(key: Key, position?: "end" | { before: Key } | { after: Key }): boolean -} -``` - -```definitions -[ - { "term": "slots", "definition": "The structure channel. An array of stable per-row readables. Its identity changes only when membership or order changes. subscribes here." }, - { "term": "slot", "definition": "The value channel. One writable signal per row identity. A streaming delta touches exactly one slot. The row's component subscribes here via useValue." }, - { "term": "values", "definition": "The aggregate channel. A lazy computed mapping slots to current values. The TUI uses it only for internal mutation snapshots and does not subscribe boundaries to it." } -] -``` - -This separation is what a proxy-based store cannot give you for free: **a value change and a structure change are different events**, published to different audiences. - -## How It CAN Be Faster: One Unique Row Update, Step by Step - -The highest-frequency reducer workload is a repeated streaming ordinal, which -the seen-part set rejects in expected `O(1)` before reading the aggregate. The -next useful comparison is a unique event that extends or completes an existing -group row. Trace that value update through both systems. - -### Before: Solid Store path - -```typescript caption="Old hot path — every access and write crosses a proxy" -setRows( - produce((draft) => { - // 1. draft is a proxy — every property read is a trap - // 2. finding the group row walks proxied array elements - // 3. the mutation writes through proxy machinery - // 4. Solid records fine-grained dependencies per touched path - append(draft, ref, part, queuedStart(draft)) - }), -) -// ...and on reconnect / revert / rebuild: -setRows(reconcile(reduce())) -// reconcile must re-derive identity from item references, -// walk every row's shape, and diff against the proxy graph -``` - -Solid's generality — arbitrary nested access, partial path writes, plain objects — is paid for on **every operation** with proxy traps, shape inspection, and dependency bookkeeping. - -### After: Quark path - -The whole collection is a closure over three pieces of state — the structure signal, the identity map, and the declared equivalence. `update` reads directly against them: - -```typescript title="packages/quark/src/keyed.ts" caption="The closure state update() operates on, plus the whole hot path" -export function make(options: { - readonly key: (value: A) => Key - readonly equivalent?: (left: A, right: A) => boolean -}): Keyed { - const slots = State.make[]>([]) // structure channel: one signal holding the slot array - const byKey = new Map>() // identity → slot address, maintained by set/insert/remove - const equivalent = options.equivalent ?? Object.is // declared sameness (sameRow in the TUI) - const values = Computed.make((previous) => { - const next = slots().map((slot) => slot()) // aggregate channel, lazy until subscribed - return same(previous, next) ? previous! : next - }) - - return { - slots, - values, - // ... - update(value) { - const key = options.key(value) // 1. one key extraction (a field read: row.id) - const slot = byKey.get(key) // 2. one Map lookup — an address, not a search - if (!slot) throw new Error(`Keyed value does not exist: ${String(key)}`) - if (equivalent(slot(), value)) return false // 3. one equivalence check — early cutoff - slot.set(value) // 4. one signal write; slots is untouched - return true - }, - // ... - } -} -``` - -Note what `update` never touches: `slots`. A value change writes one slot signal and the structure channel stays reference-identical — that single fact is the structural cutoff below. - -Four operations. No proxies, no shape discovery, no graph diff. Then the adapter forwards the change to exactly one Solid signal: - -```typescript title="packages/quark/src/solid.ts" caption="The entire Solid boundary — 8 lines" -export function useValue(readable: Readable): Accessor { - const [value, setValue] = createSignal(readable()) - onCleanup(readable.subscribe((next) => setValue(() => next))) - return value -} -``` - -### The structural cutoff — why `` never even wakes up - -Because `slots` only fires on structural change, a value-only update leaves the outer array **reference-identical**. The `` signal never fires, so keyed-list diffing never runs and the component owner survives: - -```typescript caption="Value-only transition: structure untouched, owner preserved" -const structure = rows.slots() -const groupSlot = structure[0] - -rows.update({ ...initial, completed: true }) - -rows.slots() === structure // true — receives no change at all -rows.slots()[0] === groupSlot // true — component owner survives -groupSlot().completed === true // true — the one subscribed row re-renders -``` - -This is also the **beauty** mechanism, not just speed: an expanded reasoning group keeps its local state through permission repartitioning, because refs moving between `refs` and `pending` is a value change on a stable identity — never a remount. - -### The membership cutoff — duplicate deltas avoid the aggregate entirely - -The compiled `parts` members index contains every visible part identity, -including refs nested inside groups. A duplicate streaming delta dies at -`state.hasMember("parts", id)` before reading `state.values()`. Duplicate -message and footer events use `Keyed.has` against the collection's existing key -map. For a replacement that reaches `modify`, generated equivalence remains the -final publication cutoff. - -## More Before / After, From the Actual Diff - -### Wiring the renderer: one ``, two subscription levels - -Before, every row component read its row through the store proxy, and any reconcile could disturb the list: - -```tsx title="packages/tui/src/routes/session/index.tsx" caption="Before — one reactive graph for everything" - - {(row, index) => ( - - )} - -``` - -After, `KeyedFor` subscribes to the structure channel and each row component subscribes to exactly one slot: - -```tsx title="packages/tui/src/routes/session/index.tsx" caption="After — structure and value subscriptions are separate" - - {(row, index) => ( - - )} - -``` - -### Appending a footer row - -```typescript caption="Before — mutate a proxied draft; Solid infers what changed" -setRows( - produce((draft) => { - if (draft.some((row) => row.type === "assistant-footer" && row.messageID === messageID)) return - const index = queuedStart(draft) - completePrevious(draft, index) - draft.splice(index, 0, { type: "assistant-footer", messageID }) - }), -) -``` - -```typescript caption="After — say exactly what happened: maybe one value update, then one insert" -mutate(() => { - if (state.has(footerRowID(messageID))) return - const current = state.values() - const index = queuedStart(current) - complete(current, index) // one state.update on the previous group, if open - insert(current, index, footerRow(messageID)) // one new slot + one structural publication -}) -``` - -Same policy, but the after version _names_ its effects: at most one slot value change plus one structural change, flushed together by `Transaction.run`. Nothing has to diff anything to figure that out afterward. - -### Removing a row - -```typescript caption="Before — linear search, then splice through the proxy" -setRows( - produce((draft) => { - const index = draft.findIndex((row) => row.type === "assistant-footer" && row.messageID === messageID) - if (index !== -1) draft.splice(index, 1) - }), -) -``` - -```typescript caption="After — remove by identity; the ownership law does the rest" -mutate(() => { - state.remove(footerRowID(messageID)) -}) -``` - -### Full rebuild (reconnect, revert, compaction) - -```typescript caption="Before — reconcile re-derives identity from scratch" -setRows(reconcile(reduce())) -``` - -```typescript caption="After — set() reconciles with declared identity and equivalence" -setRows(reduce()) // → state.set(next): Map lookups + sameRow checks; unchanged rows publish nothing -``` - -This is the one place the two systems do comparable O(N) work — and it's exactly the workload the checked benchmark measured. Every path above it is where Quark does structurally _less_. - -## The Cost Ledger - -| Per operation | Solid Store (`produce`/`reconcile`) | Quark `Keyed` | -| ------------------- | ------------------------------------------- | ------------------------------------------------- | -| Find the row | proxied array walk / identity re-derivation | one `Map.get` | -| Detect "no change" | proxy-graph diff per touched path | one `equivalent()` call | -| Value change | proxy writes + per-path invalidation | one signal write → one Solid signal | -| Structure unchanged | reconcile still walks the list | outer array identity preserved; `` silent | -| Structure changed | full reconcile | one new array; retained slots reused by reference | -| Batch of writes | store batching | one `Transaction.run` — settled publication | - -Same asymptotics — `O(N)` for a whole-array `set` — but far fewer instructions, allocations, and invalidations per unit of change. **The speedup is a constant-factor win purchased with a stronger contract** (unique stable keys, declared equivalence), not framework magic. The checked-in claim: ~4.7× on sparse value publication, ~6.4× on reorder, at 1,000 rows. - -## Architecture Verdict - -### What's genuinely good - -- **Deep module, tiny surface.** One small `keyed.ts` module carries the whole idea; the laws (unique key, stable key, equivalence, structural cutoff, settled publication, ownership) are explicit and testable. -- **The adapter is honest.** `useValue` is 8 lines and creates no second reconciliation system — the failure mode the design doc itself warns about. -- **Identity-as-input is the right call** for this domain: OpenCode _has_ real identities (messageID, partID) and was previously throwing that information away for Solid to rediscover. -- **Falsifiable framing.** The docs predict where Quark should lose. That's rare and worth preserving. - -### Where the architecture leaked — all fixed during review - -Every leak identified in the first pass has since been closed: - -- ~~**`JSON.stringify` keys.**~~ Keys are precomputed `id` fields built once at row construction with length-prefixed segments (`rows.ts` smart constructors); `rowKey` is a field read. -- ~~**The aggregate channel reintroduces O(N) per delta.**~~ `boundaries` now derives from the structure channel (`rows.slots()`) and reads slot values untracked (quark reads are invisible to Solid's tracker); the raw `values` readable is no longer mirrored into Solid. Per-delta boundary cost is zero; the O(N) recompute fires only on structural change or tracked `messages()` field changes. -- ~~**Boundary staleness as an implicit invariant.**~~ The untracked-read trick is now type-enforced: `messageBoundaryIDs` accepts `readonly BoundaryRow[]`, a projection of `Pick`ed identity-immutable fields, so a future dependence on a mutable row field is a compile error. -- ~~**Two grouping engines.**~~ Join-vs-insert policy is unified in `appendDecision` + `groupKind`; both the batch `reduce()` path and the incremental `appendPart` path consume it, and row construction goes through shared smart constructors. -- ~~**Benchmark not checked in / wrong hot path.**~~ `packages/quark/bench/` now exists and covers incremental `keyed.update` — including against Solid's _direct path write_, the adversarial case — plus dense updates and reorders. -- ~~**No work counters.**~~ `Keyed` accepts an optional `Metrics` sink; `OPENCODE_QUARK_METRICS=1` reports timeline counters on cleanup. -- ~~**Opaque positioning and adapter boilerplate.**~~ `insert` and `move` now share an explicit `Position` vocabulary (`"end"`, `before`, or `after`), `get` exposes stable slot addresses, and `KeyedFor` encapsulates the two-level Solid subscription. - -The remaining asymmetry — `complete` (immutable `state.update`) vs `completePrevious` (in-place mutation on a plain array under construction) — is inherent to the two contexts, three lines each, and shares the same policy. Not worth abstracting. - -None of these are structural flaws — they are integration debts. The layering itself (laws → Keyed → thin adapter → unchanged renderer) is the right shape, and it's the shape worth keeping even if every number in the current doc had to be re-measured. - -See `quark-review-recommendations.md` for the prioritized fix list. diff --git a/docs/design/quark-message-content.md b/docs/design/quark-message-content.md deleted file mode 100644 index ea6344f6c3..0000000000 --- a/docs/design/quark-message-content.md +++ /dev/null @@ -1,94 +0,0 @@ -# Quark-Owned Message Content Slots - -Status: implemented; drive parity pending - -## Summary - -Assistant message content currently lives as unkeyed arrays inside the -DataProvider Solid store. Streaming deltas mutate those arrays through -`produce`, and every consumer that resolves a part re-scans `content` -(`findLast`, `filter(type)[ordinal]`), so per-delta work scales with message -size instead of delta size. - -This experiment changes only content ownership: each assistant message's parts -become one Quark keyed collection with stable per-part slots. Streaming events -already name the slot on the wire — `ordinal` for text and reasoning, `callID` -for tools — so deltas address slots directly instead of being rediscovered. - -Message metadata (finish, usage, retry, error), the event protocol, timeline -rows, and view components' rendering logic remain unchanged. - -The experiment succeeds only if all checks pass: - -1. Existing TUI data and row tests preserve their behavior. -2. Per-delta work is O(1) in message size: one slot publication per delta, - zero structural publications, one markdown recompute scoped to the part. -3. The drive script passes unchanged. - -## Ownership Boundary - -``` -server events - | - v -DataProvider - ├── Solid store: session/message metadata (unchanged) - └── SessionContent: Keyed per assistant message <- this experiment - | | - | slots (structure) | slot (value channel) - v v - part row resolution TextPart / ReasoningPart / ToolPart -``` - -- Part identity: `text:{ordinal}` and `reasoning:{ordinal}` from event - ordinals, tool parts by `callID`. These match the synthetic part IDs the - timeline already uses, so `resolvePart`'s positional bridge disappears. -- Deltas are value-channel publications: `content.modify(key, ...)`. The - parts ``/group structure never re-runs on a delta. -- Part starts are structural insertions; `ended` events publish the - authoritative final value into the same slot. -- Full sync (`message.sync`) seeds each message's collection with `set(parts)` - derived from the fetched payload, assigning ordinals per type in order. -- Revert, session removal, and model-switch refetch drop or reseed collections. -- Cold consumers (transcript export, copy-last-response, message navigation, - permission lookup) read `values()` — plain arrays, same shapes as before. - -## Part Layout - -Tool identity fields are immutable; streamed fields are mutable. The field -diff mask means a tool status change skips text-driven work and vice versa. - -```ts -const PartLayout = Layout.keyedUnion({ - key: Layout.key("partID", Layout.string), - tag: "type", - variants: { - text: Layout.struct({ text: Layout.string }), - reasoning: Layout.struct({ text: Layout.string, state: ..., time: ... }), - tool: Layout.struct({ - id: Layout.immutable(Layout.string), - name: Layout.immutable(Layout.string), - state: ..., time: ..., executed: ..., providerState: ..., - }), - }, -}) -``` - -Store values are the client `SessionMessageAssistant*` content shapes plus the -synthetic `partID`; `values()` strips nothing because consumers already ignore -unknown fields. - -## What Is Not In Scope - -- Message metadata stays in the Solid store. -- No changes to server events or the client package types. -- No lazy or pull-based consumers; slots publish eagerly as today. -- Timeline row ownership is the previous experiment and is unchanged. - -## Measurement - -Instrument with `Keyed.Metrics`: a streaming test drives N deltas into a -message with M existing parts and asserts slot publications == N and -structural publications == parts started, independent of M. Markdown -recompute counts are asserted through the part component render counters in -the existing test harness. diff --git a/docs/design/quark-performance-model.md b/docs/design/quark-performance-model.md deleted file mode 100644 index 3220fd4dba..0000000000 --- a/docs/design/quark-performance-model.md +++ /dev/null @@ -1,532 +0,0 @@ -# Why Quark Can Be Faster Than Solid Store - -Status: experimental performance model - -## Conclusion - -Quark can be faster than the current Solid Store timeline because it accepts a -stronger contract and uses that contract to do less runtime work. - -The advantage does **not** come from TypeScript types by themselves. TypeScript -types disappear at runtime. The useful information comes from explicit runtime -inputs and API laws: - -- A row key is unique and cannot change during its ownership lifetime. -- Row equivalence is supplied before updates begin. -- A keyed list has one stable slot per key. -- Value changes and structural changes publish through separate channels. -- A transaction publishes only settled graph state. - -Those guarantees remove choices from the hot path. Quark does not need to -rediscover identity, shape, equivalence, index membership, or ownership on -every update. Less discovery means fewer comparisons, proxy traps, -allocations, writes, and downstream invalidations. - -This is a conditional claim, not a claim that Quark is universally faster than -Solid. Solid can be as fast or faster when an application performs a precise -direct store write, when lists are tiny, or when Quark receives no useful -identity information. The relevant comparison is the current OpenCode path: -whole projected records and ordered rows are reconciled into a Solid Store. - -## The Physical Argument Is “Do Less Work” - -Software cannot escape the physical costs of instructions, memory reads, -allocations, pointer writes, cache misses, and notification fan-out. An -abstraction can only become faster by reducing those costs or arranging them -more favorably. - -Quark's proposed advantage has four concrete sources: - -1. **Declare repeated decisions once.** Identity and equivalence become - preconfigured functions rather than per-event discovery. -2. **Replace searches with addresses.** Stable keys resolve to persistent - slots through a `Map`. -3. **Separate unrelated change dimensions.** An item value can change without - publishing a new list structure. -4. **Stop propagation early.** Schema equivalence and slot equivalence suppress - writes before they reach Solid or the terminal renderer. - -These are ordinary computer costs, not framework mythology. If profiling shows -that Quark executes the same work as Solid plus an adapter, the hypothesis is -false. If counters show fewer structural publications, fewer component-owner -reconciliations, and fewer proxy/store operations, the speedup has a mechanical -explanation. - -## Solid Store Must Preserve Generality - -The current timeline asks Solid Store to accept ordinary JavaScript objects and -reconcile new projections into a proxy-backed graph. - -```mermaid -flowchart LR - A[Projected rows] --> B[Solid reconcile] - B --> C[Inspect keys and shape] - C --> D[Proxy-backed Store writes] - D --> E[Dependency invalidation] - E --> F[For reconciliation] - F --> G[SessionRowView] -``` - -Solid's generality is valuable. It can react to arbitrary nested property -access, partial path writes, and plain objects without requiring a schema-owned -model. That generality also means the runtime must maintain proxy metadata and -interpret each reconciliation against the current store graph. - -Solid is not inherently inefficient. A precise call such as -`setStore(index, "value", next)` can avoid whole-list reconciliation. OpenCode's -timeline frequently receives whole projected messages or rebuilt row arrays, -however, so its current implementation uses `produce` and `reconcile` to -recover identity and preserve nested owners. - -## Quark Makes Identity an Input - -`Layout.collection` compiles identity, equivalence, and indexes once when the -timeline is created. Every row carries a collision-safe primitive `id`. Groups -also retain their immutable origin for message-boundary projection: - -```typescript -const PartRefLayout = Layout.struct({ - messageID: Layout.string, - partID: Layout.string, -}) - -const SessionRowLayout = Layout.keyedUnion({ - key: Layout.key("id", Layout.string), - tag: "type", - variants: { - message: Layout.struct({ messageID: Layout.string }), - "compaction-queued": Layout.struct({ inputID: Layout.string }), - part: Layout.struct({ ref: PartRefLayout }), - group: Layout.union({ - tag: "kind", - variants: { - reasoning: Layout.struct({ - origin: Layout.immutable(PartRefLayout), - refs: Layout.array(PartRefLayout), - completed: Layout.boolean, - }), - exploration: Layout.struct({ - origin: Layout.immutable(PartRefLayout), - refs: Layout.array(PartRefLayout), - pending: Layout.array(PartRefLayout), - completed: Layout.boolean, - }), - }, - }), - "assistant-footer": Layout.struct({ messageID: Layout.string }), - }, -}) - -type PartRef = Layout.Type -type SessionRow = Layout.Type -``` - -Factories build the ID once with length-prefixed segments. Length prefixes -avoid delimiter collisions without invoking a serializer: - -```typescript -function segment(value: string) { - return `${value.length}:${value}` -} - -function groupRow(kind: "reasoning" | "exploration", origin: PartRef): SessionRow { - const id = `g${kind === "reasoning" ? "r" : "e"}${segment(origin.messageID)}${segment(origin.partID)}` - if (kind === "reasoning") return { id, type: "group", kind, origin, refs: [origin], completed: false } - return { id, type: "group", kind, origin, refs: [origin], pending: [], completed: false } -} -``` - -Generated equivalence answers a different question: can consumers distinguish -these two values for the same identity? The layout compares rendered fields, -dispatches through the row and group tags, and ignores immutable `origin`. - -```typescript -const SessionRows = Layout.collection( - SessionRowLayout, - ({ members }) => ({ - parts: members(partIDs), - }), - { backend: "generated" }, -) - -const rows = SessionRows.make() -``` - -The `parts` index contains standalone and grouped part IDs. Duplicate stream -deltas return through `hasMember` before aggregate projection. Arbitrary -replacements derive membership automatically; the hot group-append path applies -the exact one-member delta already known from the event. - -It exposes two reactive surfaces: - -```typescript -rows.slots // changes only when keys are inserted, removed, or reordered -rows.values // changes when any current value changes -``` - -Each slot is itself a readable value. Solid `` receives `slots`, so a -group can grow or complete without replacing its component owner. Generic -aggregate consumers can subscribe to `values` when they need every value -change. - -The TUI's message-boundary projection deliberately does not subscribe to -`values`. Boundary identity uses only immutable row fields, so it tracks the -structural `slots` array and reads each slot value untracked. Value-only deltas -therefore perform no boundary work; inserts, removals, reorders, or message -changes recompute boundaries. - -Here is a complete value-only transition. The group changes, but its key and -position do not: - -```typescript -const origin = { messageID: "assistant-1", partID: "call-read" } -const initial = groupRow("exploration", origin) - -rows.set([initial]) - -const structure = rows.slots() -const groupSlot = structure[0] - -rows.modify(initial.id, (group) => ({ ...group, completed: true })) - -rows.slots() === structure // true: receives no structural change -rows.slots()[0] === groupSlot // true: component owner survives -groupSlot().completed === true // true: row value updated -``` - -A structural insertion changes the outer list but preserves every retained -slot: - -```typescript -const footer: SessionRow = { - id: "f11:assistant-1", - type: "assistant-footer", - messageID: "assistant-1", -} - -const footerSlot = rows.insert(footer) - -rows.slots().length === 2 // true -rows.slots()[0] === groupSlot // true -rows.slots()[1] === footerSlot // true -``` - -Removing and later reinserting a key begins a new ownership lifetime: - -```typescript -rows.remove(footer.id) -const nextFooterSlot = rows.insert(footer) - -nextFooterSlot === footerSlot // false -``` - -```mermaid -flowchart LR - A[Next rows] --> B[Keyed.set] - B --> C{Same keys and order?} - C -->|yes| D[Keep slots array] - C -->|no| E[Publish slots array] - B --> F{Equivalent value?} - F -->|yes| G[No slot write] - F -->|no| H[Publish one slot] - D --> I[Solid For unchanged] - E --> J[Solid For reconciles structure] - H --> K[Existing SessionRowView updates] -``` - -The key distinction is that **row identity is not inferred from row value**. -The timeline gives every reasoning or exploration group an immutable creation -key. Permission partitioning may move refs between `refs` and `pending`, but it -cannot change the group slot or reset local expanded state. - -## The `Keyed.set` Algorithm - -The current algorithm is deliberately small: - -```text -set(next): - 1. Compute every next key and reject duplicates. - 2. Look up retained slots in the persistent key map. - 3. For each next value: - a. Reuse the previous slot for its key, or create one. - b. Run generated equivalence over previous and next values. - c. Write only a changed slot. - 4. Publish the outer slot array only if membership or order changed. - 5. Synchronize declared collection indexes. - 6. Flush slot, index, and structural writes in one transaction. -``` - -The aggregate `values` readable maps the current slots to their values. It is a -computed node, so it remains lazy without subscribers and publishes one settled -array when subscribed. - -The algorithm preflights duplicate keys before any slot mutation. A rejected -set therefore cannot partially update the graph. `Map` and `Set` give keys -SameValueZero semantics, including consistent treatment of `0`/`-0` and -`NaN`. - -## The Laws That Unlock the Optimizations - -```definitions -[ - { - "term": "Unique key law", - "definition": "No two current values have the same key. This permits one Map entry and one slot per identity." - }, - { - "term": "Stable key law", - "definition": "An entity or logical row keeps its key for its lifetime. This permits component ownership and subscriptions to survive value changes." - }, - { - "term": "Equivalence law", - "definition": "If equivalent(left, right) is true, publishing right cannot change any consumer-visible meaning. This permits early cutoff." - }, - { - "term": "Structural cutoff law", - "definition": "Value-only updates preserve the exact outer slots-array identity. This prevents keyed-list reconciliation for non-structural changes." - }, - { - "term": "Settled publication law", - "definition": "Subscribers observe the state after every slot and structural write in the operation, never a partial combination." - }, - { - "term": "Ownership law", - "definition": "Removing a key removes its slot from the collection. Reinserting that key creates a fresh slot; old external references cannot attach to a new lifetime." - } -] -``` - -These laws are stronger than a plain `Array` contract. They are also -testable. Quark should reject a violated unique-key law and should have direct -tests for every other law. - -## Layout Compiles Trusted Collection Metadata - -The timeline now uses Quark's trusted in-memory `Layout`. It supports the scope -required by this experiment: - -- primitive, array, struct, discriminated-union, and keyed-union fields; -- immutable fields excluded from rendered equivalence; -- compiled key extraction and closure or generated equivalence; -- imperative multi-value membership and ordered first-match indexes; -- automatic index derivation for arbitrary replacements; -- typed member deltas for event-native mutations that know exact changes. - -Generated nested union comparison measured `1.06x-1.10x` handwritten cost, -versus `1.19x-1.22x` for closure Layout across three runs. Generation occurs -once when `SessionRows` is declared. The older `1.322x` closure result motivated -this backend; it no longer describes the production comparator. - -Automatic membership derivation is linear in an affected row's members. That -is the safe default, but it made repeated append to one growing group roughly -`27x-30x` the handwritten index path. The event already identifies the new -part, so `SessionTimeline.appendPart` supplies a typed one-member delta. Generic -completion and permission repartition retain automatic derivation. - -Numeric field positions, change masks, per-field reactive slots, models, and -columnar storage remain future research. They are not required by the TUI. - -## Asymptotics and Constants - -For a whole next array of `N` keyed values, both Quark `Keyed.set` and a general -keyed reconciliation are `O(N)`. Quark does not break a lower bound: it must -read the next keys to validate and order them. - -The observed win is currently a **constant-factor win at the same asymptotic -complexity**: - -- one key extraction per next item; previous slots are already indexed; -- one `Map` lookup per next item; -- one explicit equivalence check; -- one slot write per changed value; -- one outer write only for changed structure; -- no general nested proxy reconciliation inside Quark. - -For direct collection operations, stronger bounds are possible. An immutable -key lets `has`, `get`, `hasMember`, and `modify` find their target in expected -`O(1)` time. Work is then proportional to the changed row, affected declared -indexes, and any ordered structural array copy. `SessionTimeline` caches the -active group and queued boundary IDs; inserts and removals remain `O(N)` because -the ordered slots array must be copied. Full synchronization and revert rebuilds -remain linear `set` operations. - -## Controlled Benchmark Evidence - -The benchmarks are checked into the fork and run directly: - -```bash -cd packages/quark -bun run bench:keyed -bun run bench:row-key - -cd ../tui -bun run bench:timeline -``` - -Each invocation interleaves and rotates variants over nine measured samples -after a discarded warmup. It reports median nanoseconds per operation, median -absolute deviation, machine-readable metrics, and a checksum. - -The original whole-array benchmark established the initial direction: - -| Workload | Run | Quark | Solid Store | Quark / Solid | -| -------------------------------------------- | --: | -------: | ----------: | ------------: | -| One value changes in a 1,000-item next array | 1 | 177.2 us | 863.7 us | 0.207x | -| One value changes in a 1,000-item next array | 2 | 158.8 us | 755.6 us | 0.213x | -| Reorder a 1,000-item list | 1 | 155.3 us | 1,020.8 us | 0.157x | -| Reorder a 1,000-item list | 2 | 147.1 us | 975.6 us | 0.152x | - -Across these two invocations: - -- keyed value publication was approximately **4.7x-4.8x faster**; -- keyed reorder was approximately **6.4x-6.6x faster**. - -The checksum was stable, and both variants performed the same next-array -construction inside their timed workloads. - -The refreshed lower-level `Keyed` benchmark measures event-native updates with -the aggregate `values` channel subscribed, plus adverse workloads. It does not -include `SessionTimeline`, generated row equivalence, or compiled index -maintenance. The July 17, 2026 evidence run produced: - -| Workload | Quark | Solid | Quark / Solid | -| ------------------------------------ | -------: | -------: | ------------: | -| Direct update, no subscribers, 1,000 | 46.8 ns | 761.6 ns | 0.068x | -| Precise Solid path write, 1,000 | 54.3 ns | 327.5 ns | 0.170x | -| Subscribed aggregate, 10 rows | 608.0 ns | 7.4 us | 0.085x | -| Subscribed aggregate, 100 rows | 4.4 us | 80.0 us | 0.055x | -| Subscribed aggregate, 1,000 rows | 37.4 us | 703.1 us | 0.052x | -| Subscribed aggregate, 10,000 rows | 388.5 us | 8.1 ms | 0.044x | -| Dense 1,000-row update | 270.2 us | 2.3 ms | 0.132x | -| Unstable keys, 100 rows | 68.0 us | 245.2 us | 0.282x | - -A second independent invocation produced paired ratios of `0.084x`, -`0.170x`, `0.077x`, `0.059x`, `0.061x`, `0.042x`, `0.130x`, and `0.266x` -in the same row order. The direct-path result reproduced exactly to three -decimal places; the other workloads retained the same direction and broad -magnitude despite normal timing variance. - -The aggregate path is still `O(N)`, but it remained substantially cheaper than -the equivalent Solid Store projection. The precise direct-path case was added -specifically to favor Solid: it calls `setValues(index, "value", next)` while -Quark still constructs and compares a replacement object. The predicted Solid -win did not occur in this environment; Quark measured `0.170x`. This is a -falsified prediction for this setup, not evidence that Solid can never win a -direct-write comparison. The production TUI has no aggregate subscriber; it -tracks structural slots and immutable boundary metadata instead. - -The checked-in `SessionTimeline` benchmark compares the final domain module to -the previous handwritten `Keyed + Set` reducer mechanics and the original Solid -Store `produce` path. Across three runs: - -| Workload | SessionTimeline result | -| ----------------------------------------------------- | ----------------------: | -| Growing reasoning-group append / handwritten Quark | about `1.3x-1.5x` | -| Growing reasoning-group append / Solid `produce` | about `0.013x` | -| Duplicate part in a 1,000-ref group / Solid `produce` | about `0.0004x-0.0005x` | - -The final module pays roughly 30%-50% over the minimal handwritten Quark path -for compiled layout dispatch, domain policy, and maintained index ownership. -It remains roughly 74x-79x faster than the baseline Solid append in this -growing-group stress workload. Duplicate lookup is expected `O(1)` through the -compiled members index; the Solid baseline scans 1,000 refs, producing an -approximately 1,960x-2,500x stress-case difference. These are state-operation -results, not claims about total OpenCode response latency. - -Key extraction was measured independently over 10,000 representative rows: - -| Key strategy | Median | Paired ratio to JSON | -| --------------------------------------- | ----------: | -------------------: | -| `JSON.stringify` tuple | 109.6 ns/op | 1.000x | -| Collision-safe length-prefixed string | 25.6 ns/op | 0.205x | -| Precomputed primitive row ID field read | 8.7 ns/op | 0.075x | - -The TUI now uses the precomputed primitive ID strategy. - -## What the Benchmark Does Not Prove - -The suite compares whole-array reconciliation, direct point updates, subscribed -aggregate projection, dense changes, unstable keys, key extraction, growing -timeline groups, and duplicate deltas. It does not prove that Quark beats: - -- every Solid keyed-list primitive; -- rendering, terminal layout, or paint; -- all list sizes and mutation distributions; -- a complete OpenCode session under real provider and tool traffic. - -The end-to-end `opencode-drive` runs establish behavioral parity, including -visible terminal completion and projected reasoning/text content. Their timing -varied too widely to support a speed claim because it includes provider -simulation, process scheduling, event transport, terminal rendering, and UI -polling. - -## When Quark Should Lose - -The design predicts smaller or negative gains when: - -- lists are so small that setup and adapter costs dominate; -- keys are unstable or expensive to compute; -- equivalence is more expensive than simply publishing; -- almost every item and the structure change on every operation; -- there are no subscribers, so reactive precision has no downstream value; -- Solid receives a cheaper primitive update while Quark must perform a more - expensive projection or whole-array reconciliation; -- copying the next application-level projection dominates both runtimes; -- the Quark-to-Solid adapter duplicates work rather than cutting it off. - -These are falsifiable predictions. The current adverse suite did not find a -Solid win, including the newly added precise path write, but it preserves these -cases so future changes cannot optimize only the favorable sparse path. - -## Deterministic Tests Measure Work, Not Just Time - -The single-binary deterministic event trace counts: - -| Deterministic transition | Slot delta | Structure delta | Observed ownership behavior | -| ------------------------------ | ---------: | --------------: | -------------------------------------------- | -| First exploration part | +0 | +1 | New group slot | -| Extend exploration group | +1 | +0 | Existing group slot retained | -| Permission repartition | +1 | +0 | Existing group slot retained | -| Insert queued user row | +0 | +1 | Group remains mounted and incomplete | -| Complete promoted-input group | +1 | +0 | Existing group slot retained | -| Duplicate text delta | +0 | +0 | Returns through compiled membership index | -| Full unchanged two-row rebuild | +0 | +0 | Two equivalence suppressions, no publication | - -`Keyed` exposes optional counters for slot publications, structural -publications, and equivalence suppressions. The focused TUI trace -`completes exploration when a queued prompt is promoted` records the first five -transitions and asserts the same slot identity across extension, repartition, -and completion. `does not publish timeline rows for duplicate streaming -deltas` records the duplicate path. The direct Keyed law test records the full -unchanged rebuild. Boundary tests independently assert that value-only changes -do not invalidate structural boundary projection. - -Pure timeline tests additionally cover cached queue advancement, out-of-order -promotion, duplicate message/footer idempotence, stable group ownership, and -parity between event-native operations and snapshot reduction. Unique append -uses cached IDs and never reads `state.values()`. - -## Decision Rule - -Continue the Quark timeline experiment if deterministic replay confirms all of -the following: - -1. Behavior and component ownership remain equivalent to the Solid baseline. -2. Value-only events publish no structural changes. -3. CPU or allocation cost improves on the real event distribution. -4. The Solid adapter remains a thin boundary rather than a second reactive - system doing duplicate work. -5. The reusable `Keyed` and layout laws contain the complexity; TUI code - does not grow its own reconciliation engine. - -If those conditions fail, the standalone microbenchmarks are not sufficient -reason to replace Solid Store. If they hold, the speedup is not mysterious: -Quark is faster because OpenCode supplied stronger information and Quark used -that information to execute less work. - -The current evidence satisfies ownership, structural cutoff, adapter -thinness, and containment in `Keyed`. Controlled workloads and the -deterministic TUI trace show lower CPU work for the exercised event shapes. -That is enough to continue and land the timeline experiment, not to claim a -universal framework win. Real-session soak remains the check on whether the -measured event distribution matches production usage. diff --git a/docs/design/quark-review-recommendations.md b/docs/design/quark-review-recommendations.md deleted file mode 100644 index c38bfb1c72..0000000000 --- a/docs/design/quark-review-recommendations.md +++ /dev/null @@ -1,137 +0,0 @@ -# Quark Timeline Experiment: Review Recommendations - -Status: point-in-time review feedback; accepted performance and correctness findings implemented -Reviewed: `docs/design/quark-performance-model.md`, `docs/design/quark-tui-timeline.md`, `packages/quark/src/*`, `packages/tui/src/routes/session/rows.ts`, `packages/tui/src/routes/session/index.tsx` - -Goal: really fast and really beautiful. These are ordered recommendations for the next agent. Each item is independently actionable; the order reflects priority. - -This document preserves the audit as written. The current implementation has -checked benchmarks and adverse workloads, structural-only boundaries, -precomputed IDs, shared grouping policy, filtered permission repartition, -optional publication counters, deterministic counter tests, and an `O(1)` -seen-part duplicate path. Current evidence and commands live in -`quark-performance-model.md` and `quark-tui-timeline.md`; claims below describe -the pre-fix state. Optional API polish and broader model/index/layout work are -deferred to avoid expanding the timeline landing. - -## 1. Check in the microbenchmark - -`quark-performance-model.md` cites concrete numbers (177.2µs vs 863.7µs, etc.) but the benchmark source is not in the tree. Unreproducible numbers are the weakest part of an otherwise careful doc. - -- Add the benchmark under `packages/quark/bench/keyed.bench.ts` (or similar). -- Add the exact repro command to the "Controlled Benchmark Evidence" section of `quark-performance-model.md`. -- Add the adverse workloads promised in "When Quark Should Lose" (tiny lists, unstable keys, dense mutation, no subscribers, direct `setStore(index, "field", value)` on the Solid side). That section is currently a promise, not a suite. - -## 2. Benchmark the actual hot path: incremental `update` with subscribed `values` - -The checked-in benchmark story compares `Keyed.set(nextArray)` vs Solid `reconcile(nextArray)`. But after the migration, the streaming hot path is **incremental**: `state.update({...previous, refs: [...refs, ref]})` per delta (`packages/tui/src/routes/session/rows.ts`). Whole-array `set` only runs on reconnect/revert/compaction. - -Add a workload: one `update` on a group row, with `values` subscribed, at 100 / 1,000 / 10,000 rows. This measures what the TUI actually does per streaming delta. - -## 3. Fix the hidden O(N) per value change through `values` + `boundaries` - -The performance-model doc presents `values` as "lazy without subscribers," but in the real integration it is **always subscribed**: - -```ts -// packages/tui/src/routes/session/index.tsx (~line 190) -const boundaries = createMemo(() => messageBoundaryIDs([...rows.values()], messages())) -``` - -So every value-only change that survives `sameRow` pays: - -- O(N) recompute of the `values` computed (N slot reads + dependency tracking, `packages/quark/src/keyed.ts:22-25`) -- O(N) `same` compare -- O(N) array spread in the memo -- a full `messageBoundaryIDs` recompute - -The structural cutoff for `` is real, but this is the remaining per-delta O(N) path. Options, in increasing ambition: - -1. Change `messageBoundaryIDs` to accept `readonly SessionRow[]` so the spread copy dies (trivial). -2. Derive boundaries from `slots` plus per-slot reads so a group value change cannot invalidate them. -3. Maintain boundary info incrementally inside the same mutate operations that change structure. - -At minimum, the doc should state the real cost with a measured number instead of implying laziness the integration doesn't have. - -## 4. Replace `JSON.stringify` row keys with cheap concatenation - -`rowKey` calls `JSON.stringify` per row per `set` (`packages/tui/src/routes/session/rows.ts:439-445`). On a 10,000-row `set` that's 10,000 serializer calls plus allocations, working directly against the "compile repeated decisions once" thesis. - -Use a delimiter that cannot appear in IDs: - -```ts -function rowKey(row: SessionRow) { - if (row.type === "message") return `message\x00${row.messageID}` - if (row.type === "compaction-queued") return `compaction-queued\x00${row.inputID}` - if (row.type === "part") return `part\x00${row.ref.messageID}\x00${row.ref.partID}` - if (row.type === "assistant-footer") return `assistant-footer\x00${row.messageID}` - return `group\x00${row.kind}\x00${row.key.messageID}\x00${row.key.partID}` -} -``` - -Re-measure after; expect a meaningful constant-factor win on key extraction. Update the code sample in `quark-performance-model.md` to match. - -## 5. Unify the two grouping-policy engines - -There are now two implementations of the row grouping rules: - -- The batch rebuild path: `reduce()` builds a plain array via the old `append` / `completePrevious` helpers (`rows.ts:385+`), then calls `state.set`. -- The incremental path: `appendPart` / `appendMessage` / `complete` closures reimplement the same join/complete decisions inline (`rows.ts:147-215`). - -The performance-model doc's own decision rule #5 says "TUI code does not grow its own reconciliation engine" — this duplication is exactly that risk. Extract the shared decisions as pure functions, e.g. `joinsPreviousGroup(previous, part)` and `newRowFor(part, ref)`, and have both paths consume them. Do not lose the incremental path's precision; only deduplicate the policy. - -## 6. Filter before cloning in permission repartition - -The pending-permissions effect clones **every** exploration group on every change before `sameRow` suppresses the no-ops (`rows.ts:83-91`): - -```ts -state.values().forEach((row) => { - if (row.type !== "group" || row.kind !== "exploration") return - const next = { ...row, refs: [...row.refs], pending: [...row.pending] } - partitionPending([next], pending) - state.update(next) -}) -``` - -Filter to groups whose refs actually intersect the pending set before cloning. The clone-then-compare pattern spends the allocations the equivalence law was supposed to save. - -## 7. Add work counters to `Keyed` - -The strongest section of the performance model is "The Next Tests Must Measure Work, Not Just Time" — but `Keyed` has no instrumentation, so none of that table can be produced today. - -Add optional counters behind a debug flag in `packages/quark/src/keyed.ts`: - -- slot publications (value writes) -- structural publications (outer `slots` writes) -- equivalence suppressions (writes avoided by `equivalent`) - -Then run the deterministic trace via `script/quark-timeline-drive.ts` and fill in the expected-behavior table with real counts. This converts the decision rule from vibes to numbers. - -## 8. Demonstrate the "beautiful" claim with a recording - -The user-visible payoff — stable component ownership means no flicker, no reset of expanded reasoning groups, no scroll jumps during permission repartition — is asserted but never demonstrated. - -Capture a terminal-control recording of an expanded group surviving a permission repartition on this branch vs untouched `origin/v2`. That artifact is more persuasive than any benchmark table and belongs alongside it in the doc. - -## 9. Small API and code polish - -- ~~`insert(value, { before? })` where `before: undefined` means append reads poorly at call sites.~~ Fixed with the shared `Position` vocabulary: `"end"`, `{ before: key }`, or `{ after: key }`. -- The duplicate-key error in `Keyed.set` should name the offending key (`packages/quark/src/keyed.ts:32`). -- `update` mixes contracts: throws on missing key, returns `false` on no-op. That's defensible (missing key is a programmer error) but should be documented as a law. -- Doc correction in `quark-performance-model.md`, "Asymptotics and Constants": it claims "one key extraction per previous and next item," but the implementation only extracts **next** keys; previous slots are reached through the `byKey` map (`keyed.ts:30-52`). Fix the claim. - -## What is already good (do not churn) - -- The two-channel `slots` / `values` surface plus verbs is a genuinely clean API. Keep it. -- The laws section and the honest "When Quark Should Lose" / "What the Benchmark Does Not Prove" framing are correct and rare. Preserve that tone. -- The experimental boundary (only the row owner changes; `DataProvider`, reduction rules, and `SessionRowView` untouched) is right. Do not widen it while acting on these items. -- Duplicate-key preflight before any mutation (no partial updates) is correct; keep it. - -## Suggested execution order - -1. Check in benchmark + adverse workloads (items 1, 2) -2. Cheap key extraction, re-measure (item 4) -3. Unify grouping engines (item 5) -4. Counters + deterministic drive trace (item 7) -5. Decide the `values`/boundaries O(N) question with data (item 3) -6. Repartition filtering (item 6) -7. Recording artifact + doc corrections + polish (items 8, 9) diff --git a/docs/design/quark-tui-timeline.md b/docs/design/quark-tui-timeline.md deleted file mode 100644 index 993115f0ea..0000000000 --- a/docs/design/quark-tui-timeline.md +++ /dev/null @@ -1,447 +0,0 @@ -# Quark-Owned TUI Timeline Rows - -Status: experiment in progress - -## Summary - -The V2 TUI previously stored its rendered session-row list in a Solid Store. -This experiment changes only that owner: `SessionTimeline` stores an ordered -array of stable row slots in a compiled Quark collection. Each slot owns one -`SessionRow`, and an owner-aware adapter exposes both levels to the existing -Solid renderer. - -The event protocol, durable session data, `DataProvider`, row reduction rules, -and `SessionRowView` remain unchanged. This boundary makes the experiment easy -to compare and easy to remove. Message content remains in `DataProvider`; -`SessionTimeline` owns a finite imperative index of part IDs for its current -rows. - -The experiment succeeds only if all three checks pass: - -1. Existing TUI timeline tests preserve their behavior. -2. One `opencode-drive` script passes unchanged against untouched `origin/v2` - and this branch. -3. The Quark path improves measured timeline work without increasing visible - latency or invalidation counts. - -## The Row Owner Is the Experimental Boundary - -`DataProvider` receives server events and projects them into its existing Solid -Store. `createSessionRows` reads that projection, subscribes to timeline events, -and maintains the ordered rows consumed by `SessionRowView`. - -Before this branch, `createSessionRows` also used a Solid Store for its row -state: - -```text -server events - | - v -DataProvider Solid Store - | - v -createSessionRows - | - v -SessionRow[] Solid Store - | - v - -> SessionRowView -``` - -This branch replaces only the second store: - -```text -server events - | - v -DataProvider Solid Store - | - v -createSessionRows adapter - | - v -SessionTimeline - | - v -Layout.collection - | - v -KeyedFor outer + slot adapters - | - v -SessionRowView -``` - -The narrow boundary answers one question: does Quark improve the TUI timeline -when it owns the ordered render rows? Moving message storage at the same time -would make a regression or improvement impossible to attribute. - -## Components and Responsibilities - -| Component | Responsibility | -| --------------------------------------------- | -------------------------------------------------------------------------------------------------- | -| `packages/tui/src/context/data.tsx` | Preserve the existing server-event projection and message lookup API. | -| `packages/tui/src/routes/session/rows.ts` | Translate DataProvider state and events into timeline operations; own the Solid adapter lifecycle. | -| `packages/tui/src/routes/session/timeline.ts` | Declare row layout, reduce snapshots, cache cursors, and implement timeline domain mutations. | -| `packages/quark/src/layout.ts` | Compile row keys/equivalence and maintain declared imperative collection indexes. | -| `packages/quark/src/reactivity.ts` | Provide synchronous state updates, subscriptions, and transaction batching. | -| `packages/quark/src/keyed.ts` | Reuse per-key slots and separate value changes from structural changes. | -| `packages/quark/src/solid.ts` | Bridge Quark readables into Solid ownership and compose stable slots with `KeyedFor`. | -| `packages/tui/src/routes/session/index.tsx` | Render stable row accessors through `KeyedFor` and the existing `SessionRowView` components. | -| `script/quark-timeline-drive.ts` | Exercise the same streamed timeline scenario against baseline and fork binaries. | - -## Quark State Publishes One Synchronous Value - -`State.make(initial)` wraps an `alien-signals` signal with three operations: - -```ts -read() // track and return the current value -read.set(next) // publish one replacement value -read.update(f) // derive and publish from the current value -``` - -`subscribe` creates an `alien-signals` effect. The effect reads the signal once -to establish its dependency without calling the listener, because `useValue` -already reads the initial value. Later signal writes rerun the effect -synchronously and call the listener once with the published value. - -Before invoking the listener, `subscribe` clears alien-signals' active -subscriber and restores it afterward. Reads performed by the listener or by a -synchronous Solid update therefore cannot become accidental dependencies of -the Quark subscription. - -Transactions call `startBatch()` and `endBatch()`. Row reconciliation can -update several slots plus the outer slot array, so the row owner uses one -transaction for the complete publication. - -## `useValue` Crosses the Reactive Boundary Once - -`useValue(readable)` creates one Solid signal initialized from `readable()`. It -then subscribes to Quark and forwards each published value into that Solid -signal. The timeline uses this algorithm for the structural slot list and each -mounted row slot. Its aggregate `values` readable remains inside Quark and is -not mirrored into Solid: - -```text -outer Quark State.set(nextSlots) - | - v -alien-signals subscription - | - v -Solid setSignal(nextSlots) - | - v -rowSlots() -> - | - v - slot Quark State.set(nextRow) - | - v - row() -> SessionRowView -``` - -The adapter passes `() => nextRows` to Solid's setter. Solid treats a bare -function as an updater, so the wrapper is required when the reactive value -itself could be a function. The same generic adapter therefore works for both -arrays and callable values. - -## Full Rebuilds Use a Deterministic Reducer - -Connection, synchronization, revert, pending-compaction, and message-boundary -changes rebuild rows from the current `DataProvider` projection. The rebuild -algorithm is: - -```text -messages = loaded messages before the revert boundary -inputs = admitted input message IDs -pending = running compaction IDs + input IDs - -ordered messages = - completed messages - + running compactions - + admitted inputs - -for each ordered message: - non-assistant -> append one message row - assistant -> append/group each non-empty content part - finished assistant -> append one footer row - -partition exploration refs by pending permission call IDs -insert queued compactions before the first pending input row -``` - -The first partition puts completed transcript rows before work that has not -crossed its safe execution boundary. A non-pending message completes the group -immediately before it. This prevents reasoning or exploration groups from -visually spanning separate transcript messages. - -Assistant text and reasoning parts use their ordinal among parts of the same -type as their stable part ID, such as `text:0` or `reasoning:1`. Tool parts use -their call ID. Empty text and reasoning parts do not create rows. - -## Incremental Events Avoid Full Reduction - -Most live events update the current row array directly: - -| Event shape | Row operation | -| --------------------------------------------------------------- | --------------------------------------------------------------- | -| New user, synthetic, shell, model, agent, or compaction message | Insert a message row unless its ID already exists. | -| First non-empty text or reasoning delta | Insert or extend the corresponding part/group row. | -| Tool input started | Insert a tool row or extend an exploration group. | -| Step ended, failed, or retry scheduled | Insert an assistant footer. | -| Step started | Remove the previous retry footer. | -| Permission changed | Repartition exploration refs between visible and pending lists. | - -`SessionTimeline` caches the first queued row ID and the active incomplete group -ID. Full replacement derives both once. Live operations maintain them as work -is queued or promoted. New assistant parts and completed messages insert before -the cached queue boundary; pending user inputs append after queued work; a -running compaction becomes the new boundary. - -`append(rows, ref, part, index)` implements grouping: - -```text -reasoning part: - extend the immediately preceding reasoning group - or complete the preceding group and create a reasoning group - -read/glob/grep tool: - extend the immediately preceding exploration group - or complete the preceding group and create an exploration group - -other text or tool part: - complete the preceding group and insert a standalone part row -``` - -The grouping decision depends on the tool name, not the call ID. This matters -because call IDs can look like generated text or reasoning IDs. - -## Incremental Mutations Touch Only Affected Slots - -The old Solid `produce` path created a mutable draft of the row list. The first -Quark implementation also rebuilt a complete next array, then reconciled it. -The event-native path now calls domain operations backed by the compiled -collection: - -```text -group change -> state.modify(groupID, update) -new row -> state.insert(row, { before }) -removed footer -> state.remove(row.id) -full sync / revert -> state.set(nextRows) -``` - -Extending or completing a group creates one new group value with copied refs -and writes its existing slot. It does not copy unrelated groups or publish -structure. Permission repartitioning first checks whether a group's visible or -pending membership would actually change, then allocates arrays only for those -groups. - -Message and footer duplicates use the keyed map. Part duplicates use the -compiled `parts` members index and return before aggregate projection. Unique -group appends use cached IDs and do not read `state.values()`. The append event -also supplies the exact one-member index delta, avoiding a scan of the growing -group; arbitrary replacements retain automatic index derivation. - -## `Keyed` Preserves Row Identity - -A full rebuild creates fresh row values, but Solid `` uses item identity -to preserve child ownership. A changed group value must not remount its -`SessionRowView`, because that would reset the group's local expanded and hover -state. `` therefore receives stable Quark slots rather than row values. - -`Keyed.set(nextRows)` preserves ownership through its persistent key-to-slot -map: - -```text -1. Validate the next keys. -2. Find each retained slot through the persistent map. -3. Update that slot only when generated row equivalence reports a change. -4. Reuse the previous outer slot array when membership and order are unchanged. -``` - -Every row receives one collision-safe length-prefixed primitive ID when it is -created. `SessionRowLayout` declares that key and generates discriminated-union -equivalence. Group `origin` is immutable metadata; `refs`, `pending`, and -`completed` participate in equivalence. Reconciliation performs no -serialization: - -| Row | ID shape | -| ----------------- | ------------------------------------------------------ | -| Message | `m{length}:{messageID}` | -| Queued compaction | `c{length}:{inputID}` | -| Part | `p{message segment}{part segment}` | -| Assistant footer | `f{length}:{messageID}` | -| Group | `g{kind}{origin message segment}{origin part segment}` | - -Every group records the ref that created it as immutable `origin` metadata. -Appending refs, completing the group, and moving refs between permission -partitions cannot change its ID or boundary identity. Group equivalence compares -`completed`, every `refs` entry, and every exploration `pending` entry. - -`Keyed` publishes two readables. `slots` changes only when key membership or -order changes, so Solid `` does no structural work for a group-value -update. `values` depends on both the outer list and every slot for consumers -that require aggregate values. The TUI has no reactive `values` subscriber; it -uses the lazy aggregate for bulk permission repartitioning and snapshot/test -consumers. -Message boundaries track structural slots and messages, then read immutable -row identity fields untracked. A changed group publishes through its existing -slot and preserves the mounted Solid owner without recalculating boundaries. -The lazy aggregate is read by permission repartitioning and snapshot/test -consumers, not by unique append paths. - -Slot updates and the outer-array update run inside both a Quark transaction and -a Solid batch. The Quark transaction settles the alien-signals graph; the -surrounding Solid batch holds adapter writes until every Quark listener has -flushed. Solid therefore observes one settled state rather than a row value -from one ordering and an outer slot array from another. - -## Complexity Is Linear in Rows and Contained Refs - -Let `R` be visible rows, `M` loaded messages, `P` assistant content parts, and -`C` the total refs contained by visible groups. - -| Operation | Time | Allocation | -| ---------------------------- | -----------------------: | ---------------------------------------------: | -| Full rebuild | `O(M + P + R)` | New reduction plus reconciled slot array | -| Active group / queue lookup | Expected `O(1)` | None | -| Queue boundary advancement | `O(R)` | None | -| Group extension | `O(G)` | One copied group array; `O(1)` index delta | -| Structural insertion/removal | `O(R)` | One structural slots array | -| Duplicate part append | `O(1)` | One key string | -| Duplicate message/footer | `O(1)` | One key string | -| Footer removal | `O(R)` | One structural slot array | -| Permission repartition | `O(R + C)` | Arrays only for groups whose split moves | -| Quark-to-Solid publication | `O(1)` per changed level | One slot write and, when required, outer write | - -`Keyed.has` handles message and footer membership through the collection's -existing key map. `Layout.collection` maintains every visible part ID, -including refs nested inside groups. Full replacement rebuilds the finite index; -inserts and arbitrary modifications derive membership automatically; the hot -group append applies one typed member delta. Duplicate streaming deltas return -before reading the lazy aggregate or scanning rows. - -## Solid Owns the Adapter Lifecycle - -`createSessionRows` runs inside the Solid owner for the session route. -`useValue` reads the initial Quark value, subscribes once, and registers the -unsubscribe function with Solid's `onCleanup`. - -The existing event-bus subscriptions use the same owner cleanup. Leaving the -route therefore removes both the Quark-to-Solid subscription and all timeline -event subscriptions. Quark state does not outlive the route. - -## Behavior Must Remain Equivalent - -The experiment preserves these existing timeline rules: - -- User, synthetic, compaction, shell, model, and agent messages retain their - current ordering. -- Pending inputs and queued compactions remain after active output. -- Reasoning and exploration parts retain their grouping behavior. -- Exploration tool calls move between pending and completed partitions when - permissions change. -- Retry footers appear and disappear on the same events. -- Duplicate message, part, and footer events remain idempotent. -- Revert boundaries and message synchronization still rebuild the complete row - list from `DataProvider`. -- `SessionRowView` continues to resolve message content from `DataProvider`; - Quark owns row structure, not message content. - -Changing `createSessionRows` from an array-like Solid Store to structural -`slots` plus an internal lazy aggregate is an internal API change. The session -route and its data-test probes are the only consumers on this branch. - -## The Comparison Uses One Drive Script - -The baseline is a detached worktree at -`/Users/kit/code/open-source/opencode-quark-baseline`, pinned to the same -`origin/v2` commit as the fork. The fork is -`/Users/kit/code/open-source/opencode-quark-timeline`. - -`script/quark-timeline-drive.ts` creates one isolated project, submits one -prompt, streams one long reasoning span and one long text span as many delta -events, and waits for the final `QUARK_TIMELINE_COMPLETE` marker. The script -prints elapsed wall time and uses marker visibility as its rendered correctness -assertion. This workload stresses duplicate-part handling after the first -delta; it does not create 120 distinct timeline rows. - -After the marker renders, the script queries the same isolated service and -asserts that the projected assistant message contains the complete reasoning -span, complete text span, and `stop` finish. The API assertion covers content -that has scrolled outside the terminal viewport. - -Run the same checked script against each worktree: - -```sh -cd /Users/kit/code/open-source/opencode-drive - -packages/drive/bin/opencode-drive check \ - /Users/kit/code/open-source/opencode-quark-timeline/script/quark-timeline-drive.ts - -packages/drive/bin/opencode-drive start \ - --name quark-timeline-baseline \ - --script /Users/kit/code/open-source/opencode-quark-timeline/script/quark-timeline-drive.ts \ - --dev /Users/kit/code/open-source/opencode-quark-baseline - -packages/drive/bin/opencode-drive start \ - --name quark-timeline-fork \ - --script /Users/kit/code/open-source/opencode-quark-timeline/script/quark-timeline-drive.ts \ - --dev /Users/kit/code/open-source/opencode-quark-timeline -``` - -Wall time from one end-to-end drive run is supporting evidence, not a stable -microbenchmark. Provider simulation, terminal polling, and process scheduling -also contribute to it. A performance decision requires repeated paired runs -and row-update or invalidation counters in addition to this behavioral check. - -Seven baseline/fork behavior pairs completed successfully: - -| Pair | Baseline | Quark fork | Fork / baseline | -| ---------------------: | --------: | ---------: | --------------: | -| 1 | 11,505 ms | 6,455 ms | 0.561x | -| 2 | 4,003 ms | 4,733 ms | 1.182x | -| 3, fork first | 9,296 ms | 7,563 ms | 0.813x | -| 4, API assertions | 4,283 ms | 3,114 ms | 0.727x | -| 5, current `origin/v2` | 3,636 ms | 2,074 ms | 0.570x | -| 6, current Drive API | 2,233 ms | 1,413 ms | 0.633x | -| 7, timeline extraction | 973 ms | 825 ms | 0.848x | - -The range is too wide to support a speed claim. These runs establish that the -same streamed scenario renders and terminates on both implementations. They do -not establish that either implementation is faster. - -## Validation Status - -| Check | Current result | -| ------------------------------ | -------------------------------------------------------------------------- | -| TUI package typecheck | Pass | -| Full TUI test suite | 274 pass, 1 skip | -| Focused data and row tests | 55 pass | -| Vendored Quark tests | 24 pass | -| Shared drive scripts typecheck | Pass | -| Shared Drive behavior | Seven baseline/fork pairs pass; latest pair verifies projected content | -| Publication-counter trace | Pass; exact slot/structure deltas recorded in performance model | -| Paired performance evidence | Pass for controlled workloads; end-to-end wall time remains non-conclusive | - -## Reactive Query Caches Remain Outside This Phase - -The timeline's imperative members index is finite: entries follow current rows -and are removed synchronously. It is not a reactive query cache and renderers -do not subscribe to it. - -A later message-collection phase must not delete empty cache entries while a -live readable still references them. Safe eviction therefore needs explicit -collection ownership or subscription reference counting. That design is not a -prerequisite for the route-scoped row-state experiment. - -## Rollback Is One Seam - -Rollback removes the private `effect-quark` workspace package and restores the -Solid Store in `createSessionRows`. No protocol, persisted data, public client -API, or server projection changes need migration. - -The experiment should not expand into `DataProvider` message storage until the -current boundary has behavior parity and repeatable performance evidence. diff --git a/packages/quark/THIRD_PARTY_NOTICES.md b/packages/quark/THIRD_PARTY_NOTICES.md new file mode 100644 index 0000000000..b02efdc858 --- /dev/null +++ b/packages/quark/THIRD_PARTY_NOTICES.md @@ -0,0 +1,33 @@ +# Third-Party Notices + +## alien-signals + +The internal reactive kernel in `src/reactivity.ts` adapts the dependency +graph algorithm of [alien-signals](https://github.com/stackblitz/alien-signals) +3.2.1: the intrusive doubly-linked dependency and subscriber links with +versioned in-place reuse, and the iterative `propagate` and `checkDirty` +graph walks. alien-signals is not a runtime dependency of Quark. + +``` +MIT License + +Copyright (c) 2024-present Johnson Chu + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. +``` diff --git a/packages/quark/package.json b/packages/quark/package.json index 1548fcbb53..bd1ae13ccf 100644 --- a/packages/quark/package.json +++ b/packages/quark/package.json @@ -14,7 +14,6 @@ "test": "bun --conditions=browser test" }, "dependencies": { - "alien-signals": "3.2.1", "solid-js": "catalog:" } } diff --git a/packages/quark/src/keyed.ts b/packages/quark/src/keyed.ts index 32040703a5..575aa196bb 100644 --- a/packages/quark/src/keyed.ts +++ b/packages/quark/src/keyed.ts @@ -133,7 +133,7 @@ export namespace Keyed { function publish(slot: Writable, value: A) { const current = slot() - // alien-signals uses SameValueZero-compatible identity for primitive writes. + // The reactive kernel uses strict identity for primitive writes. if (current === value || equivalent(current, value)) { if (options.metrics) options.metrics.equivalenceSuppressions++ return false diff --git a/packages/quark/src/reactivity.ts b/packages/quark/src/reactivity.ts index d127bf52e9..7bea77e94b 100644 --- a/packages/quark/src/reactivity.ts +++ b/packages/quark/src/reactivity.ts @@ -1,5 +1,3 @@ -import { computed, effect, endBatch, setActiveSub, signal, startBatch } from "alien-signals" - export interface Readable { (): A subscribe(listener: (value: A) => void): () => void @@ -10,57 +8,466 @@ export interface Writable extends Readable { update(f: (value: A) => A): void } -function subscribe(read: () => A, listener: (value: A) => void) { - let initialized = false - return effect(() => { - const value = read() - if (!initialized) { - initialized = true - return - } - const active = setActiveSub() - try { - listener(value) - } finally { - setActiveSub(active) - } - }) -} - export namespace State { export function make(initial: A): Writable { - const state = signal(initial) - const read = (() => state()) as Writable - read.set = (value) => state(value) - read.update = (f) => { - // Read untracked: calling update inside an effect must not make the - // effect depend on (and re-trigger from) this signal. - const active = setActiveSub() - const current = state() - setActiveSub(active) - state(f(current)) + const node: StateNode = { + flags: Flags.Mutable, + value: initial, + pending: initial, + deps: undefined, + depsTail: undefined, + subs: undefined, + subsTail: undefined, } - read.subscribe = (listener) => subscribe(read, listener) + const read = (() => readState(node)) as Writable + read.set = (value) => writeState(node, value) + read.update = (f) => { + // Read untracked: calling update inside a tracked evaluation must not + // make the caller depend on (and re-trigger from) this state. + const previous = swapActiveSub(undefined) + try { + writeState(node, f(readState(node))) + } finally { + activeSub = previous + } + } + read.subscribe = (listener) => subscribeNode(node, read, listener) return read } } export namespace Computed { export function make(evaluate: (previous: A | undefined) => A): Readable { - const value = computed(evaluate) - const read = (() => value()) as Readable - read.subscribe = (listener) => subscribe(read, listener) + const node: ComputedNode = { + flags: Flags.None, + value: undefined, + evaluate, + deps: undefined, + depsTail: undefined, + subs: undefined, + subsTail: undefined, + } + const read = (() => readComputed(node)) as Readable + read.subscribe = (listener) => subscribeNode(node, read, listener) return read } } export namespace Transaction { export function run(f: () => A): A { - startBatch() + batchDepth++ try { return f() } finally { - endBatch() + if (!--batchDepth) flush() } } } + +// Internal reactive kernel. +// +// The dependency graph representation (intrusive doubly-linked dependency and +// subscriber links with versioned in-place reuse) and the iterative +// `propagate`/`checkDirty` walks are adapted from alien-signals 3.2.1 +// (https://github.com/stackblitz/alien-signals, MIT). See +// THIRD_PARTY_NOTICES.md. Quark departs from the reference in three ways: +// subscribers are fixed single-dependency watchers instead of general +// effects, orphaned computeds are detached with an iterative work queue +// instead of recursive unwatch callbacks, and reading a computed that is +// currently evaluating throws a stable cycle error instead of looping +// (stackblitz/alien-signals#118, #123). + +const enum Flags { + None = 0, + /** The node can produce a new value: states always, computeds once evaluated. */ + Mutable = 1, + /** The node is a subscription watcher delivering values to a listener. */ + Watching = 2, + /** The computed is currently evaluating; reading it again is a cycle. */ + Computing = 4, + /** The watcher is queued for the next flush. */ + Queued = 8, + /** The node's value is known stale. */ + Dirty = 16, + /** The node's value is possibly stale pending dependency revalidation. */ + Pending = 32, +} + +interface ReactiveNode { + flags: Flags + deps?: Link + depsTail?: Link + subs?: Link + subsTail?: Link +} + +interface StateNode extends ReactiveNode { + value: A + pending: A +} + +interface ComputedNode extends ReactiveNode { + value: A | undefined + evaluate: (previous: A | undefined) => A +} + +interface WatcherNode extends ReactiveNode { + read: () => unknown + listener: (value: unknown) => void +} + +interface Link { + version: number + dep: ReactiveNode + sub: ReactiveNode + prevSub: Link | undefined + nextSub: Link | undefined + prevDep: Link | undefined + nextDep: Link | undefined +} + +interface Stack { + value: A + prev: Stack | undefined +} + +let activeSub: ReactiveNode | undefined +let batchDepth = 0 +let version = 0 +let flushIndex = 0 +let queueLength = 0 +const queue: Array = [] +const orphans: Array = [] + +function swapActiveSub(sub: ReactiveNode | undefined): ReactiveNode | undefined { + const previous = activeSub + activeSub = sub + return previous +} + +function readState(node: StateNode): A { + if (node.flags & Flags.Dirty && updateState(node) && node.subs !== undefined) { + shallowPropagate(node.subs) + } + if (activeSub !== undefined) link(node, activeSub, version) + return node.value +} + +function writeState(node: StateNode, value: A): void { + if (node.pending === (node.pending = value)) return + node.flags = Flags.Mutable | Flags.Dirty + if (node.subs !== undefined) { + propagate(node.subs) + if (!batchDepth) flush() + } +} + +function readComputed(node: ComputedNode): A { + const flags = node.flags + if (flags & Flags.Computing) { + throw new Error("Reactive cycle detected: a computed depends on its own value") + } + if ( + flags & Flags.Dirty || + (flags & Flags.Pending && + (checkDirty(node.deps!, node) || ((node.flags = flags & ~Flags.Pending), false))) || + !flags + ) { + if (updateComputed(node) && node.subs !== undefined) { + shallowPropagate(node.subs) + } + } + if (activeSub !== undefined) link(node, activeSub, version) + return node.value! +} + +function updateState(node: StateNode): boolean { + node.flags = Flags.Mutable + return node.value !== (node.value = node.pending) +} + +function updateComputed(node: ComputedNode): boolean { + node.depsTail = undefined + node.flags = Flags.Mutable | Flags.Computing + const previous = swapActiveSub(node) + version++ + let completed = false + try { + const oldValue = node.value + const changed = oldValue !== (node.value = node.evaluate(oldValue)) + completed = true + return changed + } finally { + activeSub = previous + node.flags &= ~Flags.Computing + // A throwing evaluation stays dirty so the next read retries instead of + // serving a stale value with no dependency links. + if (!completed) node.flags |= Flags.Dirty + purgeDeps(node) + } +} + +function subscribeNode(node: ReactiveNode, read: () => A, listener: (value: A) => void): () => void { + // Evaluate untracked before linking so a throwing computed leaves no + // partially initialized watcher behind. + const previous = swapActiveSub(undefined) + try { + read() + } finally { + activeSub = previous + } + const watcher: WatcherNode = { + flags: Flags.Watching, + read, + listener: listener as (value: unknown) => void, + deps: undefined, + depsTail: undefined, + } + link(node, watcher, ++version) + return () => { + if (!(watcher.flags & Flags.Watching)) return + watcher.flags &= ~(Flags.Watching | Flags.Dirty | Flags.Pending) + unlink(watcher.deps!, watcher) + drainOrphans() + } +} + +function runWatcher(watcher: WatcherNode): void { + const flags = watcher.flags + watcher.flags = flags & ~(Flags.Queued | Flags.Dirty | Flags.Pending) + if (!(flags & Flags.Watching)) return + const dirty = + !!(flags & Flags.Dirty) || (!!(flags & Flags.Pending) && checkDirty(watcher.deps!, watcher)) + // Revalidation runs user code in computed evaluations, which may dispose + // this watcher; a disposed watcher must not deliver. + if (!dirty || !(watcher.flags & Flags.Watching)) return + // Listener reads stay untracked and listeners may dispose subscriptions, + // including their own. + const previous = swapActiveSub(undefined) + try { + watcher.listener(watcher.read()) + } finally { + activeSub = previous + } +} + +function flush(): void { + try { + while (flushIndex < queueLength) { + const watcher = queue[flushIndex]! + queue[flushIndex++] = undefined + runWatcher(watcher) + } + } finally { + // A throwing listener aborts this flush; the remaining watchers keep + // their Dirty/Pending flags and requeue on the next propagation. + while (flushIndex < queueLength) { + const watcher = queue[flushIndex]! + queue[flushIndex++] = undefined + watcher.flags &= ~Flags.Queued + } + flushIndex = 0 + queueLength = 0 + } +} + +function enqueue(watcher: WatcherNode): void { + watcher.flags |= Flags.Queued + queue[queueLength++] = watcher +} + +function link(dep: ReactiveNode, sub: ReactiveNode, linkVersion: number): void { + const prevDep = sub.depsTail + if (prevDep !== undefined && prevDep.dep === dep) return + const nextDep = prevDep !== undefined ? prevDep.nextDep : sub.deps + if (nextDep !== undefined && nextDep.dep === dep) { + nextDep.version = linkVersion + sub.depsTail = nextDep + return + } + const prevSub = dep.subsTail + if (prevSub !== undefined && prevSub.version === linkVersion && prevSub.sub === sub) return + const newLink: Link = { + version: linkVersion, + dep, + sub, + prevDep, + nextDep, + prevSub, + nextSub: undefined, + } + sub.depsTail = newLink + dep.subsTail = newLink + if (nextDep !== undefined) nextDep.prevDep = newLink + if (prevDep !== undefined) prevDep.nextDep = newLink + else sub.deps = newLink + if (prevSub !== undefined) prevSub.nextSub = newLink + else dep.subs = newLink +} + +function unlink(current: Link, sub: ReactiveNode): Link | undefined { + const dep = current.dep + const prevDep = current.prevDep + const nextDep = current.nextDep + const nextSub = current.nextSub + const prevSub = current.prevSub + if (nextDep !== undefined) nextDep.prevDep = prevDep + else sub.depsTail = prevDep + if (prevDep !== undefined) prevDep.nextDep = nextDep + else sub.deps = nextDep + if (nextSub !== undefined) nextSub.prevSub = prevSub + else dep.subsTail = prevSub + if (prevSub !== undefined) prevSub.nextSub = nextSub + else if ((dep.subs = nextSub) === undefined && dep.deps !== undefined) orphans.push(dep) + return nextDep +} + +// Detaches computeds that lost their last subscriber. Iterative on purpose: +// a recursive unwatch cascade overflows the stack on deep chains. Only nodes +// with dependencies enter the queue (states and dep-less computeds have +// nothing to detach). As in the reference, a computed that is read but never +// subscribed stays linked to its sources until they are collected; create +// long-lived computeds rather than per-operation ones. +function drainOrphans(): void { + while (orphans.length > 0) { + const node = orphans.pop()! + if (!("evaluate" in node) || node.depsTail === undefined) continue + node.flags = Flags.Mutable | Flags.Dirty + let current = node.depsTail as Link | undefined + while (current !== undefined) { + const prev = current.prevDep + unlink(current, node) + current = prev + } + } +} + +function purgeDeps(sub: ReactiveNode): void { + const depsTail = sub.depsTail + let dep = depsTail !== undefined ? depsTail.nextDep : sub.deps + while (dep !== undefined) { + dep = unlink(dep, sub) + } + drainOrphans() +} + +function propagate(current: Link): void { + let next = current.nextSub + let stack: Stack | undefined + + top: do { + const sub = current.sub + const flags = sub.flags + const unmarked = !(flags & (Flags.Dirty | Flags.Pending)) + if (unmarked) sub.flags = flags | Flags.Pending + if (flags & Flags.Watching && !(flags & Flags.Queued)) enqueue(sub as WatcherNode) + + if (unmarked && flags & Flags.Mutable) { + const subSubs = sub.subs + if (subSubs !== undefined) { + current = subSubs + const nextSub = subSubs.nextSub + if (nextSub !== undefined) { + stack = { value: next, prev: stack } + next = nextSub + } + continue + } + } + + if (next !== undefined) { + current = next + next = current.nextSub + continue + } + + while (stack !== undefined) { + const continuation = stack.value + stack = stack.prev + if (continuation !== undefined) { + current = continuation + next = current.nextSub + continue top + } + } + + break + } while (true) +} + +function shallowPropagate(current: Link): void { + let iterator: Link | undefined = current + do { + const sub = iterator.sub + const flags = sub.flags + if ((flags & (Flags.Pending | Flags.Dirty)) === Flags.Pending) { + sub.flags = flags | Flags.Dirty + if (flags & Flags.Watching && !(flags & Flags.Queued)) enqueue(sub as WatcherNode) + } + iterator = iterator.nextSub + } while (iterator !== undefined) +} + +function updateNode(node: ReactiveNode): boolean { + if ("evaluate" in node) return updateComputed(node as ComputedNode) + return updateState(node as StateNode) +} + +function checkDirty(current: Link, sub: ReactiveNode): boolean { + let stack: Stack | undefined + let checkDepth = 0 + let dirty = false + + top: do { + const dep = current.dep + const flags = dep.flags + + if (sub.flags & Flags.Dirty) { + dirty = true + } else if ((flags & (Flags.Mutable | Flags.Dirty)) === (Flags.Mutable | Flags.Dirty)) { + const subs = dep.subs! + if (updateNode(dep)) { + if (subs.nextSub !== undefined) shallowPropagate(subs) + dirty = true + } + } else if ((flags & (Flags.Mutable | Flags.Pending)) === (Flags.Mutable | Flags.Pending)) { + stack = { value: current, prev: stack } + current = dep.deps! + sub = dep + checkDepth++ + continue + } + + if (!dirty) { + const nextDep = current.nextDep + if (nextDep !== undefined) { + current = nextDep + continue + } + } + + while (checkDepth--) { + current = stack!.value + stack = stack!.prev + if (dirty) { + const subs = sub.subs! + if (updateNode(sub)) { + if (subs.nextSub !== undefined) shallowPropagate(subs) + sub = current.sub + continue + } + dirty = false + } else { + sub.flags &= ~Flags.Pending + } + sub = current.sub + const nextDep = current.nextDep + if (nextDep !== undefined) { + current = nextDep + continue top + } + } + + return dirty + } while (true) +} diff --git a/packages/quark/test/reactivity.test.ts b/packages/quark/test/reactivity.test.ts index 66c1f479bc..1c3b1e2224 100644 --- a/packages/quark/test/reactivity.test.ts +++ b/packages/quark/test/reactivity.test.ts @@ -1,13 +1,110 @@ import { describe, expect, it } from "bun:test" -import { createEffect, createRoot } from "solid-js" -import { State } from "../src" -import { useValue } from "../src/solid" +import { Computed, State, Transaction, type Readable } from "../src/reactivity" + +describe("reactivity", () => { + it("keeps computed values lazy", () => { + const source = State.make(1) + let evaluations = 0 + const doubled = Computed.make(() => { + evaluations++ + return source() * 2 + }) + + expect(evaluations).toBe(0) + expect(doubled()).toBe(2) + expect(doubled()).toBe(2) + expect(evaluations).toBe(1) + + source.set(2) + expect(evaluations).toBe(1) + expect(doubled()).toBe(4) + expect(evaluations).toBe(2) + }) + + it("propagates a diamond without glitches", () => { + const source = State.make(1) + const left = Computed.make(() => source() * 2) + const right = Computed.make(() => source() * 3) + const total = Computed.make(() => left() + right()) + const values: Array = [] + const dispose = total.subscribe((value) => values.push(value)) + + source.set(2) + + expect(values).toEqual([10]) + dispose() + }) + + it("tracks dynamic dependencies", () => { + const enabled = State.make(true) + const left = State.make(1) + const right = State.make(10) + const selected = Computed.make(() => (enabled() ? left() : right())) + const values: Array = [] + const dispose = selected.subscribe((value) => values.push(value)) + + left.set(2) + enabled.set(false) + left.set(3) + right.set(11) + + expect(values).toEqual([2, 10, 11]) + dispose() + }) + + it("batches transactions", () => { + const left = State.make(1) + const right = State.make(2) + const total = Computed.make(() => left() + right()) + const values: Array = [] + const dispose = total.subscribe((value) => values.push(value)) + + Transaction.run(() => { + left.set(2) + right.set(3) + }) + + expect(values).toEqual([5]) + dispose() + }) + + it("cuts off unchanged derived values", () => { + const source = State.make(1) + let parityEvaluations = 0 + let labelEvaluations = 0 + const parity = Computed.make(() => { + parityEvaluations++ + return source() % 2 + }) + const label = Computed.make(() => { + labelEvaluations++ + return parity() === 0 ? "even" : "odd" + }) + const dispose = label.subscribe(() => {}) + + source.set(3) + + expect(parityEvaluations).toBe(2) + expect(labelEvaluations).toBe(1) + dispose() + }) + + it("disposes subscriptions", () => { + const source = State.make(1) + const values: Array = [] + const dispose = source.subscribe((value) => values.push(value)) + + source.set(2) + dispose() + source.set(3) + + expect(values).toEqual([2]) + }) -describe("Quark", () => { it("does not track state read by subscription listeners", () => { const source = State.make(1) const unrelated = State.make(1) - const values: number[] = [] + const values: Array = [] const dispose = source.subscribe((value) => { unrelated() values.push(value) @@ -20,20 +117,311 @@ describe("Quark", () => { dispose() }) - it("bridges values into a Solid owner and disposes with it", () => { + it("does not track the state read by update", () => { + const trigger = State.make(0) + const updated = State.make(0) + let runs = 0 + const dispose = trigger.subscribe(() => { + runs++ + updated.update((value) => value + 1) + }) + + trigger.set(1) + updated.set(10) + + expect(runs).toBe(1) + expect(updated()).toBe(10) + dispose() + }) + + it("delivers the previous value to computed evaluation", () => { const source = State.make(1) - const values: number[] = [] + const previousValues: Array = [] + const running = Computed.make((previous) => { + previousValues.push(previous) + return (previous ?? 0) + source() + }) + + expect(running()).toBe(1) + source.set(2) + expect(running()).toBe(3) + source.set(3) + expect(running()).toBe(6) + expect(previousValues).toEqual([undefined, 1, 3]) + }) + + it("detaches and reattaches dynamic dependencies", () => { + const enabled = State.make(true) + const left = State.make(1) + const right = State.make(10) + let evaluations = 0 + const selected = Computed.make(() => { + evaluations++ + return enabled() ? left() : right() + }) + const dispose = selected.subscribe(() => {}) + + expect(evaluations).toBe(1) + enabled.set(false) + expect(evaluations).toBe(2) + + // Detached: left writes must not re-evaluate the computed. + left.set(2) + left.set(3) + expect(evaluations).toBe(2) + + // Reattached: left writes re-evaluate, right writes no longer do. + enabled.set(true) + expect(evaluations).toBe(3) + expect(selected()).toBe(3) + right.set(11) + expect(evaluations).toBe(3) + left.set(4) + expect(evaluations).toBe(4) + expect(selected()).toBe(4) + dispose() + }) + + it("notifies a diamond subscriber exactly once per write", () => { + const source = State.make(1) + const left = Computed.make(() => source() * 2) + const right = Computed.make(() => source() * 3) + const total = Computed.make(() => left() + right()) + let notifications = 0 + const dispose = total.subscribe(() => notifications++) + + source.set(2) + source.set(3) + + expect(notifications).toBe(2) + dispose() + }) + + it("batches nested transactions until the outermost ends", () => { + const left = State.make(1) + const right = State.make(2) + const total = Computed.make(() => left() + right()) + const values: Array = [] + const dispose = total.subscribe((value) => values.push(value)) + + Transaction.run(() => { + left.set(2) + Transaction.run(() => { + right.set(3) + }) + expect(values).toEqual([]) + left.set(3) + }) + + expect(values).toEqual([6]) + dispose() + }) + + it("restores batch state when a transaction throws", () => { + const source = State.make(1) + const values: Array = [] + const dispose = source.subscribe((value) => values.push(value)) + + expect(() => + Transaction.run(() => { + source.set(2) + throw new Error("boom") + }), + ).toThrow("boom") + + // The write that happened before the throw still flushes once the + // transaction unwinds, and later writes are not batched. + expect(values).toEqual([2]) + source.set(3) + expect(values).toEqual([2, 3]) + dispose() + }) + + it("supports disposing another subscription during notification", () => { + const source = State.make(1) + const first: Array = [] + const second: Array = [] + let disposeSecond = () => {} + const disposeFirst = source.subscribe((value) => { + first.push(value) + disposeSecond() + }) + disposeSecond = source.subscribe((value) => second.push(value)) + + source.set(2) + source.set(3) + + expect(first).toEqual([2, 3]) + expect(second).toEqual([]) + disposeFirst() + }) + + it("does not deliver to a watcher disposed during revalidation", () => { + const source = State.make(0) let dispose = () => {} - createRoot((rootDispose) => { - dispose = rootDispose - const value = useValue(source) - createEffect(() => values.push(value())) + const gate = Computed.make(() => { + const value = source() + if (value > 0) dispose() + return value + }) + const values: Array = [] + dispose = gate.subscribe((value) => values.push(value)) + + // Revalidating the watcher re-evaluates gate, whose evaluation disposes + // the subscription before the value could be delivered. + source.set(1) + source.set(2) + + expect(values).toEqual([]) + }) + + it("supports a subscription disposing itself during notification", () => { + const source = State.make(1) + const values: Array = [] + let dispose = () => {} + dispose = source.subscribe((value) => { + values.push(value) + dispose() }) source.set(2) - dispose() source.set(3) - expect(values).toEqual([1, 2]) + expect(values).toEqual([2]) + }) + + it("does not track State.update reading its own value", () => { + const counter = State.make(0) + const source = State.make(1) + let evaluations = 0 + const tracked = Computed.make(() => { + evaluations++ + counter.update((value) => value) + return source() + }) + const dispose = tracked.subscribe(() => {}) + + expect(evaluations).toBe(1) + counter.set(5) + expect(evaluations).toBe(1) + source.set(2) + expect(evaluations).toBe(2) + dispose() + }) + + it("recovers tracking after a computed evaluation throws", () => { + const shouldThrow = State.make(true) + const source = State.make(1) + const throwing = Computed.make(() => { + if (shouldThrow()) throw new Error("computed boom") + return source() + }) + + expect(() => throwing()).toThrow("computed boom") + + // The failed evaluation must restore the active observer so unrelated + // graphs keep tracking correctly afterwards. + const other = State.make(1) + const doubled = Computed.make(() => other() * 2) + const values: Array = [] + const dispose = doubled.subscribe((value) => values.push(value)) + other.set(2) + expect(values).toEqual([4]) + dispose() + + shouldThrow.set(false) + expect(throwing()).toBe(1) + source.set(2) + expect(throwing()).toBe(2) + }) + + it("keeps notifying other subscribers after a listener throws", () => { + const source = State.make(1) + const values: Array = [] + const disposeThrowing = source.subscribe(() => { + throw new Error("listener boom") + }) + const dispose = source.subscribe((value) => values.push(value)) + + expect(() => source.set(2)).toThrow("listener boom") + disposeThrowing() + source.set(3) + + expect(values).toContain(3) + dispose() + }) + + it("leaves no dependency links behind when subscription initialization fails", () => { + const source = State.make(1) + const throwing = Computed.make(() => { + if (source() === 1) throw new Error("init boom") + return source() + }) + const values: Array = [] + + expect(() => throwing.subscribe((value) => values.push(value))).toThrow("init boom") + + // The failed subscription must not stay linked to the graph. + source.set(2) + source.set(3) + expect(values).toEqual([]) + + // The computed itself remains usable. + expect(throwing()).toBe(3) + }) + + it("propagates deep chains without recursive stack growth", () => { + // A cold pull of an unevaluated chain necessarily nests user getter + // frames, so each layer is evaluated as it is built. The kernel-owned + // paths under test are dirty propagation and revalidation, which must + // walk the full depth iteratively. + const depth = 100_000 + const source = State.make(0) + const chain = Array.from({ length: depth }).reduce>((current) => { + const next = Computed.make(() => current() + 1) + next() + return next + }, source) + + expect(chain()).toBe(depth) + const values: Array = [] + const dispose = chain.subscribe((value) => values.push(value)) + source.set(1) + expect(values).toEqual([depth + 1]) + dispose() + }) + + it("propagates wide fan-out without recursive stack growth", () => { + const width = 50_000 + const source = State.make(0) + const nodes = Array.from({ length: width }, () => Computed.make(() => source() + 1)) + const total = Computed.make(() => nodes.reduce((sum, node) => sum + node(), 0)) + let sink = 0 + const dispose = total.subscribe((value) => { + sink = value + }) + + source.set(1) + expect(sink).toBe(width * 2) + dispose() + }) + + it("fails deterministically on cyclic computed dependencies", () => { + // Regression for stackblitz/alien-signals#123: mutually dependent + // computed graphs must throw a stable error instead of looping or + // exhausting memory. + const fieldA = State.make(false) + const fieldB = State.make(false) + const a: Readable = Computed.make(() => (b() !== true ? fieldA() : null)) + const b: Readable = Computed.make(() => (a() !== true ? fieldB() : null)) + + // Every read that reaches the cycle fails the same way; a failed + // evaluation stays dirty and retries instead of serving a stale value. + expect(() => a()).toThrow(/cycle/i) + expect(() => b()).toThrow(/cycle/i) + + fieldA.set(true) + + expect(() => a()).toThrow(/cycle/i) }) }) diff --git a/packages/tui/src/routes/session/content.ts b/packages/tui/src/routes/session/content.ts index 8692879503..bd454f84af 100644 --- a/packages/tui/src/routes/session/content.ts +++ b/packages/tui/src/routes/session/content.ts @@ -7,7 +7,7 @@ import { Keyed, Layout } from "effect-quark" * Streaming events name their part on the wire (text/reasoning by ordinal, * tools by callID); each assistant message owns one keyed collection so a * delta publishes exactly one slot instead of reconciling a content array. - * See docs/design/quark-message-content.md. + * Design record: quark repo, docs/experiments/quark-message-content.md. */ export namespace SessionContent { type ContentPart = SessionMessageAssistant["content"][number]