fix edfe desync

This commit is contained in:
shine1i 2026-02-06 11:47:29 +01:00
commit b1445febdb
2 changed files with 182 additions and 233 deletions

View file

@ -3,30 +3,29 @@
Root:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab`
This doc explains current architecture, how nodes map to payload/import, and how to add new blocks safely.
This doc reflects current code shape (React Flow UI node/edge shell + inline config split).
## 1) High-level flow
1. UI renders canvas + dialogs in:
1. Page shell + React Flow canvas:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/canvas-lab-page.tsx`
2. Add-block sheet uses registry metadata to create config objects:
2. Block picker sheet (plus/import/copy floating controls):
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/components/block-sheet.tsx`
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/blocks/registry.tsx`
3. Zustand store owns nodes/edges/configs and all mutation logic:
3. Zustand state + all graph/config mutations:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/stores/canvas-lab.ts`
4. Graph connection logic updates references + semantic edges:
4. Connection validation + edge side-effects:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/utils/graph.ts`
5. Export (preview/copy) converts in-memory graph/config to API payload:
5. Export/payload map:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/utils/payload/build-payload.ts`
6. Import reconstructs configs, nodes, edges from JSON:
6. Import/rebuild:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/utils/import/importer.ts`
## 2) Core types (single source of truth)
## 2) Core types
File:
Source of truth:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/types/index.ts`
`NodeConfig` is the main union the whole feature uses:
`NodeConfig` union:
```ts
export type NodeConfig =
@ -37,300 +36,231 @@ export type NodeConfig =
| ModelConfig;
```
Canvas node UI data (`CanvasNodeData`) is derived from config via `nodeDataFromConfig`.
`CanvasNodeData` is derived from config (`nodeDataFromConfig`), not edited directly.
## 3) Entrypoint wiring
File:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/canvas-lab-page.tsx`
Key wiring:
Current wiring:
```ts
const NODE_TYPES: NodeTypes = { builder: CanvasNode };
const EDGE_TYPES: EdgeTypes = { canvas: CanvasEdge, semantic: CanvasEdge };
const EDGE_TYPES: EdgeTypes = { canvas: DataEdge, semantic: CanvasSemanticEdge };
```
`CanvasLabPage` pulls actions/state from store and passes add handlers into `BlockSheet`:
Default data edge style uses auto path selection:
```ts
<BlockSheet
onAddSampler={addSamplerNode}
onAddLlm={addLlmNode}
onAddModelProvider={addModelProviderNode}
onAddModelConfig={addModelConfigNode}
onAddExpression={addExpressionNode}
/>
defaultEdgeOptions={{
type: "canvas",
data: { key: "name", path: "auto" },
style: { strokeWidth: 1.5, stroke: "var(--border)" },
}}
```
Preview/copy route through `buildCanvasPayload`, import route through `importCanvasPayload`.
Node click selects config (`selectConfig`), does not auto-open dialog.
Dialog opens via node `Details` button (`openConfig`) or explicit flows.
## 4) Registry-driven block system
File:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/blocks/registry.tsx`
Registry defines each block in one place:
- sheet title/icon/description
Registry owns:
- block metadata for sheet (title/icon/description)
- config factory (`createConfig`)
- config dialog (`renderDialog`)
- dialog renderer (`renderDialog`)
Example (model blocks):
If adding new `NodeConfig.kind`, keep `getBlockDefinitionForConfig` coverage complete.
```ts
{
kind: "llm",
type: "model_provider",
createConfig: (id, existing) => makeModelProviderConfig(id, existing),
renderDialog: ({ config, onUpdate }) =>
config.kind === "model_provider" ? (
<ModelProviderDialog config={config} onUpdate={(patch) => onUpdate(config.id, patch)} />
) : null,
}
```
Important: `getBlockDefinitionForConfig` must map every new `config.kind`, else dialog won't render.
## 5) Config factories + node label mapping
File:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/utils/index.ts`
Responsibilities:
- create default config objects (`makeSamplerConfig`, `makeLlmConfig`, `makeModelProviderConfig`, `makeModelConfig`, `makeExpressionConfig`)
- map `NodeConfig -> CanvasNodeData` via `nodeDataFromConfig`
- sampler set includes `category`, `subcategory`, `uniform`, `gaussian`, `bernoulli`, `datetime`, `timedelta`, `uuid`, `person`, `person_from_faker`
Example mapping:
```ts
if (config.kind === "model_provider") {
return {
title: "Model Provider",
kind: "model_provider",
subtype: config.provider_type || "Provider",
blockType: "model_provider",
name: config.name,
layoutDirection,
};
}
```
This is what controls visible node title/subtitle in the canvas.
## 6) Store responsibilities
## 5) Store responsibilities
File:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/stores/canvas-lab.ts`
Store owns:
- graph state (`nodes`, `edges`)
- config map (`configs[id]`)
- add/update/remove/connect operations
- layout direction + apply layout
- `nodes`, `edges`, `configs`, `processors`
- add/update/remove/connect logic
- `layoutDirection` + dagre apply-layout
- config selection/dialog state
Add-node pattern (all block types follow same shape):
Current config-selection API:
- `selectConfig(id)`: select node config, keep dialog closed
- `openConfig(id)`: select + open modal
```ts
const definition = getBlockDefinition("llm", "model_config");
const config = definition.createConfig(id, existing);
return buildNodeUpdate(state, config, state.layoutDirection);
```
When model config `provider` field changes, store auto-syncs semantic edge to matching provider name.
## 7) Edge semantics + connection behavior
File:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/utils/graph.ts`
Semantic edge classifier:
```ts
function isSemanticEdge(source: NodeConfig, target: NodeConfig): boolean {
if (source.kind === "model_provider" && target.kind === "model_config") return true;
return source.kind === "model_config" && target.kind === "llm";
}
```
Handle lanes:
- data edges use `data-out -> data-in` (`right -> left`)
- semantic edges use `semantic-out -> semantic-in` (`bottom -> top`)
- semantic lane only used for `model_provider -> model_config -> llm`
Connection side effects:
- `model_provider -> model_config`: set `model_config.provider = source.name`
- `model_config -> llm`: set `llm.model_alias = source.name`
- `datetime -> timedelta`: set `timedelta.reference_column_name = source.name`
- regular data edges into LLM/expression append `{{ source_name }}` refs
Edge rendering (dotted semantic edges):
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/components/canvas-edge.tsx`
```ts
const nextStyle = type === "semantic"
? { ...style, strokeDasharray: "4 4" }
: style;
```
## 8) Rename/remove propagation
File:
Add-node behavior is mode-aware via:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/stores/canvas-lab-helpers.ts`
Centralized consistency updates:
- rename updates:
- Jinja refs in `llm.prompt/system_prompt/output_format`
- expression `expr`
- subcategory parent
- `model_config.provider`
- `llm.model_alias`
- removal clears same references
New nodes:
- become selected
- set `activeConfigId`
- open dialog only for dialog-first config modes
This keeps graph fields stable when upstream nodes renamed/deleted.
## 9) Payload building (node graph -> API)
## 6) Inline vs dialog config policy
File:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/utils/payload/build-payload.ts`
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/components/inline/inline-policy.ts`
`buildCanvasPayload(configs, nodes, edges)` outputs:
Inline mode:
- sampler: `uniform`, `gaussian`, `bernoulli`, `uuid`
- `model_provider`
- `model_config`
- llm: `text`, `code`
- `expression`
Dialog mode:
- sampler: `category`, `subcategory`, `datetime`, `timedelta`, `person`, `person_from_faker`
- llm: `structured`, `judge`
Inline editors:
- `/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/components/inline/inline-sampler.tsx`
- `/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/components/inline/inline-model.tsx`
- `/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/components/inline/inline-llm.tsx`
- `/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/components/inline/inline-expression.tsx`
## 7) Node UI architecture (React Flow UI shell)
File:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/components/canvas-node.tsx`
Node shell uses feature-local RF UI primitives:
- `/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/components/rf-ui/base-node.tsx`
- `/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/components/rf-ui/labeled-handle.tsx`
Current node UX:
- `corner-squircle` + `rounded-lg` container
- inline editor shown by default for inline-capable configs
- summary text for dialog-first configs
- `Details` button opens modal dialog
- node resizer logic enabled (`NodeResizer`), visuals hidden (no corner/box affordance)
## 8) Handles + layout direction
Handle IDs (kept stable):
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/utils/handles.ts`
```ts
{
recipe: {
model_providers: [...],
model_configs: [...],
columns: [...],
processors: [...],
},
run: { rows: 5, preview: true, output_formats: ["jsonl"] },
ui: { nodes: [...], edges: [...] }
}
dataIn: "data-in"
dataOut: "data-out"
semanticIn: "semantic-in"
semanticOut: "semantic-out"
```
Current processor UI surface:
- `schema_transform` (sheet -> `Processors` -> `Schema Transform`)
- mapped to recipe processor with `build_stage: "post_batch"` and JSON `template`.
`canvas-node.tsx` switches handle positions by layout direction:
- `LR`: data left/right, semantic top/bottom
- `TB`: data top/bottom, semantic left/right
Current drop policy:
- column dialogs (`sampler` / `llm` / `expression`) expose `drop` toggle.
- payload writes column `drop` directly (preferred over drop-columns processor in v1).
After direction toggle or auto-layout, page refreshes node internals to avoid stale edge anchor offsets.
How relation is enforced:
- collect `model_alias` values used by LLM columns
- ensure each alias exists in `recipe.model_configs`
- validate `model_config.provider` points to existing provider
- validate `timedelta.reference_column_name` points to a datetime sampler
- require endpoint/provider_type only for providers that are actually referenced
- category sampler supports typed `conditional_params` in payload output
## 9) Edge architecture
This is why unused provider/config blocks can exist without blocking preview.
Data edge:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/components/rf-ui/data-edge.tsx`
## 10) Import pipeline (API -> node graph)
Features:
- label from source node data key
- `path: "auto"` chooses straight vs smoothstep/bezier based on geometry/positions
Entry file:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/utils/import/importer.ts`
Semantic edge:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/components/canvas-semantic-edge.tsx`
Order of reconstruction:
1. parse `recipe.model_providers` -> `ModelProviderConfig`
2. parse `recipe.model_configs` -> `ModelConfig`
3. parse `recipe.columns` -> sampler/llm/expression
4. build nodes with positions
5. build edges
Features:
- custom dashed smooth-step
- muted stroke styling
Edge inference file:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/utils/import/edges.ts`
Legacy mixed edge component is removed (no `canvas-edge.tsx` path in active flow).
If UI edges missing, infer edges from fields:
- `subcategory_parent` (canvas edge)
- `model_config.provider`
- `llm.model_alias`
- infer data edge from `timedelta.reference_column_name`
## 10) Connection semantics + side effects
## 11) Dialog routing and edit UIs
Config dialog shell:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/dialogs/config-dialog.tsx`
It calls:
`renderBlockDialog(config, categoryOptions, onUpdate)`
Model dialogs:
- provider:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/dialogs/models/model-provider-dialog.tsx`
- model config:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/dialogs/models/model-config-dialog.tsx`
- processors:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/dialogs/processors-dialog.tsx`
`ModelConfigDialog` and `LlmDialog` use shadcn `Combobox` fed from store configs:
- model config `provider` suggests model-provider node names
- llm `model_alias` suggests model-config aliases
- timedelta dialog suggests datetime columns for `reference_column_name`
## 12) How to add a new block (checklist)
Minimal path for a new block type:
1. Add/extend type in:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/types/index.ts`
2. Add default factory + node label mapping in:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/utils/index.ts`
3. Add block definition in:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/blocks/registry.tsx`
4. Add dialog component and route it via `renderDialog` in registry.
5. Add store add-action if block should be special-cased from sheet:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/stores/canvas-lab.ts`
6. Add payload serialization/validation in:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/utils/payload/`
7. Add import parsing + inferred edges in:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/utils/import/parsers.ts`
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/utils/import/importer.ts`
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/utils/import/edges.ts`
8. If connection has semantic meaning, extend:
File:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/utils/graph.ts`
## 13) Practical mental model
Lane rules:
- semantic lane only: `semantic-out -> semantic-in`
- data lane only: `data-out -> data-in`
- model infra nodes (`model_provider`, `model_config`) blocked from data lane
- `NodeConfig` is source-of-truth business state.
- `CanvasNodeData` is derived display state.
- registry = block metadata + factories + dialog routing.
- store = mutation orchestration.
- graph utils = connection semantics.
- payload/import utils = external contract boundary.
Semantic relations:
- `model_provider -> model_config`
- `model_config -> llm`
If one piece changes, keep all six in sync.
Connect side effects:
- provider edge sets `model_config.provider`
- model config edge sets `llm.model_alias`
- datetime edge sets `timedelta.reference_column_name`
- data edges into llm/expression append `{{ source_name }}` refs
- category -> subcategory syncs mapping scaffold
## 14) Processors roadmap (decision)
Single-incoming enforcement (competing refs pruned on connect):
- `provider`
- `model_alias`
- `reference_column_name`
- `subcategory_parent`
Current decision: **Option 3 (hybrid)**.
Multi data refs remain allowed for llm/expression prompt/expr templates.
v1 scope:
- add `drop` toggle on column blocks (sampler/llm/expression).
- add processor config surface for `schema_transform`.
- keep payload builder as single mapper to `recipe.processors`.
- keep processor state separate from node graph for now.
## 11) Canvas controls UX
Reason:
- fastest ship path.
- matches Data Designer column-level `drop`.
- avoids duplicate/complex processor edge logic in v1.
File:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/components/block-sheet.tsx`
Future option noted: **Option 2 (processor chain in graph)**.
Top-right floating controls are icon-only:
- `+` opens add-block sheet
- import icon opens import dialog
- copy icon copies recipe (brief check icon state on success)
Option 2 structure:
- add virtual node `Dataset Output`.
- processors become graph nodes: `Schema Transform`, `Drop Columns`, future processors.
- processor order derived from chain edges:
`Dataset Output -> P1 -> P2 -> ...`
- enforce chain rules:
- no cycles
- one incoming max per processor
- one outgoing max per processor
- chain must start at `Dataset Output`
All use same no-bg bordered button style (`corner-squircle`, hover primary border/icon).
Migration from option 3 -> 2:
- keep same processor schema/payload contracts.
- move order source from list/order field to edge traversal.
- UI changes mostly in canvas rendering + validation; payload adapter stays mostly same.
## 12) Dialog routing
Config modal shell:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/dialogs/config-dialog.tsx`
Routes block-specific forms through registry renderer:
`renderBlockDialog(config, categoryOptions, onUpdate)`
Current modal-only edits still live here (structured/judge/category/subcategory/etc).
## 13) Payload + import boundary
Payload map:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/utils/payload/build-payload.ts`
Import map:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/utils/import/importer.ts`
If `ui.edges` missing on import, inferred edges are built from config refs in:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/utils/import/edges.ts`
## 14) Add new block checklist
1. Add/extend config type in:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/types/index.ts`
2. Add factory + `nodeDataFromConfig` mapping in:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/utils/index.ts`
3. Add registry entry in:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/blocks/registry.tsx`
4. Add dialog and wire in registry `renderDialog`
5. Decide inline vs dialog mode; update:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/components/inline/inline-policy.ts`
6. If inline, add inline editor component under:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/components/inline/`
7. Add payload mapping/validation updates in:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/utils/payload/`
8. Add import parse/infer updates in:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/utils/import/`
9. Extend connection semantics if needed in:
`/Volumes/Expansion/projects/new-ui-prototype/studio/frontend/src/features/canvas-lab/utils/graph.ts`
## 15) Mental model
- `NodeConfig` = business truth
- `CanvasNodeData` = derived presentational truth
- registry = block metadata + factories + dialog routing
- store = orchestration + consistency
- graph utils = legal edges + side effects
- payload/import = external contract boundary
Keep all 6 synchronized when adding/changing block behavior.

View file

@ -8,6 +8,7 @@ import {
Panel,
ReactFlow,
useReactFlow,
useUpdateNodeInternals,
} from "@xyflow/react";
import { type ReactElement, useCallback, useMemo, useState } from "react";
import { useShallow } from "zustand/react/shallow";
@ -45,21 +46,39 @@ function LayoutControls({
onLayout,
onToggleDirection,
}: LayoutControlsProps): ReactElement {
const { fitView } = useReactFlow();
const { fitView, getNodes } = useReactFlow();
const updateNodeInternals = useUpdateNodeInternals();
const refreshNodeInternals = useCallback(() => {
const nodeIds = getNodes().map((node) => node.id);
if (nodeIds.length > 0) {
updateNodeInternals(nodeIds);
}
}, [getNodes, updateNodeInternals]);
const handleLayout = useCallback(() => {
onLayout();
requestAnimationFrame(() => {
fitView({ duration: 250 });
refreshNodeInternals();
requestAnimationFrame(() => {
fitView({ duration: 250 });
});
});
}, [fitView, onLayout]);
}, [fitView, onLayout, refreshNodeInternals]);
const handleToggleDirection = useCallback(() => {
onToggleDirection();
requestAnimationFrame(() => {
refreshNodeInternals();
});
}, [onToggleDirection, refreshNodeInternals]);
return (
<Panel position="top-left" className="m-3 flex items-center gap-2">
<Button size="sm" className="corner-squircle" variant="secondary" onClick={handleLayout}>
Auto layout
</Button>
<Button size="sm" className="corner-squircle" variant="outline" onClick={onToggleDirection}>
<Button size="sm" className="corner-squircle" variant="outline" onClick={handleToggleDirection}>
{direction}
</Button>
</Panel>