08cee7cc0a
Uploading many docs (or a folder) previously spawned an ingestion subprocess per file all at once, thrashing the GPU/CPU. Add a configurable concurrency limit and a folder picker. - ragIndexConcurrency setting (default 1) in the chat runtime store, persisted like the other RAG scalar settings; exposed as a 'Parallel indexing' slider (1-8) at the bottom of the sidebar Retrieval section. - New rag-index-queue.ts semaphore: each document upload acquires a slot before it starts and releases it once its ingestion job finishes (complete / error / already-indexed), so bulk uploads drain at the configured rate. Wired into both composer upload paths (use-thread-doc-uploads + shared-composer). - Folder upload: a second 'Attach a folder' button on the RAG attach control uses a webkitdirectory input; every compatible file is routed through the same queue. Multi-file select already worked (the input has 'multiple' and loops addDoc). - Content-hash dedup (shipped earlier) means re-scanning a folder skips already-indexed files. Not build/UI verified here (no bun); needs bun typecheck + a browser check of bulk/folder upload draining at the set concurrency. |
||
|---|---|---|
| .. | ||
| public | ||
| src | ||
| .gitignore | ||
| .gitkeep | ||
| .npmrc | ||
| biome.json | ||
| components.json | ||
| data-designer.openapi (1).yaml | ||
| eslint.config.js | ||
| index.html | ||
| package-lock.json | ||
| package.json | ||
| tsconfig.app.json | ||
| tsconfig.json | ||
| tsconfig.node.json | ||
| vite.config.ts | ||
| vitest.config.ts | ||