65587c2be7* Studio: Data settings tab, uploaded files manager, quant pinning, image preview fix Settings - New Data tab in the settings sidebar, under Connections. Chat data management (archived chats, confirm before deleting, exports, import, clear all) moved there from the Chat tab. - New Archive all chats action with confirmation. Archives every chat in Recents and Projects; compare pairs count as one chat. - New Uploaded files manager listing RAG documents (chats, projects, knowledge bases) and chat message attachments with location, size and date. Files can be opened in a new tab or deleted. Deleting a chat attachment keeps the message text. Backend - GET /api/rag/documents lists all uploaded RAG documents with file size plus KB and project names. - GET /api/chat/attachments lists chat message attachments; per attachment file and delete endpoints included. Model selector - Downloaded GGUF quants can be pinned from the quant row (next to the settings and delete actions). Pinned quants show at the top of On Device under a Pinned heading as model name plus a grey quant chip and load directly with one click. Non GGUF cached repos pin as a whole. - Toned down the green of the downloaded label. Fix - Clicking an image attachment in chat now opens the preview overlay. The tooltip trigger wrapper called preventDefault before composed handlers ran, which made Radix DialogTrigger skip opening. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: image previews and file type chips in uploaded files list Image attachments now show a small thumbnail (lazy loaded from the stored bytes, object URL revoked on unmount) and every row shows a grey uppercase type chip derived from the extension or content type. Non image rows keep a file icon. Name cell floors its width and clips overflow so narrow dialogs stay aligned. * Harden attachment serving, add tests, and polish pinned rows and previews - Strict base64 decoding for attachment files: corrupt payloads now return 422 instead of silently serving empty or garbled bytes; whitespace, missing padding, the URL-safe alphabet, and RFC 2397 percent-encoded data URLs are all handled - New backend test suite covering attachment listing, size accounting, malformed rows, deletion semantics, and every file-serving edge case - Pinned quant rows show a Loaded tag when that exact quant is active, and reveal unpin, settings, and delete actions on hover - Uploaded files dialog is wider and chat locations link straight to the thread the attachment belongs to - Chat image preview is now a chrome-free lightbox: dimmed backdrop, rounded image, corner close button, click outside to dismiss - File opens go through a synchronous window.open so Safari and Firefox popup blockers do not eat them * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Uploaded files: click a file to jump to its chat, square thumbs, new Data icon - Clicking a file row (thumbnail or name) now goes straight to the chat it belongs to; files without a chat open directly as before - File thumbnails pin a small 7px radius: the theme scales rounded-md up to a near circle at this size - Settings Data tab now uses the database-setting icon * Uploaded files is now a Data tab subpage instead of a popup - Manage swaps the tab body for an inline Uploaded files page with a back header, matching the rest of settings navigation - Size column header and values are left aligned like the other columns - Column widths tightened so the table fits the settings panel * Lightbox polish and Data tab row order - Image preview close button is transparent until hovered - Preview image no longer rounds its corners - Import chats now sits below Clear all chats in the Data tab * Data tab: export chats as fine-tuning data and open them in Recipes - New Fine-tuning section in Settings > Data converts every chat into a JSONL dataset in the OpenAI messages format, one conversation per line with string-only system/user/assistant turns - The Train tab detects this file as chatml natively: no column mapping and no standardization pass, and it works with train on completions since every assistant turn sits behind the chat template response marker - Consecutive same-role turns merge, trailing turns without an assistant reply drop, and reasoning, tool calls, and images are excluded so chat templates format the data cleanly - Open in Recipes stages the JSONL as a local seed upload, creates a new Data Recipe with the seed block preconfigured, and jumps to the editor * Data tab: load chats straight into the Train tab, row moved to the top - New Load in Train tab button uploads the fine-tuning JSONL through the training dataset endpoint, selects it in the training config store, and opens the Train tab with the dataset loaded and format-checked - Use chats as training data now sits at the very top of the Data tab - The Chats subheading is gone; chat rows flow directly under it * Address review findings on the uploads manager and quant pins - Deleting the last attachment stores '[]' instead of NULL: a NULL reads back as a missing field and triggers the legacy IndexedDB backfill, which resurrected the deleted attachment on the next chat load - The attachment file endpoint now serves audio: adapter parts store {data, format} raw base64 and compare chats store a bare base64 string; media type comes from the attachment contentType or the format - Compare-chat uploads live in message content parts, not attachments; the uploads list now includes those blobs via synthetic content-part ids that the same get and delete routes resolve - Deleting a quant from the expanded repo row also unpins it so a pinned row cannot try to load a file that no longer exists - Thumbnails in the uploads list fetch their blob only once the row is visible, so a long screenshot history does not download everything - Nine new backend tests cover audio serving, content-part listing, serving, deletion, and the empty-list delete behavior * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Data tab: single action dropdown with format choices for chat training data - The three fine-tune buttons collapse into one dropdown plus a run button; pick Load in Train tab, Open in Recipes, or Export JSONL, then click the arrow to run it - The dropdown's Format section adds ShareGPT and Alpaca alongside the default OpenAI messages format, ticked like a checklist; all three shapes are auto-detected by the Train tab's format check - Alpaca is single-turn, so each user to assistant pair becomes its own record with the system prompt and earlier turns carried in the input column - Shorter description on the training data row - Uploaded files rows show the size under the file name instead of a separate column, matching the tighter layout * Polish the training data action control - Run button is a true circle (icon-sm plus rounded-full) with a heavier arrow stroke - Dropdown trigger uses the shared standard chevron and a fixed width so switching actions no longer resizes the control * Shorten the training data row description * Use the standard chevron for the run button and enlarge the ticks - Run button uses the shared standard right chevron so it matches the dropdown chevron instead of the hugeicons arrow - Dropdown ticks bumped up a size for legibility * Reword the training data row description * Shorten Data Recipes to Recipes in the training data description * List Export JSONL first and rename the default format to Chat Completions * Handle legacy string content in fine-tune exports and gate Train on chat-only hosts - messageToPlainText now accepts plain-string message content, the shape legacy and imported histories store, so those conversations export instead of being skipped as having no exchange - The Load in Train tab action is disabled on chat-only hosts the same way the sidebar gates Train; the default action falls back to Export JSONL there so the run button never uploads a dataset that /studio would immediately redirect away from * Narrow the training data action dropdown slightly * Drop the format picker from the training data dropdown Chat Completions (OpenAI messages) is the only export format we ship, so the ShareGPT and Alpaca options and the Format section are removed. The export always uses the OpenAI messages shape. * Address the second round of review findings Security - Chat attachment data URLs no longer echo their embedded media type: anything that is not a plain raster image serves as octet-stream, so imported text/html or SVG payloads cannot render under the app origin - Uploaded .html/.htm RAG documents serve as text/plain for the same reason; the preview sheet only uses the file URL for PDFs Uploads manager - Remote image URLs in imported chats are no longer listed as stored uploads (nothing to serve, and delete would strip the chat reference); the delete guard mirrors the same data:-only rule - Deleting a content-part upload refetches the list since the remaining parts re-index, keeping sibling row ids current - Deleting a project document from the Data tab invalidates the project sources cache like the sources panel does - Data-tab deletions now patch the loaded thread's in-memory copy via a small event, so a later repo sync cannot write the attachment back Fine-tune export - Branch siblings from retries stay out of the exported conversation; only the selected chain converts (full exports still keep everything) - Assistant turns before the first user turn drop, preserving leading system prompts, so no unconditioned assistant targets are emitted Four new backend tests cover the media type clamp and remote-URL rows; two existing tests updated for the clamped types * Fix uploaded file lifecycle and model state * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Make archived chats a Data settings subpage * Studio: fix attachment route tests and pinned quant edge cases - test_chat_attachments: drop asyncio.run around the synchronous /attachments routes (list/get/delete are plain def, so asyncio.run raised 'a coroutine was expected' and failed the Repo tests CI job). - test_chat_attachments: align compare-chat content-part assertions with the stable content-hash id scheme (content-part-sha256-...) instead of the removed array-index ids; resolve ids from the listing. - pickers: pass disabled={deleteDisabled} to the pinned-quant delete action so a quant cannot be deleted mid model-load, matching the expanded variant rows. - pickers: build the pinned-quant existence set from the query-unfiltered cached GGUF repos (format filter still applied) so a pinned quant stays findable when the search term matches only its quant name. * Fix Studio review regressions * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Guard fine-tune export content blocks * Add Export button for archived chats Adds an Export action to the Archived chats view in Settings > Data that downloads only the archived chats as a JSON backup (their threads, messages and projects). The button sits in the archived header row and appears only when archived chats exist. * Refactor archived export into pure, testable units Split the archived-chats export into a dependency-free filter (archived-chat-export.ts) and a shared JSON download helper (download-json.ts). Skip the download when nothing is archived so a stray call never drops an empty file. No behavior change to the button. --------- Co-authored-by: shimmyshimmer <shimmyshimmer@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com> Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local> Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
62 lines
2.1 KiB
TypeScript
62 lines
2.1 KiB
TypeScript
// SPDX-License-Identifier: AGPL-3.0-only
|
|
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
|
|
|
// Minimal views over the `unknown[]` export fields we filter on.
|
|
type ExportThreadView = {
|
|
id?: string;
|
|
archived?: boolean;
|
|
projectId?: string | null;
|
|
};
|
|
type ExportMessageView = { threadId?: string };
|
|
type ExportProjectView = { id?: string };
|
|
|
|
// Full chat-export backup shape, kept structural so the pure filter below
|
|
// stays decoupled from the storage layer that produces it.
|
|
export interface ChatExportData {
|
|
exportedAt?: string;
|
|
version?: number;
|
|
threadCount: number;
|
|
projects?: unknown[];
|
|
threads: unknown[];
|
|
messages: unknown[];
|
|
}
|
|
|
|
// Restrict a full chat export to archived threads, their messages and the
|
|
// projects those threads belong to. Pure: never mutates the input, and keeps
|
|
// the original thread/message objects so the backup re-imports unchanged.
|
|
export function filterArchivedChatExport<T extends ChatExportData>(
|
|
full: T,
|
|
): { data: T; archivedCount: number } {
|
|
const archivedThreads = (full.threads as ExportThreadView[]).filter(
|
|
(thread) => thread.archived === true,
|
|
);
|
|
const archivedThreadIds = new Set(
|
|
archivedThreads
|
|
.map((thread) => thread.id)
|
|
.filter((id): id is string => typeof id === "string"),
|
|
);
|
|
const messages = (full.messages as ExportMessageView[]).filter(
|
|
(message) =>
|
|
typeof message.threadId === "string" &&
|
|
archivedThreadIds.has(message.threadId),
|
|
);
|
|
const referencedProjectIds = new Set(
|
|
archivedThreads
|
|
.map((thread) => thread.projectId)
|
|
.filter((id): id is string => typeof id === "string"),
|
|
);
|
|
const projects = (full.projects as ExportProjectView[] | undefined)?.filter(
|
|
(project) =>
|
|
typeof project.id === "string" && referencedProjectIds.has(project.id),
|
|
);
|
|
return {
|
|
data: {
|
|
...full,
|
|
threadCount: archivedThreads.length,
|
|
projects: projects ?? [],
|
|
threads: archivedThreads as unknown[],
|
|
messages: messages as unknown[],
|
|
},
|
|
archivedCount: archivedThreads.length,
|
|
};
|
|
}
|