Commit graph

133 commits

Author SHA1 Message Date
Shine1i
b08b606b21 feat(studio): studio storage roots path utilities 2026-03-09 23:48:31 +00:00
Roland Tannous
a0f03d3080 Add AGPL-3.0 SPDX headers to all source files 2026-03-09 20:17:45 +00:00
Shine1i
992e07495f Merge remote-tracking branch 'origin/nightly' into feature/fixes-client 2026-03-09 19:07:42 +01:00
Roland Tannous
65d3539bac Merge pull request #342 from unslothai/local-dataset
dataset upload
2026-03-09 21:22:23 +04:00
Roland Tannous
3745e06776 store uploaded datasets under assets/datasets/uploads instead of ~/.cache 2026-03-09 17:06:36 +00:00
Shine1i
7c153f6a53 feat(recipe-studio): remove MCP tools-related dialogs and refactor tool profile management logic 2026-03-09 17:04:15 +01:00
Shine1i
24cb20c076 feat(recipe-studio): improve UI responsiveness and fix JSON preview handling 2026-03-09 16:04:46 +01:00
Roland Tannous
8d81c991f8 switch dataset upload from base64 JSON to multipart/form-data with streamed writes 2026-03-09 13:55:45 +00:00
Roland Tannous
ee6d5c124d normalize uploaded filename extension to lowercase for consistent downstream checks 2026-03-09 13:35:55 +00:00
Roland Tannous
dd24009f06 Merge pull request #340 from unslothai/fix/auth-audio
Added auth to audio generate endpoint
2026-03-09 17:25:49 +04:00
Roland Tannous
dcedc4df56 merge nightly, resolve conflict in use-chat-model-runtime 2026-03-09 13:19:17 +00:00
Roland Tannous
f416b7aa3d training: restore YAML fallback for trust_remote_code (no UI toggle) 2026-03-09 13:10:24 +00:00
Roland Tannous
83b1ff05ef respect trust_remote_code toggle, return helpful error when required 2026-03-09 13:06:55 +00:00
Roland Tannous
5ebd4de2ef backend: resolve trust_remote_code from YAML when not set by frontend 2026-03-09 11:58:23 +00:00
Manan17
0cc5fe3de8 remove file size limit 2026-03-09 07:04:02 +00:00
Manan17
b430e23c0e dataset upload 2026-03-09 05:50:18 +00:00
Wasim Yousef Said
39a02602cf Merge pull request #273 from unslothai/feature/data-reciper-enchansments
UX + layout polish & WIP data-reciper client & backend finalization p2
2026-03-09 02:57:32 +01:00
Shine1i
4aa171b079 feat(recipe-studio, datasets): improve dataset handling and update metadata logic 2026-03-09 02:47:32 +01:00
samit
1eb68678a0 added auth to audio generate endpoint 2026-03-08 17:46:54 -07:00
Roland Tannous
1b04a40bc8 Merge pull request #328 from unslothai/fix/chat-unloading-model
fixed model unload before load without validation
2026-03-09 04:40:05 +04:00
Shine1i
d951e5aef0 merge nightly 2026-03-09 00:32:33 +01:00
samit
6aa50d353f exposed trust_remote_code through the UI 2026-03-08 16:28:56 -07:00
Roland Tannous
7db2c90cc6 merge nightly into audio branch (mock test) 2026-03-08 10:23:44 +00:00
Manan17
4faa6e9152 fixing sesame model 2026-03-07 19:06:00 +00:00
Samit
7ec41afaf0 fixed model unload before load 2026-03-06 22:01:27 -08:00
Roland Tannous
ad6739be7a fix: wait for training shutdown before export load, clear stop flag on reset
1. Export route: stop_training() only signals the subprocess — wait up to
   30s for it to actually exit before loading the export checkpoint, avoiding
   a GPU memory race.

2. Training reset: clear _should_stop so /api/train/status returns phase=idle
   instead of staying stuck on phase=stopped after a user-triggered stop.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 04:16:10 +00:00
Roland Tannous
527db4ffff feat: add OpenAI-compatible /v1/chat/completions endpoint 2026-03-06 07:48:09 +00:00
Roland Tannous
c82e3d86bb fix: unload competing subprocesses before load across all routes 2026-03-06 06:05:31 +00:00
Roland Tannous
661ac4be96 feat: subprocess-based export, pin huggingface_hub==0.36.0 2026-03-06 06:03:09 +00:00
Roland Tannous
ce9bfd7476 fix: unload inference model before training to free GPU memory
When starting training, shut down the inference subprocess first
so the training subprocess has full GPU memory available.
2026-03-05 22:28:11 +00:00
Roland Tannous
b70faf8cb7 feat: subprocess-based inference for transformers version switching
Inference now runs in a persistent subprocess, solving the same
transformers version-switching problem that was fixed for training.
The subprocess stays alive between requests (model in GPU memory)
and is only restarted when switching transformers versions.

New files:
- core/inference/worker.py: subprocess entry point with command loop
- core/inference/orchestrator.py: parent-side proxy with same API

Modified:
- core/inference/__init__.py: exports orchestrator as default backend
- routes/inference.py: removed in-process ensure_transformers_version()
2026-03-05 17:47:57 +00:00
Roland Tannous
021c3aafdd fix: handle None job_id before first training run 2026-03-05 16:59:37 +00:00
Roland Tannous
06fbeb22cf fix: remove in-process version switching from models routes 2026-03-05 16:22:32 +00:00
Roland Tannous
f90af41c5f feat: subprocess-based training for transformers version switching 2026-03-05 15:40:32 +00:00
Shine1i
6fc829e3ef merge: nightly into feature/data-reciper-enchansments 2026-03-05 14:51:08 +01:00
Shine1i
2ff1d4de36 feat(recipe-studio): normalize and slugify run_name, update job naming logic 2026-03-05 12:25:51 +01:00
Shine1i
f988202290 refactor(studio): add local data-recipe dataset selection + training wiring 2026-03-05 12:25:51 +01:00
Manan17
8203637d89 resolved merge conflicts 2026-03-05 07:59:43 +00:00
Roland Tannous
f57664e268 Merge nightly into feature/transformers-v5-support 2026-03-05 06:49:44 +00:00
Roland Tannous
e04b9d53d6 feat: parallel URL image probe with time estimate and progress reporting
- Add 200-sample parallel probe using ThreadPoolExecutor + safe_num_proc
  to estimate download speed and failure rate before full conversion
- Abort with clear error if >=30% of probe images fail to download
- Show estimated download time in the training overlay modal
- Parallel batch conversion for URL-based datasets (vs sequential for local)
- Add warning field to /check-format response for URL-based image datasets
- Display URL warning in dataset preview dialog (amber banner)
- Thread progress_callback from trainer through format_and_template_dataset
  to convert_to_vlm_format for real-time status updates
2026-03-04 23:40:38 +00:00
Roland Tannous
64889cd5fc feat: add index range dataset slicing to studio training page
Add Start/End index inputs under Advanced in the dataset card,
allowing users to slice a dataset by row range before training.
Wired end-to-end: frontend store, API payload, backend Pydantic
model, and trainer dataset loading (inclusive on both ends).
2026-03-04 23:24:09 +00:00
Roland Tannous
9333f99dd3 Revert "Add index range dataset slicing to Studio training page" 2026-03-05 03:21:07 +04:00
Roland Tannous
02b17ec6d9 feat: add index range dataset slicing to studio training page
Add Start/End index inputs under Advanced in the dataset card,
allowing users to slice a dataset by row range before training.
Wired end-to-end: frontend store, API payload, backend Pydantic
model, and trainer dataset loading (inclusive on both ends).
2026-03-04 21:48:40 +00:00
Manan17
ee23628b0c dynamic detection of audio models and fixing autoencoder issues 2026-03-04 10:44:44 +00:00
Roland Tannous
e8dfa8446d Merge pull request #289 from unslothai/fix/datasets-auth
Added auth to dataset endpoints
2026-03-04 08:21:42 +04:00
Shine1i
acf7cce4a8 merge: nightly into feature/data-reciper-enchansments
resolve setup.sh conflict by keeping nightly installer flow and preserving local data-designer plugin install via install_python_stack.py
2026-03-03 22:21:04 +01:00
Shine1i
e38911709f feat(recipe-studio): add authentication to API requests and backend routes 2026-03-03 21:32:39 +01:00
Shine1i
e2292df7da refactor(seed): package unstructured seed reader as local Data Designer plugin 2026-03-03 11:22:04 +01:00
Shine1i
a2b61fa762 merge nightly into feature/data-reciper-enchansments 2026-03-03 11:14:18 +01:00
Shine1i
4c5a2543c3 feat(seed): backend unstructured seed reader + server-side chunking, remove client chunk splitter 2026-03-03 11:11:26 +01:00