Shine1i
2ccb75f2b7
Merge remote-tracking branch 'origin/nightly' into feature/fixes-client
2026-03-09 19:07:42 +01:00
Roland Tannous
b6811bc5c4
Merge pull request #342 from unslothai/local-dataset
...
dataset upload
2026-03-09 21:22:23 +04:00
Roland Tannous
41351e1566
fix: split dataset 80/20 when eval split matches train split
2026-03-09 16:36:44 +00:00
Shine1i
542d9126cc
chore(data-recipe): bump data-designer to 0.5.2 and pin duckdb<1.5
2026-03-09 17:27:02 +01:00
Roland Tannous
2a11e79b8b
fix: restore eval_enabled early signal for subprocess training
2026-03-09 15:35:49 +00:00
Roland Tannous
c3185d5d98
fix: allow eval-only progress events through worker callback filter
2026-03-09 14:39:49 +00:00
Roland Tannous
56412f2362
include all candidate files when scanning a directory, not just the first
2026-03-09 13:52:45 +00:00
Roland Tannous
91dd7fc762
merge nightly, resolve conflict in use-chat-model-runtime
2026-03-09 13:19:17 +00:00
Roland Tannous
c719f1ba54
training: restore YAML fallback for trust_remote_code (no UI toggle)
2026-03-09 13:10:24 +00:00
Roland Tannous
7989cd4567
respect trust_remote_code toggle, return helpful error when required
2026-03-09 13:06:55 +00:00
Manan17
a49638c504
dataset upload
2026-03-09 05:50:18 +00:00
Shine1i
3b1663b1e9
feat(recipe-studio, datasets): improve dataset handling and update metadata logic
2026-03-09 02:47:32 +01:00
samit
662cb1c440
Adding trust_remote_code to the orchestrator and worker
2026-03-08 16:44:41 -07:00
Shine1i
a2dde15367
merge nightly
2026-03-09 00:32:33 +01:00
samit
86e94b5844
exposed trust_remote_code through the UI
2026-03-08 16:28:56 -07:00
Roland Tannous
a1778d6655
fix: replace is_dataset_multimodal with is_dataset_image/is_dataset_audio in training orchestrator
2026-03-08 19:40:00 +00:00
Manan17
80b704d7b7
Audio_VLM bug fix
2026-03-08 19:14:07 +00:00
Roland Tannous
7ee81dd7df
feat: route audio inference (TTS, ASR, Whisper) through orchestrator/worker subprocess
2026-03-08 18:25:27 +00:00
Roland Tannous
1435dbaf59
merge nightly into audio branch (mock test)
2026-03-08 10:23:44 +00:00
Manan17
ef714f010e
adding export support
2026-03-08 04:18:20 +00:00
Roland Tannous
a7c34b42be
fix: clear stale model state on failed inference subprocess reload
2026-03-07 23:32:53 +00:00
Roland Tannous
4f766bbe25
fix: reset checkpoint metadata on failed export checkpoint reload
2026-03-07 23:29:34 +00:00
Manan17
6487f81113
check fir gated repo
2026-03-07 21:32:50 +00:00
Roland Tannous
8454e6dd2b
fix: scope dataloader_num_workers=0 to Windows + transformers 5.x only
2026-03-07 17:55:59 +00:00
Roland Tannous
ef9184c731
fix: prevent training hang on Windows by adding triton-windows support
2026-03-07 17:53:36 +00:00
Roland Tannous
e25705a211
fix: propagate PYTHONPATH to child subprocesses, revert tokenizer patching
2026-03-07 11:28:24 +00:00
Roland Tannous
9330588015
fix: patch TokenizersBackend in export output after save_pretrained
2026-03-07 10:57:51 +00:00
Roland Tannous
76c78afb8f
fix: patch TokenizersBackend by model name - Qwen3.5→Qwen2Tokenizer, GLM→PreTrainedTokenizer
2026-03-07 10:29:59 +00:00
Roland Tannous
d60cd2843f
fix: patch Qwen3.5 broken tokenizer_class TokenizersBackend across all backends
2026-03-07 09:43:25 +00:00
Roland Tannous
bd60562145
fix: bump transformers 5.x pin from 5.1.0 to 5.2.0 for Qwen3.5 support
2026-03-07 09:10:09 +00:00
Roland Tannous
0b3397cc3a
fix: fail fast if runtime pip install of transformers 5.x fails
2026-03-07 08:40:25 +00:00
Roland Tannous
f3aeceeb24
fix: join prior pump thread before starting new training job
2026-03-07 08:37:03 +00:00
Roland Tannous
728420b290
fix: drain stale events from resp_queue after generation cancel
2026-03-07 08:12:16 +00:00
Roland Tannous
609e3168a1
fix: serialize generation with _gen_lock to prevent concurrent queue readers
...
Two overlapping /chat/completions requests could both read from the shared
resp_queue, consuming and dropping each other's token events. Replace the
request_id filtering (which silently dropped non-matching messages) with a
threading.Lock that serializes generation — correct for single-GPU inference.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 04:06:51 +00:00
Roland Tannous
9470957bb9
fix: log final GGUF file locations after relocation
2026-03-06 18:04:27 +00:00
Roland Tannous
5a828ebd43
fix: increase export timeout to 1 hour for large model GGUF conversion
2026-03-06 17:59:42 +00:00
Shine1i
5e5feb5c00
feat(recipe-studio, validators): tweak OXC validator with lint suppression support and improve error normalization logic
2026-03-06 09:40:53 +01:00
Roland Tannous
c3bc19494f
fix: pin huggingface_hub==1.3.0 in .venv_t5 (satisfies transformers 5.x)
2026-03-06 06:19:28 +00:00
Roland Tannous
6b32af0bdc
feat: subprocess-based export, pin huggingface_hub==0.36.0
2026-03-06 06:03:09 +00:00
Shine1i
93063c3212
feat(recipe-studio, validators): extend OXC validator with code shape support and integrate into recipe studio
2026-03-06 02:04:05 +01:00
Shine1i
cf8cb9109b
feat(recipe-studio): add support for inference_extra_body configuration with collapsible UI and enhanced validation logic
2026-03-05 23:33:48 +01:00
Roland Tannous
31334cece1
fix: indentation error in orchestrator load_model
2026-03-05 19:43:30 +00:00
Shine1i
3cafc0506e
feat(data-recipes, validators): extend OXC validator with linting mode support and integrate new modes into recipe studio
2026-03-05 20:19:31 +01:00
Roland Tannous
5bd6fac80e
fix: always spawn fresh subprocess per model load
...
Reusing a subprocess after unsloth patches torch internals causes
inspect.getsource() failures when loading a different model type.
Each load now gets a clean Python interpreter.
2026-03-05 19:15:37 +00:00
Roland Tannous
7fc563731a
fix: use mp.Event for instant cross-process generation cancel
...
Replaces cmd_queue-based cancel polling with a shared mp.Event.
Fixes two issues:
- Loading a new model while generating no longer hangs (cancel is instant)
- Subprocess shuts down cleanly after explicit stop generation
2026-03-05 18:54:17 +00:00
Shine1i
552eb06bed
feat(data-recipes, validators): add OXC validator runtime and integration with recipe studio
2026-03-05 19:48:26 +01:00
Roland Tannous
4eabc74f34
feat: subprocess-based inference for transformers version switching
...
Inference now runs in a persistent subprocess, solving the same
transformers version-switching problem that was fixed for training.
The subprocess stays alive between requests (model in GPU memory)
and is only restarted when switching transformers versions.
New files:
- core/inference/worker.py: subprocess entry point with command loop
- core/inference/orchestrator.py: parent-side proxy with same API
Modified:
- core/inference/__init__.py: exports orchestrator as default backend
- routes/inference.py: removed in-process ensure_transformers_version()
2026-03-05 17:47:57 +00:00
Roland Tannous
842c05e75a
fix: lazy imports in core/__init__ to prevent subprocess importing ML libs early
2026-03-05 16:56:45 +00:00
Roland Tannous
878f8f3924
fix: remove UnslothTrainer/get_trainer from core __init__ exports
2026-03-05 15:57:07 +00:00
Roland Tannous
f8bd4303f7
feat: subprocess-based training for transformers version switching
2026-03-05 15:40:32 +00:00
