Commit graph

21 commits

Author SHA1 Message Date
Roland Tannous
9fd08a3f25 debug: fix dataset access - result is a dict, use dataset['dataset'] 2026-03-10 13:19:31 +00:00
Roland Tannous
a5d9f611e6 debug: improve sample preview with type info and traceback 2026-03-10 12:56:24 +00:00
Roland Tannous
10ba97c34c debug: switch to print() for subprocess visibility 2026-03-10 12:49:01 +00:00
Roland Tannous
19cbf575ac debug: add temporary log statements for dataset preview and VLM instruction 2026-03-10 12:35:55 +00:00
Roland Tannous
afad614bfa feat: add LLM-assisted dataset detection using ephemeral GGUF helper
Uses Qwen2.5-3B-Instruct Q8_0 via LlamaCppBackend to complement
heuristic-based dataset detection when heuristics are uncertain.

- New llm_assist.py: VLM instruction generation, column classification,
  and user-friendly warning generation for dataset issues
- Pre-cache helper GGUF on FastAPI startup (background thread)
- Reorder training pipeline: dataset processing runs BEFORE model load
  to avoid VRAM contention (detect → dataset → model → train)
- Add pre_detect_and_load_tokenizer() for lightweight detection
- LLM warnings on VLM conversion failures (broken URLs, missing images)
- LLM column classification fallback when heuristics return unknown
- Graceful degradation: all paths unchanged when helper unavailable
2026-03-10 09:20:45 +00:00
Roland Tannous
22eb0eea29 Revert "Merge pull request #347 from unslothai/feature/studio-storage-roots"
This reverts commit e9c7b97d23, reversing
changes made to b75cc9b959.
2026-03-10 01:52:47 +00:00
Shine1i
b08b606b21 feat(studio): studio storage roots path utilities 2026-03-09 23:48:31 +00:00
Roland Tannous
a0f03d3080 Add AGPL-3.0 SPDX headers to all source files 2026-03-09 20:17:45 +00:00
Roland Tannous
a7d78d16be fix: restore eval_enabled early signal for subprocess training 2026-03-09 15:35:49 +00:00
Roland Tannous
d85176ba1a fix: allow eval-only progress events through worker callback filter 2026-03-09 14:39:49 +00:00
Roland Tannous
f416b7aa3d training: restore YAML fallback for trust_remote_code (no UI toggle) 2026-03-09 13:10:24 +00:00
Roland Tannous
83b1ff05ef respect trust_remote_code toggle, return helpful error when required 2026-03-09 13:06:55 +00:00
samit
6aa50d353f exposed trust_remote_code through the UI 2026-03-08 16:28:56 -07:00
Roland Tannous
7db2c90cc6 merge nightly into audio branch (mock test) 2026-03-08 10:23:44 +00:00
Roland Tannous
dcebfe718a fix: prevent training hang on Windows by adding triton-windows support 2026-03-07 17:53:36 +00:00
Roland Tannous
c882a3d2f7 fix: propagate PYTHONPATH to child subprocesses, revert tokenizer patching 2026-03-07 11:28:24 +00:00
Roland Tannous
f101befca7 fix: bump transformers 5.x pin from 5.1.0 to 5.2.0 for Qwen3.5 support 2026-03-07 09:10:09 +00:00
Roland Tannous
2c4c598832 fix: fail fast if runtime pip install of transformers 5.x fails 2026-03-07 08:40:25 +00:00
Roland Tannous
7e59440029 fix: pin huggingface_hub==1.3.0 in .venv_t5 (satisfies transformers 5.x) 2026-03-06 06:19:28 +00:00
Roland Tannous
661ac4be96 feat: subprocess-based export, pin huggingface_hub==0.36.0 2026-03-06 06:03:09 +00:00
Roland Tannous
f90af41c5f feat: subprocess-based training for transformers version switching 2026-03-05 15:40:32 +00:00