ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
976 commits 391 branches 58 tags 304 MiB
Commit graph

976 commits

This branch
This branch
All branches
Author SHA1 Message Date
Roland Tannous
f7ca361c5c feat: add LLM-assisted dataset detection using ephemeral GGUF helper 
Uses Qwen2.5-3B-Instruct Q8_0 via LlamaCppBackend to complement
heuristic-based dataset detection when heuristics are uncertain.

- New llm_assist.py: VLM instruction generation, column classification,
  and user-friendly warning generation for dataset issues
- Pre-cache helper GGUF on FastAPI startup (background thread)
- Reorder training pipeline: dataset processing runs BEFORE model load
  to avoid VRAM contention (detect → dataset → model → train)
- Add pre_detect_and_load_tokenizer() for lightweight detection
- LLM warnings on VLM conversion failures (broken URLs, missing images)
- LLM column classification fallback when heuristics return unknown
- Graceful degradation: all paths unchanged when helper unavailable
 2026-03-10 09:20:45 +00:00
Roland Tannous
daa50d0756 Revert "Merge pull request #347 from unslothai/feature/studio-storage-roots" 
This reverts commit 6b43e33ff1, reversing
changes made to 9edadaf21f.
 2026-03-10 01:52:47 +00:00
Roland Tannous
6b43e33ff1 Merge pull request #347 from unslothai/feature/studio-storage-roots 
update studio storage roots
 2026-03-10 05:49:42 +04:00
Roland Tannous
9edadaf21f Merge pull request #350 from unslothai/fix/vision-datasets-fix 
Fix VLM dataset detection and conversion
 2026-03-10 05:48:27 +04:00
Roland Tannous
8488c2b1df fix: fall back to auto-detection when user VLM mapping fails 
Instead of erroring out when custom_format_mapping fails conversion,
clear it and let auto-detection try. Handles stale cached mappings.
 2026-03-10 01:42:25 +00:00
Roland Tannous
dd6c38cc7b fix: probe image column candidates when multiple exist 
When multiple image columns are found, probes them (HEAD for URLs,
os.path.exists for paths) and picks the first that works.
Skips probing when top candidate is PIL/dict (score >= 75).
 2026-03-10 01:38:33 +00:00
Roland Tannous
81adc47b6e fix: prefer URL image columns over bare filenames, add value-based fallback 
find_image_column now scores candidates by resolvability (PIL > dict > URL > path)
and has a Pass 2 value-based fallback for columns not matching image keywords.
Fixes phiyodr/coco2017 picking file_name (unresolvable) over coco_url (resolvable).
 2026-03-10 01:36:19 +00:00
Roland Tannous
d6803de35a fix: detect list-of-strings text columns and pick random element for VLM conversion 
Handles datasets like phiyodr/coco2017 where captions is a list of strings.
 2026-03-10 01:32:19 +00:00
Roland Tannous
0b8325ab96 feat: add ShareGPT+image VLM format support and improve image column detection 
- Detect and convert ShareGPT/ChatML conversations with <image> placeholders
- Add file_name/filename as image column keywords
- Detect image paths and URLs by value (string ending in .jpg/.png/etc)
 2026-03-10 01:27:36 +00:00
Roland Tannous
56d02a3b57 fix: use word-boundary matching for image/audio column detection 
Substring matching caused false positives like 'pic' in 'topic',
leading to non-deterministic image column selection.
 2026-03-10 00:38:02 +00:00
Manan17
32569fc8a8 shifting setup & co inside studio 2026-03-09 23:48:31 +00:00
Shine1i
109db14817 feat(studio): add auth-specific paths and integrate auth database location 2026-03-09 23:48:31 +00:00
Shine1i
958bdef43e fix(studio): update temporary directory path to use system temp dir 2026-03-09 23:48:31 +00:00
Shine1i
5301514775 feat(studio): studio storage roots path utilities 2026-03-09 23:48:31 +00:00
Roland Tannous
32bbccc573 fix: resolve bare-filename images via HF repo lookup 
Datasets like VQAonline store image filenames (e.g. "img.png") without
the directory prefix. Build a basename→repo_path lookup using
list_repo_files, then resolve each file via hf_hub_download.
 2026-03-09 23:37:00 +00:00
Roland Tannous
c272c4f844 fix: prefer tabular files over archives in Tier 1 dataset preview 
Tier 1 check-format was picking images.zip over testmini.parquet,
causing wrong columns (image/label) and broken VLM mapping.
Also log first VLM conversion failure instead of swallowing silently.
 2026-03-09 22:00:20 +00:00
Roland Tannous
65413c95fb Merge pull request #349 from unslothai/license/agpl3-studio 
Add AGPL-3.0 SPDX headers to all source files
 2026-03-10 00:30:29 +04:00
Roland Tannous
d882678fe4 Add AGPL-3.0 SPDX headers to all source files 2026-03-09 20:17:45 +00:00
Roland Tannous
198ca7efce Merge pull request #348 from unslothai/license/agpl3-studio 
Main license file for studio codebase
 2026-03-09 23:38:08 +04:00
Roland Tannous
ac2906f357 Add AGPL-3.0 license to studio folder 2026-03-09 19:36:25 +00:00
Wasim Yousef Said
a4bc6330a0 Merge pull request #344 from unslothai/style/ui-feedback 
Refine UI spacing, icons, and border radius per feedback
 2026-03-09 19:24:12 +01:00
Wasim Yousef Said
971ef40d85 Merge pull request #345 from unslothai/feature/fixes-client 
feat(studio): fix chat code block actions and some training view changes
 2026-03-09 19:22:19 +01:00
Shine1i
1fe8995f1c feat(recipe-studio): add support for managing tools by provider in tool profiles 2026-03-09 19:19:14 +01:00
Shine1i
2ccb75f2b7 Merge remote-tracking branch 'origin/nightly' into feature/fixes-client 2026-03-09 19:07:42 +01:00
Roland Tannous
b6811bc5c4 Merge pull request #342 from unslothai/local-dataset 
dataset upload
 2026-03-09 21:22:23 +04:00
Roland Tannous
022bafaf92 store uploaded datasets under assets/datasets/uploads instead of ~/.cache 2026-03-09 17:06:36 +00:00
Roland Tannous
ae89101e81 Revert "narrow stale selection guard to only skip clearing for uploaded files" 
This reverts commit fbcd111a70.
 2026-03-09 16:51:30 +00:00
Roland Tannous
ffeefd15d1 Merge pull request #346 from unslothai/fix/eval-loss-worker-filtering 
fix: eval loss broken after subprocess isolation refactor
 2026-03-09 20:43:19 +04:00
Roland Tannous
41351e1566 fix: split dataset 80/20 when eval split matches train split 2026-03-09 16:36:44 +00:00
Shine1i
542d9126cc chore(data-recipe): bump data-designer to 0.5.2 and pin duckdb<1.5 2026-03-09 17:27:02 +01:00
Roland Tannous
6eba6fff43 fix: disable Start Training when eval_steps set without eval split 2026-03-09 16:20:00 +00:00
Shine1i
1f37b76b19 feat(recipe-studio): remove MCP tools-related dialogs and refactor tool profile management logic 2026-03-09 17:04:15 +01:00
Roland Tannous
2a11e79b8b fix: restore eval_enabled early signal for subprocess training 2026-03-09 15:35:49 +00:00
Shine1i
c67f4ba29f feat(recipe-studio): improve UI responsiveness and fix JSON preview handling 2026-03-09 16:04:46 +01:00
Roland Tannous
c3185d5d98 fix: allow eval-only progress events through worker callback filter 2026-03-09 14:39:49 +00:00
Shine1i
1ae8fb402b feat(studio): centralize chart styling and formatting 2026-03-09 15:34:15 +01:00
Roland Tannous
fbcd111a70 narrow stale selection guard to only skip clearing for uploaded files 2026-03-09 14:01:02 +00:00
Roland Tannous
1d06e2f54c switch dataset upload from base64 JSON to multipart/form-data with streamed writes 2026-03-09 13:55:45 +00:00
Roland Tannous
56412f2362 include all candidate files when scanning a directory, not just the first 2026-03-09 13:52:45 +00:00
Shine1i
d66bc2760b feat(studio): rework chart settings with a new preferences store and revamped settings UI 2026-03-09 14:47:20 +01:00
Roland Tannous
c998227fec add client-side file size validation before upload 2026-03-09 13:38:00 +00:00
Roland Tannous
4c5ded4c52 normalize uploaded filename extension to lowercase for consistent downstream checks 2026-03-09 13:35:55 +00:00
imagineer99
8fa021e839 style: reduce border radius on onboarding summary cards 2026-03-09 13:33:00 +00:00
Roland Tannous
19b2a0a0da Merge pull request #340 from unslothai/fix/auth-audio 
Added auth to audio generate endpoint
 2026-03-09 17:25:49 +04:00
Roland Tannous
87c8d7b3da Merge pull request #338 from unslothai/fix/trust-code 
Exposed trust_remote_code through the UI
 2026-03-09 17:19:56 +04:00
Roland Tannous
91dd7fc762 merge nightly, resolve conflict in use-chat-model-runtime 2026-03-09 13:19:17 +00:00
Roland Tannous
c719f1ba54 training: restore YAML fallback for trust_remote_code (no UI toggle) 2026-03-09 13:10:24 +00:00
imagineer99
d1c544b705 style: remove playground sidebar right border 2026-03-09 13:08:59 +00:00
Roland Tannous
7989cd4567 respect trust_remote_code toggle, return helpful error when required 2026-03-09 13:06:55 +00:00
Shine1i
95f9e0ba41 feat(studio): add support for code block actions including copy and download options in markdown blocks 2026-03-09 13:07:54 +01:00