ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
983 commits 391 branches 58 tags 304 MiB
Commit graph

727 commits

Author SHA1 Message Date
Roland Tannous
7f1fd28acd debug: decode first sample after train_on_completions masking 2026-03-10 14:08:14 +00:00
Roland Tannous
49b29fb1fd debug: fix dataset access - result is a dict, use dataset['dataset'] 2026-03-10 13:19:31 +00:00
Roland Tannous
21cd9f9d02 debug: improve sample preview with type info and traceback 2026-03-10 12:56:24 +00:00
Roland Tannous
a36c073770 debug: switch to print() for subprocess visibility 2026-03-10 12:49:01 +00:00
Roland Tannous
97612af993 debug: add temporary log statements for dataset preview and VLM instruction 2026-03-10 12:35:55 +00:00
Roland Tannous
5d471d7e4a feat: add AI Assist button for user-triggered column classification 
Move LLM-assisted column mapping from silent /check-format automation
to an explicit "AI Assist" button in the dataset mapping dialog. This
makes the feature transparent and user-controlled.

- Remove llm_classify_columns() from check_dataset_format() (heuristic-only)
- Remove auto-save suggested_mapping from use-training-actions.ts
- Add POST /api/datasets/ai-assist-mapping endpoint (receives preview
  samples from frontend, no dataset re-loading needed)
- Add AiAssistMappingRequest/Response models
- Add aiAssistMapping() frontend API function
- Add Sparkles AI Assist button to DatasetMappingCard with loading state
- Wire up handleAiAssist handler in dataset-preview-dialog.tsx
 2026-03-10 11:09:01 +00:00
Roland Tannous
0ec340d3e1 fix: LLM-assisted mapping flows from /check-format to training 
- Frontend auto-saves suggested_mapping into datasetManualMapping when
  check-format returns requires_manual_mapping=false, so the mapping
  flows to training via custom_format_mapping (no redundant AI calls)
- Backend returns meaningful warning when column detection fails
  (LLM-generated or static fallback) for both text and VLM datasets
- /check-format endpoint merges check_dataset_format warnings with
  existing URL-based image detection warnings
 2026-03-10 09:58:58 +00:00
Roland Tannous
f7ca361c5c feat: add LLM-assisted dataset detection using ephemeral GGUF helper 
Uses Qwen2.5-3B-Instruct Q8_0 via LlamaCppBackend to complement
heuristic-based dataset detection when heuristics are uncertain.

- New llm_assist.py: VLM instruction generation, column classification,
  and user-friendly warning generation for dataset issues
- Pre-cache helper GGUF on FastAPI startup (background thread)
- Reorder training pipeline: dataset processing runs BEFORE model load
  to avoid VRAM contention (detect → dataset → model → train)
- Add pre_detect_and_load_tokenizer() for lightweight detection
- LLM warnings on VLM conversion failures (broken URLs, missing images)
- LLM column classification fallback when heuristics return unknown
- Graceful degradation: all paths unchanged when helper unavailable
 2026-03-10 09:20:45 +00:00
Roland Tannous
daa50d0756 Revert "Merge pull request #347 from unslothai/feature/studio-storage-roots" 
This reverts commit 6b43e33ff1, reversing
changes made to 9edadaf21f.
 2026-03-10 01:52:47 +00:00
Roland Tannous
6b43e33ff1 Merge pull request #347 from unslothai/feature/studio-storage-roots 
update studio storage roots
 2026-03-10 05:49:42 +04:00
Roland Tannous
8488c2b1df fix: fall back to auto-detection when user VLM mapping fails 
Instead of erroring out when custom_format_mapping fails conversion,
clear it and let auto-detection try. Handles stale cached mappings.
 2026-03-10 01:42:25 +00:00
Roland Tannous
dd6c38cc7b fix: probe image column candidates when multiple exist 
When multiple image columns are found, probes them (HEAD for URLs,
os.path.exists for paths) and picks the first that works.
Skips probing when top candidate is PIL/dict (score >= 75).
 2026-03-10 01:38:33 +00:00
Roland Tannous
81adc47b6e fix: prefer URL image columns over bare filenames, add value-based fallback 
find_image_column now scores candidates by resolvability (PIL > dict > URL > path)
and has a Pass 2 value-based fallback for columns not matching image keywords.
Fixes phiyodr/coco2017 picking file_name (unresolvable) over coco_url (resolvable).
 2026-03-10 01:36:19 +00:00
Roland Tannous
d6803de35a fix: detect list-of-strings text columns and pick random element for VLM conversion 
Handles datasets like phiyodr/coco2017 where captions is a list of strings.
 2026-03-10 01:32:19 +00:00
Roland Tannous
0b8325ab96 feat: add ShareGPT+image VLM format support and improve image column detection 
- Detect and convert ShareGPT/ChatML conversations with <image> placeholders
- Add file_name/filename as image column keywords
- Detect image paths and URLs by value (string ending in .jpg/.png/etc)
 2026-03-10 01:27:36 +00:00
Roland Tannous
56d02a3b57 fix: use word-boundary matching for image/audio column detection 
Substring matching caused false positives like 'pic' in 'topic',
leading to non-deterministic image column selection.
 2026-03-10 00:38:02 +00:00
Manan17
32569fc8a8 shifting setup & co inside studio 2026-03-09 23:48:31 +00:00
Shine1i
109db14817 feat(studio): add auth-specific paths and integrate auth database location 2026-03-09 23:48:31 +00:00
Shine1i
958bdef43e fix(studio): update temporary directory path to use system temp dir 2026-03-09 23:48:31 +00:00
Shine1i
5301514775 feat(studio): studio storage roots path utilities 2026-03-09 23:48:31 +00:00
Roland Tannous
32bbccc573 fix: resolve bare-filename images via HF repo lookup 
Datasets like VQAonline store image filenames (e.g. "img.png") without
the directory prefix. Build a basename→repo_path lookup using
list_repo_files, then resolve each file via hf_hub_download.
 2026-03-09 23:37:00 +00:00
Roland Tannous
c272c4f844 fix: prefer tabular files over archives in Tier 1 dataset preview 
Tier 1 check-format was picking images.zip over testmini.parquet,
causing wrong columns (image/label) and broken VLM mapping.
Also log first VLM conversion failure instead of swallowing silently.
 2026-03-09 22:00:20 +00:00
Roland Tannous
d882678fe4 Add AGPL-3.0 SPDX headers to all source files 2026-03-09 20:17:45 +00:00
Roland Tannous
ac2906f357 Add AGPL-3.0 license to studio folder 2026-03-09 19:36:25 +00:00
Wasim Yousef Said
a4bc6330a0 Merge pull request #344 from unslothai/style/ui-feedback 
Refine UI spacing, icons, and border radius per feedback
 2026-03-09 19:24:12 +01:00
Shine1i
1fe8995f1c feat(recipe-studio): add support for managing tools by provider in tool profiles 2026-03-09 19:19:14 +01:00
Shine1i
2ccb75f2b7 Merge remote-tracking branch 'origin/nightly' into feature/fixes-client 2026-03-09 19:07:42 +01:00
Roland Tannous
b6811bc5c4 Merge pull request #342 from unslothai/local-dataset 
dataset upload
 2026-03-09 21:22:23 +04:00
Roland Tannous
022bafaf92 store uploaded datasets under assets/datasets/uploads instead of ~/.cache 2026-03-09 17:06:36 +00:00
Roland Tannous
ae89101e81 Revert "narrow stale selection guard to only skip clearing for uploaded files" 
This reverts commit fbcd111a70.
 2026-03-09 16:51:30 +00:00
Roland Tannous
41351e1566 fix: split dataset 80/20 when eval split matches train split 2026-03-09 16:36:44 +00:00
Shine1i
542d9126cc chore(data-recipe): bump data-designer to 0.5.2 and pin duckdb<1.5 2026-03-09 17:27:02 +01:00
Roland Tannous
6eba6fff43 fix: disable Start Training when eval_steps set without eval split 2026-03-09 16:20:00 +00:00
Shine1i
1f37b76b19 feat(recipe-studio): remove MCP tools-related dialogs and refactor tool profile management logic 2026-03-09 17:04:15 +01:00
Roland Tannous
2a11e79b8b fix: restore eval_enabled early signal for subprocess training 2026-03-09 15:35:49 +00:00
Shine1i
c67f4ba29f feat(recipe-studio): improve UI responsiveness and fix JSON preview handling 2026-03-09 16:04:46 +01:00
Roland Tannous
c3185d5d98 fix: allow eval-only progress events through worker callback filter 2026-03-09 14:39:49 +00:00
Shine1i
1ae8fb402b feat(studio): centralize chart styling and formatting 2026-03-09 15:34:15 +01:00
Roland Tannous
fbcd111a70 narrow stale selection guard to only skip clearing for uploaded files 2026-03-09 14:01:02 +00:00
Roland Tannous
1d06e2f54c switch dataset upload from base64 JSON to multipart/form-data with streamed writes 2026-03-09 13:55:45 +00:00
Roland Tannous
56412f2362 include all candidate files when scanning a directory, not just the first 2026-03-09 13:52:45 +00:00
Shine1i
d66bc2760b feat(studio): rework chart settings with a new preferences store and revamped settings UI 2026-03-09 14:47:20 +01:00
Roland Tannous
c998227fec add client-side file size validation before upload 2026-03-09 13:38:00 +00:00
Roland Tannous
4c5ded4c52 normalize uploaded filename extension to lowercase for consistent downstream checks 2026-03-09 13:35:55 +00:00
imagineer99
8fa021e839 style: reduce border radius on onboarding summary cards 2026-03-09 13:33:00 +00:00
Roland Tannous
19b2a0a0da Merge pull request #340 from unslothai/fix/auth-audio 
Added auth to audio generate endpoint
 2026-03-09 17:25:49 +04:00
Roland Tannous
91dd7fc762 merge nightly, resolve conflict in use-chat-model-runtime 2026-03-09 13:19:17 +00:00
Roland Tannous
c719f1ba54 training: restore YAML fallback for trust_remote_code (no UI toggle) 2026-03-09 13:10:24 +00:00
imagineer99
d1c544b705 style: remove playground sidebar right border 2026-03-09 13:08:59 +00:00
Roland Tannous
7989cd4567 respect trust_remote_code toggle, return helpful error when required 2026-03-09 13:06:55 +00:00