ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
unsloth/studio/backend/core/training
History
Roland Tannous afad614bfa feat: add LLM-assisted dataset detection using ephemeral GGUF helper 
Uses Qwen2.5-3B-Instruct Q8_0 via LlamaCppBackend to complement
heuristic-based dataset detection when heuristics are uncertain.

- New llm_assist.py: VLM instruction generation, column classification,
  and user-friendly warning generation for dataset issues
- Pre-cache helper GGUF on FastAPI startup (background thread)
- Reorder training pipeline: dataset processing runs BEFORE model load
  to avoid VRAM contention (detect → dataset → model → train)
- Add pre_detect_and_load_tokenizer() for lightweight detection
- LLM warnings on VLM conversion failures (broken URLs, missing images)
- LLM column classification fallback when heuristics return unknown
- Graceful degradation: all paths unchanged when helper unavailable
2026-03-10 09:20:45 +00:00
..
__init__.py Add AGPL-3.0 SPDX headers to all source files 2026-03-09 20:17:45 +00:00
trainer.py feat: add LLM-assisted dataset detection using ephemeral GGUF helper 2026-03-10 09:20:45 +00:00
training.py Add AGPL-3.0 SPDX headers to all source files 2026-03-09 20:17:45 +00:00
worker.py feat: add LLM-assisted dataset detection using ephemeral GGUF helper 2026-03-10 09:20:45 +00:00