unsloth/studio/backend/utils/datasets
Daniel Han-Chen c6c4378f38 Fix/adjust diffusion: round 23 P1+P2 batch for PR #5754
P1 #1 + #2 + #6: extended the chat / diffusion / training
identifier hardening to every export-side request model.
ExportCommonOptions (parent of ExportMergedModelRequest /
ExportBaseModelRequest / ExportLoRAAdapterRequest) now applies
_no_control_chars and _reject_embedded_hf_token to repo_id and
base_model_id; ExportGGUFRequest gets the same on its repo_id
plus a control-char check on quantization_method; and
LoadCheckpointRequest validates checkpoint_path. Previously
"/api/export/*" accepted newline-smuggled identifiers and
URL-form ``hf_xxxxx`` tokens that flowed into log lines.

P1 #3 + #4: ``_run_with_helper`` and ``_run_multi_pass_advisor``
now use a shared ``_gpu_workload_busy_for_helper`` that gates on
diffusion (round 22 already), training, AND export. The round 22
guard only checked diffusion, so the dataset helper / advisor
could still load llama-server on top of an active training run
or a resident export checkpoint. Each step fails closed
(unverifiable status counts as busy) so the user's primary
workload is preserved.

P1 #5: PublishDatasetRequest in models/data_recipe.py also
applies the identifier hardening to repo_id; the publish path
previously accepted control characters and URL-form tokens.

P1 #7-10: added _validate_logged_identifier helper to
routes/models.py and applied it to the path / query parameter
endpoints that flow into logger.info(...) calls --
``/config/{model_name}``, ``/check-vision/{model_name}``,
``/check-embedding/{model_name}``, ``/gguf-variants``. Mapped
the validator's ValueError to HTTP 422 so the client sees the
same shape as a Pydantic validation failure.

P2 #11 + #12: ``Loading diffusion model %s`` and
``Diffusion load failed for %s`` log lines route ``repo_id`` /
``effective_base`` through ``_display_repo_id`` (collapses
absolute local paths to the leaf, still scrubs HF tokens)
instead of plain ``_redact_hf_tokens``. The error path was
already collapsed in the user-facing 400 / RuntimeError, but
the structured-log lines kept the full path.

All 97 diffusion + training-validation + related tests pass
locally.
2026-05-25 11:20:05 +00:00
..
__init__.py Studio: tools, thinking blocks, code execution and web search for safetensors (#5520) 2026-05-19 06:30:17 -07:00
chat_templates.py MLX training support for Studio on Apple Silicon (#5340) 2026-05-14 05:24:20 -07:00
data_collators.py fix: install.sh Mac Intel compatibility + Studio no-torch support (#4624) 2026-03-27 02:09:21 -07:00
dataset_utils.py feat(studio): add Continued Pretraining (CPT) as a training method (#4677) 2026-05-06 13:38:35 +04:00
format_conversion.py fix(studio): avoid UnicodeEncodeError on Windows cp1252 consoles (#4699) 2026-03-30 06:40:47 -07:00
format_detection.py Final cleanup 2026-03-12 18:28:04 +00:00
llm_assist.py Fix/adjust diffusion: round 23 P1+P2 batch for PR #5754 2026-05-25 11:20:05 +00:00
model_mappings.py Studio: tools, thinking blocks, code execution and web search for safetensors (#5520) 2026-05-19 06:30:17 -07:00
raw_text.py feat(studio): add Continued Pretraining (CPT) as a training method (#4677) 2026-05-06 13:38:35 +04:00
vlm_processing.py Final cleanup 2026-03-12 18:28:04 +00:00