f4873182e0Adds studio/backend/utils/datasets/dataset_none_detect.py, a standalone scanner that reports None/empty content turns in alpaca, chatml, sharegpt, and gptoss datasets without modifying data, plus generator and runner scripts under tests/utils. Depends only on the datasets library and is not wired into the package init, so it stays import-light. |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| chat_templates.py | ||
| data_collators.py | ||
| dataset_none_detect.py | ||
| dataset_utils.py | ||
| format_conversion.py | ||
| format_detection.py | ||
| llm_assist.py | ||
| model_mappings.py | ||
| raw_text.py | ||
| vlm_processing.py | ||