Trim ~255 MB of unused packages from Studio setup (#4395)
* Comment out large unused packages from Studio setup requirements Audited all packages installed by `unsloth studio setup` against actual imports in unsloth, unsloth_zoo, and studio/backend. The following have zero imports anywhere and are the largest offenders by disk size: - gradio (148 MB) in studio.txt -- Studio uses React + FastAPI, not Gradio - executorch (41.5 MB) in extras-no-deps.txt -- no imports found - scikit-learn (31.8 MB) in extras.txt -- no imports found - MeCab (19.9 MB) in extras.txt -- Japanese tokenizer, no imports found - coremltools (10.2 MB) in extras.txt -- Apple CoreML, no imports found - uroman (4.0 MB) in extras.txt -- romanization tool, no imports found Total savings: ~255 MB (~32% of the 805 MB installed by setup). Each line is commented out with the package size annotated so they can be re-enabled easily if needed in the future. * Restore scikit-learn -- needed by sentence_transformers sentence_transformers is installed with --no-deps in extras-no-deps.txt, so its sklearn dependency is not auto-resolved. Multiple modules in sentence_transformers import sklearn at the top level (evaluation, util/similarity), so removing scikit-learn would break embedding jobs.
This commit is contained in:
parent
ca87669937
commit
3a28446a54
3 changed files with 6 additions and 6 deletions
|
|
@ -8,7 +8,7 @@ snac
|
|||
# TRL and related packages
|
||||
trl==0.23.1
|
||||
git+https://github.com/meta-pytorch/OpenEnv.git
|
||||
executorch>=1.0.1
|
||||
# executorch>=1.0.1 # 41.5 MB - no imports in unsloth/zoo/studio
|
||||
torch-c-dlpack-ext
|
||||
sentence_transformers==5.2.0
|
||||
transformers==4.57.1
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ tomli-w
|
|||
|
||||
# ExecuTorch dependencies
|
||||
ruamel.yaml
|
||||
coremltools
|
||||
# coremltools # 10.2 MB - Apple CoreML, no imports in unsloth/zoo/studio
|
||||
expecttest
|
||||
flatbuffers
|
||||
hydra-core
|
||||
|
|
@ -15,7 +15,7 @@ pytest<9.0
|
|||
pytest-json-report
|
||||
pytest-rerunfailures==15.1
|
||||
pytest-xdist
|
||||
# Also needed by sentence_transformers
|
||||
# Also needed by sentence_transformers (installed with --no-deps in extras-no-deps.txt)
|
||||
scikit-learn==1.7.1
|
||||
|
||||
# Additional extras
|
||||
|
|
@ -26,8 +26,8 @@ omegaconf
|
|||
einx
|
||||
pyloudnorm
|
||||
openai-whisper
|
||||
uroman
|
||||
MeCab
|
||||
# uroman # 4.0 MB - romanization, no imports found
|
||||
# MeCab # 19.9 MB - Japanese tokenizer, no imports found
|
||||
loguru
|
||||
flatten_dict
|
||||
ffmpy
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@ datasets==4.3.0
|
|||
pyjwt
|
||||
easydict
|
||||
addict
|
||||
gradio>=4.0.0
|
||||
# gradio>=4.0.0 # 148 MB - Studio uses React + FastAPI, not Gradio
|
||||
huggingface-hub==0.36.2
|
||||
structlog>=24.1.0
|
||||
diceware
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue