798e73e770
Qwen-family models (Qwen3.5, etc.) use a tiktoken-based tokenizer. When Studio routes these models through the transformers 5.x overlay (.venv_t5), the tokenizer instantiation fails with: ValueError: Couldn't instantiate the backend tokenizer... You need to have sentencepiece or tiktoken installed This happens because .venv_t5 was set up with only transformers and huggingface_hub (both --no-deps). The main venv usually has tiktoken, but on Windows it sometimes does not. Fix: install tiktoken into .venv_t5 in both setup.sh and the runtime fallback (_ensure_venv_t5_exists). Uses --no-deps in the runtime path since regex/requests are always present in the main venv. |
||
|---|---|---|
| .. | ||
| assets | ||
| auth | ||
| core | ||
| loggers | ||
| models | ||
| plugins | ||
| requirements | ||
| routes | ||
| state | ||
| tests | ||
| utils | ||
| __init__.py | ||
| colab.py | ||
| main.py | ||
| run.py | ||