Replaces `enabled`/`ref_strategy` with unified `mode` parameter in ASFTStreamingConfig. Adds "auto", "seq", "batch", "hybrid", and "off" modes with automatic fallback logic. Implements seq_kv_cache streaming with KV cache reuse and batch microbatching support. Updates notebook defaults to use `mode="auto"` for optimal VRAM reduction. Adds comprehensive tests for mode routing, fallback behavior, and backward compatibility. |
||
|---|---|---|
| .. | ||
| qlora | ||
| saving | ||
| utils | ||
| __init__.py | ||
| test_asft.py | ||
| test_get_model_name.py | ||
| test_model_registry.py | ||
| test_raw_text.py | ||
| test_unsloth_cli.py | ||