ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
unsloth/tests
History
danielhanchen 944c417ab2 tests: add gemma4_flex_bench for flex fast_inference throughput 
Throughput harness for the flex engine (UNSLOTH_FAST_INFERENCE=1) across
a batch-size sweep. Captures CUDA graphs per bucket and reports
post-warmup tok/s. Pairs with tests/gemma4_fast_inference_parity.py
(HF-naive parity) and tests/gemma4_fast_bench.py (vLLM nightly path) so
the three engines can be compared on the same prompt pool and new-token
budget.

Measured on B200 bf16 greedy, 64 new tokens, bs=16:

    E2B        645 tok/s   (28x HF naive 22.9)
    E4B        492 tok/s   (25x HF naive 19.7)
    31B        198 tok/s   (15x HF naive 13.5)
    26B-A4B    391 tok/s   (24x HF naive 16.2, 5.7x vLLM nightly 68.1)
2026-04-23 12:33:09 +00:00
..
python Add configurable PyTorch mirror via UNSLOTH_PYTORCH_MIRROR env var (#5024) 2026-04-15 11:39:11 +04:00
qlora Revert "[pre-commit.ci] auto fixes from pre-commit.com hooks" 2025-12-01 07:24:58 -08:00
saving Add regression test for shell injection fix in GGML conversion (#4773) 2026-04-02 00:10:47 -07:00
sh Add configurable PyTorch mirror via UNSLOTH_PYTORCH_MIRROR env var (#5024) 2026-04-15 11:39:11 +04:00
studio/install Add ROCm test suite for PR #4720 (#4824) 2026-04-11 04:44:13 -07:00
utils feat: Add cactus QAT scheme support (#4679) 2026-04-15 07:40:03 -07:00
__init__.py Qwen 3, Bug Fixes (#2445) 2025-04-30 22:38:39 -07:00
flex_fastlm_bench.py inference: add AGPLv3 license headers 2026-04-21 13:19:01 +00:00
flex_fastlm_smoke.py inference: add AGPLv3 license headers 2026-04-21 13:19:01 +00:00
flex_gemma4_moe_merge_parity.py flex: add Gemma 4 MoE inference support (bf16 + LoRA) 2026-04-23 08:28:53 +00:00
flex_gemma4_parity.py flex: add Gemma 4 MoE inference support (bf16 + LoRA) 2026-04-23 08:28:53 +00:00
flex_gpt_oss_parity.py tests: extend flex_gpt_oss_parity with flex_eager + LoRA variants 2026-04-23 06:54:41 +00:00
flex_lazy_batch_smoke.py [pre-commit.ci] auto fixes from pre-commit.com hooks 2026-04-21 14:58:27 +00:00
flex_lazy_live_smoke.py [pre-commit.ci] auto fixes from pre-commit.com hooks 2026-04-21 14:58:27 +00:00
flex_moe_bench.py tests: add --chat_template to vLLM + HF benches; switch vLLM to llm.chat 2026-04-23 04:53:33 +00:00
flex_moe_merge_parity.py tests: flex_moe_merge_parity.py — numerical + perf check for MoE LoRA merge 2026-04-23 04:23:09 +00:00
flex_moe_micro_bench.py tests/flex_moe_micro_bench: coord_descent compile_opts profile 2026-04-22 18:31:15 +00:00
flex_moe_parity.py flex/moe: compile_walker option — 2x decode throughput on Qwen3 MoE 2026-04-22 17:27:41 +00:00
flex_moe_smoke.py flex: fix Qwen3 MoE smoke regressions (dtype / MoE MLP / peft patching) 2026-04-22 10:46:44 +00:00
flex_moe_vllm_bench.py tests: add --chat_template to vLLM + HF benches; switch vLLM to llm.chat 2026-04-23 04:53:33 +00:00
flex_sleep_mode_smoke.py [pre-commit.ci] auto fixes from pre-commit.com hooks 2026-04-21 14:58:27 +00:00
gemma4_fast_inference_parity.py tests: add gemma4_fast_inference_parity for vLLM-vs-HF greedy check 2026-04-23 11:39:07 +00:00
gemma4_flex_bench.py tests: add gemma4_flex_bench for flex fast_inference throughput 2026-04-23 12:33:09 +00:00
run_all.sh fix: add tokenizers to no-torch deps and TORCH_CONSTRAINT for arm64 macOS py313+ (#4748) 2026-04-01 06:12:17 -07:00
test_cli_export_unpacking.py studio: stream export worker output into the export dialog (#4897) 2026-04-14 08:55:43 -07:00
test_get_model_name.py feat: Add support for OLMo-3 model (#4678) 2026-04-15 07:39:11 -07:00
test_loader_glob_skip.py Add unit tests for HfFileSystem glob skip guard (#4854) 2026-04-06 08:54:36 -07:00
test_model_registry.py Revert "[FIX] Vllm guided decoding params (#3662)" 2025-12-01 05:43:45 -08:00
test_raw_text.py fix: check find() return value before adding offset in try_fix_tokenizer (#4923) 2026-04-09 06:15:46 -07:00