e62085a3d6The ChatCompletionRequest Pydantic model defaulted repetition_penalty to 1.1 when clients omitted the field. This silently forced llama-server to perform per-token repetition scanning, dropping streaming throughput from ~225 TPS to ~172 TPS (a 24% penalty). The Studio frontend always sends repetition_penalty=1.0 explicitly, so UI users were unaffected. But any API client hitting /v1/chat/completions without setting the field (curl, third-party integrations, Open WebUI, etc.) would get the slow path. Benchmarked on Qwen3.5-4B Q4_K_XL, GPU 0: - repeat_penalty=1.0: 225.2 TPS - repeat_penalty=1.1: 172.7 TPS (24% slower) - LM Studio (which applies rp internally): 170.8 TPS This aligns the Pydantic default with the frontend default (1.0), generate_chat_completion's function signature default (1.0), and llama-server's own default (1.0). |
||
|---|---|---|
| .. | ||
| assets | ||
| auth | ||
| core | ||
| loggers | ||
| models | ||
| plugins | ||
| requirements | ||
| routes | ||
| state | ||
| storage | ||
| tests | ||
| utils | ||
| __init__.py | ||
| _platform_compat.py | ||
| colab.py | ||
| main.py | ||
| run.py | ||