ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
unsloth/studio/backend/core/inference
History
Roland Tannous 0e7c8a2e5e Switch GGUF backend from /v1/completions to /v1/chat/completions 
Fixes two bugs:
1. Chat template tags (<|im_start|>, <|im_end|>) leaking into output
   because /v1/completions treated them as literal text
2. Image hallucination because image_b64 was never passed to llama-server

Now llama-server handles chat templates natively and receives images
as OpenAI-format multimodal content parts for vision models.
2026-02-24 19:21:01 +04:00
..
__init__.py Add GGUF model inference via llama-server backend 2026-02-24 17:40:05 +04:00
inference.py feat: clear unsloth_compiled_cache on startup, shutdown, and between model loads 2026-02-23 07:26:22 +00:00
llama_cpp.py Switch GGUF backend from /v1/completions to /v1/chat/completions 2026-02-24 19:21:01 +04:00