samit
862b4100d2
deleted duplicate definitions
2026-02-27 06:00:28 -08:00
Manan17
168957a87a
Aggregating sharded models, showing fit/oom for quantizations
2026-02-27 08:23:15 +00:00
Roland Tannous
01082b84e5
Merge branch 'nightly' into feat/gguf-llama-cpp-inference
2026-02-25 16:06:03 +04:00
Roland Tannous
0e7c8a2e5e
Switch GGUF backend from /v1/completions to /v1/chat/completions
...
Fixes two bugs:
1. Chat template tags (<|im_start|>, <|im_end|>) leaking into output
because /v1/completions treated them as literal text
2. Image hallucination because image_b64 was never passed to llama-server
Now llama-server handles chat templates natively and receives images
as OpenAI-format multimodal content parts for vision models.
2026-02-24 19:21:01 +04:00
Roland Tannous
ef1cd3ac98
Use llama-server -hf mode, add GGUF variant selector, fix vision detection
...
Replace Python-side GGUF download with llama-server's native -hf flag for
HuggingFace repos. Add frontend variant picker so users can choose
quantization (Q4_K_M, Q8_0, BF16, etc.) with file sizes. Fix vision
detection via mmproj files instead of hardcoding is_vision=False.
2026-02-24 19:03:06 +04:00
Roland Tannous
08aeeaee4b
Fix llama-server: build in-tree, fix path resolution, add LD_LIBRARY_PATH
2026-02-24 18:19:29 +04:00
Roland Tannous
a40ebb1aab
Add GGUF model inference via llama-server backend
2026-02-24 17:40:05 +04:00
Manan17
aeb198f52d
Fixing base model export issue for vlms
2026-02-24 01:34:11 +00:00
Roland Tannous
dbbcdb4f09
feat: clear unsloth_compiled_cache on startup, shutdown, and between model loads
2026-02-23 07:26:22 +00:00
Roland Tannous
c051e3d532
fix: load proper vision processor from base model when FastVisionModel returns raw tokenizer, add tokenize=False to vision chat template
2026-02-21 04:40:29 +00:00
Manan17
f6ebeb1d42
Mapping proper tokenizer for VLMs
2026-02-21 01:57:05 +00:00
Manan17
3fa9e773c2
fixed the vlm's text only errors
2026-02-20 22:23:26 +00:00
Manan17
fdeccec259
Fixing compare feature
2026-02-19 20:15:44 +00:00
Shine1i
0db7da96cc
feat: support disabling top-k sampling with -1 and standardize normalization logic
...
- Updated top-k parameter range to accept -1 in models and frontend.
- Added utility to normalize top-k for backend compatibility.
2026-02-16 21:33:24 +01:00
Roland Tannous
909955767b
feat: add min_p sampling parameter to /chat/completions generation pipeline
2026-02-16 06:33:17 +00:00
Shine1i
571959e383
feat: add cancelation support for chat generation and streaming tasks
2026-02-15 18:23:27 +01:00
Roland Tannous
be3934860f
strip extra debug statements
2026-02-14 19:23:51 +00:00
Roland Tannous
3ff3def555
replace model unloading and peft loading mechanism for compare feature
2026-02-14 19:18:49 +00:00
Roland Tannous
e7ae901737
del model.peft_config instead of using model.delete_adapter
2026-02-14 17:32:15 +00:00
Roland Tannous
7d8e991c1f
added print statements for activate_lora_adapter
2026-02-14 17:25:37 +00:00
Roland Tannous
6fefbe9f0b
swipped logger for print statements as logger isn't propagating
2026-02-14 17:21:26 +00:00
Roland Tannous
d0b94eae75
added logging
2026-02-14 17:09:07 +00:00
Roland Tannous
b5c8136957
exclude default from model.delete_adapter
2026-02-14 17:03:52 +00:00
Roland Tannous
35a6e40268
_apply_adapter_state now calls revert_to_base_model and activate_lora_adapter properly
2026-02-14 16:57:24 +00:00
Roland Tannous
f67ee58347
feat(inference): add use_adapter field for per-request adapter toggling in compare mode
2026-02-14 14:52:13 +00:00
Roland Tannous
418a374125
migrate _generate_vision_response to use TextIteratorStreamer + background thread
2026-02-14 09:30:32 +00:00
Roland Tannous
da1cde971c
use get_device() for device selection and clear_gpu_cache() for GPU memory cleanup in inference, trainer, and export
2026-02-11 16:56:52 +00:00
Roland Tannous
b4ec0389f0
refactor/inference-api-routes-part-1
2026-02-03 16:57:57 +00:00
Roland Tannous
62ddcfa019
Refactor [dataset_utils.py](cci:7://file:///home/support/new-ui-prototype/studio/backend/utils/datasets/dataset_utils.py:0:0-0:0) into focused modules
2026-02-03 14:38:02 +00:00
Roland Tannous
544d6944d1
root studio folder
2026-02-02 09:13:49 +00:00
