c8057d911b* fix: system prompt was dropped in unsloth text and vision inference * refactor: simplify system prompt message construction * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: use multimodal typed content parts for vision system message and add fallback The system message content must use typed content parts ([{"type": "text", "text": ...}]) instead of a plain string to match the multimodal processor contract (consistent with the audio path). Plain strings cause some processors (e.g. LLaVA) to silently drop the system prompt. Also wraps processor.apply_chat_template in try/except so models that reject the system role gracefully fall back to no system message with a warning log. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: capture and log original exception in vision system prompt fallback --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| audio_codecs.py | ||
| defaults.py | ||
| inference.py | ||
| llama_cpp.py | ||
| orchestrator.py | ||
| tools.py | ||
| worker.py | ||