d5758ecf58
All four reproduced before fixing, and re-measured after. - Proxy-only egress was declared offline. With HTTP(S)_PROXY set, the proxy resolves the hub host, so a failing local lookup says nothing. The DNS shortcut now stands down whenever a proxy applies (and honours NO_PROXY), letting the proxy-aware probe decide. Measured: endpoint probe reachable through the proxy while the guard still forced offline. - The training worker kept its own inline probe hardcoded to huggingface.co, so a reachable HF_ENDPOINT mirror set lifetime offline flags. It now uses the shared endpoint- and proxy-aware helper. - /models/check-vision, /models/config and /picker/chat-template ran the probe even for local paths, which never reach the hub. Measured 0.9s of pure latency per request; now skipped via _hf_offline_if_unreachable_for. DNS/endpoint/proxy helpers now live in utils.utils so llama_cpp and the training worker share one implementation instead of three copies. The static pin in test_offline_inference_parent moved with the probe: the worker block must delegate to the shared helper and must not hardcode a host, and the daemon-thread/no-setdefaulttimeout property is pinned on dns_host_dead where it now lives. Offline path unchanged: load 686s -> 6s, /models/config 378s -> 0s, and a local-path vision check is back to 0s.
51 lines
1.9 KiB
Python
51 lines
1.9 KiB
Python
# SPDX-License-Identifier: AGPL-3.0-only
|
|
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
|
|
|
from __future__ import annotations
|
|
|
|
import asyncio
|
|
from typing import Optional
|
|
|
|
from fastapi import APIRouter, Body, Depends, Query
|
|
|
|
from auth.authentication import get_current_subject
|
|
from hub.dependencies import get_hf_token
|
|
|
|
from ..schemas import (
|
|
MAX_CHAT_TEMPLATE_BYTES,
|
|
ModelTemplateResponse,
|
|
ValidateChatTemplateRequest,
|
|
ValidateChatTemplateResponse,
|
|
)
|
|
from ..service import read_default_chat_template, validate_chat_template
|
|
|
|
router = APIRouter()
|
|
|
|
|
|
@router.post("/validate-chat-template", response_model = ValidateChatTemplateResponse)
|
|
async def validate_chat_template_route(
|
|
body: ValidateChatTemplateRequest = Body(...),
|
|
current_subject: str = Depends(get_current_subject),
|
|
) -> ValidateChatTemplateResponse:
|
|
return await asyncio.to_thread(validate_chat_template, body.template)
|
|
|
|
|
|
@router.get("/chat-template/{model_name:path}", response_model = ModelTemplateResponse)
|
|
async def get_default_chat_template_route(
|
|
model_name: str,
|
|
gguf_variant: Optional[str] = Query(None),
|
|
hf_token: Optional[str] = Depends(get_hf_token),
|
|
current_subject: str = Depends(get_current_subject),
|
|
) -> ModelTemplateResponse:
|
|
# Cached repos resolve from disk, but a cache miss falls through to the hub and
|
|
# offline that costs one retry backoff per candidate template file.
|
|
from core.inference.llama_cpp import _hf_offline_if_unreachable_for
|
|
|
|
def _read():
|
|
with _hf_offline_if_unreachable_for(model_name):
|
|
return read_default_chat_template(model_name, hf_token, gguf_variant)
|
|
|
|
template = await asyncio.to_thread(_read)
|
|
if template is not None and len(template.encode("utf-8")) > MAX_CHAT_TEMPLATE_BYTES:
|
|
template = None
|
|
return ModelTemplateResponse(model_name = model_name, chat_template = template)
|