4eabc74f34
Inference now runs in a persistent subprocess, solving the same transformers version-switching problem that was fixed for training. The subprocess stays alive between requests (model in GPU memory) and is only restarted when switching transformers versions. New files: - core/inference/worker.py: subprocess entry point with command loop - core/inference/orchestrator.py: parent-side proxy with same API Modified: - core/inference/__init__.py: exports orchestrator as default backend - routes/inference.py: removed in-process ensure_transformers_version() |
||
|---|---|---|
| .. | ||
| backend | ||
| frontend | ||
| tests | ||