unsloth/studio/backend/core/inference
Daniel Han d7a78599e6 studio: optimize llama.cpp build with static linking, ccache, ninja
Benchmarked BUILD_SHARED_LIBS ON vs OFF:
- Build time identical (~55s both on 192-core B200)
- Static: 75MB self-contained binary, no LD_LIBRARY_PATH needed
- Shared: 7.4MB + .so deps, needs lib path management
- Both work correctly

Optimizations applied to LlamaCppBuilder:
- BUILD_SHARED_LIBS=OFF: static binary, simpler deployment
- ccache: 27x faster rebuilds (55s -> 2s with warm cache)
- Ninja generator: parallel builds
- CMAKE_CUDA_ARCHITECTURES: build only for detected GPU arch
- CMAKE_CUDA_FLAGS=--threads=0: multi-threaded CUDA compilation
- GGML_NATIVE=ON: native CPU optimizations
- LLAMA_BUILD_TESTS=OFF, LLAMA_BUILD_EXAMPLES=OFF: skip unused targets
- LLAMA_BUILD_SERVER=ON: build only what we need
2026-03-16 14:15:14 +00:00
..
__init__.py Final cleanup 2026-03-12 18:28:04 +00:00
audio_codecs.py studio: per-model inference defaults, GGUF slider fix, reasoning toggle (#4325) 2026-03-16 06:37:55 -07:00
defaults.py chat only with gguf for mac devices (#4300) 2026-03-15 23:20:48 +04:00
inference.py studio: per-model inference defaults, GGUF slider fix, reasoning toggle (#4325) 2026-03-16 06:37:55 -07:00
llama_cpp.py studio: per-model inference defaults, GGUF slider fix, reasoning toggle (#4325) 2026-03-16 06:37:55 -07:00
llama_cpp_builder.py studio: optimize llama.cpp build with static linking, ccache, ninja 2026-03-16 14:15:14 +00:00
orchestrator.py [pre-commit.ci] auto fixes from pre-commit.com hooks 2026-03-16 02:46:56 -07:00
worker.py Set repetition_penalty default to 1.0 (disabled) everywhere 2026-03-16 02:46:56 -07:00