CLAUDE.md — bigboy-alma-deploy

This file is read automatically at the start of every Claude Code session in this repo. It captures standing decisions so they don't need to be re-explained or re-litigated each session. If something here conflicts with a task doc you've been handed, the task doc wins for that session — but flag the conflict rather than silently picking one.

What this repo is

Deployment automation for BigBoy — a Ryzen 5 / RTX 5060 Ti (16GB) AI inference server, running AlmaLinux 10.2. BigBoy also serves as the reference/testbed machine for CE's future RHEL + llama.cpp client deployments — meaning decisions made here are expected to generalize, not just work once.

Source of truth for status

planning/STATUS.md is authoritative for what's actually done vs. pending. Read it before starting work. Update it when a phase's status changes — don't leave it stale.

Standing technical decisions (do not re-derive or second-guess these)

OS: AlmaLinux 10.2, moved from NixOS due to RTX 5060 Ti driver friction on Nix. RHEL-family chosen fleet-wide for AI-server workloads for compliance/audit reasons (EU AI Act relevance), not just this one box.
Inference engine: llama.cpp, built from source — deliberately not Ollama. Reasons: full control over CUDA build flags and quantization, built-in web UI via llama-server removes the need for a separate Open WebUI layer. Ollama also just wraps llama.cpp's ggml engine underneath on NVIDIA/Linux anyway, so switching back would add a layer of indirection without avoiding the dependency.
GPU target: -DCMAKE_CUDA_ARCHITECTURES=120 (Blackwell / sm_120). Requires NVIDIA driver ≥570. Do not treat any specific driver version number as a fixed target — an earlier planning estimate of 595.84 in group_vars/bigboy.yml was just that, an estimate. Phase 4 uses AlmaLinux's precompiled open-kmod path (dnf install almalinux-release-nvidia-driver then nvidia-driver-cuda nvidia-open-kmod), which installs whatever AlmaLinux's own NVIDIA driver repo currently ships — record the actual installed version (from nvidia-smi) back into group_vars/bigboy.yml once confirmed.
Known hazard: MXFP4-quantized models have open compilation issues on sm_120 as of mid-2026. Stick to standard GGUF quants — Q4_K_M or Q5_K_M — from established quantizers (Bartowski, Unsloth namespaces on Hugging Face). This sidesteps the issue entirely; it is not a performance preference, it's a build-stability one.
Version discipline: llama.cpp has no semantic versioning — continuous build-tagged releases only. Never build against master/HEAD. Every build pins a specific tag, recorded in group_vars/bigboy.yml.
Build flags: standard only — cmake -B build -DGGML_CUDA=ON -DCMAKE_BUILD_TYPE=Release -DCMAKE_CUDA_ARCHITECTURES=120. No exotic tuning flags (e.g. -DLLAMA_CUDA_MMV_Y) unless a specific, documented problem requires one. We are already deep in non-standard territory (Blackwell + Alma + sovereignty logging); the inference engine build itself should be as boring and reproducible as possible.
Serving: llama-server under systemd, bound to 127.0.0.1, fronted by nginx (reuse the existing CE reverse-proxy pattern from jahnet — don't invent a new one). --api-key enforced even on LAN-only. --cont-batching --parallel N for concurrent chat sessions.
SELinux: stays enforcing. If early bring-up needs a permissive discovery pass to collect AVC denials via ausearch/audit2allow, that's a temporary diagnostic state, not a resting state — flip back to enforcing before considering the phase done.

Style requirements for any script in this repo

Follow the CE OS Bash Style Guide (Updated_Bash_Style_Guide in the CE project knowledge) for every script:

Attribution header (dwarves first, then John A. Hoeven / Claude AI), licence (Unlicense), version, status
No set -e — every operation checked and logged explicitly
Cleanup trap registered before work begins
Single confirmation prompt before any system-modifying action
No silent failures — every error path logs and either hard-fails or warns
Never assume root; invoke sudo explicitly for privileged steps only

This repo is a single-purpose ops repo for one Alma server, not part of the CEOS multi-distro installer framework — so skip the ce_env.conf / pkg_* / CE_PRIV abstraction layer specifically. Use sudo directly. Everything else in the style guide applies.

Explicitly deferred — do not build unless a task doc asks for it

An auditable testing/release system for llama.cpp (versioned releases, btrfs snapshot safety net, promote/rollback via symlink swap, a Forgejo mirror that only ever receives BigBoy-validated tags so client deployments never pull raw upstream) was designed in detail on 2026-07-16, but is not yet built or integrated into the phase plan. Draft, untested scripts from that design may exist under scripts/draft/. Do not wire them into the active Ansible roles or treat them as representing current repo state — they're a reference for later, not a task in progress. Bringing BigBoy up and running is the current priority; this gets picked up afterward.

Per-client agent playbook pattern — not a fixed pipeline to build once and reuse. CE's own Ambrosiana deployment and coding-assistant setup serve as proven reference implementations (working examples of agent-role design, recon checks, ingestion orchestration), but each client deployment gets its own bespoke agent-role setup based on their actual use case — captured as a documented, repeatable Ansible playbook for reinstall recovery and potential future hive federation, not copied wholesale from CE's own internal setup. The Ansible-docs-as-RAG-corpus discussion from 2026-07-17 (Phi-4 Mini for license/scraping-permission/ malicious-content recon, Claude Code for chunking and ingestion orchestration, Qwen3 0.6B for style-guide formatting) is one example of the design thinking that goes into building a specific instance of this pattern — not a template to copy verbatim into a client deploy. Current priority for BigBoy specifically is the base AI Server Deployment: Phase 4 (NVIDIA driver) → Phase 5 (llama.cpp + Mistral + llama-server's built-in web UI, chat confirmed working). No agent roles, no RAG corpus, no ingestion pipeline get built at this stage — that's all downstream of an actual use case being decided, which hasn't happened yet.

Scope discipline

Task docs will name a specific phase or task from STATUS.md. Do the named task. Don't refactor, "improve," or extend adjacent phases that are already marked complete or reviewed, even if something nearby looks improvable — flag it instead and let it be a deliberate decision, not a side effect.

8 KiB Raw Blame History