bigboy-alma-deploy/CLAUDE.md

# CLAUDE.md — bigboy-alma-deploy

This file is read automatically at the start of every Claude Code session in
this repo. It captures standing decisions so they don't need to be
re-explained or re-litigated each session. If something here conflicts with
a task doc you've been handed, the task doc wins for that session — but
flag the conflict rather than silently picking one.

## What this repo is

Deployment automation for BigBoy — a Ryzen 5 / RTX 5060 Ti (16GB) AI
inference server, running AlmaLinux 10.2. BigBoy also serves as the
reference/testbed machine for CE's future RHEL + llama.cpp client
deployments — meaning decisions made here are expected to generalize, not
just work once.

## Source of truth for status

`planning/STATUS.md` is authoritative for what's actually done vs. pending.
Read it before starting work. Update it when a phase's status changes —
don't leave it stale.

## Standing technical decisions (do not re-derive or second-guess these)

- **OS**: AlmaLinux 10.2, moved from NixOS due to RTX 5060 Ti driver
  friction on Nix. RHEL-family chosen fleet-wide for AI-server workloads
  for compliance/audit reasons (EU AI Act relevance), not just this one box.
- **Inference engine**: llama.cpp, built from source — deliberately not
  Ollama. Reasons: full control over CUDA build flags and quantization,
  built-in web UI via `llama-server` removes the need for a separate
  Open WebUI layer. Ollama also just wraps llama.cpp's ggml engine
  underneath on NVIDIA/Linux anyway, so switching back would add a layer
  of indirection without avoiding the dependency.
- **GPU target**: `-DCMAKE_CUDA_ARCHITECTURES=120` (Blackwell / sm_120).
  Requires NVIDIA driver ≥570. **Do not treat any specific driver version
  number as a fixed target** — an earlier planning estimate of `595.84`
  in `group_vars/bigboy.yml` was just that, an estimate. Phase 4 uses
  AlmaLinux's precompiled open-kmod path (`dnf install
  almalinux-release-nvidia-driver` then `nvidia-driver-cuda
  nvidia-open-kmod`), which installs whatever AlmaLinux's own NVIDIA
  driver repo currently ships — record the *actual* installed version
  (from `nvidia-smi`) back into `group_vars/bigboy.yml` once confirmed.
- **Known hazard**: MXFP4-quantized models have open compilation issues on
  sm_120 as of mid-2026. Stick to standard GGUF quants — Q4_K_M or Q5_K_M —
  from established quantizers (Bartowski, Unsloth namespaces on Hugging
  Face). This sidesteps the issue entirely; it is not a performance
  preference, it's a build-stability one.
- **Version discipline**: llama.cpp has no semantic versioning — continuous
  build-tagged releases only. Never build against `master`/HEAD. Every
  build pins a specific tag, recorded in `group_vars/bigboy.yml`.
- **Build flags**: standard only — `cmake -B build -DGGML_CUDA=ON
  -DCMAKE_BUILD_TYPE=Release -DCMAKE_CUDA_ARCHITECTURES=120`. No exotic
  tuning flags (e.g. `-DLLAMA_CUDA_MMV_Y`) unless a specific, documented
  problem requires one. We are already deep in non-standard territory
  (Blackwell + Alma + sovereignty logging); the inference engine build
  itself should be as boring and reproducible as possible.
- **Serving**: `llama-server` under systemd, bound to `127.0.0.1`, fronted
  by nginx (reuse the existing CE reverse-proxy pattern from jahnet — don't
  invent a new one). `--api-key` enforced even on LAN-only. `--cont-batching
  --parallel N` for concurrent chat sessions.
- **SELinux**: stays enforcing. If early bring-up needs a permissive
  discovery pass to collect AVC denials via `ausearch`/`audit2allow`,
  that's a temporary diagnostic state, not a resting state — flip back to
  enforcing before considering the phase done.

## Style requirements for any script in this repo

Follow the CE OS Bash Style Guide (`Updated_Bash_Style_Guide` in the CE
project knowledge) for every script:
- Attribution header (dwarves first, then John A. Hoeven / Claude AI),
  licence (Unlicense), version, status
- No `set -e` — every operation checked and logged explicitly
- Cleanup trap registered before work begins
- Single confirmation prompt before any system-modifying action
- No silent failures — every error path logs and either hard-fails or warns
- Never assume root; invoke `sudo` explicitly for privileged steps only

This repo is a single-purpose ops repo for one Alma server, not part of the
CEOS multi-distro installer framework — so skip the `ce_env.conf` /
`pkg_*` / `CE_PRIV` abstraction layer specifically. Use `sudo` directly.
Everything else in the style guide applies.

## Explicitly deferred — do not build unless a task doc asks for it

An auditable testing/release system for llama.cpp (versioned releases,
btrfs snapshot safety net, promote/rollback via symlink swap, a Forgejo
mirror that only ever receives BigBoy-validated tags so client
deployments never pull raw upstream) was designed in detail on
2026-07-16, but is **not yet built or integrated into the phase plan**.
Draft, untested scripts from that design may exist under `scripts/draft/`.
Do not wire them into the active Ansible roles or treat them as
representing current repo state — they're a reference for later, not a
task in progress. Bringing BigBoy up and running is the current priority;
this gets picked up afterward.

**Per-client agent playbook pattern** — not a fixed pipeline to build
once and reuse. CE's own Ambrosiana deployment and coding-assistant setup
serve as proven reference implementations (working examples of agent-role
design, recon checks, ingestion orchestration), but each client
deployment gets its own bespoke agent-role setup based on their actual
use case — captured as a documented, repeatable Ansible playbook for
reinstall recovery and potential future hive federation, not copied
wholesale from CE's own internal setup. The Ansible-docs-as-RAG-corpus
discussion from 2026-07-17 (Phi-4 Mini for license/scraping-permission/
malicious-content recon, Claude Code for chunking and ingestion
orchestration, Qwen3 0.6B for style-guide formatting) is one example of
the design thinking that goes into building a specific instance of this
pattern — not a template to copy verbatim into a client deploy. **Current
priority for BigBoy specifically is the base AI Server Deployment**:
Phase 4 (NVIDIA driver) → Phase 5 (llama.cpp + Mistral + `llama-server`'s
built-in web UI, chat confirmed working). No agent roles, no RAG corpus,
no ingestion pipeline get built at this stage — that's all downstream of
an actual use case being decided, which hasn't happened yet.

## Scope discipline

Task docs will name a specific phase or task from `STATUS.md`. Do the
named task. Don't refactor, "improve," or extend adjacent phases that are
already marked complete or reviewed, even if something nearby looks
improvable — flag it instead and let it be a deliberate decision, not a
side effect.