Updates pre-deploy
This commit is contained in:
parent
432100a45a
commit
71dd3c9e23
GPG key ID:
306E507219506D4E
5 changed files with 121 additions and 114 deletions
171
CLAUDE.md
171
CLAUDE.md
|
|
@ -1,96 +1,103 @@
|
|||
# CLAUDE.md — bigboy-alma-deploy
|
||||
# Task: Phase 4 + 5 bring-up — NVIDIA driver and llama.cpp
|
||||
|
||||
This file is read automatically at the start of every Claude Code session in
|
||||
this repo. It captures standing decisions so they don't need to be
|
||||
re-explained or re-litigated each session. If something here conflicts with
|
||||
a task doc you've been handed, the task doc wins for that session — but
|
||||
flag the conflict rather than silently picking one.
|
||||
## Scope
|
||||
|
||||
## What this repo is
|
||||
Bring BigBoy from its current state (Phase 3 complete — filesystems mounted)
|
||||
through Phase 4 (NVIDIA driver) and Phase 5 (llama.cpp built and serving),
|
||||
per `planning/STATUS.md`. This is a bring-up task, not the auditable
|
||||
release-pipeline work — see `CLAUDE.md` for why that's explicitly out of
|
||||
scope right now.
|
||||
|
||||
Deployment automation for BigBoy — a Ryzen 5 / RTX 5060 Ti (16GB) AI
|
||||
inference server, running AlmaLinux 10.2. BigBoy also serves as the
|
||||
reference/testbed machine for CE's future RHEL + llama.cpp client
|
||||
deployments — meaning decisions made here are expected to generalize, not
|
||||
just work once.
|
||||
## Before starting
|
||||
|
||||
## Source of truth for status
|
||||
- Read `CLAUDE.md` at repo root if you haven't already this session.
|
||||
- Read `planning/STATUS.md` for the current authoritative state of every
|
||||
phase — confirm Phase 3 is actually complete before proceeding.
|
||||
- Read `group_vars/bigboy.yml` for hardware facts (drive UUIDs, target
|
||||
driver version, GPU architecture) — don't re-derive these.
|
||||
|
||||
`plannng/STATUS.md` is authoritative for what's actually done vs. pending.
|
||||
Read it before starting work. Update it when a phase's status changes —
|
||||
don't leave it stale.
|
||||
## Phase 4 — NVIDIA driver
|
||||
|
||||
## Standing technical decisions (do not re-derive or second-guess these)
|
||||
Use AlmaLinux's **precompiled open kernel module** path, not manual DKMS.
|
||||
Confirmed via NVIDIA's own AlmaLinux installation guide (2026-07-17): this
|
||||
is the AlmaLinux-recommended method, works regardless of Secure Boot
|
||||
state, and needs none of the kernel-devel/kernel-headers/DKMS/GCC
|
||||
compilation machinery the kickstart's package list was originally
|
||||
written assuming. "Open" (not proprietary/closed) kernel modules are the
|
||||
right choice on Blackwell-generation cards like the RTX 5060 Ti
|
||||
regardless of precompiled-vs-DKMS.
|
||||
|
||||
- **OS**: AlmaLinux 10.2, moved from NixOS due to RTX 5060 Ti driver
|
||||
friction on Nix. RHEL-family chosen fleet-wide for AI-server workloads
|
||||
for compliance/audit reasons (EU AI Act relevance), not just this one box.
|
||||
- **Inference engine**: llama.cpp, built from source — deliberately not
|
||||
Ollama. Reasons: full control over CUDA build flags and quantization,
|
||||
built-in web UI via `llama-server` removes the need for a separate
|
||||
Open WebUI layer. Ollama also just wraps llama.cpp's ggml engine
|
||||
underneath on NVIDIA/Linux anyway, so switching back would add a layer
|
||||
of indirection without avoiding the dependency.
|
||||
- **GPU target**: `-DCMAKE_CUDA_ARCHITECTURES=120` (Blackwell / sm_120).
|
||||
Requires NVIDIA driver ≥570; target driver is `595.84` (see
|
||||
`group_vars/bigboy.yml`).
|
||||
- **Known hazard**: MXFP4-quantized models have open compilation issues on
|
||||
sm_120 as of mid-2026. Stick to standard GGUF quants — Q4_K_M or Q5_K_M —
|
||||
from established quantizers (Bartowski, Unsloth namespaces on Hugging
|
||||
Face). This sidesteps the issue entirely; it is not a performance
|
||||
preference, it's a build-stability one.
|
||||
- **Version discipline**: llama.cpp has no semantic versioning — continuous
|
||||
build-tagged releases only. Never build against `master`/HEAD. Every
|
||||
build pins a specific tag, recorded in `group_vars/bigboy.yml`.
|
||||
- **Build flags**: standard only — `cmake -B build -DGGML_CUDA=ON
|
||||
-DCMAKE_BUILD_TYPE=Release -DCMAKE_CUDA_ARCHITECTURES=120`. No exotic
|
||||
tuning flags (e.g. `-DLLAMA_CUDA_MMV_Y`) unless a specific, documented
|
||||
problem requires one. We are already deep in non-standard territory
|
||||
(Blackwell + Alma + sovereignty logging); the inference engine build
|
||||
itself should be as boring and reproducible as possible.
|
||||
- **Serving**: `llama-server` under systemd, bound to `127.0.0.1`, fronted
|
||||
by nginx (reuse the existing CE reverse-proxy pattern from jahnet — don't
|
||||
invent a new one). `--api-key` enforced even on LAN-only. `--cont-batching
|
||||
--parallel N` for concurrent chat sessions.
|
||||
- **SELinux**: stays enforcing. If early bring-up needs a permissive
|
||||
discovery pass to collect AVC denials via `ausearch`/`audit2allow`,
|
||||
that's a temporary diagnostic state, not a resting state — flip back to
|
||||
enforcing before considering the phase done.
|
||||
```bash
|
||||
# Enables the AlmaLinux NVIDIA driver repo + CRB + NVIDIA CUDA repo +
|
||||
# EPEL in one step
|
||||
sudo dnf install almalinux-release-nvidia-driver
|
||||
|
||||
## Style requirements for any script in this repo
|
||||
# Compute-only / headless — no desktop GL/X components, matches this
|
||||
# hardware's actual role
|
||||
sudo dnf install nvidia-driver-cuda nvidia-open-kmod
|
||||
|
||||
Follow the CE OS Bash Style Guide (`Updated_Bash_Style_Guide` in the CE
|
||||
project knowledge) for every script:
|
||||
- Attribution header (dwarves first, then John A. Hoeven / Claude AI),
|
||||
licence (Unlicense), version, status
|
||||
- No `set -e` — every operation checked and logged explicitly
|
||||
- Cleanup trap registered before work begins
|
||||
- Single confirmation prompt before any system-modifying action
|
||||
- No silent failures — every error path logs and either hard-fails or warns
|
||||
- Never assume root; invoke `sudo` explicitly for privileged steps only
|
||||
sudo reboot
|
||||
```
|
||||
|
||||
This repo is a single-purpose ops repo for one Alma server, not part of the
|
||||
CEOS multi-distro installer framework — so skip the `ce_env.conf` /
|
||||
`pkg_*` / `CE_PRIV` abstraction layer specifically. Use `sudo` directly.
|
||||
Everything else in the style guide applies.
|
||||
- Confirm `nouveau` is blacklisted (should already be handled by the
|
||||
kickstart — verify, don't assume).
|
||||
- **Do not treat `group_vars/bigboy.yml`'s recorded `595.84` as a version
|
||||
to install for** — that was an early planning estimate. The precompiled
|
||||
path installs whatever AlmaLinux's own NVIDIA driver repo currently
|
||||
ships. Record the *actual* installed version (from `nvidia-smi`'s
|
||||
output) back into `group_vars/bigboy.yml` once confirmed, replacing the
|
||||
placeholder.
|
||||
- `gcc` from the kickstart package list is still needed — not for the
|
||||
driver anymore, but for compiling llama.cpp against CUDA in Phase 5.
|
||||
No change needed there.
|
||||
- **Done when**: `nvidia-smi` runs cleanly and reports the RTX 5060 Ti.
|
||||
|
||||
## Explicitly deferred — do not build unless a task doc asks for it
|
||||
## Phase 5 — llama.cpp
|
||||
|
||||
An auditable testing/release system for llama.cpp (versioned releases,
|
||||
btrfs snapshot safety net, promote/rollback via symlink swap, a Forgejo
|
||||
mirror that only ever receives BigBoy-validated tags so client
|
||||
deployments never pull raw upstream) was designed in detail on
|
||||
2026-07-16, but is **not yet built or integrated into the phase plan**.
|
||||
Draft, untested scripts from that design may exist under `scripts/draft/`.
|
||||
Do not wire them into the active Ansible roles or treat them as
|
||||
representing current repo state — they're a reference for later, not a
|
||||
task in progress. Bringing BigBoy up and running is the current priority;
|
||||
this gets picked up afterward.
|
||||
1. **Pick a pinned build tag.** Check
|
||||
`https://github.com/ggml-org/llama.cpp/releases` (or `git ls-remote
|
||||
--tags`) for a recent, stable-looking tag — do not build against
|
||||
`master`. Record the chosen tag in `group_vars/bigboy.yml` once decided.
|
||||
2. **Shallow clone at that tag**, build with the standard flags from
|
||||
`CLAUDE.md` (`-DGGML_CUDA=ON -DCMAKE_BUILD_TYPE=Release
|
||||
-DCMAKE_CUDA_ARCHITECTURES=120`).
|
||||
3. **Model**: pull a Q4_K_M or Q5_K_M GGUF from Bartowski or Unsloth on
|
||||
Hugging Face for initial testing — a 7B-class instruct model is
|
||||
sufficient to validate the pipeline end to end. Do not use an
|
||||
MXFP4-quantized model (see `CLAUDE.md` — known sm_120 build hazard).
|
||||
4. **Serve** via `llama-server` under systemd:
|
||||
- Bind `127.0.0.1`, not `0.0.0.0`
|
||||
- `--api-key` set (placeholder value is fine for initial bring-up,
|
||||
but note in the PR/commit that it needs a real value before any
|
||||
external exposure)
|
||||
- `--cont-batching --parallel 4` (or a reasonable default — this isn't
|
||||
the tuning pass, just needs to not be single-request-only)
|
||||
- nginx reverse proxy in front, following the existing pattern already
|
||||
used for Nextcloud/Forgejo on jahnet — don't invent a new nginx
|
||||
pattern for this
|
||||
5. **Firewall**: only the reverse-proxy port needs opening; the
|
||||
`llama-server` port itself stays localhost-only.
|
||||
|
||||
## Scope discipline
|
||||
## Done criteria for this task
|
||||
|
||||
Task docs will name a specific phase or task from `STATUS.md`. Do the
|
||||
named task. Don't refactor, "improve," or extend adjacent phases that are
|
||||
already marked complete or reviewed, even if something nearby looks
|
||||
improvable — flag it instead and let it be a deliberate decision, not a
|
||||
side effect.
|
||||
- `nvidia-smi` confirms GPU visible and driver loaded
|
||||
- `systemctl status llama-server` shows active/running
|
||||
- `curl http://127.0.0.1:<port>/health` (or equivalent) returns healthy
|
||||
- One real inference request through the API returns a coherent response
|
||||
- The chosen build tag is recorded in `group_vars/bigboy.yml`
|
||||
- `planning/STATUS.md` updated to reflect Phase 4 and 5 as complete
|
||||
|
||||
## Explicitly out of scope for this task
|
||||
|
||||
- Do not build the update/promote/prune/publish scripts referenced in
|
||||
`CLAUDE.md` — that's deferred work, not this task.
|
||||
- Do not build Open WebUI (Phase 8) — `llama-server`'s built-in UI covers
|
||||
the chat-first use case; Phase 8 is likely to be dropped entirely, but
|
||||
that's a separate decision, not part of this task.
|
||||
- Do not modify the Phase 3 filesystem/mount layout — it's already
|
||||
reviewed and complete. If something about it seems to be causing a
|
||||
problem in Phase 4/5, flag it rather than changing it directly.
|
||||
- Do not touch SELinux policy beyond what's needed to get services
|
||||
running (permissive discovery pass if genuinely needed, per
|
||||
`CLAUDE.md` — but this should be a temporary diagnostic step, not a
|
||||
final state, and not a deep dive into custom policy authoring).
|
||||
24
README.md
24
README.md
|
|
@ -6,10 +6,10 @@
|
|||
|
||||
## Quick Links
|
||||
|
||||
- **Architecture & Design:** `./plannng/ARCHITECTURE.md`
|
||||
- **Deployment Status:** `./plannng/STATUS.md`
|
||||
- **Hardware Manifest:** `./plannng/HARDWARE.md`
|
||||
- **Sovereignty Policy:** `./plannng/SOVEREIGNTY-POLICY.md`
|
||||
- **Architecture & Design:** `./planning/ARCHITECTURE.md`
|
||||
- **Deployment Status:** `./planning/STATUS.md`
|
||||
- **Hardware Manifest:** `./planning/HARDWARE.md`
|
||||
- **Sovereignty Policy:** `./planning/SOVEREIGNTY-POLICY.md`
|
||||
- **Claude Code standing context:** `./CLAUDE.md`
|
||||
- **Current task handoff:** `./tasks/`
|
||||
|
||||
|
|
@ -66,7 +66,7 @@ holds only an abandoned NixOS install attempt. Nothing described as
|
|||
"complete" below has been validated on real hardware yet — it means
|
||||
*designed, written, and reviewed*, not *running*.
|
||||
|
||||
See `./plannng/STATUS.md` for the authoritative, current per-phase state.
|
||||
See `./planning/STATUS.md` for the authoritative, current per-phase state.
|
||||
Don't rely on this README for phase-by-phase status — it will drift out
|
||||
of date faster than `STATUS.md` is maintained.
|
||||
|
||||
|
|
@ -77,7 +77,7 @@ of date faster than `STATUS.md` is maintained.
|
|||
- Ansible main playbook scaffold (`ansible/site.yml`, 14-phase
|
||||
orchestration)
|
||||
- Phase 3 role (filesystem mounting) — written and reviewed
|
||||
- Hardware inventory & UUID manifest (`plannng/HARDWARE.md`)
|
||||
- Hardware inventory & UUID manifest (`planning/HARDWARE.md`)
|
||||
|
||||
### Ready to build, once Phase 1–3 actually run on hardware
|
||||
|
||||
|
|
@ -130,7 +130,7 @@ of date faster than `STATUS.md` is maintained.
|
|||
bigboy-alma-deploy/
|
||||
├── CLAUDE.md # standing context, read by Claude Code every session
|
||||
├── README.md # this file
|
||||
├── alma10-minimal-bigboy.ks # canonical kickstart — this copy, not plannng/'s
|
||||
├── alma10-minimal-bigboy.ks # canonical kickstart — this copy, not planning/'s
|
||||
├── install-phases.txt # 15-phase (0-14) plan, high level
|
||||
├── ansible/
|
||||
│ ├── site.yml # main playbook
|
||||
|
|
@ -138,7 +138,7 @@ bigboy-alma-deploy/
|
|||
│ ├── group_vars/bigboy.yml # hardware facts: UUIDs, driver version, pinned llama.cpp tag
|
||||
│ └── roles/
|
||||
│ └── phase-3-filesystems/ # written, reviewed, not yet run on hardware
|
||||
├── plannng/ # (sic — established directory name, not a typo to fix casually)
|
||||
├── planning/ # (sic — established directory name, not a typo to fix casually)
|
||||
│ ├── ARCHITECTURE.md
|
||||
│ ├── HARDWARE.md
|
||||
│ ├── SOVEREIGNTY-POLICY.md
|
||||
|
|
@ -156,12 +156,12 @@ bigboy-alma-deploy/
|
|||
## Getting Started
|
||||
|
||||
### To review design
|
||||
1. Read `./plannng/ARCHITECTURE.md` — design rationale, 14-phase breakdown
|
||||
2. Read `./plannng/HARDWARE.md` — component specs, UUIDs, network config
|
||||
3. Read `./plannng/STATUS.md` — the real current state, phase by phase
|
||||
1. Read `./planning/ARCHITECTURE.md` — design rationale, 14-phase breakdown
|
||||
2. Read `./planning/HARDWARE.md` — component specs, UUIDs, network config
|
||||
3. Read `./planning/STATUS.md` — the real current state, phase by phase
|
||||
|
||||
### To pick up work
|
||||
1. Check `./plannng/STATUS.md` for what's actually next
|
||||
1. Check `./planning/STATUS.md` for what's actually next
|
||||
2. Look in `./tasks/` for an existing scoped task doc covering it
|
||||
3. If none exists, Claude Desktop drafts one before Claude Code starts —
|
||||
don't hand Claude Code a phase without a task doc scoping it
|
||||
|
|
|
|||
|
|
@ -7,7 +7,7 @@
|
|||
# USB/OEMDRV: label a second USB "OEMDRV", place this file at its root as ks.cfg
|
||||
# HTTP: at boot, edit kernel line: inst.ks=http://<workbench_ip>:8000/alma10-minimal-bigboy.ks
|
||||
#
|
||||
# Reference: plannng/alma-phase1-install-workflow.md
|
||||
# Reference: planning/alma-phase1-install-workflow.md
|
||||
#
|
||||
# BEFORE USE — fill in:
|
||||
# BEFORE USE — fill in:
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@
|
|||
successfully. Phase 2 (Ansible) not yet run.
|
||||
|
||||
This is a working log of decisions made and why — not a step-by-step
|
||||
runbook. For the actual procedures, see `plannng/` and the kickstart file
|
||||
runbook. For the actual procedures, see `planning/` and the kickstart file
|
||||
itself. Nothing in this document is sensitive; no hashes, passwords, or
|
||||
credentials appear here even in redacted form.
|
||||
|
||||
|
|
@ -21,7 +21,7 @@ It had passed through a Forgejo migration and a rename (`bigboy-alma` →
|
|||
outside it in an untracked parent directory. Diffed identical; the
|
||||
untracked copy was archived rather than deleted outright, in case
|
||||
anything in its history mattered later.
|
||||
- **Two copies of the kickstart file** (repo root and `plannng/`) —
|
||||
- **Two copies of the kickstart file** (repo root and `planning/`) —
|
||||
diffed down to a single trailing-newline difference. Root copy kept as
|
||||
canonical.
|
||||
- **A stale `README.md`** — the version in the live repo had been reduced
|
||||
|
|
|
|||
|
|
@ -4,14 +4,14 @@
|
|||
|
||||
Bring BigBoy from its current state (Phase 3 complete — filesystems mounted)
|
||||
through Phase 4 (NVIDIA driver) and Phase 5 (llama.cpp built and serving),
|
||||
per `plannng/STATUS.md`. This is a bring-up task, not the auditable
|
||||
per `planning/STATUS.md`. This is a bring-up task, not the auditable
|
||||
release-pipeline work — see `CLAUDE.md` for why that's explicitly out of
|
||||
scope right now.
|
||||
|
||||
## Before starting
|
||||
|
||||
- Read `CLAUDE.md` at repo root if you haven't already this session.
|
||||
- Read `plannng/STATUS.md` for the current authoritative state of every
|
||||
- Read `planning/STATUS.md` for the current authoritative state of every
|
||||
phase — confirm Phase 3 is actually complete before proceeding.
|
||||
- Read `group_vars/bigboy.yml` for hardware facts (drive UUIDs, target
|
||||
driver version, GPU architecture) — don't re-derive these.
|
||||
|
|
@ -85,7 +85,7 @@ sudo reboot
|
|||
- `curl http://127.0.0.1:<port>/health` (or equivalent) returns healthy
|
||||
- One real inference request through the API returns a coherent response
|
||||
- The chosen build tag is recorded in `group_vars/bigboy.yml`
|
||||
- `plannng/STATUS.md` updated to reflect Phase 4 and 5 as complete
|
||||
- `planning/STATUS.md` updated to reflect Phase 4 and 5 as complete
|
||||
|
||||
## Explicitly out of scope for this task
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue