# AI Services on AlmaLinux — Ollama, Open WebUI, Cockpit for BigBoy

**Reference for installing and configuring inference/management services on AlmaLinux 10.2**

---

## Service Architecture

| Service | Port | Purpose | Runs As |
|---------|------|---------|---------|
| **Ollama** | 11434 | Local inference engine (localhost only) | ollama (system user) |
| **Open WebUI** | 8080 | Web interface for Ollama (LAN-facing) | open-webui (system user) |
| **Cockpit** | 9090 | System management dashboard (LAN-facing) | cockpit (systemd managed) |

---

## Installation

### Ollama

**Install from official repository:**
```bash
curl -fsSL https://ollama.ai/install.sh | sh
```

This:
- Downloads and installs the ollama binary
- Creates `ollama` system user
- Installs systemd service unit
- Enables and starts the service

**Verify installation:**
```bash
ollama --version
systemctl status ollama
```

**Pull a model (after GPU driver is confirmed working):**
```bash
ollama pull mistral                # Mistral Small 3.1 7B (smallest, fastest)
# Or for larger model:
ollama pull mistral:24b            # 24B variant if VRAM available
```

---

### Open WebUI

**Install from package (if available in repos):**
```bash
sudo dnf install -y open-webui
```

**If not in repos, install via Python/pip (alternative):**
```bash
sudo dnf install -y python3 python3-pip
pip install --user open-webui
```

**Or: Run as a container (preferred for isolation):**
```bash
sudo dnf install -y podman
podman run -d --name open-webui \
  -p 8080:8080 \
  -v open-webui:/app/backend/data \
  ghcr.io/open-webui/open-webui:latest
```

**Verify it's running:**
```bash
systemctl status open-webui
# Or if containerized:
podman ps | grep open-webui
```

**Test connection:**
```bash
curl http://localhost:8080
# Should return HTML (the Open WebUI frontend)
```

---

### Cockpit

**Install:**
```bash
sudo dnf install -y cockpit cockpit-podman cockpit-pcp
```

**Enable and start:**
```bash
sudo systemctl enable cockpit.socket
sudo systemctl start cockpit.socket
```

**Verify it's listening:**
```bash
sudo ss -tlnp | grep 9090
# Expected: tcp LISTEN ... :9090 ... cockpit
```

**Access via browser (from another machine on LAN):**
```
https://192.168.0.240:9090
# Login as john (with sudo privileges)
```

---

## Configuration

### Ollama — Environment Variables

**Location:** `/etc/default/ollama` (create if doesn't exist)

```bash
sudo cat > /etc/default/ollama <<'EOF'
# Ollama configuration for BigBoy

# GPU/CUDA settings
CUDA_VISIBLE_DEVICES=0           # Use GPU 0 (the RTX 5060 Ti)
OLLAMA_MAX_LOADED_MODELS=1       # Never load multiple models into VRAM simultaneously
OLLAMA_KEEP_ALIVE=5m             # Unload model after 5 minutes of inactivity

# Memory management
OLLAMA_MAX_QUEUE=4               # Queue up to 4 requests, don't reject
OLLAMA_FLASH_ATTENTION=1         # Use flash attention (lower peak VRAM)
OLLAMA_GPU_OVERHEAD=536870912    # Reserve 512MB explicitly for GPU overhead

# Listening
OLLAMA_HOST=127.0.0.1:11434      # Localhost only (Open WebUI is the LAN interface)

# Logging (optional)
OLLAMA_DEBUG=0                   # Set to 1 for verbose debug logs
EOF
```

**Reload after editing:**
```bash
sudo systemctl daemon-reload
sudo systemctl restart ollama
```

---

### Open WebUI — Configuration

**Connect to local Ollama:**
```
Open WebUI web interface → Settings → Backend
  OLLAMA_API_BASE_URL: http://localhost:11434
```

**Optional: Persist configuration in environment:**
```bash
sudo cat >> /etc/default/open-webui <<'EOF'
OLLAMA_BASE_URL=http://127.0.0.1:11434
EOF
```

---

### Cockpit — HTTPS Certificate

**Cockpit requires HTTPS. Certificate is auto-generated on first start:**
```bash
ls -la /etc/cockpit/ws-certs.d/
# Should see auto-generated certificate
```

**On first connection, browser will warn about self-signed cert — accept it.**

**Optional: Use a real certificate (not needed for internal use):**
```bash
# Place your certificate and key in /etc/cockpit/ws-certs.d/
sudo cp your-cert.crt /etc/cockpit/ws-certs.d/
sudo cp your-key.key /etc/cockpit/ws-certs.d/
sudo systemctl restart cockpit
```

---

## Service Management

### Check status of all three:
```bash
systemctl status ollama
systemctl status open-webui      # Or podman ps if containerized
systemctl status cockpit
```

### Start/stop/restart:
```bash
sudo systemctl start ollama
sudo systemctl stop ollama
sudo systemctl restart ollama

# Same for open-webui and cockpit
```

### Enable on boot:
```bash
sudo systemctl enable ollama
sudo systemctl enable open-webui
sudo systemctl enable cockpit
```

### View logs:
```bash
journalctl -u ollama -f           # Follow Ollama logs
journalctl -u open-webui -f
journalctl -u cockpit -f
```

---

## Verification Checklist

**Run after all three services are installed and started:**

1. **Ollama is running and accessible:**
   ```bash
   curl http://localhost:11434/api/version
   # Expected: {"version": "x.x.x"}
   ```

2. **GPU is recognized by Ollama:**
   ```bash
   ollama list
   # Should show any pulled models
   ```

3. **Open WebUI can connect to Ollama:**
   ```bash
   curl http://localhost:8080
   # Should return HTML (not a connection error)
   ```

4. **Cockpit is listening on 9090:**
   ```bash
   sudo ss -tlnp | grep 9090
   ```

5. **All ports are firewalled correctly** (see `alma-firewall.md`):
   ```bash
   sudo firewall-cmd --list-ports
   # Should show 8080/tcp, 9090/tcp (11434 NOT exposed to LAN)
   ```

---

## Troubleshooting

**Ollama won't start:**
- Check NVIDIA driver is loaded: `nvidia-smi`
- Check CUDA environment: `ollama --version` and `nvidia-smi`
- View error logs: `journalctl -u ollama -e`

**Open WebUI can't connect to Ollama:**
- Verify Ollama is listening: `curl http://localhost:11434/api/version`
- Check Open WebUI logs: `journalctl -u open-webui -e`
- Verify firewall isn't blocking localhost (it shouldn't): `sudo firewall-cmd --list-all`

**Cockpit won't open in browser:**
- Verify it's listening: `sudo ss -tlnp | grep 9090`
- Try `https://` not `http://` (HTTPS required)
- Check firewall allows 9090 from your client IP

**GPU memory exhaustion:**
- Set `OLLAMA_MAX_LOADED_MODELS=1` to avoid multiple models in VRAM
- Reduce `OLLAMA_KEEP_ALIVE` to unload faster
- Monitor GPU memory: `nvidia-smi` or `watch -n 1 nvidia-smi`

---

## Next Steps

1. Verify all three services are running (see Verification Checklist)
2. Configure firewall rules (see `alma-firewall.md`)
3. Test inference: `ollama pull mistral && ollama run mistral "Hello"`
4. Access Open WebUI from another machine: `http://192.168.0.240:8080`
5. Log into Cockpit: `https://192.168.0.240:9090`