Add man pages styleguide ai and reorganize directory structure
This commit is contained in:
parent
b1796a3270
commit
ff95bab903
2 changed files with 218 additions and 0 deletions
218
man-pages-styleguide/man-pages-styleguide-ai.md
Normal file
218
man-pages-styleguide/man-pages-styleguide-ai.md
Normal file
|
|
@ -0,0 +1,218 @@
|
|||
# CE Man Page Reformatting Styleguide — AI Corpus (v1.0)
|
||||
|
||||
## 0. Purpose and scope
|
||||
|
||||
Governs reformatting of individual man pages (Sections 1–8) into the
|
||||
CE RAG corpus for AI consumption: microSLM retrieval (Qwen 0.6B) and
|
||||
frontier reasoning over retrieval (Claude).
|
||||
|
||||
Out of scope: enthusiast-tutorial transformation of man pages. Separate
|
||||
future project, separate voice rules.
|
||||
|
||||
Sibling document (not yet authored): OHIOD-readable variant. Same rule
|
||||
body (Sections 1–8 below), Section 9 (Voice) replaced only. Authored
|
||||
after this version passes beta testing.
|
||||
|
||||
---
|
||||
|
||||
## 1. File boundary
|
||||
|
||||
One source document = one file.
|
||||
|
||||
| Source type | File unit |
|
||||
|---|---|
|
||||
| Individual man page | One page = one file |
|
||||
| Multi-chapter book (ProGit, Tao of Tmux) | One chapter = one file |
|
||||
|
||||
Rationale: file boundary = smallest unit upstream treats as independently
|
||||
coherent and independently revised.
|
||||
|
||||
Naming: `[source]-[identifier]-rag.md`
|
||||
Examples: `man-pages-conventions-v1-rag.md`, `progit-v2-ch03-branching-in-git-rag.md`
|
||||
|
||||
---
|
||||
|
||||
## 2. Chunk mechanism
|
||||
|
||||
One file, many chunks. Chunk = database retrieval unit (SQLite FTS5/BM25
|
||||
+ embeddings via `agent-indexer`), not a filesystem unit.
|
||||
|
||||
Default: chunk boundary = header boundary. One `##`/`###` heading = one
|
||||
atomic unit = one chunk.
|
||||
|
||||
Override: explicit delimiter comment where heading structure and
|
||||
atomicity disagree (a single heading spans two content kinds).
|
||||
|
||||
```
|
||||
<!-- CHUNK_START --> ... <!-- CHUNK_END -->
|
||||
```
|
||||
|
||||
Atomicity rule: a chunk boundary falls wherever source content changes
|
||||
kind (rule vs. example vs. table vs. exception list), not at a word count.
|
||||
|
||||
---
|
||||
|
||||
## 3. Format
|
||||
|
||||
Markdown body. YAML frontmatter per chunk (`---`-delimited, standard
|
||||
frontmatter convention).
|
||||
|
||||
Rejected: JSON (escaping fragility on quoted code/chars), YAML-as-whole-
|
||||
document (whitespace/colon fragility on prose), epub (pagination format,
|
||||
requires unpacking before RAG use).
|
||||
|
||||
```markdown
|
||||
---
|
||||
ris: unassigned
|
||||
source: man-pages(7) > STYLE GUIDE > Preferred terms
|
||||
kind: rule-table
|
||||
scope: exhaustive
|
||||
authority: normative
|
||||
related: [fcntl(2), open(2)]
|
||||
---
|
||||
|
||||
[chunk body]
|
||||
```
|
||||
|
||||
### Frontmatter fields
|
||||
|
||||
| Field | Values | Purpose |
|
||||
|---|---|---|
|
||||
| `ris` | code \| `unassigned` | Rosetta Indexing System pointer. Retrofit pending category scheme decision. |
|
||||
| `source` | breadcrumb string | Document > section path |
|
||||
| `kind` | `definition` \| `reference` \| `rule-table` \| `prose` \| `example` \| `pointer` \| `manifest` | Content shape — governs retrieval/formatting treatment |
|
||||
| `scope` | `exhaustive` \| `illustrative` | Is this list/example complete, or a sample? Primary anti-hallucination signal for frontier. |
|
||||
| `authority` | `normative` \| `editorial` | Verbatim upstream rule vs. AIB restructuring |
|
||||
| `related` | list of `name(section)` | Cross-references, write-once. Resolved to corpus pointers at ingestion/re-index time by `agent-librarian`/`agent-indexer`, not authored as resolved. |
|
||||
|
||||
---
|
||||
|
||||
## 4. Granularity guidance
|
||||
|
||||
No hard token cap. Constraint is Section 2 (one kind of information per
|
||||
chunk), not length.
|
||||
|
||||
Soft flag: chunks exceeding ~150–200 words warrant a second look — likely
|
||||
signals a chunk that should have split at a content-kind boundary.
|
||||
|
||||
---
|
||||
|
||||
## 5. Table-conversion rule
|
||||
|
||||
**Trigger:** 3+ instances of a consistent-field mapping (item→rule,
|
||||
category→treatment, term→form), regardless of source presentation
|
||||
(prose, list, or already tabular).
|
||||
|
||||
**Canonical shapes** (reuse, do not invent per-table schemas):
|
||||
|
||||
| Shape | Columns |
|
||||
|---|---|
|
||||
| Mapping | `Item \| Rule \| Notes` |
|
||||
| Applicability | `Category \| Applies to \| Description` |
|
||||
|
||||
**Scope preservation:** table conversion must not upgrade an open-ended
|
||||
source list ("...and so on") to `scope: exhaustive`. Tabularizing an
|
||||
illustrative list keeps `scope: illustrative`.
|
||||
|
||||
**Never tabularize:** sequential or conditional content — content
|
||||
encoding order-dependence or a decision tree (when to use `.SS` vs. a
|
||||
custom heading; alphabetization rules for ERRORS). Table format strips
|
||||
the relationship that is the actual content.
|
||||
|
||||
---
|
||||
|
||||
## 6. Per-section transform table
|
||||
|
||||
Applies uniformly to all 20 canonical man-page sections. Every routine
|
||||
runs against every source page unconditionally; absence is a normal
|
||||
return value, not an error condition.
|
||||
|
||||
| Section | KIND | Transform |
|
||||
|---|---|---|
|
||||
| NAME | `definition` | Verbatim, single chunk, never split. `scope: exhaustive`. |
|
||||
| SYNOPSIS | `reference` | Code-fenced, structure preserved literally. One chunk per feature-test-macro variant if applicable. `scope: exhaustive`. |
|
||||
| CONFIGURATION | `prose` / `rule-table` | Prose by default; table only if 3+ configurable items (Section 5 rule). |
|
||||
| DESCRIPTION | mixed, split by content | Split at topic shift → `prose` chunks. Embedded flag/option enumerations → separate `rule-table` chunk. |
|
||||
| OPTIONS | `rule-table` | `Option \| Effect \| Notes`. Split into self-contained option groups if exceeding granularity guidance. |
|
||||
| EXIT STATUS | `rule-table` | `Value \| Meaning`. `scope: exhaustive`. |
|
||||
| RETURN VALUE | `prose` / `rule-table` | Contrastive statement default; table for 3+ cases. |
|
||||
| ERRORS | `rule-table` | `Errno \| Condition \| Notes`. Preserve duplicate errno rows per distinct condition — do not deduplicate by name. Alphabetical order. `authority: normative`. |
|
||||
| ENVIRONMENT | `prose` / `rule-table` | Threshold per Section 5. |
|
||||
| FILES | `prose` / `rule-table` | Threshold per Section 5. |
|
||||
| VERSIONS | `rule-table` | `Version \| Change`, chronological row order. |
|
||||
| ATTRIBUTES | `rule-table` | `Interface \| Attribute \| Value`. `authority: normative`, `scope: exhaustive`. |
|
||||
| CONFORMING TO | `prose` | Short list, period-terminated. Table only if per-standard caveats reach 3+. |
|
||||
| NOTES | `prose` | Split at topic shift. Named subsections (Linux Notes, Glibc Notes) = separate chunks. |
|
||||
| BUGS | `reference-list` / `rule-table` | List if short; table if 3+ consistent bug→status entries. |
|
||||
| EXAMPLES | `example` | Code + explanation never split across chunks. Shell session logs preserve bold-input/plain-output distinction explicitly. `scope: illustrative` always. `authority` per source of the example. |
|
||||
| AUTHORS | `pointer` | `AUTHORS: See man {section} {page}.` `scope: exhaustive`, `authority: editorial`. |
|
||||
| REPORTING BUGS | `pointer` | Same template. |
|
||||
| COPYRIGHT | `pointer` | Same template. |
|
||||
| SEE ALSO | *(not a body chunk)* | Populates `related:` frontmatter field directly from source. Zero transformation judgment. |
|
||||
|
||||
---
|
||||
|
||||
## 7. Manifest chunk
|
||||
|
||||
One per document. Not a per-section stub. Records section presence/
|
||||
absence as a single fact set.
|
||||
|
||||
```
|
||||
Sections present: NAME, SYNOPSIS, DESCRIPTION, OPTIONS, EXIT STATUS, SEE ALSO
|
||||
Sections absent from source: CONFIGURATION, RETURN VALUE, ERRORS, ENVIRONMENT,
|
||||
FILES, VERSIONS, ATTRIBUTES, CONFORMING TO, NOTES, BUGS, EXAMPLES, AUTHORS,
|
||||
REPORTING BUGS, COPYRIGHT
|
||||
```
|
||||
|
||||
`kind: manifest`, `scope: exhaustive`. Replaces the need for absent-
|
||||
section stubs (10–14 near-zero-value chunks per page at 1/14th the cost).
|
||||
|
||||
---
|
||||
|
||||
## 8. Pipeline shape
|
||||
|
||||
**Scan (deterministic, no model):** regex/parser match against the
|
||||
canonical 20-name list. Produces manifest data. `.SH` markers are
|
||||
unambiguous; no reasoning required for standard headings.
|
||||
|
||||
**Orchestrator (7B-tier reasoning, e.g. Mistral):** routing decisions
|
||||
only — custom/nonstandard headings, threshold judgment on borderline
|
||||
table-conversion cases, splitting mixed-kind sections (DESCRIPTION with
|
||||
embedded tables). Styleguide (this document) is its reference-of-truth.
|
||||
Not used for section presence/absence — that's the deterministic scan's
|
||||
job.
|
||||
|
||||
**Extraction (single agent, 20 prompt-library entries):** one
|
||||
deterministic extractor agent; each of the 20 sections is a prompt
|
||||
template in `/var/lib/cervello-elettrico/prompts/`, not a separate agent
|
||||
identity. System-user isolation governs privilege boundaries (filesystem/
|
||||
network scope), not per-task identity — all 20 transforms share an
|
||||
identical permission profile, so one agent suffices.
|
||||
|
||||
**Retrieval (Qwen 0.6B, drone-tier):** consumes finished chunks only.
|
||||
Not involved in scan, routing, or extraction — those require more context
|
||||
and reasoning depth than 0.6B provides.
|
||||
|
||||
---
|
||||
|
||||
## 9. Voice (AI-corpus variant — swap point for OHIOD sibling)
|
||||
|
||||
AI-primary. Human-legible as a byproduct of being plain markdown, not
|
||||
as a design goal. Non-binary text is the only legibility floor; no
|
||||
tradeoff toward friendlier phrasing at any point.
|
||||
|
||||
- No rhetorical framing ("it's worth noting," "as mentioned above")
|
||||
- No tutorial voice — match man-pages(7)'s own register, don't soften it
|
||||
- `authority: editorial` chunks match `authority: normative` chunks in
|
||||
density; the frontmatter field carries the provenance distinction, not tone
|
||||
- Definitions: bare contrastive statements, not explanations
|
||||
- Pointer-stub sections: template only, zero padding
|
||||
(`AUTHORS: See man 7 man-pages.` — not "not reproduced in this corpus")
|
||||
|
||||
---
|
||||
|
||||
## Version history
|
||||
|
||||
| Version | Date | Notes |
|
||||
|---|---|---|
|
||||
| v1.0 | July 2026 | Initial AI-corpus styleguide. Derived from man-pages(7) reformatting design session. Pending: RIS retrofit, beta testing, OHIOD sibling. |
|
||||
Loading…
Add table
Add a link
Reference in a new issue