Styleguides/man-pages-styleguide/man-pages-styleguide-ai.md

8.9 KiB
Raw Permalink Blame History

CE Man Page Reformatting Styleguide — AI Corpus (v1.0)

0. Purpose and scope

Governs reformatting of individual man pages (Sections 18) into the CE RAG corpus for AI consumption: microSLM retrieval (Qwen 0.6B) and frontier reasoning over retrieval (Claude).

Out of scope: enthusiast-tutorial transformation of man pages. Separate future project, separate voice rules.

Sibling document (not yet authored): OHIOD-readable variant. Same rule body (Sections 18 below), Section 9 (Voice) replaced only. Authored after this version passes beta testing.


1. File boundary

One source document = one file.

Source type File unit
Individual man page One page = one file
Multi-chapter book (ProGit, Tao of Tmux) One chapter = one file

Rationale: file boundary = smallest unit upstream treats as independently coherent and independently revised.

Naming: [source]-[identifier]-rag.md Examples: man-pages-conventions-v1-rag.md, progit-v2-ch03-branching-in-git-rag.md


2. Chunk mechanism

One file, many chunks. Chunk = database retrieval unit (SQLite FTS5/BM25

  • embeddings via agent-indexer), not a filesystem unit.

Default: chunk boundary = header boundary. One ##/### heading = one atomic unit = one chunk.

Override: explicit delimiter comment where heading structure and atomicity disagree (a single heading spans two content kinds).

<!-- CHUNK_START --> ... <!-- CHUNK_END -->

Atomicity rule: a chunk boundary falls wherever source content changes kind (rule vs. example vs. table vs. exception list), not at a word count.


3. Format

Markdown body. YAML frontmatter per chunk (----delimited, standard frontmatter convention).

Rejected: JSON (escaping fragility on quoted code/chars), YAML-as-whole- document (whitespace/colon fragility on prose), epub (pagination format, requires unpacking before RAG use).

---
ris: unassigned
source: man-pages(7) > STYLE GUIDE > Preferred terms
kind: rule-table
scope: exhaustive
authority: normative
related: [fcntl(2), open(2)]
---

[chunk body]

Frontmatter fields

Field Values Purpose
ris code | unassigned Rosetta Indexing System pointer. Retrofit pending category scheme decision.
source breadcrumb string Document > section path
kind definition | reference | rule-table | prose | example | pointer | manifest Content shape — governs retrieval/formatting treatment
scope exhaustive | illustrative Is this list/example complete, or a sample? Primary anti-hallucination signal for frontier.
authority normative | editorial Verbatim upstream rule vs. AIB restructuring
related list of name(section) Cross-references, write-once. Resolved to corpus pointers at ingestion/re-index time by agent-librarian/agent-indexer, not authored as resolved.

4. Granularity guidance

No hard token cap. Constraint is Section 2 (one kind of information per chunk), not length.

Soft flag: chunks exceeding ~150200 words warrant a second look — likely signals a chunk that should have split at a content-kind boundary.


5. Table-conversion rule

Trigger: 3+ instances of a consistent-field mapping (item→rule, category→treatment, term→form), regardless of source presentation (prose, list, or already tabular).

Canonical shapes (reuse, do not invent per-table schemas):

Shape Columns
Mapping Item | Rule | Notes
Applicability Category | Applies to | Description

Scope preservation: table conversion must not upgrade an open-ended source list ("...and so on") to scope: exhaustive. Tabularizing an illustrative list keeps scope: illustrative.

Never tabularize: sequential or conditional content — content encoding order-dependence or a decision tree (when to use .SS vs. a custom heading; alphabetization rules for ERRORS). Table format strips the relationship that is the actual content.


6. Per-section transform table

Applies uniformly to all 20 canonical man-page sections. Every routine runs against every source page unconditionally; absence is a normal return value, not an error condition.

Section KIND Transform
NAME definition Verbatim, single chunk, never split. scope: exhaustive.
SYNOPSIS reference Code-fenced, structure preserved literally. One chunk per feature-test-macro variant if applicable. scope: exhaustive.
CONFIGURATION prose / rule-table Prose by default; table only if 3+ configurable items (Section 5 rule).
DESCRIPTION mixed, split by content Split at topic shift → prose chunks. Embedded flag/option enumerations → separate rule-table chunk.
OPTIONS rule-table Option | Effect | Notes. Split into self-contained option groups if exceeding granularity guidance.
EXIT STATUS rule-table Value | Meaning. scope: exhaustive.
RETURN VALUE prose / rule-table Contrastive statement default; table for 3+ cases.
ERRORS rule-table Errno | Condition | Notes. Preserve duplicate errno rows per distinct condition — do not deduplicate by name. Alphabetical order. authority: normative.
ENVIRONMENT prose / rule-table Threshold per Section 5.
FILES prose / rule-table Threshold per Section 5.
VERSIONS rule-table Version | Change, chronological row order.
ATTRIBUTES rule-table Interface | Attribute | Value. authority: normative, scope: exhaustive.
CONFORMING TO prose Short list, period-terminated. Table only if per-standard caveats reach 3+.
NOTES prose Split at topic shift. Named subsections (Linux Notes, Glibc Notes) = separate chunks.
BUGS reference-list / rule-table List if short; table if 3+ consistent bug→status entries.
EXAMPLES example Code + explanation never split across chunks. Shell session logs preserve bold-input/plain-output distinction explicitly. scope: illustrative always. authority per source of the example.
AUTHORS pointer AUTHORS: See man {section} {page}. scope: exhaustive, authority: editorial.
REPORTING BUGS pointer Same template.
COPYRIGHT pointer Same template.
SEE ALSO (not a body chunk) Populates related: frontmatter field directly from source. Zero transformation judgment.

7. Manifest chunk

One per document. Not a per-section stub. Records section presence/ absence as a single fact set.

Sections present: NAME, SYNOPSIS, DESCRIPTION, OPTIONS, EXIT STATUS, SEE ALSO
Sections absent from source: CONFIGURATION, RETURN VALUE, ERRORS, ENVIRONMENT,
FILES, VERSIONS, ATTRIBUTES, CONFORMING TO, NOTES, BUGS, EXAMPLES, AUTHORS,
REPORTING BUGS, COPYRIGHT

kind: manifest, scope: exhaustive. Replaces the need for absent- section stubs (1014 near-zero-value chunks per page at 1/14th the cost).


8. Pipeline shape

Scan (deterministic, no model): regex/parser match against the canonical 20-name list. Produces manifest data. .SH markers are unambiguous; no reasoning required for standard headings.

Orchestrator (7B-tier reasoning, e.g. Mistral): routing decisions only — custom/nonstandard headings, threshold judgment on borderline table-conversion cases, splitting mixed-kind sections (DESCRIPTION with embedded tables). Styleguide (this document) is its reference-of-truth. Not used for section presence/absence — that's the deterministic scan's job.

Extraction (single agent, 20 prompt-library entries): one deterministic extractor agent; each of the 20 sections is a prompt template in /var/lib/cervello-elettrico/prompts/, not a separate agent identity. System-user isolation governs privilege boundaries (filesystem/ network scope), not per-task identity — all 20 transforms share an identical permission profile, so one agent suffices.

Retrieval (Qwen 0.6B, drone-tier): consumes finished chunks only. Not involved in scan, routing, or extraction — those require more context and reasoning depth than 0.6B provides.


9. Voice (AI-corpus variant — swap point for OHIOD sibling)

AI-primary. Human-legible as a byproduct of being plain markdown, not as a design goal. Non-binary text is the only legibility floor; no tradeoff toward friendlier phrasing at any point.

  • No rhetorical framing ("it's worth noting," "as mentioned above")
  • No tutorial voice — match man-pages(7)'s own register, don't soften it
  • authority: editorial chunks match authority: normative chunks in density; the frontmatter field carries the provenance distinction, not tone
  • Definitions: bare contrastive statements, not explanations
  • Pointer-stub sections: template only, zero padding (AUTHORS: See man 7 man-pages. — not "not reproduced in this corpus")

Version history

Version Date Notes
v1.0 July 2026 Initial AI-corpus styleguide. Derived from man-pages(7) reformatting design session. Pending: RIS retrofit, beta testing, OHIOD sibling.