mirror of
https://github.com/borgbackup/borg.git
synced 2026-09-01 14:13:19 +02:00
8550349a77The document still described borg 1.x in many places. All claims were re-verified against the code (and a live scratch repository): - repository layout: borgstore namespaces (archives/ cache/ config/ index/ keys/ locks/ packs/) instead of the borg1 nested data/ tree; document the index/ namespace and the cache/ objects. - repo objects: the 49-byte header (magic, version, chunk id, meta and data sizes) and storage inside packs; container details deferred to the packs document. - manifest: version 2, archives always empty, item_keys inside config. - archives directory: entries keyed by archive ID with empty content; no duplicate-name counters; soft-delete via the .del suffix. - archive timestamps: time/start/end (time_end is borg1-only). - item fields synced with item.pyx (target, bsdflags, size, inode, ACL keys; legacy read-only keys noted). - files cache: document the digests field and the actual location. - argon2 salt is 128 bits, not 256. - key envelope: repository_id refers to the config/id store object; optional label field added. - locking: drop the borg1 fslocking cache-lock section; storelocking is the only locking borg2 does. - cache config: actual keys + [integrity] section; a corrupt files cache is discarded and rebuilt, not fatal. - compaction: describe the current index-driven algorithm; drop the "reappeared IDs" claim. - id-hash modes: --id-hash for encrypted modes vs the fixed authenticated-*/none-* modes. - default chunker examples use fastcdc; fix the broken "cache" link. - internals.rst: mention the rabin-aes/toeplitz-aes/goldilocks-aes chunkers; compression is done by RepoObj, not the key object. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
52 lines
2 KiB
ReStructuredText
52 lines
2 KiB
ReStructuredText
.. include:: global.rst.inc
|
|
.. _internals:
|
|
|
|
Internals
|
|
=========
|
|
|
|
The internals chapter describes and analyzes most of the inner workings
|
|
of Borg.
|
|
|
|
Borg uses a low-level, key-value store, the :ref:`repository`, and
|
|
implements a more complex data structure on top of it, which is made
|
|
up of the :ref:`manifest <manifest>`, :ref:`archives <archive>`,
|
|
:ref:`items <item>` and data :ref:`chunks`.
|
|
|
|
Each repository can hold multiple :ref:`archives <archive>`, which
|
|
represent individual backups that contain a full archive of the files
|
|
specified when the backup was performed.
|
|
|
|
Deduplication is performed globally across all data in the repository
|
|
(multiple backups and even multiple hosts), both on data and file
|
|
metadata, using :ref:`chunks` created by the chunker using a
|
|
content-defined chunking algorithm - the Gear rolling hash of FastCDC_
|
|
("fastcdc" chunker, the default), Buzhash_ ("buzhash" and "buzhash64"
|
|
chunker) or a universal hash followed by an AES pseudo-random function
|
|
("rabin-aes", "toeplitz-aes" and "goldilocks-aes" chunker) - or a simpler
|
|
fixed block size algorithm ("fixed" chunker).
|
|
|
|
To perform the repository-wide deduplication, a hash of each
|
|
chunk is checked against the :ref:`chunks index <index>`, which is a
|
|
hash table of all chunks that already exist.
|
|
|
|
.. figure:: internals/structure.png
|
|
:figwidth: 100%
|
|
:width: 100%
|
|
|
|
Layers in Borg. At the very top, commands are implemented, using
|
|
a data access layer provided by the Archive and Item classes.
|
|
Below that, the RepoObj class compresses the data (using a Compressor,
|
|
see class RepoObj in ``repoobj.py``) and then hands it to the "key"
|
|
object, which provides the authenticated encryption. The "key" object
|
|
represents the sole trust boundary in Borg.
|
|
The lowest layer is the repository accessed via class Repository.
|
|
Repository uses ``borgstore`` internally.
|
|
|
|
.. toctree::
|
|
:caption: Internals contents
|
|
|
|
internals/security
|
|
internals/data-structures
|
|
internals/chunker
|
|
internals/packs
|
|
internals/frontends
|