borg/.claude/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Machine-specific settings (e.g. the local virtualenv path) live in the gitignored CLAUDE.local.md:

@CLAUDE.local.md

Branch context

This checkout is based on 1.4-maint, a stable maintenance branch (main development happens on master). The current branch is either 1.4-maint itself or a branch forked from it (e.g. to work on a bug fix).

Only backport data-loss/corruption/security/forward-compat/doc fixes here — most PRs should target master instead.

On the 1.4 line, src/borg/archiver.py is one large monolithic file (all do_* command implementations); on master it is split into separate modules in src/borg/archiver/ package. Don't be surprised by the difference if you're comparing against upstream docs or memory of the codebase.

Build

Borg has Cython/C extensions and must be built before it can run.

source <project-virtualenv>/bin/activate  # see @CLAUDE.local.md for the local path
pip install -r requirements.d/development.txt
pip install -e .  # builds Cython/C extensions in place

Requires Python >=3.10, Cython >=3.0.3, pkgconfig, and dev headers for OpenSSL, lz4, zstd, xxhash.

Env vars to point the build at non-standard lib locations: BORG_OPENSSL_PREFIX, BORG_LIBLZ4_PREFIX, BORG_LIBZSTD_PREFIX, BORG_LIBXXHASH_PREFIX.

After editing any .pyx/.c file, rebuild the extensions:

python scripts/make.py clean && pip install -e .

If you are requested to build a BINARY (the usual borg build does not build a binary as it is Python), a pyinstaller-made binary is meant. On this 1.4-maint branch:

• POSIX: pyinstaller --clean --distpath=dist/binary/ scripts/borg.exe.spec

Writing tests

Always write pytest-style tests unless unittest-style is explicitly required.

Writing docstrings, comments, documentation

The docs shall always be in sync with the current code, algorithms and data structures.

The docs must:

• describe how it works, what it does, in its CURRENT state.
• use "TODO:" comments to refer to future tasks

The docs must not:

• describe what the code did in the past
• describe something that the code is NOT doing in its current state.

Running code or tests

Always activate the project virtualenv first (see @CLAUDE.local.md for its local path).

Tests

# Run everything (never omit --benchmark-skip unless you specifically want the slow benchmarks).
# Running all the tests without parallelizing them is very slow, the parallel variant should be preferred.
pytest --benchmark-skip -v -rs -n auto  # parallel variant
pytest --benchmark-skip -v -rs          # sequential (slow)

# Single file / single test
pytest src/borg/testsuite/archiver.py
pytest src/borg/testsuite/archiver.py::TestClass::test_method

# Via tox (matches CI; needs fakeroot for full coverage, most tests still work without it)
fakeroot -u tox                                     # all tests
fakeroot -u tox --recreate                          # after changing tox.ini
fakeroot -u tox -e py310                            # one Python version only
fakeroot -u tox borg.testsuite.locking              # one test module
fakeroot -u tox borg.testsuite.locking -- -k 'not Timer'   # -- requires the module path too

Test env vars: BORG_TESTS_IGNORE_MODES (disable mode/permission tests), BORG_FUSE_IMPL=llfuse|pyfuse3 (select FUSE implementation for mount tests).

Tests live in src/borg/testsuite/, see python_files in pyproject.toml.

macOS users without a Linux box can run the Linux test matrix via Podman: ./scripts/linux-run tox -e py311-pyfuse3.

Lint

tox -e ruff        # or: ruff check .
pre-commit install # one-time, so pre-commit hooks run automatically on commit

Style: PEP 8 at 120 columns (not 79). Comments/docstrings that are full sentences end with a period. Per-file ruff ignores (mostly E501/F401/legacy E722/E741) are listed in pyproject.toml under [tool.ruff.per-file-ignores] — check there before "fixing" a lint warning in an old file; it may be intentionally exempted.

Python code files must never have trailing blanks at the end of lines. Python code files must always end with exactly one linefeed character at the end of the file.

Committing

The working tree has a lot of stray untracked scratch files at the repo root — these are the user's own working notes, not build artifacts. Never do git add -A / git add .; stage only the files you intentionally changed.

Architecture

Borg is a client encrypting/deduplicating/compressing data before it ever reaches the repository, so the repository storage layer never sees plaintext or has any notion of files/archives.

Layering, bottom to top:

• repository.py — a transactional, append-only key→value store (Repository). Keys are 32-byte IDs; the repo doesn't know what an "archive" or "file" is. Segments in data/ are the journal; a segment ordering + COMMIT entries define transaction boundaries and crash consistency. The HashIndex (_hashindex.c/hashindex.pyx) maps object IDs to segment+offset. See docs/internals/data-structures.rst for the full on-disk format spec — read it before touching repository/segment/index code.
• remote.py — RemoteRepository (client) / RepositoryServer (server side of borg serve) implement the same key-value API over a RPC protocol on stdin/stdout (used for ssh:// repos). Every method callable remotely must be allowlisted in RepositoryServer.rpc_methods.
• crypto/key.py, crypto/low_level.pyx — encryption/authentication (AES + HMAC-SHA256, or ChaCha20-Poly1305/BLAKE2) and key file handling (keyfile vs repokey storage). id_hash used for deduplication depends on the encryption mode.
• compress.pyx, chunkers/ (buzhash, buzhash64, fastcdc in Cython/C) — per-chunk compression and content-defined chunking; chunking is what makes dedup insensitive to shifted/inserted bytes.
• cache.py, cache_sync/ — the local chunks/files cache (which chunk IDs already exist in the repo, refcounts) used to decide what needs uploading; synced against the repository's authoritative state via cache_sync (C-accelerated msgpack stream unpacking).
• archive.py — Archive/Manifest: the next layer up, turning the flat KV store into archives made of Items (files/dirs/links, see item.pyx) referencing chunk lists. The manifest (helpers/manifest.py) is itself an object in the repo listing all archives.
• archiver.py — the CLI: argument parsing (build_parser, one subparsers.add_parser(...) block per subcommand) and one do_<command> method per subcommand, e.g. do_create, do_check. This is the file to search first when changing or adding a CLI command/flag.
• platform/ — per-OS backends (Linux/Darwin/FreeBSD/NetBSD/Windows) for xattrs, ACLs, syncing, process info; platform/__init__.py picks the right implementation at import time.
• helpers/ — grab-bag of utilities used across layers (arg/format parsing, msgpack helpers, progress reporting, filesystem helpers); check here before writing a new utility, it may exist.

Client/server split: most do_* archiver commands operate against a Repository or RemoteRepository transparently — the choice of local vs. ssh:// is resolved by Location parsing in helpers/ and doesn't otherwise change command logic. borg serve on the remote end runs RepositoryServer, which allowlists and dispatches RPCs to a local Repository.

Docs as source of truth for on-disk formats: docs/internals/data-structures.rst documents the repository/segment/index/manifest/archive/item/cache formats in detail and is more authoritative than guessing from code — read the relevant section before changing serialization or the transaction log. CLI usage docs (docs/usage/) and man pages are generated from the argparse definitions in archiver.py (python scripts/make.py build_usage/build_man); don't hand-edit generated usage docs, and don't feel obligated to regenerate them for every CLI change — that's normally done at release time.

Logging conventions

Use the correct log level (debug only for debugging; info/warning/error/critical as appropriate).

When directly prompting the user (e.g. Y/N confirmations), write straight to stderr, not through the logger and not to stdout (stdout may be piped). Info-level volume is controlled via flags like --stats/--list feeding topic loggers — see _setup_implied_logging() in archiver.py.