Backup Operations

Operations for backing up and restoring databases. For how the backup system works, its limitations, and worked examples, see the Backups Overview.

All backup operations require a super_user role when invoked through a running server; run offline (from the CLI with the server stopped), they are governed by filesystem permissions instead. All accept a database parameter that defaults to data. The engine badge on each operation indicates which storage engines support it: the managed-backup operations require RocksDB, while get_backup works with both RocksDB and LMDB.

Operation	Description	Role Required
create_backup	Creates a managed, incremental directory backup of a database (job)	super_user
list_backups	Lists the managed backups for a database	super_user
verify_backup	Verifies a managed backup's integrity (job)	super_user
delete_backup	Deletes a single managed backup	super_user
purge_backups	Deletes all but the newest keep_count managed backups	super_user
restore_backup	Restores a database from a managed backup (job)	super_user
get_backup	Streams a full snapshot of a database in the response for download	super_user

Running backup operations from the CLI

Every operation on this page can be run from the CLI under its operation name, for example harper create_backup database=data. When Harper is running, the CLI forwards the operation to the server; when it is stopped, the command operates directly on the database and backup files. get_backup is the exception — it streams from a running server and has no offline form.

Like any CLI operation, backup commands also accept a remote target=<url> to run against another instance instead of the local one (see Remote Operations). With a remote target the operation is always forwarded — there is no offline path.

A target set with HARPER_CLI_TARGET or CLI_TARGET is always forwarded too, including one that harper login added to a .env file. harper logout does not remove a .env entry, so delete that line to run commands offline.

A target saved by harper login is also forwarded, with one exception. If the saved target is a loopback address (localhost, 127.x or ::1) and the local server is stopped:

• create_backup, list_backups and verify_backup run offline against this installation's files.
• restore_backup, delete_backup and purge_backups fail without changing anything, because a loopback address can belong to a different instance, such as an SSH tunnel, a container or another local install.

To run one of the failing commands on this installation's files, first run harper logout <target>, which also removes that saved login. To run it on the saved target, pass target=<url>.

Offline invocation matters most for restore: an in-place restore_backup of the system database, or of any database a loaded component keeps open, must be run with the server stopped. See "when can a database be restored".

create_backup

Added in: v5.2.0 Engine: RocksDB

Creates an incremental directory backup of the database under the configured backup root (storage.backupPath, default <rootPath>/backup). Through a running server this runs as a background job: the operation returns a job_id immediately, and get_job reports the outcome including the new backup_id, size, and timestamp.

Backups of the same database share unchanged RocksDB data files within the repository. Each backup also copies the transaction-log snapshot. Blob snapshots use hard links when the source and destination filesystem support them, and copy the files otherwise. Pass exclude_blobs: true to skip the blob snapshot. Measure physical repository growth to plan capacity, and use purge_backups to manage retention.

{ "operation": "create_backup", "database": "data" }

harper create_backup database=data

list_backups

Added in: v5.2.0 Engine: RocksDB

Returns the managed backups for a database, each with:

• backup_id — the monotonic integer identifier.
• timestamp — creation time in seconds since the epoch.
• size — bytes of the backup's RocksDB file payloads.
• file_count — number of RocksDB files (some are shared across backups).

size and file_count come from the RocksDB backup engine and exclude the transaction-log and blob snapshots, so they undercount the repository's true on-disk footprint. Returns an empty array if no backups have been created yet.

{ "operation": "list_backups", "database": "data" }

harper list_backups database=data

verify_backup

Added in: v5.2.0 Engine: RocksDB

Verifies a managed backup. backup_id is required.

• RocksDB files: checks file sizes and, when verify_checksum is true, checksums. Checksum verification takes longer.
• Transaction log: always checks the snapshot's framing.
• Blob snapshot: checks that a snapshot is present if the backup recorded blobs. A missing snapshot fails verification. Individual blob sizes, checksums, and readability are not checked, so damaged blob contents can pass verification.

Through a running server, verification runs as a background job. Poll get_job for its result.

{ "operation": "verify_backup", "database": "data", "backup_id": 1, "verify_checksum": true }

harper verify_backup database=data backup_id=1 verify_checksum=true

delete_backup

Added in: v5.2.0 Engine: RocksDB

Deletes a single managed backup. Files shared with other backups are reference-counted and removed only when no remaining backup references them. backup_id is required.

{ "operation": "delete_backup", "database": "data", "backup_id": 1 }

harper delete_backup database=data backup_id=1

purge_backups

Added in: v5.2.0 Engine: RocksDB

Deletes all but the newest keep_count managed backups, returning the number deleted and the number remaining.

{ "operation": "purge_backups", "database": "data", "keep_count": 3 }

harper purge_backups database=data keep_count=3

restore_backup

Added in: v5.2.0 Changed in: v5.4.0 Engine: RocksDB

Restores a database in place from a managed backup. backup_id defaults to the latest backup. The audit/transaction log is restored alongside the data, and — for a database with file-backed blobs — the blob roots are purged and rewritten from the backup's blob snapshot.

Added in: v5.3.0 allow_engine_only (optional, default false) permits restoring a backup created with exclude_blobs: true. Without it, that restore is refused, including when using target_database. Enabling it accepts the risk that restored records reference missing or unrelated blob files; recover and validate those blobs separately.

Through a running server this runs as a background job: Harper closes the database across all worker threads, checks the backup, restores it, and reloads it. This works only when no loaded component is holding the database open — if one does, the job ends in ERROR (surfaced by get_job) telling you to restore offline. Restoring the system database is rejected up front, before a job is created, as is target_database while the server is running. These cases require running the command from the CLI with the server stopped. See when can a database be restored?

As of v5.3.1, subscriptions opened before a restore do not carry over to the restored data. When Harper reloads the database, each one ends, and its last message is a DatabaseGenerationChangedError (status code 409, code DATABASE_GENERATION_CHANGED). Resubscribe to resynchronize against the restored state. A restore that fails before it changes anything reloads the database as it was; those subscriptions end instead with a retryable DatabaseClosingError (status code 503, code DATABASE_CLOSING); resubscribe to continue.

As of v5.4.0, a restore checks the backup before it deletes anything. It verifies the checksum of every database file in the backup (not blob files), checks the backup's transaction logs, and opens each of its table files to confirm this version of Harper can read their format and compression. For a backup that captured blobs, it also checks that the blob snapshot is present. A backup that fails any of these is refused: the restore fails with an error saying it did not modify the database, followed by the cause, and the database keeps its data and its blobs. A rerun over an earlier restore that did not finish is the exception: that database is already incomplete, so it stays blocked until a restore succeeds, and the error says so. verify_backup does not open the table files, so a backup it reports as valid can still be refused here for a format this version cannot read.

The check reads the whole backup once before the restore copies it, so a restore takes longer than before but needs no extra disk space. Through a running server, the check runs after the database is closed, so the database is unavailable while the backup is checked as well as while it is copied. Writes during that time fail, as they do for the rest of the restore, rather than succeeding and then being lost. The contents of blob files are not checked: a blob that cannot be read fails the restore while the blobs are copied, after the database files have been replaced, and the database stays marked incompletely restored until a rerun succeeds.

Before v5.4.0

Previously the restore deleted the database first, so a backup this version could not read, such as one taken by a newer version of Harper, destroyed the database it was meant to restore.

From the CLI with the server stopped, target_database=<name> restores into a separate database instead of overwriting the source. The target must not already exist, or must be an empty directory; Harper picks the new database up on the next start.

If a restore is interrupted (crash, power loss) after it has started replacing the database's files, Harper marks the database as incompletely restored and refuses to load it on the next start, logging an incomplete-restore error. Recover by rerunning restore_backup for the same database and backup_id — do not try to load or hand-repair the directory. As of v5.4.0, an interruption while the backup is still being checked leaves the destination's files and any preexisting incomplete-restore marker unchanged; a previously healthy database loads normally on the next start.

{ "operation": "restore_backup", "database": "data", "backup_id": 1 }

harper restore_backup database=data backup_id=1
# restore into a new database instead of overwriting the source (server stopped)
harper restore_backup database=data backup_id=1 target_database=data_restored

get_backup

Engines: RocksDB, LMDB

Streams a full snapshot of the specified database in the HTTP response for download — there is no server-side artifact and nothing to clean up. The server must be running; this is the one backup operation with no offline form. Behavior depends on the storage engine:

• RocksDB Changed in: v5.2.0 — streams a tar archive of the current database state — all tables, the transaction log, and any file-backed blobs — gzipped by default (gzip is a RocksDB-only option and compresses the snapshot substantially). Pass "gzip": false for a plain tar, or "exclude_blobs": true to leave blobs out. The snapshot is always the current state; downloading a specific historical backup_id is not supported — to move a retained managed backup off-host, copy its whole per-database backup repository (see Limitations). Restoring a RocksDB snapshot by hand takes an extra step when the database has blobs — see download a snapshot and restore it manually.
• LMDB — streams the .mdb file, without external blob files. Specify "table" for a single table or "tables" for a set; a table-filtered download includes the audit store only with "include_audit": true. These options are LMDB-only. For databases with file-backed blobs, use a coordinated filesystem or volume backup of the database and all blob roots with Harper stopped; see LMDB recovery.

{ "operation": "get_backup", "database": "data" }

From the CLI, pass out=<path> to choose the output file (defaults to a name derived from the response):

harper get_backup database=data out=./data.tar.gz
harper get_backup database=data target=https://node-2.example.com:9925 out=./data.tar.gz