# General overview With BorgBackup, I do a backup of my whole $HOME. However, some files (all located on ~/.local/share/) occur a rights error. Probably cause it’s ephemeral files belonging to root, I think. Here is an example of these files:

/home/fauve/.local/share/tracker/data/tracker-store.ontology.journal: open: [Errno 13] Permission denied: 'tracker-store.ontology.journal'
E /home/fauve/.local/share/tracker/data/tracker-store.ontology.journal
/home/fauve/.local/share/tracker/data/tracker-store.journal: open: [Errno 13] Permission denied: 'tracker-store.journal'
E /home/fauve/.local/share/tracker/data/tracker-store.journal
/home/fauve/.local/share/tracker/data/.meta.isrunning: open: [Errno 13] Permission denied: '.meta.isrunning'
E /home/fauve/.local/share/tracker/data/.meta.isrunning
/home/fauve/.local/share/keyrings: dir_open: [Errno 13] Permission denied: 'keyrings'
E /home/fauve/.local/share/keyrings
/home/fauve/.local/share/gnome-shell: dir_open: [Errno 13] Permission denied: 'gnome-shell'
E /home/fauve/.local/share/gnome-shell
/home/fauve/.local/share/evolution: dir_open: [Errno 13] Permission denied: 'evolution'
E /home/fauve/.local/share/evolution
/home/fauve/.local/share/sounds: dir_open: [Errno 13] Permission denied: 'sounds'

# The problem All the other files are fine saved. But these files occur a shell error. So, it’s a bit annoying cause I always get a “terminating with warning status, rc 1” at the end of backup process. As I use Borg inside a script, if the script get any error, it don’t continue. # What I did ## Exclude them explicitly I first try to explicitly exclude them with Borg’s --exclude’s option. But it’s not really useful, cause it’s not always the same files. It depend from several circumstances as I see. So I can’t really match them all. ## Exclude the whole directory ~/.local/share/ As all the annoying files comme from ~/.local/share/, I thank to fully ignore it. But after [sevral](https://unix.stackexchange.com/questions/797136/local-share-is-set-for-what-and-can-i-ignore-it-at-backups/797194#797194) asks, it seems not really possible. # The question So is it possible with Borg’s options to just ignore all files|directories according to this two conditions: 1. Located inside ~/.local/share (or child) ; 2. The current user have no right to read them.

I use Borgmatic(version 1.9.5) in order to dump backups (fs & databases). TODO this, I must create 2 backups repositories : local folder and cloud storage. Into my machine, the archive is generated without issue. For remote server case, I have a cloud archive hosted by ovh. Basicaly, here the credentials : - Host Name : gateways.storage. .cloud.ovh.net - User Name : pca - Password : . . Unfortunatly, the archive is not stored into the distant volume. Here an extract of the config file :

repositories:
- path: /data/borgbackup
  label: local backup
- path: ssh//:@/./
  label: remote backup

When I execute the command (borgmatic init), I'm invited to put the password. output :

using builtin fallback logging configuration
35 self tests completed in 0.06 seconds
SSH command line: ['ssh', 'pca@gateways.storage.***.cloud.ovh.net', 'borg', 'serve', '--umask=077', '--debug']
pca@gateways.storage.gra.cloud.ovh.net's password:
Remote: exec request failed on channel 0
Connection closed by remote host. Is borg working on the server?

On remote url, I tried with rsync (same result). Also, I tryed with sshfs (ssh fileSystem) for having a mount folder and procede like a local folder, neither this case: Failed to create/acquire the lock /mnt/backup-cloud/lock.exclusive (timeout). Questions - Do you have any clues about it? - Can I store the password into the config file ? or better mechanism (ssh key) ?

I am fully aware that questions like this one have been asked multiple times. However, since reading lots of documentation of various backup tools didn't answer all my questions, I think its fine to describe my scenario here to get some input. I want to develop a backup strategy that suits my needs. Here is a quick rundown of the requirements: - Open-Source Software - Commandline support (Not just a GUI) - I want to backup and restore from/to multiple devices. (All devices are running linux) - I want to be able to backup specific files/folders, and ignore others - I want to be able to backup to multiple destinations, e.g.: - An external HDD - Some off-site server (In case my house burns down) - I would like to automate backups, e.g. using cron - I would like that backups can be encrypted properly - I would like the backups to be deduplicating, so, only the delta to the previous backup is stored. - When restoring from a backup I'd like to be able to directly use the restored backup as a new point to backup new changes. (Think git-repository, where you can directly push new changes after you cloned the repository to a new system). After conducting some research, there are two programs that for the most part seem to be a good fit: - borgbackup - restic With borgbackup I see two problems: - I can restore files in two ways: Either I fuse-mount the backup or directly extract the backup. However, when restoring, I always get just get the files. When I restore a backup to a new system, I have to set all the repositories up fresh again, in order to run new backups from the new host? That is very inconvenient. (Again: I would love a similar behavior like a git repository, where the restored data/repository directly can be used to backup (push) new data.) - I don't think its possible to specify multiple endpoints for backups. E.g. I can backup my data to a remote server, but I can't also sync to my external HDD. (See: https://borgbackup.readthedocs.io/en/stable/faq.html#can-i-backup-from-multiple-servers-into-a-single-repository) . The way I see it, cold storage backups, like an external HDD doesn't seem to go to well with borg. With restic: - The same as with borg, essentially. So, my question is, how can I achieve a backup solution that works for the scenario I described above. Are borg and restic the right tools for the job here? If so, how do I deal with the problems I described? If not, is there other backup software that would be better here?

I have upgraded my Debian server (from bullseye to bookworm), hosted at Hetzner. I use their Storage Box + borg as one of means of backup. However, since the upgrade, all my attempts to run borg end with Connection timed out. I am at a loss how to recover from this. Version change of borg: 1.1.16-3 -> 1.2.4-1 (note: this is Debian versioning, but should correspond to actual borg versions Script called (abridged):

BACKUP_DIR=/mnt/backup/mysql
export BORG_REPO=ssh://XYZ@XYZ.your-storagebox.de:23/./my-dir
export BORG_PASSPHRASE='mysecurepassphrase'

borg create                         \
--stats                         \
--list                          \
--show-rc                       \
--compression lz4               \
 ::'{hostname}-db-{now}'        \
 $BACKUP_DIR

output as provided when run with --debug

Sun Feb 25 07:53:31 PM CET 2024 Starting backup

using builtin fallback logging configuration
33 self tests completed in 0.05 seconds
SSH command line: ['ssh', '-p', '23', 'XYZ@XYZ.your-storagebox.de', 'borg-1.2', 'serve', '--debug']
Remote: ssh: connect to host XYZ.your-storagebox.de port 23: Connection timed out
Connection closed by remote host. Is borg working on the server?
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/borg/archiver.py", line 5213, in main
    exit_code = archiver.run(args)
                ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/borg/archiver.py", line 5144, in run
    return set_ec(func(args))
                  ^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/borg/archiver.py", line 161, in wrapper
    repository = RemoteRepository(location.omit_archive(), create=create, exclusive=argument(args, exclusive),
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/borg/remote.py", line 578, in __init__
    raise ConnectionClosedWithHint('Is borg working on the server?') from None
borg.remote.ConnectionClosedWithHint: Connection closed by remote host. Is borg working on the server?

Platform: Linux ABC 6.1.0-18-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.76-1 (2024-02-01) x86_64
Linux: Unknown Linux
Borg: 1.2.4  Python: CPython 3.11.2 msgpack: 1.0.3 fuse: None [pyfuse3,llfuse]
PID: 42470  CWD: /root
sys.argv: ['/usr/bin/borg', 'create', '--remote-path=borg-1.2', '--debug', '--progress', '--stats', '--list', '--show-rc', '--compression', 'lz4', '::{hostname}-db-{now}', 'my-dir']
SSH_ORIGINAL_COMMAND: None

terminating with error status, rc 2

Important note: The Storage box is available, as I am able to backup another machine on it (through borg)

I have used an external disk to backup a laptop, for several months, using Borg-Vorta, under Linux Mint 21.3. The laptop has had a major failure, and is no more accessible. Now I have a new laptop, on which I have installed the same system, I have created the same user, and I have installed Borg-Vorta. Is there a way to access the old backups, from the external disk? I haven't done any backup of cryptographic keys from the first machine, and on that machine I used the default configuration (i.e. using the ssh key). -- Since I have the feeling the answer is simply "no", could you please explain me how to not end up in this situation again? Can you explain how to "export" the crypto keys, so to access the same backup repository on a different machine?

Using borg backup, it is quite easy to restore (extract ) a specific file X from an archive to the current location . using $ borg extract /path/to/repo::my-files home/USERNAME/X I like to extract that file from all my archives. A single borg call cannot do that. How to script this so that for each of the archives (timecoded), the respective file X(timecoded) gets extracted?

What should we keep in mind when using CACHEDIR.TAG and .nobackup? Are these synonyms, or should we expect that programs handle these differently?

I want to backup backintime-data from an internet-facing, remote machine running debian 11 (src). That is the hardlinks-part from the title. I want to back it up to a (very dumb) NAS (dest) in the internal network. Connecting to both via ssh-keys and running the backup is a third machine running debian 10. I'd like to use borgbackup - to stay with one solution for the all the machines (also: compression, encryption), but I'm open for other ideas. For accessing remote data in borgbackup, use of sshfs is suggested in their docs . But sshfs can't handle hardlinks on the reading side. Rsync should be/is able to handle hardlinks but can only handle one remote connection. Using sshfs on the dest-part of rsync throws errors ( rsync: chown "dest/filename" failed: No such file or directory (2) and rsync: mknod "dest/filename" failed: Operation not permitted (1)). The sshfs-option -o sshfs_sync is used. So I thought piping rsync into borgbackup via the stdin option, but -again according to borgbackup docs - this omits metadata (seemingly even group-/ownership) from the file. I still have to play with this answer: https://unix.stackexchange.com/a/183516/134408 So, what could I do? What are my other options?

I'm trying to setup an automated backup system to backup my Raspberry Pi's data to my Mac Mini. However, calling neither borg init nor borg create from the RPi to the remote Mac Mini repo host seems to reach the borg server running on the Mac Mini. On the Mac Mini:

% sudo borg serve --debug --restrict-to-path /Users/borg/BorgRepos/RetroPie
$LOG DEBUG borg.logger Remote: using builtin fallback logging configuration
$LOG DEBUG borg.archiver Remote: 33 self tests completed in 0.12 seconds

On the RPi:

$ sudo borg create --debug --stats borg@octolen:/Users/borg/BorgRepos/RetroPie2::Friday2 RetroPie
using builtin fallback logging configuration
35 self tests completed in 0.59 seconds
SSH command line: ['ssh', 'borg@octolen', 'borg', 'serve', '--umask=077', '--debug']
Password:
Remote: zsh:1: command not found: borg
Connection closed by remote host. Is borg working on the server?
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/borg/archiver.py", line 4455, in main
exit_code = archiver.run(args)
File "/usr/lib/python3/dist-packages/borg/archiver.py", line 4387, in run
return set_ec(func(args))
File "/usr/lib/python3/dist-packages/borg/archiver.py", line 134, in wrapper
make_parent_dirs=make_parent_dirs, args=args)
File "/usr/lib/python3/dist-packages/borg/remote.py", line 577, in __init__
raise ConnectionClosedWithHint('Is borg working on the server?') from None
borg.remote.ConnectionClosedWithHint: Connection closed by remote host. Is borg working on the server?
Platform: Linux retropie2 5.10.103-v7l+ #1529 SMP Tue Mar 8 12:24:00 GMT 2022 armv7l
Linux: debian 10.13 
Borg: 1.1.9 Python: CPython 3.7.3
PID: 28539 CWD: /home/pi
sys.argv: ['/usr/bin/borg', 'create', '--debug', '--stats', 'borg@octolen:/Users/borg/BorgRepos/RetroPie2::Friday2', 'RetroPie']
SSH_ORIGINAL_COMMAND: None

I did add /usr/bin/borg to the Mac's firewall's list of applications permitted to allow incoming connections. What am I missing here?

I backup all my files (with borgbackup) every 2-3 months to an external harddisk I store off-site. The problem I want to solve is how to protect the files that changed since the last backup. These are probably not big and could be stored on a free tier cloud storage. I was thinking: if should be possible to find all files that were changed since the last backup (maybe with find *all files newer than the file I touched after the last backup*), create a hardlink to a folder on my harddisk and backup this folder to the cloud. Am I forgetting something? How would such a script look like? Is there a simpler way?

I have a borg backup system for some time now. The target is cloud storage, accessed by ssh. This is SSH command line from debug: SSH command line: ['ssh', '-i', '/root/.ssh/id_rsa', '-p', '23', '@.your-storagebox.de', 'borg', 'serve', '--umask=077', '--debug'] The cloud storage was full and it caused a borg prune command to stop responding. In the meantime I have killed the process for that command, deleted the borg cache, and have broken the lock. If I try to run borg info or borg check, the command stops after

Remote: Verified integrity of /home/backup/hints.8441

Here is the output of the debug:

borg check --debug 
using builtin fallback logging configuration
35 self tests completed in 0.36 seconds
SSH command line: ['ssh', '-i', '/root/.ssh/id_rsa', '-p', '23', '@.your-storagebox.de', 'borg', 'serve', '--umask=077', '--debug']
Remote: using builtin fallback logging configuration
Remote: 33 self tests completed in 0.08 seconds
Remote: using builtin fallback logging configuration
Remote: Initialized logging system for JSON-based protocol
Remote: Resolving repository path b'/./backup'
Remote: Resolved repository path to '/home/backup'
Remote: Starting repository check
Remote: Verified integrity of /home/backup/index.8441
Remote: Verified integrity of /home/backup/hints.8441

After that, the borg hangs (I have waited for a few hours now)

Currently I have Borg archives being creative hourly, via a cron script, for directories I'd like to be backed up. However, this is not exactly ideal. Ideally, I'd like backups to be created more than hourly while that portion of the filesystem is active. Conversely, sometimes one of the directories might stay unchanged for weeks at a time - so, creating hourly archives is just a waste of space, and I don't want to be manually switching portions off or on all the time. From what I've been able to search, linux does not really have any robust system for continuous backups. If this conclusion is erroneous, then please do point me in the right direction. But if that is true, I'd like to do the next best thing - have an archive only be created if a file has changed (i.e., checksums on the live filesystem do not match those of files in the archive... ideally). Is there a way that a script, program option, or some other method can accomplish this? Thank you.

I am trying to use borg backup and am running into an issue with SSH. I am backing up some root files such as /var/www/html, therefore I require root permissions, so I need to run the backup script as root, but when I run it as root the SSH key doesn't seem to work, and as it's a script I'd like to run with cron it means I cant be entering the password each time. Note that the SSH keys work when I run the script from my user, but then the backup doesn't work as some of the file locations are permission denied. I am backing up to a storage box with Hetzner. My Script: #!/usr/bin/env bash LOG="/var/log/borg/backup.log" BACKUP_USER="XXXXXXX" REPOSITORY_DIR="backups/Node1" REPOSITORY="ssh://${BACKUP_USER}@${BACKUP_USER}.your-storagebox.de:23/./${REPOSITORY_DIR}" exec > >(tee -i ${LOG}) exec 2>&1 echo "###### Backup started: $(date) ######" echo "Transfer files ..." BORG_PASSPHRASE='XXXXXXXXXX' borg create -v --stats \ $REPOSITORY::'{now:%Y-%m-%d_%H:%M}' \ /root \ /etc \ /var/www \ /home \ --exclude /dev \ --exclude /proc \ --exclude /sys \ --exclude /var/run \ --exclude /run \ --exclude /lost+found \ --exclude /mnt \ --exclude /var/lib/lxcfs echo "###### Backup ended: $(date) ######"

I'm trying to run the borg mount in Borg Backup, but it's saying that fusermount3 is not installed. fuse: failed to exec fusermount3: No such file or directory Googling this problem isn't helping. I have fuse installed: fuse is already the newest version (2.9.9-3). as is libfuse: libfuse2 is already the newest version (2.9.9-3). I'm running Linux Mint 20 with Kernel 5.4.0-113-generic

Using borg create , I want to exclude a folder, but with some exceptions. Specifically, I want to exclude all paths subordinate to the AppData folder, except for the ones that are bookmark related, which would be paths that contain these strings: 1. places.sqlite 2. bookmarkbackups 3. Bookmarks My thoughts were, first **exclude** what you don't want, and then **include** exceptions: borg create --list --dry-run --exclude /*/AppData/* --include /*places.sqlite* --include /*bookmarkbackups* --include /*Bookmarks* /path/to/rep::$(date '+%Y-%m-%d') /windows/user/folder Apparently, there is no --include argument. So, how do I achieve this?

When the borg create command is used with the --progress argument, it output like this: 5.50 GB O 5.10 GB C 23.95 kB D 15600 N /path/to/current/file/being/processed I was able to locate what the abbreviations above stand for here : > The --progress option shows (from left to right) Original, Compressed > and Deduplicated (O, C and D, respectively), then the Number of files > (N) processed so far, followed by the currently processed path. -source Yet, I'm still not certain about the meaning of Deduplicated as it is used here. In the example above, Deduplicated's value is 23.95 kB. Does that mean that out of the 5GB of data, processed so far, only 23.95 kB of it needed to be transferred because that's the only unique data found since my last archival into the repository?

I need to backup data using Borg Backup from a remote server to a local server (raspberry-pi like - it is actually Yunohost) equipped with a 2 slots storage bay. I would like to have identical copies of the backup data on each disk. Any idea how to do so? I've never done anything like this. Should I create two repositories, one on each disk, and check on Borg's settings to sync them? Or should I set a software raid 1 between both hdds?

I've been trying to setup a borg automated backup script following the documentation. For some reason my script fails to see its variables while I'm trying to write to a sudo protected encrypted drive. For example, something like this works as the script as the script doesn't ask me a password again. #!/bin/sh DATE=$(date --iso-8601)-$(hostname) TARGET=~/Documents/borg export BORG_PASSPHRASE="whatever" borg init --encryption=repokey $TARGET borg create $TARGET::$DATE-$$ ~/Desktop/TestFolder/1 However when I mount an external FDE implemented drive, it ignores the exported password and and weirdly enough the "repokey" encryption mode. #!/bin/sh DATE=$(date --iso-8601)-$(hostname) TARGET=~/X/borg export BORG_PASSPHRASE="whatever" sudo borg init --encryption=repokey $TARGET sudo borg create $BORG_OPTS $TARGET::$DATE-$$ ~/Desktop/TestFolder/1 I'm fairly new to Linux so this is probably an easy fix but I couldn't do it on my own when I referred to other answers. Please help.

I recently had to restore the disk with **borg**. So via live linux, I entered my disk file system, deleted everything (root **/**), and restored a backup of a specific date. *Now I ask myself:* **do you need to align the disk or the files?** (*inodes*, etc... in this case, some *inodes* have also changed). That is, I know that some files (mainly system files) cannot be copied normally with the classic "cp". In this case a "cp" command was not used but a copy was made anyway. Don't the files have to *"move them to the beginning of the disk"?* Is there a difference, in this case, between a **HDD** and an **SSD**? (my system is on ***SSD***)....and, can i copy the system with the classic **cp**? I would like more detailed explanations on this. Thanks and apologize for my english.

I want to backup some files that can only be read as root to a remote repository that only my user has access to. If I run borg as myself using /etc as my source I'll get permission-denied errors when trying to read some files like /etc/crypttab. If I run borg as root with sudo, the files can be read but I can't write to the repository because my local root user doesn't have ssh access to the remote repository and it doesn't have the repo key for that repository (ie /root/.borg/ doesn't exist). To recap:

$ borg create ssh://me@remote:/backup/my_repo.borg::demo /etc

will error because I can't read some files in /etc, which I _do want backed up_. And

$ sudo borg create ssh://me@remote:/backup/my_repo.borg::demo /etc

will fail because root doesn't have neither access to ssh://me@remote nor the repokey. Now an obvious work around would be give root ssh access and the repo key but I'm wondering if there is a better solution.