borg passphrase is incorrect issue, Linux vs. macOS client

[andreaseberharter]

Have you checked borgbackup docs, FAQ, and open GitHub issues?

Yes

Is this a BUG / ISSUE report or a QUESTION?

QUESTION

System information. For client/server mode post info for both machines.

Your borg version (borg -V).

Client: borg 1.4.0
Server: hetzner storage box

Operating system (distribution) and version.

macOS Sonama, homebrew install

Hardware / network configuration, and filesystems used.

How much data is handled by borg?

20GB

Full borg commandline that lead to the problem (leave away excludes and passwords)

borg mount ssh://[email protected]:23/~/repro.borg /local/mnt/dir

Describe the problem you're observing.

I have a client running Ubuntu 24.04, borg 1.4.0 which is backing up daily to a hetzner storage box server. I understand how I can restore data from the repository to the client our mount it by following the commands in the FAQ

I want to mount the repository on my macOS client, but running the same command and specifying the correct borg-passphrase I always get error message: passphrase supplied in BORG_PASSPHRASE, by BORG_PASSCOMMAND or via BORG_PASSPHRASE_FD is incorrect.

Can you reproduce the problem? If so, describe how. If not, describe troubleshooting steps you took before opening the issue.

I have read through the docs and FAQs and search the web but I can not find the correct documentation. I am sure this has been done before and I am just missing a simple step :S Please be so kind and point me to the right direction.

I can access the hetzner storage box via SSH using a ssh_key, so it is not a SSH issue. I found similar questions on reddit, do I have to init the repository locally to mount it? I am so confused...

Include any warning/errors/backtraces from the system logs

[ThomasWaldmann]

Do you have any chars outside pure ASCII or accent chars in your pw?

While it is not a problem for borg to process them, it might be a problem for you to reproduce the exact same utf-8 byte sequence due to keyboard and normalisation differences.

If that is the case, change the pw and just use 0..9 a..z A..Z.

[andreaseberharter]

Thank you Thomas for your quick reply :)

Yes, I do have a crazy amount of special chars, maybe a bit excessive :S

Now I am getting the same error message when I try to change the passphrase from the Linux client executing:
borg key change-passphrase -v ssh://[email protected]:23/~/repro.borg

I used borgmatic to setup the borg repository and store the password in $HOME/.config/borg/.borg-passphrase. So, I also tried:

export BORG_REPO="ssh://[email protected]:23/~/repro.borg"
export BORG_PASSCOMMAND="cat $HOME/.config/borg/.borg-passphrase"
borg key change-passphrase

with the same result passphrase supplied in BORG_PASSPHRASE, by BORG_PASSCOMMAND or via BORG_PASSPHRASE_FD is incorrect.

Is there any other way I could change the passphrase?

[ThomasWaldmann]

export BORG_PASSCOMMAND="cat $HOME/.config/borg/.borg-passphrase"

That looks incorrect, guess you meant BORG_PASSPHRASE?

[andreaseberharter]

Yes, of course, sorry my mistake, I meant export BORG_PASSPHRASE="cat $HOME/.config/borg/.borg-passphrase"

The funny thing though, if I run borgmatic list or borgmatic info there is no error message and I get the returns as expected:

borgmatic list
storagebox: Listing archives
...
backup-2024-10-25T00:43:54.373614   Fri, 2024-10-25 00:43:55 [4c0def73e780d0b60849310fe04bd47ea939eacf48f01c7a977377e2f3748497]
backup-2024-10-26T01:32:45.015218   Sat, 2024-10-26 01:32:46 [b459c20b7e18c97b5b865493a33700c476763c81024a9279b302a29a5f807f13]
backup-2024-10-27T02:45:22.548788   Sun, 2024-10-27 02:45:24 [dd751c3d57d115482d432c8a38931b05b8cf0807cda7fd2e92474e4356a9a8a8]
...

[andreaseberharter]

This is interesting, I installed borgmatic on macOS client, and copied over the .borg-passphrase and borgmatic/config.yaml config files

brew install borgmatic
~/.config/borg/.borg-passphrase
~/.config/borgmatic/config.yaml

Now I can can list and mount the repository:

borgmatic list --match-archives "*"   
borgmatic mount --mount-point ~/repro

Well, it works now to access the repository from macOS client, but I still don't understand why it does not work with borg alone.

[jdchristensen]

export BORG_PASSCOMMAND="cat $HOME/.config/borg/.borg-passphrase"

That looks incorrect, guess you meant BORG_PASSPHRASE?

Shouldn't BORG_PASSPHRASE actually contain the passphrase, not a command for producing the passphrase? And BORG_PASSCOMMAND would contain a command that produces the passphrase? So the quoted export above looks sensible to me. Or one could do export BORG_PASSPHRASE=`cat $HOME/.config/borg/.borg-passphrase` to do command substitution.

[ThomasWaldmann]

Ehrm, yeah, @jdchristensen is of course correct.

Somehow I misread and thought the shell was executing the command, but with simple double-quotes, it doesn't.

[andreaseberharter]

Thank you @jdchristensen, I have tried the following commands on the command line:

export BORG_PASSCOMMAND="cat $HOME/.config/borg/.borg-passphrase"   # BORG_PASSCOMMAND
export BORG_PASSPHRASE="cat $HOME/.config/borg/.borg-passphrase"    # double quotes "
export BORG_PASSPHRASE='cat $HOME/.config/borg/.borg-passphrase'    # single quotes '
export BORG_PASSPHRASE="super long borg-passphrase"                 # just the passphrase

All give me the same error when running borg info afterwards.

Also, if I don't export BORG_PASSPHRASE I am being prompted for it, once the connection has been established - not sure if pasting works, because I could never type the passphrase on the keyboard.

Is there any way I can increase debug info to understand what am I doing wrong?

[ThomasWaldmann]

line 1 and line 4 should work, 2 and 3 are incorrect.

Please note that the last command in @jdchristensen's post has backticks, not single quotes.

[jdchristensen]

Also note that BORG_PASSPHRASE overrides BORG_PASSCOMMAND, so if you have a bogus setting in BORG_PASSPHRASE, then BORG_PASSCOMMAND will probably be ignored and things won't work.

[ThomasWaldmann]

Related: #8496

[andreaseberharter]

Thank you all for your help.

Unfortunately I continue to be unable to run borg info even if I set the passphrase like export BORG_PASSPHRASE=`cat $HOME/.config/borg/.borg-passphrase` or export BORG_PASSPHRASE="super long borg-passphrase"

I am able to access the repro via borgmatic though, where I have the following in the config.yaml:
encryption_passphrase: export BORG_PASSCOMMAND="cat $HOME/.config/borg/.borg-passphrase"

I guess I will try to start from scratch with a simpler passphrase and see if that works.

As my current passphrase is not accepted I also can not change it with borg - borgmatic does not have the option key change-passphrase as far as I know.

It would be good though, to be able to access the repro directly from borg without borgmatic, because also Vorta is throwing the same error.

[ThomasWaldmann]

Does borgmatic do anything special with the passphrase, @witten?

[witten]

borgmatic doesn't do anything special with the passphrase—other than it being interpreted through YAML if it's defined directly in the configuration file. But that shouldn't be the case with the BORG_PASSPHRASE environment variable, which is passed through to Borg directly.

Are you sure your shell isn't interpreting the passphrase though? And you're not using the borgmatic container, are you? There's a known issue with BORG_PASSPHRASE and special characters that's hopefully soon to be fixed.

I am able to access the repro via borgmatic though, where I have the following in the config.yaml:
encryption_passphrase: export BORG_PASSCOMMAND="cat $HOME/.config/borg/.borg-passphrase"

I assume you mean encryption_passcommand here? encryption_passphrase must contain either the literal passphrase or, say, a reference to an environment variable or constant containing it. Also, there's no need to export an environment variable with encryption_passcommand... It's supposed to be a command that returns the passphrase on stdout. (And under the hood, borgmatic passes that command to Borg via environment variable.) See both the borgmatic docs and the Borg docs for more information.

As my current passphrase is not accepted I also can not change it with borg - borgmatic does not have the option key change-passphrase as far as I know.

It will in the upcoming 1.9.0 release!

[andreaseberharter]

Thank you @witten, indeed I am using encryption_passphrase: ... in my config.yaml.

Here is the config from the Ubuntu 24.04 client:

source_directories:
    - /home
    - /etc
    - /var/www

repositories:
    - path: ssh://[email protected]/./repro.borg
      label: storagebox

exclude_patterns:
    - /home/*/.cache

encryption_passphrase: export BORG_PASSCOMMAND="cat $HOME/.config/borg/.borg-passphrase"

ssh_command: ssh -p 23

keep_daily: 7
keep_weekly: 4
keep_monthly: 6

checks:
    - name: repository
    - name: archives
      frequency: 2 weeks

mariadb_databases:
    - name: database
      username: username
      password: asdfasdf
      format: sql

healthchecks:
    ping_url: https://hc-ping.com/####

The config.yaml on my macOS client looks the same but only includes repositories: and encryption_passphrase

I installed borg 1.4.0 as a standalone binary:

wget https://github.com/borgbackup/borg/releases/download/1.4.0/borg-linux-glibc236
cp borg-linux-glibc236 /usr/local/bin/borg 
etc.

and borgmatic as described in other ways to install

apt install borgmatic
borgmatic --version
1.8.3

So, I do have a configuration error in my config.yaml, thank you for pointing that out. I will update the configs and try again.

I am grateful for your help and sorry to keep you all occupied without any real contribution... but it remains a mystery why borgmatic works and borg does not.

Is there anything I could try to debug this?

[witten]

I actually wouldn't expect borgmatic to work with this configuration, so I'm not sure what's going on here! That's because encryption_passphrase does not accept a command. Rather, it must be the literal passphrase (either inline or interpolated with a variable). If you'd like to use a command, then I recommend using encryption_passcommand instead. But again, you don't need to export anything. borgmatic takes care of that for you.

[witten]

I just realized something.. Did you by chance initialize your repo with this configuration? Because if so, then your passphrase may be literally set to the string export BORG_PASSCOMMAND="cat $HOME/.config/borg/.borg-passphrase" and not the contents of ~/.config/borg/.borg-passphrase. If this is true, this would also explain why your passphrase isn't working with Borg.

My recommendation is first verify this theory and, if your passphrase is indeed that string, change it immediately because you've shared it on the internet. 😄 And plus, if you get borgmatic and Borg using the same passphrase, then that would likely solve your issue.

[andreaseberharter]

Haha, yes I did initialize the repro with this configuration!

How do I export the string with double quotes as my passphrase though? Like this:
export BORG_PASSPHRASE=export BORG_PASSCOMMAND="cat $HOME/.config/borg/.borg-passphrase"

[witten]

I would expect single quotes to handle that in most shells:

export BORG_PASSPHRASE='export BORG_PASSCOMMAND="cat $HOME/.config/borg/.borg-passphrase"'

[andreaseberharter]

Wonderful, that worked! Thank you so much, it all makes sense now :D

[andreaseberharter]

Passphrase is changed too :) Thank you again for your help!