object_store Azure Government using OAuth

[zadeluca]

Which part is this question about
object_store Azure

Describe your question
I have reviewed this issue #4853 which also pertains to using Azure Government. That example uses shared key authentication. For OAuth (ClientSecretOAuthProvider), I believe it is also necessary to set authority_host to authority_hosts::AZURE_GOVERNMENT in order for the token_url to be correct.

Without this setting, I am currently getting this error "AADSTS900382: Confidential Client is not supported in Cross Cloud request. I understand it is possible to override using MicrosoftAzureBuilder::with_authority_host(), however I am using delta-rs which requires that all configuration is provided though storage_options with keys defined in AzureConfigKey. This currently does not allow setting AuthorityHost.

Does that sound correct? If so, is it possible to expose AuthorityHost through AzureConfigKey?

Additional context
I have only tried in Python using delta-rs. I am going to attempt to use arrow-rs directly in order to verify that setting AuthorityHost does indeed resolve the error mentioned above. I am a Rust beginner so it may take some effort.