Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Auto-configure the Postgres application_name when using Docker Compose #42460

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

nosan
Copy link
Contributor

@nosan nosan commented Sep 26, 2024

gh-40772

I added since 3.4.0 everywhere but maybe it could be merged earlier if everything is ok.

Thanks in advance

@spring-projects-issues spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged label Sep 26, 2024
@nosan nosan force-pushed the 40772 branch 3 times, most recently from 2ac675e to 381e85a Compare September 27, 2024 07:24
@nosan
Copy link
Contributor Author

nosan commented Sep 27, 2024

Should I also configure ApplicationName in PostgresR2dbcDockerComposeConnectionDetailsFactory?

@nosan
Copy link
Contributor Author

nosan commented Sep 27, 2024

Should I also configure ApplicationName in PostgresR2dbcDockerComposeConnectionDetailsFactory?

Auto-configure for r2dbc as well.

@nosan nosan force-pushed the 40772 branch 2 times, most recently from 0bf5cc3 to 6d627e5 Compare September 27, 2024 11:26
Copy link
Member

@snicoll snicoll left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR @nosan. I feel that the testing bits of it are too involved and we should rather attempt to test the logic in simpler tests, if possible. Let us know if you have time to explore that route.

@@ -57,6 +57,38 @@ void runWithBitnamiImageCreatesConnectionDetails(JdbcConnectionDetails connectio
assertConnectionDetails(connectionDetails);
}

@DockerComposeTest(composeFile = "postgres-compose-application-name-label.yaml", image = TestImage.POSTGRESQL,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Those tests shouldn't need to be full integration tests. The only thing that we assert here is that the ConnectionDetails auto-configured bean has the correct state. I can see that we don't have such tests so far but I wonder if it wouldn't be useful to put something in place so that we don't need all the docker machinery.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. For this to warrant being a Docker-based test, I think it should somehow check that setting the application_name has had the desired effect on the Postgres side of things.

I'd then prefer that we only have a single Docker-based test that checks this and that the precedence of the different sources of the application name is unit tested without involving Docker. Right now we're adding a lot to the build time and I don't think the extra test coverage warrants that increase.

@@ -60,6 +61,35 @@ void runWithBitnamiImageCreatesConnectionDetails(R2dbcConnectionDetails connecti
assertConnectionDetails(connectionDetails);
}

@DockerComposeTest(composeFile = "postgres-compose-application-name-label.yaml", image = TestImage.POSTGRESQL,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto. I can see this also changes the test infrastructure quite a bit and I wonder if that's warranted for this change.

return appendApplicationName(parameters);
}

private String appendApplicationName(String parameters) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is quite involved. I wonder if there wouldn't be a way to simplify this. And if not, perhaps getParameters may need to evolve so that such a handling of the raw string is unnecessary.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will take a look at how it could be done better and easy to read. I have the same feeling that it looks quite messy.

@snicoll snicoll added the status: waiting-for-feedback We need additional information before we can continue label Sep 30, 2024
@@ -57,6 +57,38 @@ void runWithBitnamiImageCreatesConnectionDetails(JdbcConnectionDetails connectio
assertConnectionDetails(connectionDetails);
}

@DockerComposeTest(composeFile = "postgres-compose-application-name-label.yaml", image = TestImage.POSTGRESQL,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. For this to warrant being a Docker-based test, I think it should somehow check that setting the application_name has had the desired effect on the Postgres side of things.

I'd then prefer that we only have a single Docker-based test that checks this and that the precedence of the different sources of the application name is unit tested without involving Docker. Right now we're adding a lot to the build time and I don't think the extra test coverage warrants that increase.

@nosan
Copy link
Contributor Author

nosan commented Sep 30, 2024

Thank you for your feedback @snicoll and @wilkinsona

  1. I can add unit tests for PostgresJdbcDockerComposeConnectionDetails and PostgresDbR2dbcDockerComposeConnectionDetails without involving docker, to verify that all options are set correctly.
  2. I can add a docker integration test for PostgresJdbcDockerComposeConnectionDetailsFactory and PostgresR2dbcDockerComposeConnectionDetailsFactory to verify the application name is correctly set.
	@DockerComposeTest(composeFile = "postgres-compose.yaml", image = TestImage.POSTGRESQL,
			properties = "spring.application.name=my-app")
	void runCreatesConnectionDetailsWithAutoConfiguredApplicationName(JdbcConnectionDetails connectionDetails)
			throws ClassNotFoundException {
		assertThat(connectionDetails.getUsername()).isEqualTo("myuser");
		assertThat(connectionDetails.getPassword()).isEqualTo("secret");
		assertThat(connectionDetails.getJdbcUrl()).startsWith("jdbc:postgresql://").endsWith("?ApplicationName=my-app");
		assertThat(getApplicationName(connectionDetails)).isEqualTo("my-app");
	}

	private String getApplicationName(JdbcConnectionDetails connectionDetails) throws ClassNotFoundException {
		JdbcTemplate template = getJdbcTemplate(connectionDetails);
		return template.queryForObject("select current_setting('application_name')", String.class);
	}

and

@DockerComposeTest(composeFile = "postgres-compose.yaml", image = TestImage.POSTGRESQL,
			properties = "spring.application.name=my-app")
	void runCreatesConnectionDetailsWithAutoConfiguredApplicationName(R2dbcConnectionDetails connectionDetails) {
		assertConnectionDetails(connectionDetails);
		assertThat(getApplicationName(connectionDetails)).isEqualTo("my-app");
	}

	private String getApplicationName(R2dbcConnectionDetails connectionDetails) {
		ConnectionFactoryOptions connectionFactoryOptions = connectionDetails.getConnectionFactoryOptions();
		return DatabaseClient.create(ConnectionFactories.get(connectionFactoryOptions))
			.sql("select current_setting('application_name')")
			.map((row, metadata) -> row.get(0))
			.first()
			.map(Objects::toString)
			.block(Duration.ofSeconds(30));
	}

Does it make sense?

@spring-projects-issues spring-projects-issues added status: feedback-provided Feedback has been provided and removed status: waiting-for-feedback We need additional information before we can continue labels Sep 30, 2024
@nosan
Copy link
Contributor Author

nosan commented Sep 30, 2024

Also, should I revert my DockerComposeTestExtension changes? I can test the application name is set via docker-compose labels.

@nosan nosan force-pushed the 40772 branch 2 times, most recently from 35a5ae3 to 54ee498 Compare September 30, 2024 19:51
@nosan
Copy link
Contributor Author

nosan commented Sep 30, 2024

PR has been updated.

  • I reverted my DockerComposeTestExtension and JdbcUrlBuilder changes.
  • I added unit tests for both ConnectionDetails and removed docker-compose integration tests.
  • I left only two docker-compose integration tests for checking that application_name had the desired effect on the Postgres side.

@nosan nosan requested review from wilkinsona and snicoll September 30, 2024 19:58
@nosan nosan force-pushed the 40772 branch 3 times, most recently from 2e0e3a5 to 3a5a70a Compare October 6, 2024 09:55
*
* @author Dmytro Nosan
*/
class PostgresJdbcUrl {
Copy link
Member

@wilkinsona wilkinsona Oct 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We discussed this today and it feels a bit overly complicated to us. We think the same could be achieved with something similar to the code that disables encryption for SQL Server. One addition that's probably needed here is to encode the application name as its user-provided input and may contain characters that are restricted in a query string.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wilkinsona
I initially used this variant

private static String getJdbcUrl(RunningService service, String database, Environment environment) {
			String jdbcUrl = jdbcUrlBuilder.build(service, database);
			return appendApplicationName(jdbcUrl, environment);
		}

private static String appendApplicationName(String jdbcUrl, Environment environment) {
	if (hasParameter(jdbcUrl, "ApplicationName")) {
		return jdbcUrl;
	}
	String applicationName = environment.getProperty("spring.application.name");
	if (!StringUtils.hasText(applicationName)) {
		return jdbcUrl;
	}
	return appendParameter(jdbcUrl, "ApplicationName", applicationName);
}

private static String appendParameter(String jdbcUrl, String name, String value) {
	StringBuilder jdbcUrlBuilder = new StringBuilder(jdbcUrl);
	if (!jdbcUrl.contains("?")) {
		jdbcUrlBuilder.append('?');
	}
	else if (!jdbcUrl.endsWith("&")) {
		jdbcUrlBuilder.append('&');
	}
	return jdbcUrlBuilder.append(name)
			.append('=')
			.append(URLEncoder.encode(value, StandardCharsets.UTF_8))
			.toString();
}

private static boolean hasParameter(String jdbcUrl, String parameterName) {
	return jdbcUrl.contains("?%s=".formatted(parameterName))
			|| jdbcUrl.contains("&%s=".formatted(parameterName));
}
		

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

@nosan nosan requested a review from wilkinsona October 10, 2024 13:17
@nosan nosan force-pushed the 40772 branch 2 times, most recently from 73c945b to 9ad5cd0 Compare October 23, 2024 21:28
@snicoll snicoll removed the status: waiting-for-triage An issue we've not yet triaged label Nov 5, 2024
@snicoll snicoll added type: enhancement A general enhancement and removed status: feedback-provided Feedback has been provided labels Nov 5, 2024
@snicoll snicoll added this to the 3.5.x milestone Nov 5, 2024
@snicoll snicoll self-assigned this Nov 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: enhancement A general enhancement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants