gRPC headers are not proxied correctly

[bosh-bs]

Describe the bug

The setup is the following: a gRPC server that is behind YARP, and a client that calls the server.
The server has a gRPC bidirectional streaming method exposed. Before the messages are sent on this call, the client and the server have to perform the header exchange:

1. The client sends the headers to the server.
2. The server sends the headers to the client.
3. After the client receives the headers from the server, a regular message exchange starts (e.g., the client sends a message to the server, etc.).

What I observe is that step (3) never happens, as the client never receives the headers from the server.

If the client calls the server directly (without YARP), then step (3) happens, and the call procceeds.

To Reproduce

I've implemented a minimal repro here:
https://github.com/bosh-3shape/YarpGrpcHeadersRepro

Further technical details

• YARP.ReverseProxy 2.1.0
• Happens both on Windows and Linux. Have not tried on MacOS.

[MihaZupan]

Thank you for the great repro project!

This does appear to be a YARP issue where we're not flushing the response headers until we receive some content.
Relevant section:

reverse-proxy/src/ReverseProxy/Forwarder/HttpForwarder.cs

Lines 269 to 280 in 8b53dac

	// NOTE: it may *seem* wise to call `context.Response.StartAsync()` at this point
	// since it looks like we are ready to send back response headers
	// (and this might help reduce extra delays while we wait to receive the body from the destination).
	// HOWEVER, this would produce the wrong result if it turns out that there is no content
	// from the destination -- instead of sending headers and terminating the stream at once,
	// we would send headers thinking a body may be coming, and there is none.
	// This is problematic on gRPC connections when the destination server encounters an error,
	// in which case it immediately returns the response headers and trailing headers, but no content,
	// and clients misbehave if the initial headers response does not indicate stream end.
	// :: Step 7-B: Copy response body Client ◄-- Proxy ◄-- Destination
	var (responseBodyCopyResult, responseBodyException) = await CopyResponseBodyAsync(destinationResponse.Content, context.Response.Body, activityCancellationSource);