Can the fallback being AverageValue metric type cause the HPA to scale down below the fallback value

[RRethy]

Since the fallback uses an AverageValue metric, and AverageValue metrics work by doing a label selector then filtering some of them out, during a deploy with maxUnavailable 0% and maxSurge 100% we could potentially have twice as many available pods which would cause the HPA to think we need to scale down to a value below the fallback.

I also looked at the source code for the HPA indicates it looks at the scale sub-resource on the deployment to get available pods which I also would think could result in a scale down.

I haven't been able to recreate this in testing, so I'm wondering if I'm missing something.