This is a tip of an iceberg of better sampling, but the most critical
case. Up-sampling in general may in the implementation allocate a larger
temporary buffer than its input. Of course this makes little semantic
sense here: after all, the actual information can not increase by this.

If one dimension increases while the other decreases the unfortunate
consequence is that callers may somewhat reasonably expect a small
buffer but internally will get a very large buffer. The approach of
swapping sampling orders accordingly (first down, then up) might address
the memory issue but is lossy.

So instead let's fix the most pressing issue: if no information was
present in the input, nothing can be lost and we can pretend to perform
everything by a very small intermediate image.