Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add sample/interpolate nearest #1962

Merged
merged 1 commit into from
Aug 4, 2023
Merged

Add sample/interpolate nearest #1962

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion src/imageops/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,8 @@ pub use self::affine::{

/// Image sampling
pub use self::sample::{
blur, filter3x3, interpolate_bilinear, resize, sample_bilinear, thumbnail, unsharpen,
blur, filter3x3, interpolate_bilinear, interpolate_nearest, resize, sample_bilinear,
sample_nearest, thumbnail, unsharpen,
};

/// Color operations
Expand Down
84 changes: 82 additions & 2 deletions src/imageops/sample.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -304,7 +304,7 @@ where
out
}

/// Linearly bisample from an image using coordinates in [0,1].
/// Linearly sample from an image using coordinates in [0,1].
pub fn sample_bilinear<P: Pixel>(
img: &impl GenericImageView<Pixel = P>,
u: f32,
Expand All @@ -328,6 +328,45 @@ pub fn sample_bilinear<P: Pixel>(
)
}

/// Sample from an image using coordinates in [0,1], taking the nearest coordinate.
pub fn sample_nearest<P: Pixel>(
img: &impl GenericImageView<Pixel = P>,
u: f32,
v: f32,
) -> Option<P> {
if ![u, v].iter().all(|c| (0.0..=1.0).contains(c)) {
return None;
}

let (w, h) = img.dimensions();
let ui = w as f32 * u - 0.5;
let ui = ui.max(0.).min((w.saturating_sub(1)) as f32);

let vi = h as f32 * v - 0.5;
let vi = vi.max(0.).min((h.saturating_sub(1)) as f32);
interpolate_nearest(img, ui, vi)
}

/// Linearly bisample from an image using coordinates in [0,w-1] and [0,h-1].
pub fn interpolate_nearest<P: Pixel>(
img: &impl GenericImageView<Pixel = P>,
x: f32,
y: f32,
) -> Option<P> {
let (w, h) = img.dimensions();
if w == 0 || h == 0 {
return None;
}
if !(0.0..=((w - 1) as f32)).contains(&x) {
return None;
}
if !(0.0..=((h - 1) as f32)).contains(&y) {
return None;
}

Some(img.get_pixel(x.round() as u32, y.round() as u32))
Comment on lines +360 to +367
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should be rounding before checking whether the index is in bounds. For instance, (-0.3, -0.3) should round to (0,0) and thus be accepted

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If that's the case then, I think that we should also do the same for bilinear interpolation, otherwise it would lead to inconsistent boundary conditions.

Not that we have to go with the convention of glsl, but for reference
https://www.khronos.org/opengl/wiki/Sampler_Object#:~:text=If%20GL_NEAREST%20is%20used%2C%20then,between%20the%20nearest%20adjacent%20samples.

If a user were to implement wrapping outside of this, then -0.3 could be considered as both 0 and w-1

Copy link
Contributor

@fintelia fintelia Jul 22, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I'm realizing that sample_linear is already inconsistent. Bilinear sampling at uv coordinates (0,0) should produce the top left pixel's color if doing "clamp-to-edge" and should be the average of the four corner pixels for "wrapping".

Specifically, the ui.max(0.).min((w - 1) as f32) implements the "clamp to edge".

Copy link
Contributor Author

@JulianKnodt JulianKnodt Jul 22, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you mean inconsistent with interpolate_bilinear or sample_nearest?

It only clamps to edge to handle numerical inconsistencies, but both methods should return None if it's out of bounds.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean that sample_linear already can't be used to implement wrapping sample, since not matter what arguments the user calls it with, they won't be able to interpolate between pixels on opposite sides of the image.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's true... One alternative is to provide a separate function but that seems not as clean (akin to wrapping add, saturating, checked).
There was also providing an enum as an option, which forces the user to think about it, but may be unfriendly to people learning the library.

I looked at how Vulkan handles sampling, and they have an object with a number of settings to sample from an image. Maybe it makes sense to attach these to an object which has sensible defaults (clamp to border), but can be changed if the user needs precise control. It will also be easier to modify in the future from a backcompat perspective if new features need to be added.

}

/// Linearly bisample from an image using coordinates in [0,w-1] and [0,h-1].
pub fn interpolate_bilinear<P: Pixel>(
img: &impl GenericImageView<Pixel = P>,
Expand Down Expand Up @@ -961,7 +1000,7 @@ where

#[cfg(test)]
mod tests {
use super::{resize, sample_bilinear, FilterType};
use super::{resize, sample_bilinear, sample_nearest, FilterType};
use crate::{GenericImageView, ImageBuffer, RgbImage};
#[cfg(feature = "benchmarks")]
use test;
Expand Down Expand Up @@ -1006,6 +1045,25 @@ mod tests {
assert!(sample_bilinear(&img, -0.1, -0.1).is_none());
}
#[test]
#[cfg(feature = "png")]
fn test_sample_nearest() {
use std::path::Path;
let img = crate::open(&Path::new("./examples/fractal.png")).unwrap();
assert!(sample_nearest(&img, 0., 0.).is_some());
assert!(sample_nearest(&img, 1., 0.).is_some());
assert!(sample_nearest(&img, 0., 1.).is_some());
assert!(sample_nearest(&img, 1., 1.).is_some());
assert!(sample_nearest(&img, 0.5, 0.5).is_some());

assert!(sample_nearest(&img, 1.2, 0.5).is_none());
assert!(sample_nearest(&img, 0.5, 1.2).is_none());
assert!(sample_nearest(&img, 1.2, 1.2).is_none());

assert!(sample_nearest(&img, -0.1, 0.2).is_none());
assert!(sample_nearest(&img, 0.2, -0.1).is_none());
assert!(sample_nearest(&img, -0.1, -0.1).is_none());
}
#[test]
fn test_sample_bilinear_correctness() {
use crate::Rgba;
let img = ImageBuffer::from_fn(2, 2, |x, y| match (x, y) {
Expand Down Expand Up @@ -1038,6 +1096,28 @@ mod tests {
Some(Rgba([0, 0, 128, 128]))
);
}
#[test]
fn test_sample_nearest_correctness() {
use crate::Rgba;
let img = ImageBuffer::from_fn(2, 2, |x, y| match (x, y) {
(0, 0) => Rgba([255, 0, 0, 0]),
(0, 1) => Rgba([0, 255, 0, 0]),
(1, 0) => Rgba([0, 0, 255, 0]),
(1, 1) => Rgba([0, 0, 0, 255]),
_ => panic!(),
});

assert_eq!(sample_nearest(&img, 0.0, 0.0), Some(Rgba([255, 0, 0, 0])));
assert_eq!(sample_nearest(&img, 0.0, 1.0), Some(Rgba([0, 255, 0, 0])));
assert_eq!(sample_nearest(&img, 1.0, 0.0), Some(Rgba([0, 0, 255, 0])));
assert_eq!(sample_nearest(&img, 1.0, 1.0), Some(Rgba([0, 0, 0, 255])));

assert_eq!(sample_nearest(&img, 0.5, 0.5), Some(Rgba([0, 0, 0, 255])));
assert_eq!(sample_nearest(&img, 0.5, 0.0), Some(Rgba([0, 0, 255, 0])));
assert_eq!(sample_nearest(&img, 0.0, 0.5), Some(Rgba([0, 255, 0, 0])));
assert_eq!(sample_nearest(&img, 0.5, 1.0), Some(Rgba([0, 0, 0, 255])));
assert_eq!(sample_nearest(&img, 1.0, 0.5), Some(Rgba([0, 0, 0, 255])));
}

#[bench]
#[cfg(all(feature = "benchmarks", feature = "tiff"))]
Expand Down
Loading