add test for transformations

README.md

@@ -51,10 +51,10 @@ Please take a look at [distributed documentation](http://aeon.nervanasys.com/ind
 
 ## Code coverage
 
-    Code coverage in aeon depends on llvm-cov and lcov.
-    Report will be generated in html-coverage-report/index.html
+Code coverage in aeon depends on `llvm-cov` and `lcov`.
+Report will be generated in html-coverage-report/index.html
 
-    Example:
+Example:
 
     sudo apt-get install llvm lcov
     mkdir build
@@ -64,17 +64,17 @@ Please take a look at [distributed documentation](http://aeon.nervanasys.com/ind
     # If you want to generate report when unit test fails: make -i coverage
     make coverage
 
-### To install Aeon:
+## To install Aeon:
 
     git clone https://github.com/NervanaSystems/aeon.git
 
-##### For Python 2.7
+### For Python 2.7
 
     cd aeon
     pip install -r requirements.txt
     mkdir -p build && cd $_ && cmake .. && pip install .
 
-##### For Python 3.n
+### For Python 3.n
 
     cd aeon
     pip3 install -r requirements.txt
@@ -85,6 +85,16 @@ Note: if installing system wide (as opposed to within a virtual environment) you
 Now continue on to the [user guide](http://aeon.nervanasys.com/index.html/user_guide.html) to get started using aeon. Or to the
 [developer guide](http://aeon.nervanasys.com/index.html/developer_guide.html) to developing custom loaders/transformers.
 
+## Unit Testing
+To run unit tests after building with make, execute inside `build` directory:
+
+    make runtest
+
+To run python tests execute inside `test` directory:
+
+    pytest
+
+
 # Documentation
 
 The complete documentation for aeon is available [here](http://aeon.nervanasys.com).

doc/source/augment_image.rst

@@ -24,7 +24,7 @@ aeon performs a series of customizable transformations on the image before provi
 1. Rotate the image by a random angle drawn from a uniform distribution between the provided ranges (parameter ``angle (int, int)``), and fill the padded regions with zeros.
 2. Resize the image proportionally making the shorter dimension equal to the provided size (parameter ``resize_short_size``).
 3. Take a random crop of the image. The size of the crop is controlled by the parameters ``scale`` and ``do_area_scale``. Suppose the width is the short-side of the input image. By default, the crop width will then be a ``scale`` fraction of the width of the image. Optionally, if ``do_area_scale`` is enabled, then total area of the crop will be a ``scale`` fraction of the input image area. If ``resize_short_size`` was specified, the scale is relative to the resized image. The proportions of the crop box match that of the output shape, unless horizontal_distortion is required.
-4. Adds padding for the image with ``padding`` number of pixels on each side. It then takes random crop of the padded image. The cropbox size is equal to original image size (not changing the output image size then). It cannot be combined with previous crop step, so `crop_enabled` has to be set to `false`.
+4. Adds padding for the image with ``padding`` number of pixels on each side. It then takes random crop of the padded image. The cropbox size is equal to original image size (not changing the output image size then). It cannot be combined with previous crop step, so `crop_enable` has to be set to `false`.
 5. Resize the cropped image to the desired output shape, defined by the parameters ``height`` and ``width``.
 6. If required, apply any transformations (e.g. lighting, horizontal flip, photometric distortion)
 7. Standardize the image (:math:`(I/255 - mean) / stddev`).
@@ -36,17 +36,17 @@ aeon performs a series of customizable transformations on the image before provi
    :escape: ~
 
    flip_enable (bool) | False | Apply horizontal flip with probability 0.5.
-   scale (float, float) | (1, 1) | Fraction of image short-side length to take the crop.
-   do_area_scale (bool) | False | Determine the crop size as a fraction of total area instead of short-side length.
    angle (int, int) | (0, 0) | Rotate image by a random angle drawn from the provided ranges. Angle is in degrees.
    lighting (float, float) | (0.0, 0.0) |  Mean and Standard deviation for distribution of values used as input to colorspace perturbation as described in  Krizhevksy et al.  Default value of 0.0 std means that no colorspace perturbation is applied.  Otherwise, three random variables are drawn from the normal distribution and used to create a pixel color bias based on pre-trained principal components.
    horizontal_distortion (float, float) | (1, 1) | Change the aspect ratio by scaling the image width by a random factor.
    contrast (float, float) | (1.0, 1.0) |  Boundaries of a uniform distribution from which to draw a contrast adjustment factor.  A contrast adjustment factor of 1.0 results in no change to the contrast of the image.  Values less than 1 decrease the contrast, while values greater than 1 increase the contrast.  Recommended boundaries for random contrast perturbation are (0.9 and 1.1).
    brightness (float, float) | (1.0, 1.0) | Boundaries of a uniform distribution from which to draw a brightness adjustment factor.  A brightness adjustment factor of 1.0 results in no change to the brightness of the image.  Values less than 1 decrease the brightness, while values greater than 1 increase the brightness.  Recommended boundaries for random brightness perturbation are (0.9 and 1.1).
-   hue (int) | (0, 0) | Boundaries of a uniform distribution from which to draw a hue adjustment factor. Factors are multiples of 2 degrees, so a full turn is 180. Recommented boundaries for random hue shift are (-18, 18).
+   hue (int, int) | (0, 0) | Boundaries of a uniform distribution from which to draw a hue adjustment factor. Factors are multiples of 2 degrees, so a full turn is 180. Recommented boundaries for random hue shift are (-18, 18).
    saturation (float, float) | (1.0, 1.0) | Boundaries of a uniform distribution from which to draw a saturation adjustment factor.  A saturation adjustment factor of 1.0 results in no change to the saturation of the image.  Values less than 1 decrease the saturation, while values greater than 1 increase the saturation.  Recommended boundaries for random saturation perturbation are (0.9 and 1.1)
+   crop_enable (bool) | True | Crop the input image. using ``center``, ``scale``, ``do_area_scale``.
    center (bool) | False | Take the center crop of the image. If false, a randomly located crop will be taken.
-   crop_enable (bool) | True | Crop the input image. using ``center``, ``scale`` and ``do_area_scale``
+   scale (float, float) | (1, 1) | Fraction of image short-side length to take the crop.
+   do_area_scale (bool) | False | Determine the crop size as a fraction of total area instead of short-side length.
    padding (int) | 0 | Size of the padding added to the image. Cannot be combined with ``crop_enable`` set to ``true``, because it crops the padded image to adjust to the original input image size.
    fixed_aspect_ratio (bool) | False | Maintain fixed aspect ratio when copying the image to the output buffer. This may result in padding of the output buffer.
    fixed_scaling_factor (float) | | If set, uses fixed scaling factor for output image size.

src/augment_image.cpp

@@ -67,6 +67,13 @@ augment::image::param_factory::param_factory(nlohmann::json js)
                     "method in augmentations.");
             }
 
+            if (!crop_origin.empty() && (crop_origin.size() != 2 || crop_origin[0] < 0 || crop_origin[1] < 0))
+            {
+                throw invalid_argument(
+                    "Invalid crop_origin provided. This argument is for testing only - do not use "
+                    "it! It accepts two non-negative integers.");
+            }
+
             // Now fill in derived
             if (flip_enable)
             {
@@ -198,16 +205,24 @@ shared_ptr<augment::image::params> augment::image::param_factory::make_params(
 
             std::uniform_real_distribution<float> scale2{scale_min, scale_max};
 
-            float target_area = sqrt(input_height * input_width * scale2(random));
-            out_shape.width  *= target_area;
-            out_shape.height *= target_area;
+            float target_size = sqrt(input_height * input_width * scale2(random));
+            cv::Size2f cropbox_size = out_shape * target_size;
 
             float c_off_x = crop_offset(random);
             float c_off_y = crop_offset(random);
 
-            cv::Size2f cropbox_size = out_shape;
-            cv::Point2i cropbox_origin = nervana::image::cropbox_shift(input_size, cropbox_size, c_off_x, c_off_y);
-            settings->cropbox = cv::Rect(cropbox_origin, cropbox_size);
+            cv::Point2f cropbox_origin;
+            if (crop_origin.size() == 2) {
+                cropbox_origin.x = crop_origin[0];
+                cropbox_origin.y = crop_origin[1];
+            } else {
+                cropbox_origin = nervana::image::cropbox_shift(input_size, cropbox_size, c_off_x, c_off_y);
+            }
+            // explicitly round cropbox_origin and cropbox_size to nearest integer
+            settings->cropbox = cv::Rect(std::round(cropbox_origin.x),
+                                         std::round(cropbox_origin.y),
+                                         std::round(cropbox_size.width),
+                                         std::round(cropbox_size.height));
         }
         else
         {
@@ -243,9 +258,18 @@ shared_ptr<augment::image::params> augment::image::param_factory::make_params(
             float c_off_x = crop_offset(random);
             float c_off_y = crop_offset(random);
 
-            cv::Point2i cropbox_origin =
-                nervana::image::cropbox_shift(input_size, cropbox_size, c_off_x, c_off_y);
-            settings->cropbox = cv::Rect(cropbox_origin, cropbox_size);
+            cv::Point2f cropbox_origin;
+            if (crop_origin.size() == 2) {
+                cropbox_origin.x = crop_origin[0];
+                cropbox_origin.y = crop_origin[1];
+            } else {
+                cropbox_origin = nervana::image::cropbox_shift(input_size, cropbox_size, c_off_x, c_off_y);
+            }
+            // explicitly round cropbox_origin and cropbox_size to nearest integer
+            settings->cropbox = cv::Rect(std::round(cropbox_origin.x),
+                                         std::round(cropbox_origin.y),
+                                         std::round(cropbox_size.width),
+                                         std::round(cropbox_size.height));
         }
     }
 

src/augment_image.hpp

@@ -146,6 +146,8 @@ class nervana::augment::image::param_factory : public json_configurable
 
     bool                do_area_scale                 = false;
     bool                crop_enable                   = true;
+    // crop_origin is for testing only. Do not use it!
+    std::vector<int>    crop_origin                   = {};
     bool                fixed_aspect_ratio            = false;
     std::vector<double> mean                          = {};
     std::vector<double> stddev                        = {};
@@ -235,6 +237,7 @@ class nervana::augment::image::param_factory : public json_configurable
         ADD_SCALAR(interpolation_method, mode::OPTIONAL),
         ADD_SCALAR(do_area_scale, mode::OPTIONAL),
         ADD_SCALAR(crop_enable, mode::OPTIONAL),
+        ADD_SCALAR(crop_origin, mode::OPTIONAL),
         ADD_SCALAR(expand_probability, mode::OPTIONAL),
         ADD_SCALAR_WITH_KEY(m_emit_constraint_type, "emit_constraint_type", mode::OPTIONAL),
         ADD_SCALAR_WITH_KEY(

src/etl_image.cpp

@@ -145,9 +145,12 @@ shared_ptr<image::decoded>
 cv::Mat image::transformer::transform_single_image(shared_ptr<augment::image::params> img_xform,
                                                    cv::Mat& single_img) const
 {
-    // img_xform->dump(cout);
     cv::Mat rotatedImage;
-    image::rotate(single_img, rotatedImage, img_xform->angle);
+    if (img_xform->angle % 360 == 0) {
+        rotatedImage = single_img;
+    } else {
+        image::rotate(single_img, rotatedImage, img_xform->angle);
+    }
 
     cv::Mat expandedImage;
     if (img_xform->expand_ratio > 1.0)
@@ -168,7 +171,6 @@ cv::Mat image::transformer::transform_single_image(shared_ptr<augment::image::pa
 
     cv::Mat croppedImage = resizedShortImage(img_xform->cropbox);
     image::add_padding(croppedImage, img_xform->padding, img_xform->padding_crop_offset);
-
     cv::Mat resizedImage;
     image::resize(croppedImage,
                   resizedImage,

src/image.cpp

@@ -53,25 +53,18 @@ namespace
 void image::rotate(
     const cv::Mat& input, cv::Mat& output, int angle, bool interpolate, const cv::Scalar& border)
 {
-    if (angle == 0)
+    cv::Point2i pt(input.cols / 2, input.rows / 2);
+    cv::Mat     rot = cv::getRotationMatrix2D(pt, angle, 1.0);
+    int         flags;
+    if (interpolate)
     {
-        output = input;
+        flags = cv::INTER_LINEAR;
     }
     else
     {
-        cv::Point2i pt(input.cols / 2, input.rows / 2);
-        cv::Mat     rot = cv::getRotationMatrix2D(pt, angle, 1.0);
-        int         flags;
-        if (interpolate)
-        {
-            flags = cv::INTER_LINEAR;
-        }
-        else
-        {
-            flags = cv::INTER_NEAREST;
-        }
-        cv::warpAffine(input, output, rot, input.size(), flags, cv::BORDER_CONSTANT, border);
+        flags = cv::INTER_NEAREST;
     }
+    cv::warpAffine(input, output, rot, input.size(), flags, cv::BORDER_CONSTANT, border);
 }
 
 void image::add_padding(cv::Mat& input, int padding, cv::Size2i crop_offset)
@@ -261,12 +254,12 @@ cv::Size2f image::cropbox_area_scale(const cv::Size2f& in_size,
     return result;
 }
 
-cv::Point2i image::cropbox_shift(const cv::Size2f& in_size,
+cv::Point2f image::cropbox_shift(const cv::Size2f& in_size,
                                  const cv::Size2f& crop_box,
                                  float             xoff,
                                  float             yoff)
 {
-    cv::Point2i result;
+    cv::Point2f result;
     result.x = (in_size.width - crop_box.width) * xoff;
     result.y = (in_size.height - crop_box.height) * yoff;
     return result;

src/image.hpp

@@ -61,7 +61,7 @@ namespace nervana
         cv::Size2f cropbox_area_scale(const cv::Size2f& in_size,
                                       const cv::Size2f& cropbox_size,
                                       float             scale);
-        cv::Point2i cropbox_shift(const cv::Size2f&, const cv::Size2f&, float, float);
+        cv::Point2f cropbox_shift(const cv::Size2f&, const cv::Size2f&, float, float);
 
         class photometric
         {

test/test_data/manifest_for_compare.tsv

@@ -0,0 +1,13 @@
+@FILE	STRING
+flowers.jpg	0
+img_2112_70.jpg	1
+mushroom.jpg	2
+climbing.jpg	3
+deer_head.jpg	4
+fruits.jpg	5
+glass.jpg	6
+mountain_town.jpg	7
+squirrel.jpg	8
+target.jpg	9
+toys.jpg	10
+welding.jpg	11