parallelizing classification and stitching

[aryarm]

goal: parallelize the classification of each image and then stitch the classification masks together, rather than applying the classification to a large, stitched image

motivation:

1. the classification step will be much faster and allows us more flexibility when creating pipelines
2. we'll be able to avoid the effect of running the classifier on artifacts created by the stitched image, (like black holes, etc)

problem: the stitching software doesn't tell us how it stitched the images together

challenges:

1. how do we stitch classification masks together geometrically?
2. when classification masks overlap, how should we resolve diverging classifications?

[aryarm]

Potential Solutions

There exists an open-source alternative to Microsoft Composite Image Editor. Hugin works on all platforms and has both a GUI and command-line interface. It can output stitched images as multiple files in which each file is a cropped/transformed version of one of the the original images (see the Hugin FAQ). We could run the classifier on each separate output image and then stitch the masks together the same way we do the output images (ie using tiffcp). This would sidestep challenge 2 above, since the images we classify won't be overlapping to begin with.

Equivalently, we could simply take the large stitched image that we would get from Microsoft Composite Image Editor, break it up into pieces, and run the classifier on each piece. This approach wouldn't let us run the classifier on each original image exactly, but does it matter?

However, there might be a benefit to having some control over how we resolve conflicts that would normally occur from overlapping masks. The two solutions suggested above delegate this problem to the image stitcher, but we should consider whether the classifier performs better when run on the original images rather than the stitched images (because, for example, stitching creates blurred regions).

[aryarm]

@mdonaldsonmatasci, what are your thoughts? Am I misunderstanding the problem in any way?

[mdonaldsonmatasci]

This is a good summary of the problem. A couple notes: one is that the software we were using is not Microsoft Composite Image Editor but Agisoft Photoscan (much more powerful). The second is that there is another motivation that is missing, which is that the stitched map contains artifacts of stitching, e.g. blurriness, black holes. This is why breaking up the stitched image into smaller pieces wouldn't be as good a solution.