Path Mappings and Branch Mappings in MergeTreeBarycenter and MergeTreeDistanceMatrtix

[floWetzels]

This PR adds state and data files for the path mapping distance and branch mapping distance used in the MergeTreeBarycenter and MergeTreeDistanceMatrix modules. The examples come from the VIS23 paper "Merge Tree Geodesics and Barycenters with Path Mappings".

It accompanies the following TTK pull request: topology-tool-kit/ttk#1025

[julien-tierny]

Thanks a lot Florian!
Can you please go over the contributing instructions for this repository (https://github.com/topology-tool-kit/ttk-data/blob/dev/CONTRIBUTING.md) and add the missing items? (e.g. python script, doc, etc.).

Thanks a lot!

[floWetzels]

Thanks a lot Florian! Can you please go over the contributing instructions for this repository (https://github.com/topology-tool-kit/ttk-data/blob/dev/CONTRIBUTING.md) and add the missing items? (e.g. python script, doc, etc.).

Thanks a lot!

A quick question regarding the docs: where are the screenshots supposed to be saved? They are referenced in the other markdown files, but as far as I can see they are not present in the data repository and the references point to the ttk website.

@julien-tierny

[julien-tierny]

hi florian, thanks for your message.
the images are hosted on the ttk website (https://github.com/topology-tool-kit/topology-tool-kit.github.io). you can leave a blank link there for the moment.

[floWetzels]

ok, thanks! so they will be generated automatically?

[julien-tierny]

not exactly, but we can generate them after the fact.

[floWetzels]

Ok, sounds reasonable.

One more question: could you have a look at the current python states? Since the output of the MergeTreeClustering is a vtk multiblock, I had to save the barycenter and mapping in that format, which is not very nice in my opinion. Would you prefer to keep it like this or to add some code that extracts the unstructured grids? I am not sure if we should prefer a cleaner output or the simplicity of the python code. (I think csv outputs are not reasonable here, as not all scalars seem to be saved.)

[julien-tierny]

Hi Florian,
thanks for your input.
I'm starting to play with your state files. I have a question:

• For the state mergeTreeBarycenter_branchMapping.pvsm, the matching is hard to interpret visually (see the screenshot below). Isn't there a way to play with the visualization parameters, such that the sub-tree of the foreground trees gets displayed to the left (instead of the right)? That would minimize the crossings within the matchings, which would help for the understanding I guess.
  

thanks for your feedback!

[julien-tierny]

here's a quick review.

[julien-tierny]

It looks like the first output is the mapping (based on the above text and the python script).
Is that correct?

[floWetzels]

Yes, the description wasn't correct here. The file name was wrong, too. I fixed it now. Thanks!

[floWetzels]

Hi Florian, thanks for your input. I'm starting to play with your state files. I have a question:

* For the state `mergeTreeBarycenter_branchMapping.pvsm`, the matching is hard to interpret visually (see the screenshot below). Isn't there a way to play with the visualization parameters, such that the sub-tree of the foreground trees gets displayed to the left (instead of the right)? That would minimize the crossings within the matchings, which would help for the understanding I guess.
  ![Screenshot_20240506_082702](https://private-user-images.githubusercontent.com/3018628/328095514-5b91c6b6-d3f7-400a-b3e7-b55f84bb91de.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTQ5OTQ0OTYsIm5iZiI6MTcxNDk5NDE5NiwicGF0aCI6Ii8zMDE4NjI4LzMyODA5NTUxNC01YjkxYzZiNi1kM2Y3LTQwMGEtYjNlNy1iNTVmODRiYjkxZGUucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDUwNiUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA1MDZUMTExNjM2WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9ODFmMDEyMWUzNzhkZTk2ODAxODliMzI0MGFhN2FhNDYwYTIyMjIzZGZlMmU3MGViY2I2MTVjOWJkY2I5YmEwZSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.ZuPIne7aPACIkBYMkoXiOouUG0JiFGjxdVF4-OHcjss)

thanks for your feedback!

Good point, I will try some different setups. I think the layout parameters won't do a lot, but I could try extracting different members of the ensemble.

[julien-tierny]

Hi Florian, thanks for your input. I'm starting to play with your state files. I have a question:

* For the state `mergeTreeBarycenter_branchMapping.pvsm`, the matching is hard to interpret visually (see the screenshot below). Isn't there a way to play with the visualization parameters, such that the sub-tree of the foreground trees gets displayed to the left (instead of the right)? That would minimize the crossings within the matchings, which would help for the understanding I guess.
  ![Screenshot_20240506_082702](https://private-user-images.githubusercontent.com/3018628/328095514-5b91c6b6-d3f7-400a-b3e7-b55f84bb91de.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTQ5OTQ0OTYsIm5iZiI6MTcxNDk5NDE5NiwicGF0aCI6Ii8zMDE4NjI4LzMyODA5NTUxNC01YjkxYzZiNi1kM2Y3LTQwMGEtYjNlNy1iNTVmODRiYjkxZGUucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDUwNiUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA1MDZUMTExNjM2WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9ODFmMDEyMWUzNzhkZTk2ODAxODliMzI0MGFhN2FhNDYwYTIyMjIzZGZlMmU3MGViY2I2MTVjOWJkY2I5YmEwZSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.ZuPIne7aPACIkBYMkoXiOouUG0JiFGjxdVF4-OHcjss)

thanks for your feedback!

Good point, I will try some different setups. I think the layout parameters won't do a lot, but I could try extracting different members of the ensemble.

Maybe you can try to play with the "Important Pair Threshold"? (to make the front subtree switch from the right to the left side)

[floWetzels]

Maybe you can try to play with the "Important Pair Threshold"? (to make the front subtree switch from the right to the left side)

Yes that worked, I have pushed the updates.

[floWetzels]

Another question regarding the outputs of the python scripts: Are they just for testing/CI purposes or should they also yield nice renderings? Because at the moment I only went for objects that are useful for verifying the output, e.g. only the mapping in the branch mapping example.

[julien-tierny]

Another question regarding the outputs of the python scripts: Are they just for testing/CI purposes or should they also yield nice renderings? Because at the moment I only went for objects that are useful for verifying the output, e.g. only the mapping in the branch mapping example.

for the python script, I guess the idea is to output something that is useful for an analysis in batch mode (irrespective of the actual visualization). the barycenter (or the matchings) and the distance matrices are good in that perspective.

[julien-tierny]

Thanks for the quick updates!

For the branchMapping example, there's something that is not clear to me.
You use MergeTreeClustering with 10 trees on its input, right? But as far as I understand, it cannot compute a barycenter, then it just computes a matching between 2 of these 10 input trees. Is that correct? If so, how do you choose which trees to compare?

[floWetzels]

Thanks for the quick updates!

For the branchMapping example, there's something that is not clear to me. You use MergeTreeClustering with 10 trees on its input, right? But as far as I understand, it cannot compute a barycenter, then it just computes a matching between 2 of these 10 input trees. Is that correct? If so, how do you choose which trees to compare?

Yes, that's correct. I think it's always the first two trees, but maybe @MatPont knows more. The same behavior is applied for the original Merge Tree Edit Distance, or in general if the computeBarycenter flag is set to false. Maybe that's something we should change or maybe it needs clarification in the documentation, I don't know.

I chose to keep the whole ensemble to have a unified input for the two BDIED examples. I also think it's a good way to showcase how the filter operates without the computeBarycenter flag.

Do you want to keep the example as is or do you want me to change it?

[julien-tierny]

I think the example is good as is, but it'd be nice to clarify the behavior of the filter in the documentation (i.e., for these backends which do not support barycenter computation, only a matching between the first two trees will be computed).
this could be clarified in the paraview filter xml, the headers of the vtk filter, and the header of the base code.

[floWetzels]

Shouldn't it be formulated differently? This behavior happens for all distances if the computeBarycenter flag is not set. You can check that in the state file by first selecting a supported distance and then setting the flag to false. Then you get the same behavior with any distance.

I think at the moment, if the user want to compute a barycenter with an unsupported distance, the output is simply empty and an error message is printed to console. The parameters are no longer corrected automatically. This is actually something that @MatPont wanted to discuss with you, as we run into some weird workflows when deactivating specific parameter combinations.

[julien-tierny]

Hi @floWetzels ,

sorry it took me some time to come back to this review.
I've been playing with your paraview state files, and I found two weird things (cf screenshot)

1. For the "Matchings" output, the "Field Data" should contain the actual distance between the considered trees (in the screenshot: 43.2154, bottom left, second spreadsheet view from the bottom). However, when I look at the numerical values of the distance matrix (bottom right), I don't find this value anywhere :( Is the Field Data correctly set by your code?
2. For the "Matchings" output, the Cell Data Array "Cost" is supposed to contain the cost for its matching. But with your backend, this value is at 0 for all cells. Could that be fixed in the code?

Thanks a lot for your feedback!
Best,

[julien-tierny]

The same seems to happen for the path mapping backend...

[floWetzels]

Hi @julien-tierny, good catch!

Regarding the field data, I am really confused. I didn't even know about that feature and never touched it. For me, the problem actually appears for all four distances. I had a quick look at the vtk layer code and didn't find anything odd. Could be that something is wrong with the parameters in the state file, it uses two different filters after all. But I couldn't find anything there neither... @MatPont Do you have any idea on this?

Regarding the cost array, I think this was a deliberate choice. I don't remember it exactly, but I think it had something to do with the incompatible matching types and their conversion. I am not sure though, could also be a bug. I you want me to, I could have a look next week. I don't know if I'll find the time during EuroVis, but I can try. Just let me know.

[julien-tierny]

hi @floWetzels ,
thanks for your feedback. yes, that'd be great if you could have a look into these two things!

[floWetzels]

Hi @julien-tierny,
I think I found the reason for the field data inconsistencies. I have to talk to @MatPont though, something seems really wrong. So it might take some time.

[MatPont]

After discussing with @floWetzels it seems that all current todos are done, and the problems discussed in this PR are fixed except one thing that @floWetzels will take care of.

Basically, the cost array in the matching output for the branch mapping backend is not filled for the interior nodes (it is only for the leaves and the root for the moment), their values should be copied from the corresponding leaf of their branch.
Moreover, line 639 of core/base/mergeTreeClustering/BranchMappingDistance.h the cost array is defined as a vector of int instead of a vector of double. (std::vector<int> matchedCost(tree1->getNumberOfNodes(), -1);)

[julien-tierny]

thanks a lot for the feedback!
please let me know when these final edits are in.
take care!

[floWetzels]

I think all issues are fixed now. Can you have another look, @julien-tierny?