Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add notebook comparing EPIC and quanTIseq #117

Closed
sjspielman opened this issue Jan 13, 2025 · 3 comments · Fixed by #122
Closed

Add notebook comparing EPIC and quanTIseq #117

sjspielman opened this issue Jan 13, 2025 · 3 comments · Fixed by #122
Assignees

Comments

@sjspielman
Copy link
Member

sjspielman commented Jan 13, 2025

Following #116, we will want to add notebook that explore the EPIC results and compare them to quanTIseq.

Part of this comparison, at a minimum, should entail what proportion of "cells" in each dataset were classified vs. remain other/unknown. We'll want to compare this among results for both EPIC references and quanTIseq.

It might also be helpful to bring some of the quanTIseq exploration we performed in the quanTIseq notebook into this one so we can have a clear comparison between methods. Edit: In particular, we'll want a stacked barplot of cell types including "other"; this is not precisely in the other notebook (we have a stacked barplot without other), but we'll do it this way here.

@sjspielman
Copy link
Member Author

I'm beginning to write up the second notebook comparing EPIC and quanTIseq inferences, and I'm not exactly sure what the right way is to compare cell type proportions across references/methods. Since each reference has a different set of cell types, this will influence the values themselves. I had originally thought maybe it makes sense to renormalize each dataset without "other," but since "other" isn't directly comparable between references, I'm not convinced this changes the situation. All the ways I can think to compare feel too much like "apples to oranges" and not very robust since they are each conditioned on different cell type distributions.

Any thoughts on quantitatively comparing individual fractions @allyhawkins? I'm starting to think that maybe we can really only do this qualitatively, and this next notebook would more serve to see all results in one spot. But then again if that's all we can do, maybe it actually suffices to export select PNGs from the individual method exploration notebooks?

@allyhawkins
Copy link
Member

I think you could still compare the "other" fraction and state the caveat that the you may be missing some cell types that are not available in the reference for a specific method.

There are some cell types that are the same across methods (macrophages, B cells, T cells, Monocytes, etc). I would directly compare the proportions that get classified as each of the cell types that overlap.

I also agree that part of the benefit of this notebook is to consolidate the findings. I think with something like this we could pick one we think is doing better and include that in the main figures and put the other method in the supplemental, so having the same figures with both methods will be helpful.

@sjspielman
Copy link
Member Author

There are some cell types that are the same across methods (macrophages, B cells, T cells, Monocytes, etc). I would directly compare the proportions that get classified as each of the cell types that overlap.

This is more or less what I was setting up to do - just focus on what they have in common. I do think this will be a relatively short notebook overall either way.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants