Background subtraction

[tischi]

@metavibor @constantinpape @imagirom
I am opening this issue to collect and discuss ideas about background subtraction.

[constantinpape]

Just linking this related issue here:
sciai-lab/batchlib#126

[tischi]

Median well background distributions (using inverse of serum based cell segmentation)

[tischi]

Background values in H01 wells vs median of all wells, using the serum based cell segmentation:

  plate_name                                   H01                      Median                      Roman
  <chr>                                           <dbl>                      <dbl>
1 plate1rep3_20200505_100837…                     1588.                      1644.                     1380.36
2 plate2rep3_20200507_094942…                     1840.                      1861.                     1650.12
3 plate5rep3_20200507_113530…                     2206.                      1658.                     1428.63
4 plate6rep2_wp_20200507_131…                     4952.                      5012.                     4372.64
5 plate8rep1_20200425_162127…                     1633.                      1632.                     1882.13 **
6 plate8rep2_20200502_182438…                     1962.                      1697.                     2161.61 **
7 plate9_2rep1_20200506_1633…                     1588.                        NA                     1430.98                     
8 titration_plate_20200403_1…                     1195.                      1213.                    206.22

[constantinpape]

@metavibor
I am using the following values for fixed background subtraction right now:

Default:
'serum_IgG': 1300, 'serum_IgA': 1800
Plates: plate6rep2_wp_20200507_131032_010, titration_plate_20200403_154849
'serum_IgG': 5000, 'serum_IgA': 7000

Judging from the results @tischi posted this is wrong for the titration plate. (I think you said to use the alternative values there as well, but maybe I misread this).
In any case, this explains the weird results we have lately seen for the titration plate.

In general, I would much more prefer a background estimated from the data if we can do this in a reliable manner, because it eliminates this source of error.

[tischi]

If I did not do a mistake with my table manipulations it looks like as if the measured background in the H01 wells is higher in some plates than the plate median, which I find surprising.

Maybe someone could use the PlateViewer to check what happened there?

(note: we are not using the dilated nuclei yet, so maybe this makes no sense to look at yet...)

[imagirom]

I posted some preliminary results using the dilated nulcei at sciai-lab/batchlib#126

[tischi]

I manually copied @imagirom's results into above table. The values look very sensible! 👏
However two looks too high: plate8rep*.
We should look what happened there (modulo copy&paste mistake on my side).

[imagirom]

If I did not do a mistake with my table manipulations it looks like as if the measured background in the H01 wells is higher in some plates than the plate median, which I find surprising.

In some plates there is serum staining in the H1 Well, see my comment at sciai-lab/batchlib#126. @metavibor Do you have a list of plates where the serum in the H1 well is stained? For these plates, is there another well without serum staining for those plates?

Another problem might be that we do not have any images from the H1 well for some plates.

[imagirom]

However two looks too high: plate8rep*.
We should look what happened there (modulo copy&paste mistake on my side).

It looks like the serum was stained in the H1 well of these plates. E.g. for plate8rep1:

I used the pixels in the light region for the background computation

[tischi]

New idea: Take the minimum measured background value across all wells. Typically this should be the H01 well if it exists.

[constantinpape]

New idea for background measurement:

• take the background of the dimmest well (measured as minimum median background over the robust background (dilated nuclei))
• in the new plates this should always be H01
• corner case: not enough background -> should only accept this value if there are enough background pixels
• corner case: all channels are empty -> we exoect no nuclei because DAPI is empty
• from Roman, dilate nuclei by 50 pixel

[constantinpape]

* from Roman, dilate nuclei by 50 pixel

We discussed this more and decided to dilate by some factor of the median nucleus radius.
Roman will compute the factor from data.

[tischi]

by some factor of the median nucleus radius.

very good idea!

[imagirom]

With an erosion radius of 2.5 times the square root of the median nucleus size, only 0.3% of cells would not be completely excluded from the background mask. For a radius of 2 times that it would be about 1.2%. I calculated that using 5 random images per plate.

To get an Idea on how much the median nucleus size varies, both per plate and per image:

[constantinpape]

With an erosion radius of 2.5 times the square root of the average nucleus size, only 0.3% of cells would not be completely excluded from the background mask.

Ok, 2.5 sounds good then.