Issue with NIRSpec barshadow correction in MOS long slit data

[stscijgbot-jp]

Issue JP-3796 was created on JIRA by Christian Hayes:

As a result of looking into helpdesk ticket INC0205578 (PID 5064 obs 1) we noticed that there is striping in calibrated MOS long slit data, which appears to be an aliasing effect of a mis-aligned bar shadow correction.

I've attached an example here (jw01444003001_04102_00001_nrs1_cal, reduced with jwst 1.16.0, jwst_1298.pmap) showing cal spectra with barshadow correction on on the left and no barshadow correction on the right (where the barshadow can be seen as the periodic dip in light in between shutters).  It appears that the mis-alignment is less severe along some parts of the longslit.  The spectra in the barshadowed region between shutters looks smoother near the top of the screen shot - though there may still be some residual barshadow correction that isn't quite right.  Towards the bottom of the screen shot, the barshadow appears to be misaligned with the shutters, amplifying the barshadow effect.

[stscijgbot-jp]

Comment by David Law on JIRA:

Christian Hayes Is this something that just requires a reference file update, or is there something about the code that also needs to change?

[stscijgbot-jp]

Comment by Christian Hayes on JIRA:

I believe this is a code related issue, because the barshadow reference files are for a single shutter or a 3-shutter slitlet and the code will use these to stitch together the appropriate number of shutters to make slitlets of variable length (so it looks like the stitching isn't scaling up correctly).

 

[stscijgbot-jp]

Comment by Melanie Clarke on JIRA:

It looks like there are two approximations being made that are not scaling well to the very long "slitlet" in these data.

The first is related to JP-1341: there are two different definitions of slit height. The barshadow correction needs the full bar-to-bar height, but the slit coordinates in the WCS use the open area height. The barshadow code currently hard-codes the ratio between the full and open heights as 1.15, but the value actually varies a bit by quadrant. Using the equation in JP-1341 and jwst_nirspec_msa_0008.asdf, the Q4 scale should be closer to 1.1435773. Using this value gets the bars much closer to the right places.

The second is that the barshadow code shifts the slit y-coordinates by the mean value for the bounding box in order to put the center of the constructed shadow at the center of the slit. This is correct if the center of the bounding box is also the center of the middle shutter, but may be a little offset if not. In this case, it introduced a shift of ~1 pixel. Changing the code to shift the constructed shadow to the fiducial shutter instead of the center of the bounding box fixes this offset as well.

I'm attaching a quick screenshot of a subsection of the data, showing the current misalignment and the fixed version. In this section of the array, the barshadow correction was off by about half a shutter, almost exactly misaligned with the real shadows.

I’m not sure yet what the best way is to get the slit scaling issue fixed – right now, I just replaced the hard-coded value with some better estimates by quadrant from the above MSA file. I’d like to do a little more research on the issues in JP-1341 to see if we can do better than that without too much extra complexity.

[stscijgbot-jp]

Comment by Melanie Clarke on JIRA:

I have a proposed fix in jwst and stdatamodels PRs here:
#9085
spacetelescope/stdatamodels#379

This change would add slit scaling factors for open to full slit height conversion to the slit datamodel (slit.slit_xscale and slit.slit_yscale, FITS keywords SLTXSCL and SLTYSCL).  The slit_yscale value would be used in the barshadow correction in place of the hard-coded value of 1.15.  If the value is not available in the datamodel (e.g. existing intermediate products), the old value will be used as a fallback.

Christian Hayes - Before we proceed, I'd like to make sure NIRSpec is happy with:

the scale calculation I implemented

the plan to store the scales in the datamodel

the application of this scale factor to the barshadow algorithm.

I'll tag you in the draft PRs.  Let me know if you need more info or want to discuss.

[stscijgbot-jp]

Comment by Christian Hayes on JIRA:

Thanks Melanie Clarke for looking into this.  On a first scan this looks reasonable (and explains why I was seeing that the Q3 barshadow corrections looked more reasonable with a scale closer to 1.15), but I'll take a closer look at the PR and get in touch with the NIRSpec MOS experts. We'll let you know if we have any questions or concerns.