Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JP-3827: Remove unused error from ramp data #334

Merged
merged 4 commits into from
Feb 5, 2025

Conversation

melanieclarke
Copy link
Contributor

@melanieclarke melanieclarke commented Jan 29, 2025

Resolves JP-3827

This PR removes references to the unused error array from the 4D ramp models. In the jwst pipeline, error values are not meaningful until after ramp fitting.

I also did a little minor cleaning for trivial things I noticed while making this change. I'll note any significant differences.

Requires:
spacetelescope/stdatamodels#384
spacetelescope/jwst#9109

Tasks

  • update or add relevant tests
  • update relevant docstrings and / or docs/ page
  • Does this PR change any API used downstream? (if not, label with no-changelog-entry-needed)
    • write news fragment(s) in changes/: echo "changed something" > changes/<PR#>.<changetype>.rst (see below for change types)
    • run regression tests with this branch installed ("git+https://github.com/<fork>/stcal@<branch>")
news fragment change types...
  • changes/<PR#>.apichange.rst: change to public API
  • changes/<PR#>.bugfix.rst: fixes an issue
  • changes/<PR#>.general.rst: infrastructure or miscellaneous change

Copy link

codecov bot commented Jan 29, 2025

Codecov Report

Attention: Patch coverage is 73.68421% with 5 lines in your changes missing coverage. Please review.

Project coverage is 86.16%. Comparing base (70ce6f8) to head (61c6ea0).
Report is 6 commits behind head on main.

Files with missing lines Patch % Lines
src/stcal/ramp_fitting/gls_fit.py 0.00% 2 Missing ⚠️
src/stcal/ramp_fitting/likely_algo_classes.py 0.00% 1 Missing ⚠️
src/stcal/ramp_fitting/ramp_fit.py 0.00% 1 Missing ⚠️
tests/test_ramp_fitting_likely_fit.py 66.66% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #334      +/-   ##
==========================================
+ Coverage   85.50%   86.16%   +0.65%     
==========================================
  Files          50       50              
  Lines        9398     9322      -76     
==========================================
- Hits         8036     8032       -4     
+ Misses       1362     1290      -72     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@github-actions github-actions bot added the documentation Improvements or additions to documentation label Jan 30, 2025
@melanieclarke
Copy link
Contributor Author

melanieclarke commented Jan 30, 2025

jwst tests here: https://github.com/spacetelescope/RegressionTests/actions/runs/13039620799
romancal tests here: https://github.com/spacetelescope/RegressionTests/actions/runs/13058971688

jwst tests show expected changes, romancal tests pass.


ramp_data_slice.set_arrays(data, err, groupdq, pixeldq)
ramp_data_slice.set_arrays(data, groupdq, pixeldq, average_dark_current)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The gls_fit algorithm is unused and only partially implemented. I noticed in passing that the set_arrays call had incorrect arguments here, so I added the average dark current. Changes in this file should have no impact to any current data processing.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. The gls_fit algorithm should be superseded by the likelihood-based one--it should be the identical calculation, but with orders of magnitude better performance. I think we can consider permanently removing gls_fit.

@@ -278,6 +278,8 @@ def calc_bias(self, countrates, sig, cvec, da=1e-7):
Bias of the best-fit count rate from using cvec plus the observed
resultants to estimate the covariance matrix.
"""
raise NotImplementedError('Bias calculations are not implemented.')
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed in passing that this method currently has a code error: it calls a fit_ramps function which is not defined in this module. I added the NotImplementedError to make it clear that it is not working, but this method is not called anywhere. We should decide if it should be removed or fixed in a separate PR.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. I think it can be removed. The calculation is in my repo fit-ramp, but I do not think it is necessary to include in the jwst pipeline.

Copy link
Contributor

@t-brandt t-brandt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me; thanks for getting this done so quickly.

@tapastro
Copy link
Collaborator

tapastro commented Feb 5, 2025

We have a JWST approval from Ken; I'm going to hold off reviewing until someone from Roman gives the go-ahead, just make sure they're aware of the changes we're making.

Copy link
Collaborator

@schlafly schlafly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

romancal regtests are passing, looks good to me, thanks!

@ddavis-stsci
Copy link
Collaborator

The romancal regression tests pass on this branch,
https://github.com/spacetelescope/RegressionTests/actions/runs/13159969400
So I think it's good to go from the Roman side.

@melanieclarke melanieclarke merged commit 9bd365d into spacetelescope:main Feb 5, 2025
24 of 26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dark_current documentation Improvements or additions to documentation jump ramp_fitting testing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants