Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider adding the SSIM statistics to the output of MET #3031

Open
22 tasks
JohnHalleyGotway opened this issue Dec 5, 2024 · 0 comments
Open
22 tasks

Consider adding the SSIM statistics to the output of MET #3031

JohnHalleyGotway opened this issue Dec 5, 2024 · 0 comments
Labels
alert: NEED ACCOUNT KEY Need to assign an account key to this issue alert: NEED CYCLE ASSIGNMENT Need to assign to a release development cycle alert: NEED MORE DEFINITION Not yet actionable, additional definition required MET: AI/ML Verification Verification of Artificial Intelligence/Machine Learning models MET: Statistics priority: medium Medium Priority requestor: Navy/NRL Naval Research Laboratory type: enhancement Improve something that it is currently doing

Comments

@JohnHalleyGotway
Copy link
Collaborator

Describe the Enhancement

MET #3024 adds new columns to gradient line type (GRAD) from Grid-Stat to help quantify "sharpness", especially in AI/ML output, using the methods defined in this paper:
An Investigation of Metrics to Evaluate the Sharpness in AI-Generated Meteorological Imagery

That paper also mentions SSIM statistics:

  • Structural Similarity Index Measure (ssim) is a similarity measure between two images based on a weighted combination of three simpler comparisons: luminance, contrast, and structure. If this were well-defined, we could consider adding it to the CNT line type.
    See reference on Wikipedia

Recommend considering adding the SSIM statistic, along with its luminance, contrast, and structure components.

These could be added to the existing continuous statistics (CNT) line type, but more thought should be given to where it belongs.

Time Estimate

Estimate the amount of work required here.
Issues should represent approximately 1 to 3 days of work.

Sub-Issues

Consider breaking the enhancement down into sub-issues.

  • Add a checkbox for each sub-issue here.

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

Define the source of funding and account keys here or state NONE.

Define the Metadata

Assignee

  • Select engineer(s) or no engineer required
  • Select scientist(s) or no scientist required

Labels

  • Review default alert labels
  • Select component(s)
  • Select priority
  • Select requestor(s)

Milestone and Projects

  • Select Milestone as a MET-X.Y.Z version, Consider for Next Release, or Backlog of Development Ideas
  • For a MET-X.Y.Z version, select the MET-X.Y.Z Development project

Define Related Issue(s)

Consider the impact to the other METplus components.

Enhancement Checklist

See the METplus Workflow for details.

  • Complete the issue definition above, including the Time Estimate and Funding Source.
  • Fork this repository or create a branch of develop.
    Branch name: feature_<Issue Number>_<Description>
  • Complete the development and test your changes.
  • Add/update log messages for easier debugging.
  • Add/update unit tests.
  • Add/update documentation.
  • Push local changes to GitHub.
  • Submit a pull request to merge into develop.
    Pull request: feature <Issue Number> <Description>
  • Define the pull request metadata, as permissions allow.
    Select: Reviewer(s) and Development issue
    Select: Milestone as the next official version
    Select: MET-X.Y.Z Development project for development toward the next official release
  • Iterate until the reviewer(s) accept and merge your changes.
  • Delete your fork or branch.
  • Close this issue.
@JohnHalleyGotway JohnHalleyGotway added type: enhancement Improve something that it is currently doing priority: medium Medium Priority requestor: Navy/NRL Naval Research Laboratory alert: NEED MORE DEFINITION Not yet actionable, additional definition required alert: NEED ACCOUNT KEY Need to assign an account key to this issue alert: NEED CYCLE ASSIGNMENT Need to assign to a release development cycle MET: Statistics labels Dec 5, 2024
@JohnHalleyGotway JohnHalleyGotway added the MET: AI/ML Verification Verification of Artificial Intelligence/Machine Learning models label Dec 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
alert: NEED ACCOUNT KEY Need to assign an account key to this issue alert: NEED CYCLE ASSIGNMENT Need to assign to a release development cycle alert: NEED MORE DEFINITION Not yet actionable, additional definition required MET: AI/ML Verification Verification of Artificial Intelligence/Machine Learning models MET: Statistics priority: medium Medium Priority requestor: Navy/NRL Naval Research Laboratory type: enhancement Improve something that it is currently doing
Projects
None yet
Development

No branches or pull requests

1 participant