Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Area cropping inside cutout results in metadata error #11

Closed
Sooph-knmi opened this issue Jul 30, 2024 · 1 comment
Closed

Area cropping inside cutout results in metadata error #11

Sooph-knmi opened this issue Jul 30, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@Sooph-knmi
Copy link

What happened?

When cropping is used inside Cutout, the following error message is produced when calling on the metadata property:

NotImplementedError: subclass_metadata_specific() must be implemented in derived class Cropping

It seems that the metadata is missing in this class. It should give the metadata values inherited from the data that is being cropped.

What are the steps to reproduce the bug?

step 1:

In a jupiter notebook:
from anemoi.datasets import open_dataset as od
Then call the function with the following parameters:

dsorig = od(cutout=[{"dataset":"s3://ml-datasets/aifs-ea-an-oper-0001-mars-n320-1979-2022-6h-v6.zarr", "area": [58.8758931, -7.5, 43.13815694, 23.25], "reorder": ['q_50', 'q_100', 'q_150', 'q_200', 'q_250', 'q_300', 'q_400', 'q_500', 'q_600', 'q_700', 'q_850', 'q_925', 'q_1000', 'z_50', 'z_100', 'z_150', 'z_200', 'z_250', 'z_300', 'z_400', 'z_500', 'z_600', 'z_700', 'z_850', 'z_925', 'z_1000', 'sp', 'msl', 't_50', 't_100', 't_150', 't_200', 't_250', 't_300', 't_400', 't_500', 't_600', 't_700', 't_850', 't_925', 't_1000', '2t', 'u_50', 'u_100', 'u_150', 'u_200', 'u_250', 'u_300', 'u_400', 'u_500', 'u_600', 'u_700', 'u_850', 'u_925', 'u_1000', '10u', 'v_50', 'v_100', 'v_150', 'v_200', 'v_250', 'v_300', 'v_400', 'v_500', 'v_600', 'v_700', 'v_850', 'v_925', 'v_1000', '10v', 'w_50', 'w_100', 'w_150', 'w_200', 'w_250', 'w_300', 'w_400', 'w_500', 'w_600', 'w_700', 'w_850', 'w_925', 'w_1000', '2d', 'sdor', 'slor', 'cp', 'tp', 'z', 'cos_latitude', 'cos_longitude', 'sin_latitude', 'sin_longitude', 'cos_julian_day', 'cos_local_time', 'sin_julian_day', 'sin_local_time', 'insolation', 'tcw', 'skt', 'lsm']},
    {"dataset": "s3://ml-datasets/aifs-ea-an-oper-0001-mars-o96-1979-2022-6h-v5.zarr", "reorder": ['q_50', 'q_100', 'q_150', 'q_200', 'q_250', 'q_300', 'q_400', 'q_500', 'q_600', 'q_700', 'q_850', 'q_925', 'q_1000', 'z_50', 'z_100', 'z_150', 'z_200', 'z_250', 'z_300', 'z_400', 'z_500', 'z_600', 'z_700', 'z_850', 'z_925', 'z_1000', 'sp', 'msl', 't_50', 't_100', 't_150', 't_200', 't_250', 't_300', 't_400', 't_500', 't_600', 't_700', 't_850', 't_925', 't_1000', '2t', 'u_50', 'u_100', 'u_150', 'u_200', 'u_250', 'u_300', 'u_400', 'u_500', 'u_600', 'u_700', 'u_850', 'u_925', 'u_1000', '10u', 'v_50', 'v_100', 'v_150', 'v_200', 'v_250', 'v_300', 'v_400', 'v_500', 'v_600', 'v_700', 'v_850', 'v_925', 'v_1000', '10v', 'w_50', 'w_100', 'w_150', 'w_200', 'w_250', 'w_300', 'w_400', 'w_500', 'w_600', 'w_700', 'w_850', 'w_925', 'w_1000', '2d', 'sdor', 'slor', 'cp', 'tp', 'z', 'cos_latitude', 'cos_longitude', 'sin_latitude', 'sin_longitude', 'cos_julian_day', 'cos_local_time', 'sin_julian_day', 'sin_local_time', 'insolation', 'tcw', 'skt', 'lsm']}],
  end= 2020,
  frequency= 6,
  drop=  [])

step 2:

Calling the metadata will produce an error

dsorig.metadata()

Version

0.4.3

Platform (OS and architecture)

Linux, Visual Studio Cose

Relevant log output

{
	"name": "NotImplementedError",
	"message": "subclass_metadata_specific() must be implemented in derived class Cropping",
	"stack": "---------------------------------------------------------------------------
NotImplementedError                       Traceback (most recent call last)
Cell In[24], line 1
----> 1 dsorig.metadata()

File ~/data1/miniconda3/envs/anemoi/lib/python3.12/site-packages/anemoi/datasets/data/dataset.py:187, in Dataset.metadata(self)
    179         return os.path.basename(v)
    180     return v
    182 return tidy(
    183     dict(
    184         version=anemoi.datasets.__version__,
    185         shape=self.shape,
    186         arguments=self.arguments,
--> 187         specific=self.metadata_specific(),
    188         frequency=self.frequency,
    189         variables=self.variables,
    190         start_date=self.dates[0].astype(str),
    191         end_date=self.dates[-1].astype(str),
    192     )
    193 )

File ~/data1/miniconda3/envs/anemoi/lib/python3.12/site-packages/anemoi/datasets/data/select.py:87, in Select.metadata_specific(self, **kwargs)
     86 def metadata_specific(self, **kwargs):
---> 87     return super().metadata_specific(indices=self.indices, **kwargs)

File ~/data1/miniconda3/envs/anemoi/lib/python3.12/site-packages/anemoi/datasets/data/forwards.py:93, in Forwards.metadata_specific(self, **kwargs)
     91 def metadata_specific(self, **kwargs):
     92     return super().metadata_specific(
---> 93         forward=self.forward.metadata_specific(),
     94         **self.subclass_metadata_specific(),
     95         **kwargs,
     96     )

File ~/data1/miniconda3/envs/anemoi/lib/python3.12/site-packages/anemoi/datasets/data/forwards.py:93, in Forwards.metadata_specific(self, **kwargs)
     91 def metadata_specific(self, **kwargs):
     92     return super().metadata_specific(
---> 93         forward=self.forward.metadata_specific(),
     94         **self.subclass_metadata_specific(),
     95         **kwargs,
     96     )

File ~/data1/miniconda3/envs/anemoi/lib/python3.12/site-packages/anemoi/datasets/data/forwards.py:186, in Combined.metadata_specific(self, **kwargs)
    181 def metadata_specific(self, **kwargs):
    182     # We need to skip the forward superclass
    183     # TODO: revisit this
    184     return Dataset.metadata_specific(
    185         self,
--> 186         datasets=[d.metadata_specific() for d in self.datasets],
    187         **kwargs,
    188     )

File ~/data1/miniconda3/envs/anemoi/lib/python3.12/site-packages/anemoi/datasets/data/masked.py:116, in Cropping.metadata_specific(self, **kwargs)
    115 def metadata_specific(self, **kwargs):
--> 116     return super().metadata_specific(area=self.area, **kwargs)

File ~/data1/miniconda3/envs/anemoi/lib/python3.12/site-packages/anemoi/datasets/data/forwards.py:94, in Forwards.metadata_specific(self, **kwargs)
     91 def metadata_specific(self, **kwargs):
     92     return super().metadata_specific(
     93         forward=self.forward.metadata_specific(),
---> 94         **self.subclass_metadata_specific(),
     95         **kwargs,
     96     )

File ~/data1/miniconda3/envs/anemoi/lib/python3.12/site-packages/anemoi/datasets/data/forwards.py:102, in Forwards.subclass_metadata_specific(self)
    101 def subclass_metadata_specific(self):
--> 102     raise NotImplementedError(
    103         f\"subclass_metadata_specific() must be implemented in derived class {self.__class__.__name__}\"
    104     )

NotImplementedError: subclass_metadata_specific() must be implemented in derived class Cropping"
}

Accompanying data

No response

Organisation

KNMI

@Sooph-knmi Sooph-knmi added the bug Something isn't working label Jul 30, 2024
@JPXKQX
Copy link
Member

JPXKQX commented Sep 24, 2024

PR #53 fix this.

@JPXKQX JPXKQX closed this as completed Sep 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants