Question about params in mpr metadata

[carla-terboven]

Hello, I am using data = yadg.extractors.extract(filetype='eclab.mpr', path=file_path) to extract data from mpr files. It works like a charm, but I have a question about the meaning of some of the values in the metadata.

As far as I understand it, data.attrs.get('original_metadata') consists of 3 parts: settings, params and log.

Currently I am interested in the params and in all items with _vs.

From my mps files I know that under "vs." there are the 3 possibilities Ref, <None>, Eoc.
However, when I read my mpr files with yadg, I get numbers in the _vs entries.
The most common numbers so far are 0, 2, 3, 4, 50, 110, 112, 123, 82838, 2147483648, 2516582400.

So far, the mapping from number to Ref, <None>, Eoc is not yet clear to me. For example, the 0 can sometimes mean Ref or None. Unfortunately, I have not yet been able to find a correct pattern. Is the mapping known or does anyone know where to look up the mapping from number to Ref, <None>, Eoc? Or are there any plans to integrate this mapping into yadg?

PS: I hope it is ok to ask questions in this issues section. If there is some kind of forum or so I can also ask there.

[PeterKraus]

Hi Carla, yes, this is something that I know about. In the mpr files, these values are stored as integers, while if you were to export your data as a mpt file, you would see a string representation of the same data. The lack of this mapping is a known long-standing issue (see #12). It's very time consuming to figure out the mapping and nobody has really cared about this so far, so it's been a low priority. I do believe that there shouldn't be many more than a few values for each parameter - the I_range is the obvious exception.

Any contribution here is welcome. Basically, the param_map here would have to be extended.

As a side note, any parameter value highter than 256 is likely to be a bug (most parameters are supposedly 1 byte unsigned integers, so 0 - 255). Please let me know about any case where you get such high values in a separate issue.

[carla-terboven]

Hello Peter,

I'm beginning to understand the structure of your code. So I can add the mapping to the param_map for the EC-Lab files I currently have. But for this I would have to get the authorization to push on a branch in this repo. Or I can simply share my current status here.

"E_vs": (
        ("Eref", 0),
        ("Eref", 112),
    ),
    "E1_vs": (
        ("Eref", 0),
        ("Eref", 110),
        ("Eref", 135),
    ),
    "E2_vs": (
        ("Eref", 123),
    ),
    "Ef_vs": (
        ("Eoc", 0),
        ("Eref", 1),
        ("Eoc", 3),
        ("Eoc", 50)
    ),
    "Ei_vs": (
        ("Eoc", 0),
        ("Eref", 4),
    ),
    "EL_vs": (
        ("Eref", 0),
        ("Eoc", 102),
    ),
    "Is_vs": (
        ("None", 0),
        ("None", 2),
        ("None", 82838),
        ("None", 2147483648),
        ("None", 2516582400),
    ),

As you can see, the values I can contribute are unfortunately far from complete. I am therefore a bit afraid that if the mapping is added like this, a wave of people will come with the error element 'xxxxx' for parameter 'xxxxx_vs' not understood. And unfortunately it is not possible for me to try out much what other params could be out there. Should I then map the values myself after using yadg when evaluating the data? Or should we start with the list I shared above? What is your opinion on this?

[carla-terboven]

The values above 255 for Is_vs occur in some GEIS measurements. But I have not yet been able to find out anything more precise. I'll see if I can open another issue when I understand more.

[PeterKraus]

That's a great start, thank you. We can work around the current limitations down the line. Please feel free to fork, make a branch with changes in your own fork, and then submit a PR to upstream via github. We can also move the discussion there.

[PeterKraus]

The values above 255 for Is_vs occur in some GEIS measurements. But I have not yet been able to find out anything more precise. I'll see if I can open another issue when I understand more.

I am fairly certain that must be a bug. It might be that the byte-length of the parameter data structure is not correct, so the first parameter set gets parsed "correctly", but the following ones are broken. Let's have a look at the offending GEIS file.

[carla-terboven]

That's a great start, thank you. We can work around the current limitations down the line. Please feel free to fork, make a branch with changes in your own fork, and then submit a PR to upstream via github. We can also move the discussion there.

Please find PR #191 with all reasonable values below 255 that are currently available to me.

[PeterKraus]

Implemented in #193 and #196.