Retina Image in E2E file

[ly1998117]

• eyepy version:
• Python version: 3.11
• Operating System: MacOS

Description

I have an RGB Retina E2E file where the head information is readable, but the main image cannot be read. The Image data.type of the main image is 33620225, which differs from other common types like 33620481 or 35652097. How can I interpret or process this 33620225 type image data?

The E2E file is attached. 

H1AMD328.E2E.zip

[Oli4]

Dear @ly1998117,

Do you know the shape of the contained image? I find 3 slices (probably the RGB channels), but the image height and width seem to be encoded differently than in the other formats.

You can use the code below to read the bytes containing the header and one channel of your image.

import eyepy as ep
from eyepy.io import HeE2eReader
from eyepy.io.he.e2e_format import TypesEnum
import numpy as np

with HeE2eReader("H1AMD328.E2E") as e2e_reader:
    slices = e2e_reader.series[0].slices
    folder = slices[0].folders[TypesEnum.image][0] # slices[0] selects probably one of the channels
    b = folder.get_bytes() # Read the complete bytes including the header of the folder

data = np.frombuffer(b, dtype=np.uint8)

You can try to decode the bytes to an image by experimenting with different encodings (int8, uint8, float32, etc.), image shapes (this is easier if you already know the shape), and maybe even byte/bit offsets since the header might contain values of different data types.

Let me know if you figured it out, then I would include it in eyepy, so it works out of the box.

Best,
Olivier

[ly1998117]

Hi @olivier,

I’ve investigated the issue further. It seems that the 3 slices actually correspond to 3 images instead of representing the RGB channels， since I have tried another EYE file which has only 2 slices.

Additionally, I noticed that the size of the converted JPEG images by HEIDELBERG EYE EXPLORER is significantly larger than the slice data read by eyepy. This could be related to how the data is being decoded or encoded during the conversion process.

For your reference, I’ve uploaded the EYE files and the corresponding JPEG images to Google Drive. Feel free to take a look and let me know if you spot anything that could explain the size discrepancy.

Let me know if you’d like me to continue exploring this or provide any further details.

Best regards,
Yang.

[ly1998117]

By the way, I would like to understand why the Heidelberg Eye Explorer outputs OCT images differently compared to eyepy.

1. IR + OCT Combined Visualization:

• In Heidelberg Eye Explorer, the IR image and OCT slices are combined, with green lines overlaid to indicate the scanning positions for the cross-sectional OCT slices. However, in eyepy, the IR and OCT images are output separately.
• The green lines in Heidelberg Eye Explorer represent the scanning positions of the OCT (Optical Coherence Tomography) slices relative to the IR image.

How can I replicate this combined visualization (IR + green scan lines + OCT slices) using eyepy?

2. FA and ICGA Image Concatenation:
   I’ve noticed that in some cases, images such as FA (Fluorescein Angiography) and ICGA ( Indocyanine Green Angiography ) are concatenated together in Heidelberg Eye Explorer.

• How can I determine which images should be concatenated when working with eyepy? ( I guess the shooting time. )

Hi @olivier,

I’ve investigated the issue further. It seems that the 3 slices actually correspond to 3 images instead of representing the RGB channels， since I have tried another EYE file which has only 2 slices.

Additionally, I noticed that the size of the converted JPEG images by HEIDELBERG EYE EXPLORER is significantly larger than the slice data read by eyepy. This could be related to how the data is being decoded or encoded during the conversion process.

For your reference, I’ve uploaded the EYE files and the corresponding JPEG images to Google Drive. Feel free to take a look and let me know if you spot anything that could explain the size discrepancy.

Let me know if you’d like me to continue exploring this or provide any further details.

Best regards, Yang.

[Oli4]

Hi @olivier,

I’ve investigated the issue further. It seems that the 3 slices actually correspond to 3 images instead of representing the RGB channels， since I have tried another EYE file which has only 2 slices.

Additionally, I noticed that the size of the converted JPEG images by HEIDELBERG EYE EXPLORER is significantly larger than the slice data read by eyepy. This could be related to how the data is being decoded or encoded during the conversion process.

For your reference, I’ve uploaded the EYE files and the corresponding JPEG images to Google Drive. Feel free to take a look and let me know if you spot anything that could explain the size discrepancy.

Let me know if you’d like me to continue exploring this or provide any further details.

Best regards, Yang.

You are right the slices are 3 different images and I also figured how to read them. The header of the folder is only 16 bytes long here and the remaining bytes contain a JFIF file which can be read with PIL. If I find the time I'll add it to eyepy, but for now you can use the code below to read the images.

import eyepy as ep
from eyepy.io import HeE2eReader
from eyepy.io.he.e2e_format import TypesEnum
from io import BytesIO
from PIL import Image

with HeE2eReader("H1AMD328.E2E") as e2e_reader:
    slices = e2e_reader.series[0].slices
    folder = slices[0].folders[TypesEnum.image][0] # slices[0] selects probably one of the channels
    b = folder.get_bytes() # Read the complete bytes including the header of the folder
    img = Image.open(io.BytesIO(b[16:])))

Best Olivier

[ly1998117]

Hi @olivier,
I’ve investigated the issue further. It seems that the 3 slices actually correspond to 3 images instead of representing the RGB channels， since I have tried another EYE file which has only 2 slices.
Additionally, I noticed that the size of the converted JPEG images by HEIDELBERG EYE EXPLORER is significantly larger than the slice data read by eyepy. This could be related to how the data is being decoded or encoded during the conversion process.
For your reference, I’ve uploaded the EYE files and the corresponding JPEG images to Google Drive. Feel free to take a look and let me know if you spot anything that could explain the size discrepancy.
Let me know if you’d like me to continue exploring this or provide any further details.
Best regards, Yang.

You are right the slices are 3 different images and I also figured how to read them. The header of the folder is only 16 bytes long here and the remaining bytes contain a JFIF file which can be read with PIL. If I find the time I'll add it to eyepy, but for now you can use the code below to read the images.

import eyepy as ep
from eyepy.io import HeE2eReader
from eyepy.io.he.e2e_format import TypesEnum
from io import BytesIO
from PIL import Image

with HeE2eReader("H1AMD328.E2E") as e2e_reader:
    slices = e2e_reader.series[0].slices
    folder = slices[0].folders[TypesEnum.image][0] # slices[0] selects probably one of the channels
    b = folder.get_bytes() # Read the complete bytes including the header of the folder
    img = Image.open(io.BytesIO(b[16:])))

Best Olivier

Hi @olivier,

Thank you so much for your insightful reply and for pointing me in the right direction. Your code snippet also works perfectly, and I was able to load the images using PIL as you suggested. I’ll keep an eye on any further improvements for the excellent eyepy library, but in the meantime, I’ll continue using the workaround you provided.

Thanks again for your support, and I appreciate you taking the time to share your findings.

Best regards,
Yang

[Oli4]

By the way, I would like to understand why the Heidelberg Eye Explorer outputs OCT images differently compared to eyepy.

1. IR + OCT Combined Visualization:

• In Heidelberg Eye Explorer, the IR image and OCT slices are combined, with green lines overlaid to indicate the scanning positions for the cross-sectional OCT slices. However, in eyepy, the IR and OCT images are output separately.
• The green lines in Heidelberg Eye Explorer represent the scanning positions of the OCT (Optical Coherence Tomography) slices relative to the IR image.

How can I replicate this combined visualization (IR + green scan lines + OCT slices) using eyepy?

2. FA and ICGA Image Concatenation:
   I’ve noticed that in some cases, images such as FA (Fluorescein Angiography) and ICGA ( Indocyanine Green Angiography ) are concatenated together in Heidelberg Eye Explorer.

• How can I determine which images should be concatenated when working with eyepy? ( I guess the shooting time. )

Hi @olivier,
I’ve investigated the issue further. It seems that the 3 slices actually correspond to 3 images instead of representing the RGB channels， since I have tried another EYE file which has only 2 slices.
Additionally, I noticed that the size of the converted JPEG images by HEIDELBERG EYE EXPLORER is significantly larger than the slice data read by eyepy. This could be related to how the data is being decoded or encoded during the conversion process.
For your reference, I’ve uploaded the EYE files and the corresponding JPEG images to Google Drive. Feel free to take a look and let me know if you spot anything that could explain the size discrepancy.
Let me know if you’d like me to continue exploring this or provide any further details.
Best regards, Yang.

The E2E file contains the data and the Heidelberg viewer shows this data in a particular way. You can use matplotlib in python to show the NIR localizer next to one of the B-scans and also indicate the B-scan position on the NIR. Have a look at the EyeVolume EyeEnface and EyeBscan objects, they provide plotting functions which might be usefull for you. I have never tried to visualize the radial Scan layout you show, but the grid layout where Bscans are stacked should work out of the box.

For the FA and ICGA Image Concatenatio, I have no experience with this data.

[ly1998117]

Hi @olivier,
I’ve investigated the issue further. It seems that the 3 slices actually correspond to 3 images instead of representing the RGB channels， since I have tried another EYE file which has only 2 slices.
Additionally, I noticed that the size of the converted JPEG images by HEIDELBERG EYE EXPLORER is significantly larger than the slice data read by eyepy. This could be related to how the data is being decoded or encoded during the conversion process.
For your reference, I’ve uploaded the EYE files and the corresponding JPEG images to Google Drive. Feel free to take a look and let me know if you spot anything that could explain the size discrepancy.
Let me know if you’d like me to continue exploring this or provide any further details.
Best regards, Yang.

You are right the slices are 3 different images and I also figured how to read them. The header of the folder is only 16 bytes long here and the remaining bytes contain a JFIF file which can be read with PIL. If I find the time I'll add it to eyepy, but for now you can use the code below to read the images.

import eyepy as ep
from eyepy.io import HeE2eReader
from eyepy.io.he.e2e_format import TypesEnum
from io import BytesIO
from PIL import Image

with HeE2eReader("H1AMD328.E2E") as e2e_reader:
    slices = e2e_reader.series[0].slices
    folder = slices[0].folders[TypesEnum.image][0] # slices[0] selects probably one of the channels
    b = folder.get_bytes() # Read the complete bytes including the header of the folder
    img = Image.open(io.BytesIO(b[16:])))

Best Olivier

I just modify the source code, and it works now.

class DataType:
    localizer = 33620481
    oct = 35652097
    retina = 33620225

class RetinaAdapter(construct.Adapter):
    def _decode(self, obj: bytes, context, path):
        data = np.array(Image.open(io.BytesIO(obj)), dtype='uint8')
        return data

    def _encode(self, obj: np.ndarray, context, path):
        return obj.astype(np.uint8).tobytes()
Retina = RetinaAdapter(construct.Bytes(construct.this.n_values))
@dataclasses.dataclass
class Type1073741824(DataclassMixin, TypeMixin):
    """Image data Stores various kinds of images.

    Size: variable

    Notes:
    Different kinds of images are stored in this structure. Currently we know the following types:

    * 33620481: LocalizerNIR (`int8u`)
    * 35652097: Bscan (`float16u`)

    The custom `float16u` used to store the Bscan data, has no sign, a 6-bit exponent und 10-bit mantissa.
    """
    size: int = csfield(construct.Int32ul, doc='Size of the data')
    type: int = csfield(construct.Int32ul, doc='Type of the data')
    n_values: int = csfield(construct.Int32ul, doc='Number of values in the data')
    height: int = csfield(construct.Int32ul, doc='Height of the image')
    width: t.Optional[int] = csfield(If(lambda ctx: ctx.type != DataType.retina, construct.Int32ul), doc='Width of the image')
    data: t.Any = csfield(construct.Switch(construct.this.type, {
        DataType.localizer: LocalizerNIR,
        DataType.oct: Bscan,
        DataType.retina: Retina,
    },
                                           default=construct.Bytes(construct.this.size)),
                          doc='Image data')

I also discovered that the type TypesEnum.2:2 in the EYE file ‘FFA+ICG+OCT.E2E’ I shared is the same structure as the RGB Retina image (a JFIF file, which can be read using PIL). This file stores the corresponding thumbnail infrared (IR) image (128x128) of the OCT slice. I hope this finding can help you to improve this library.
TypesEnum.3:3 records the shooting information, like time and angle of FA|ICGA or ART .