Accuracy

[deepsea6034625]

Hello.
First of all, thanks for releasing this excellent project.

I tried to segment lenses from images.
But as you can see here, accuracy is not good.
Is there any way to improve the accuracy?

![BOSS_BOSS0521S_003_01](https://github.com/mantasu/glasses-detector/assets/15572169


4/4b480698-bf43-4778-8398-d282be9bd8f4)

Hope to please help me.
Thanks again.

[deepsea6034625]

One more.

I used this code.

import numpy as np
from glasses_detector import GlassesSegmenter

# Initialize full segmentation model of size large
seg_full = GlassesSegmenter(kind='lenses', size='medium') # TODO: change medium to large

# Process the directory of images and save the results in various ways
seg_full.process_dir('images', 'results', format='img', batch_size=6, pbar=False)

[mantasu]

Better Weights

Hi @deepsea920415, unfortunately, there's not much that can be done because the dataset is very small. Based on the data (standalone glasses) you are showing, I tried training a new large model which seems to have slightly improved accuracy.

segmenter = GlassesSegmenter(kind="lenses", size="large", weights="path/to/weights.pth")

Here are two weight files:

• Same lenses dataset, larger model: segmentation-lenses-large.pth
• Fine-tuned on data without people: segmentation-lenses-large-standalone-glasses.pth

Square Inputs

Also, you might want to pad your image from both sides to make sure it is a square:

OPTION 1: preprocess image files

import os
from PIL import Image
from glasses_segmenter import GlassesSegmenter

# Both directories must already exist
INPUT_DIR = "path/to/non_square"
PREPROCESS_DIR = "path/to/square"

def make_image_square(image_path):
    img = Image.open(image_path)
    width, height = img.size
    target_size = max(width, height)
    new_img = Image.new(img.mode, (target_size, target_size), img.getpixel((0,0)))
    left_padding = (target_size - width) // 2
    top_padding = (target_size - height) // 2
    new_img.paste(img, (left_padding, top_padding))
    return new_img
    
for filename in os.listdir(INPUT_DIR):
    img = make_image_square(os.path.join(INPUT_DIR, filename))
    img.save(os.path.join(PREPROCESS_DIR, filename))
    
segmenter = GlassesSegmenter(kind="lenses", size="large", weights="path/to/weights.pth")
segmenter.process_dir(PREPROCESS_DIR)

OPTION 2: extend GlassesSegmenter

from glasses_detector import GlassesSegmenter

class MyGlassesSegmenter(GlassesSegmenter):
    @staticmethod
    def make_image_square(image_path):
        img = Image.open(image_path)
        width, height = img.size
        target_size = max(width, height)
        new_img = Image.new(img.mode, (target_size, target_size), img.getpixel((0,0)))
        left_padding = (target_size - width) // 2
        top_padding = (target_size - height) // 2
        new_img.paste(img, (left_padding, top_padding))
        return new_img
    
    def predict(self, image_paths, *args, **kwargs):
        # WARNING: image_paths must be a list of paths!
        images = [self.make_image_square(path) for path in image_paths]
        return super().predict(images, *args, **kwargs)
        
segmenter = MyGlassesSegmenter(kind="lenses", size="large", weights="path/to/weights.pth")
segmenter.process_dir("path/to/non_square")

Further Improvements

To further improve accuracy, here are some further suggestions (they are, however, beyond the scope of this package):

• Use more data, and experiment with various data augmentation techniques
• Use dilation and erosion to smoothen the masks, e.g., close segmentation holes
• Subtract frames segmenter mask from full segmenter mask to get a "helper" mask
• Tune for specific segmenter architecture and hyperparaeters (e.g., using optuna, ray, pytorch lightning packages)
• Use cross-validation because the dataset is very small
• Train an ensemble of models, possibly with varying architectures

[deepsea6034625]

Thank you.
I'll test with them.

[deepsea6034625]

The accuracy is improved a lot.
But still some bad samples.

But totally, I think overall accuracy is good.
If we add some more dataset, the accuracy will be better.
Thanks again.

[mantasu]

Yup, training on more data is the best way to go!