Face-api vs. FaceRes vs. SSR-Net for non-realtime use?

[sadluck]

First off, thank you for the awesome libraries!

I'm trying to get the best possible age & gender estimates in a ~50k small profile picture dataset (mostly 150x150px). I modified the nodejs example in vladmandic/face-api and it works ok(ish), but then I found your comment saying that "models used by face-api are pretty old" which has led me here. Which combination of models would you recommend for such a task in Human? Is SSR-Net better than the default FaceRes? Speed is not critical, but of course appreciated.

Thank you!

[vladmandic]

Thanks!

Regarding FaceAPI, honstely its still not bad at all, especially given its age.

Regarding your use case, its not something I've focused on, but its worth special considerations.
Comparing models in Human, there are actually 3 - FaceRES, GEAR and SSRNet

• GEAR: Ive added it a while ago, but havent tested it much
• SSRNet: Best choice for really tiny inputs as its trained on 64x64 images
• FaceRES: Most precise (its trained on 224x224) and complete model
  But its quite sensitive to face cropping
  Now, if input is higher resolution, its easy to crop it as desired
  But if input is already very low resolution, any additional cropping can create problems
  Anyhow, this is model I'm focusing on and variable cropping is something I'm working on right now

Can you share a part of your dataset? I'd be interested in testing solutions as I'm improving 'Human`?

[01coding]

@vladmandic Is it possible to replace hairstyles?

[vladmandic]

@01coding what do you mean?

[vladmandic]

@01coding if you mean can a library replace hairstyle automatically, answer is no, it's not intended to do that. but it can be used to detect face outline and then you can position/scale/draw your desired hiarstyle on it. but you have to do it yourself.

[sadluck]

Can you share a part of your dataset?

I'm afraid that's not possible at this time as it's part of a research project. I'll make sure to let you know if/when it becomes possible. The dataset is honestly nothing special. I would love to test the individual models side by side on it and share the results, though. The dataset contains about 2k images that are age-labeled, so there's some semi-reliable ground truth to test against. Is it possible to set Human up (in nodejs) to use multiple models for the same thing (e.g. age detection) in a single run?

And the most basic of questions - how do I switch face description from the default combined model to individual models at all? With the additional models downloaded, I thought I'd just change the inside of face: {} in myConfig (I'm trying the demo\nodejs\node.js example) to something like:
description: { enabled: false}, age: { enabled: true, modelPath: 'age.json'} but that's probably not how it's done. Sorry to be hacking here, not much of a javascript/node guy :)

[vladmandic]

its been a while since i've played with gear or ssrnet since most of my focus has been on faceres
as a result, those two module got pretty broken

anyhow, i've just fixed them and new version of human is on github

and a quick test app at test/test-gear.js to try each of the 3 models on input image or all images in input folder
(you can see how you select each model there)

you need to download models separately since they are not included in the default distribution due to overal size considerations
(i only include default models in default distribution)
url is: https://github.com/vladmandic/human-models/tree/main/models

but...all 3 models require further tuning regarding ideal face crop (tunable via config.face.detector.cropFactor, default is 1.6),
i'd appreciate if you test and let me know which values give you best results

once best values are known, i can place them directly in the library (can be different for each model)

[vladmandic]

I wrote a quick test to compare results:

dataset: UTKFace In-the-wild variation
sample: Random shuffled 500 images

Results

• age estimation is performed for age range of 16..60 and disabled otherwise
• tfjs-node is missing kernel implementation required for rotation correction thus precision in browser is expected to be slightly higher

1. FaceRes

• works with any crop factors with very little variation in precision
• model seems biased as majority of errors are with non-white ethnicity
• model is sensitive to facial obstructions like wearing glasses or mask

results: { genderAccuracy: 98, raceAccuracy: 0, ageError: { '<10%': 33, '<25%': 78, '<50%': 99, '<99%': 100 } }

2. GEAR

• default crop factor works ok, but any variation results in precision drop: 10% variation => 5% drop
• age algorithm can be further tuned as its weight based vs strict numeric value
• only model that also provides race info, although precision is questionable

results: { genderAccuracy: 93, raceAccuracy: 56, ageError: { '<10%': 32, '<25%': 73, '<50%': 92, '<99%': 100 } }

3. SSRNet

• default crop factor works ok, but any variation results in precision drop: 10% variation => 5% drop

results: { genderAccuracy: 88, raceAccuracy: 0, ageError: { '<10%': 26, '<25%': 57, '<50%': 96, '<99%': 99 } }