Importing new model

[key2]

Hello,
I was trying to import an antispoof based on MobileNetV2 into human.
The model takes a [1, 224, 224, 3] values from [-1, 1] reversed RGB.
So far I am able to find a face, crop and resize it to 224, 224 and convert. But since the model was trained with opencv, using res10_300x300_ssd_iter_140000, the way it cuts and resize seems to have an importance. Therefore I don't get the same result using the face-detection of human.
How would one get the same box when face detecting, as on opencv ? any thoughts?

[vladmandic]

most likely biggest difference is face crop factor - opencv performs a really tight crop by default and i prefer to have a wider crop.
you can adjust crop scale using config.face.scale (default is 1.4 which means expand margins from tight crop by 40%).

if that works, you can take a look to implement a custom crop for your model.

good example is in emotion module as it takes existing detected face and performs additional tight crop on it to fit the model training data:

human/src/gear/emotion.ts

Lines 51 to 57 in cca0102

	if (config.face.emotion['crop'] > 0) { // optional crop
	const crop = config.face.emotion['crop'];
	const box = [[crop, crop, 1 - crop, 1 - crop]]; // empyrical values for top, left, bottom, right
	t.resize = tf.image.cropAndResize(image, box, [0], [inputSize, inputSize]);
	} else {
	t.resize = tf.image.resizeBilinear(image, [inputSize, inputSize], false);
	}

[key2]

yes but even then, opencv takes the box from the chin line, while in this case we take it from a bit lower.
But that's what I thought, from what I understand, one need to empirically play with those crop and offset value until we find the best match between the 2. Also what would be the best equivalent of cv.resize() the cropAndResize or bilinear ?

[vladmandic]

But that's what I thought, from what I understand, one need to empirically play with those crop and offset value until we find the best match between the 2

unfortunate, but true.

Also what would be the best equivalent of cv.resize() the cropAndResize or bilinear ?

cv.resize supports multiple resize interpolation modes, but for all intended purposes tf.image.resizeBilinear can be considered equal.
tf.image.cropAndResize is basically crop followed by resizeBilinear in one operation.

[key2]

Yep, with the difference that it has to be converted to a 0-1.0 value.

By the way, I don't have your experience in this field. Would it be possible to convert the res10 coffee model used by opencv to run it directly with tensorflow.js ? or is it only feasible by importing opencv.js

[vladmandic]

Would it be possible to convert the res10 coffee model used by opencv to run it directly with tensorflow.js

only tool that i know of that did even a semi-decent job in converting caffe models was https://github.com/Microsoft/MMdnn, but it hasn't been maintained for couple of years now. given that model in question is also quite old, it might be possible to convert it to tensorflow saved model format and from its a simple step to convert to tfjs graph model format.

one caveat is that mmdnn tool does cross-model conversions by going through onnx common format which does have some problems with alignments. solvable, but annoying.

all-in-all, likely possible but non-trivial.

or is it only feasible by importing opencv.js

i've never used opencv.js, so cant say much.