[Fix] in Xnnpack EP, the conversion for fused activation param isn't correct

[mszhanyi]

Description

In Xnnpack EP, the activation_param's conversion isn't correct for Fp16 model
Sometimes, it may cause an exception that "lower bound must be below upper bound"
Because CPU EP doesn't support FP16 activation fusion now, so the newly added test skips the comparison of the test result.

Motivation and Context

Test Cases

2024-12-23T09:17:39.5038317Z [ RUN      ] XnnpackEP.TestNhwcConvReluFusion_FP16
2024-12-23T09:17:39.5079188Z �[0;93m2024-12-23 09:17:39.505334389 [W:onnxruntime:TestNhwcConvReluFusion_FP16, session_state.cc:1263 VerifyEachNodeIsAssignedToAnEp] Some nodes were not assigned to the preferred execution providers which may or may not have an negative impact on performance. e.g. ORT explicitly assigns shape related ops to CPU to improve perf.�[m
2024-12-23T09:17:39.5080635Z �[0;93m2024-12-23 09:17:39.505405629 [W:onnxruntime:TestNhwcConvReluFusion_FP16, session_state.cc:1265 VerifyEachNodeIsAssignedToAnEp] Rerunning with verbose output on a non-minimal build will show node assignments.�[m
2024-12-23T09:17:39.5099292Z [       OK ] XnnpackEP.TestNhwcConvReluFusion_FP16 (5 ms)
2024-12-23T09:17:39.5100453Z [ RUN      ] XnnpackEP.TestNhwcConvReluClipFusion_FP16
2024-12-23T09:17:39.5145494Z [       OK ] XnnpackEP.TestNhwcConvReluClipFusion_FP16 (5 ms)

Linux XnnPack on ARM64
https://dev.azure.com/onnxruntime/onnxruntime/_build/results?buildId=1576767&view=logs&j=317a4805-d006-5aac-2a82-238793a22a22&t=5481769e-95ca-5a84-acab-996d49e28b59

[wejoncy]

What if GetType(arg, arg_type) failed here?

[skottmckay]

Generally type info is always available, so I think this is ok. Shape info may be missing depending on the model.

The Conv op looks to be setup to allow fp32, u8, s8 and optionally fp16. Should this also handle u8 and s8 or should ClipReluChecker limit fusion to fp32 and fp16?

[mszhanyi]

So far, core runtime Clip fusion only supports float too.

onnxruntime/onnxruntime/core/optimizer/utils.cc

Lines 335 to 349 in c6ba7ed

	if (initializer) {
	Initializer i(*initializer, graph.ModelPath());
	switch (initializer->data_type()) {
	case ONNX_NAMESPACE::TensorProto_DataType_FLOAT:
	value = *i.data<float>();
	break;
	// double isn't currently supported
	// case ONNX_NAMESPACE::TensorProto_DataType_DOUBLE:
	// value = static_cast<float>(*i.data<double>());
	// break;
	case ONNX_NAMESPACE::TensorProto_DataType_FLOAT16:
	value = math::halfToFloat(i.data<MLFloat16>()->val);
	break;
	default:
	ORT_THROW("Unexpected data type for Clip input of ", initializer->data_type());

.
Shall we update them together?

[mszhanyi]

cc @snnn

[skottmckay]

I'd leave the core Clip fusion as-is for now. Can be a separate PR if we think there's a use-case that would benefit.

Are you planning on updating ClipReluChecker to limit the types?

[skottmckay]

Not quite following. There should still be valid output from the CPU EP even if it doesn't fuse, so why can't we use verify_outputs?

Suggested change

	// So far, CPU EP doensn't support Fp16 Conv fusion, so verify_outputs is skipped.
	RunAndVerifyOutputsWithEP(ort_model_path, "TestNhwcConvReluClipFusion_FP16", std::move(ep), feeds, params, {}, false);
	// So far, CPU EP doesn't support Fp16 Conv fusion, so verify_outputs is skipped.
	RunAndVerifyOutputsWithEP(ort_model_path, "TestNhwcConvReluClipFusion_FP16", std::move(ep), feeds, params, {}, false);

[mszhanyi]

thx, fixed

[mszhanyi]

So far, CPU EP doesn't implement FP16 Clip fusion. The output verification fails because it looks CPU EP falls back to FP32 Clip.

onnxruntime/onnxruntime/core/providers/cpu/fp16/fp16_activations.h

Lines 74 to 77 in e0e8304

	// TODO Add the following activations:
	// MlasTanhActivation,
	// MlasLogisticActivation,
	// MlasClipActivation,

To verify the Xnnpack FP16 conv fusion correctness, I add a new test with a new FP16 model ( with only Conv+Relu).
Current test (Conv+Clip+Relu) is kept because I want to make sure that Conv+Clip fusion can run, that is, the activition parameters are added correctly.

[github-actions]

You can commit the suggested changes from lintrunner.

[github-actions]

You can commit the suggested changes from lintrunner.

[github-actions]

Suggested change

	: value.float_data()[0];
	}
	}
	: value.float_data()[0];
	}
	}