Issues with Zbb tests in rv32i_m suite

[neworderofjamie]

I may be missing something here but I think the Zbb tests are being generated incorrectly. For example, for CLZ:

TEST_RD_OP(clz, x31, x31, 0x00000000, 0x55555555, x1, 0, x2)  

0x55555555 has 1 rather than 0 leading zeros. Infact the correctval for all tests of clz, cpop and ctz are 0.

[allenjbaum]

I am guessing this is a case where the actual result doesn’t match the result parameter in the macro. The test itself doesn’t care what the result is, since ACTs merely compare the result with what Sail will compute. The macro has a result parameters generated by CTG, but ( for arcane reasons I don’t understand) this is a best effort only, and defaults to zero if it doesn’t know what the result should be. Tests pay no attention to that value (since the real computed result is generated by Sail)- unless the assertion macro is enabled for debug. We really should have either a warning if that value is non-zero for ops where CTG doesn’t know the answer or we should augment GTG with the correct answer (assuming there is only one answer).

…

On Monday, October 21, 2024, neworderofjamie ***@***.***> wrote: I may be missing something here but I think the Zbb tests are being generated incorrectly. For example, for CLZ: TEST_RD_OP(clz, x31, x31, 0x00000000, 0x55555555, x1, 0, x2) 0x55555555 has 1 rather than 0 leading zeros. Infact the correctval for all tests of clz, cpop and ctz are 0. — Reply to this email directly, view it on GitHub <#540>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHPXVJSSWLJMK7Q72XCUINLZ4UZPNAVCNFSM6AAAAABQKU2YMOVHI2DSMVQWIX3LMV43ASLTON2WKOZSGYYDGMRQGA2DANY> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[UmerShahidengr]

The correctval value is placed by CTG without actually calculating it, but the macro TEST_RD_OP(defined here) is not using correctval while checking the result. Since the tests are signature based, so the accuracy of the tests depends on if signatures (results) are matching with Sail and Spike.

[neworderofjamie]

Thanks for all your responses. In short, as the manual says, correctval generation is WIP in CTG and results are normally verified by comparing the in-memory output to Sail (as such in the Sail plugin, the RVMODEL_IO_ASSERT_GPR_EQ which TEST_RD_OP uses via TEST_CAST to check correctval is empty)

[allenjbaum]

We should add to the list of things to do a tool to back-annotate tests with Sail results (which will only work for tests that are implementation option independent....

…

On Wed, Oct 30, 2024 at 4:03 AM neworderofjamie ***@***.***> wrote: Thanks for all your responses. In short, as the manual says, correctval generation is WIP in CTG and results are normally verified by comparing the in-memory output to Sail (as such in the Sail plugin, the RVMODEL_IO_ASSERT_GPR_EQ which TEST_RD_OP uses via TEST_CAST to check correctval is empty) — Reply to this email directly, view it on GitHub <#540 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHPXVJR6QW6QX6MR22FLK4DZ6C4HTAVCNFSM6AAAAABQKU2YMOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINBWGYYTCNRZGI> . You are receiving this because you commented.Message ID: ***@***.***>