Failed test assertions cause test timeout instead of assertion failures

[mrubli]

I am unable to get failed unit test assertions to output the actual error condition. Instead, what happens is that the test times out with no hint of the actual error.

This is easily reproducible with the included lower-case_spec.js test:

Actual behavior

As long as no tests fail everything is fine:

$ npm run env -- mocha examples/lower-case_spec.js 

> [email protected] env …/node-red-node-test-helper
> env "mocha" "examples/lower-case_spec.js"

  lower-case Node
    ✓ should be loaded
    ✓ should be loaded in exported flow
    ✓ should make payload lower case

  3 passing (40ms)

Let's change the test a bit to make an assertion fail:

diff --git a/examples/lower-case_spec.js b/examples/lower-case_spec.js
index 39dc55e..3171fe7 100644
--- a/examples/lower-case_spec.js
+++ b/examples/lower-case_spec.js
@@ -36,7 +36,7 @@ describe('lower-case Node', function () {
       var n2 = helper.getNode("n2");
       var n1 = helper.getNode("n1");
       n2.on("input", function (msg) {
-        msg.should.have.property('payload', 'uppercase');
+        msg.should.have.property('payload', 'uPPercase');
         done();
       });
       n1.receive({ payload: "UpperCase" });

Note how the test times out without a trace of the assertion error:

$ npm run env -- mocha examples/lower-case_spec.js 

> [email protected] env …/node-red-node-test-helper
> env "mocha" "examples/lower-case_spec.js"

  lower-case Node
    ✓ should be loaded
    ✓ should be loaded in exported flow
    1) should make payload lower case

  2 passing (2s)
  1 failing

  1) lower-case Node
       should make payload lower case:
     Error: Timeout of 2000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (…/node-red-node-test-helper/examples/lower-case_spec.js)

Expected behavior

I would expect the test to fail something like this:

  lower-case Node
    ✓ should be loaded
    ✓ should be loaded in exported flow
    1) should make payload lower case

  2 passing (37ms)
  1 failing

  1) lower-case Node
       should make payload lower case:
     AssertionError: expected Object { payload: 'uppercase' } to have property payload of 'uPPercase' (got 'uppercase')
      at Assertion.fail (node_modules/should/cjs/should.js:275:17)
      at Assertion.value [as property] (node_modules/should/cjs/should.js:356:19)
      at Context.<anonymous> (examples/lower-case_spec.js:37:22)

Given how the test helper works I could imagine that these exceptions get swallowed by Node-RED itself, so maybe this is a design limitation. But if there's a way to workaround it or even a way to fix the problem that would be great. I haven't been able to figure it out.

[knolleary]

Yes, the error gets swallowed by the internals.

The workaround is to wrap tests code in a try/catch so you can pass the failure to the done function:

n2.on("input", function (msg) {
    try {
       msg.should.have.property('payload', 'uPPercase');
       done();
    } catch(err) {
       done(err);
   }
});

[mrubli]

@knolleary Thanks a lot for the quick response, that worked great! I could have sworn having tried something like that at one point ...

I've created PR #46 that updates both the lower-case example and the documentation to include this trick.