Use node-canary to run interpreter-generated JS tests

[sbc100]

No description provided.

[backes]

Don't we need an opt-out mechanism to be able to land tests that node/V8 doesn't currently pass?

[sbc100]

Don't we need an opt-out mechanism to be able to land tests that node/V8 doesn't currently pass?

Yes, good point. The comment I'm replacing here seems to suggest that just commenting this line out with a TODO is one way to skip these days when working on such a proposal.

Perhaps we should separate the "run tests in interp" and "run tests under node/v8" so that its easy to comment out just the later?

[rossberg]

A proper opt-out mechanism for in the test runner would be useful. We also talked about this in the context of individual (negative) tests that conflict with a yet unfinished proposal. But designing and implementing this properly seems like a lot of work.

As for separate targets, wouldn't that still involve commenting out a line?

[rossberg]

Suggested change

	NODE=$HOME/node-v${NODE_VERSION}-linux-x64/bin/node
	cd interpreter && opam exec make "JS=$NODE --wasm-staging" ci
	NODE="$HOME/node-v${NODE_VERSION}-linux-x64/bin/node --wasm-staging"
	cd interpreter && opam exec make JS=$NODE ci

[backes]

Yeah, I see that it's quite some work, but I think long-term we really need to be able to skip individual tests. Otherwise we always have the risk of introducing bigger problems like #1828.

Splitting is at least a step in the right direction, but would not have prevented #1828.

[rossberg]

@backes:

Splitting is at least a step in the right direction

Not sure I see how. AFAICS, it does not bring us any closer. I mean, I'm not opposed, but I don't think it makes a difference one way or the other, other than complicating the test runner in different ways.

In any case, I think neither should block the current PR, which still is a net improvement.

[sbc100]

It seems like ref_null.wast is failing under node, @backes do you know if this is know failure of current v8 against the wasm-3.0 spec tests?

[sbc100]

Apparently there is a fix in v8 that landed just this morning that might be fix this, so I guess we are already running into the issue of needing to selectively disable tests. I'll give this PR another try once a new node-canary is published.