-
Notifications
You must be signed in to change notification settings - Fork 6.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
scripts: twister: Fix NOTRUN in test_only #80221
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -354,6 +354,12 @@ def json_report(self, filename, version="NA", platform=None, filters=None): | |
elif instance.status == TwisterStatus.SKIP: | ||
suite["status"] = TwisterStatus.SKIP | ||
suite["reason"] = instance.reason | ||
elif instance.status == TwisterStatus.NOTRUN: | ||
suite["status"] = TwisterStatus.NOTRUN | ||
suite["reason"] = instance.reason | ||
else: | ||
suite["status"] = TwisterStatus.NONE | ||
suite["reason"] = 'Unknown Instance status.' | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. same as above |
||
|
||
if instance.status != TwisterStatus.NONE: | ||
suite["execution_time"] = f"{float(handler_time):.2f}" | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -652,6 +652,9 @@ def load_from_file(self, file, filter_platform=[]): | |
self.hwm | ||
) | ||
|
||
if self.options.test_only and not instance.run: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. hmm, what is the reason for this ? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. seems ok, this is my test result.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. according to the fix logic it is platform naming issue, which is not related to test status.
and rebase to latest it fails.
and apply #80346 also works for short name, but there is CI issue need fix There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
This will remove unrunnable test instances from the To illustrate: Imagine that a Previously, the second instance would be read and included in the report as a I'm currently checking rebase locally. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
I'm not sure it is good. In |
||
continue | ||
|
||
instance.metrics['handler_time'] = ts.get('execution_time', 0) | ||
instance.metrics['used_ram'] = ts.get("used_ram", 0) | ||
instance.metrics['used_rom'] = ts.get("used_rom",0) | ||
|
@@ -669,9 +672,9 @@ def load_from_file(self, file, filter_platform=[]): | |
instance.status = TwisterStatus.NONE | ||
instance.reason = None | ||
instance.retries += 1 | ||
# test marked as passed (built only) but can run when | ||
# --test-only is used. Reset status to capture new results. | ||
elif status == TwisterStatus.PASS and instance.run and self.options.test_only: | ||
# test marked as built only can run when --test-only is used. | ||
# Reset status to capture new results. | ||
elif status == TwisterStatus.NOTRUN and instance.run and self.options.test_only: | ||
instance.status = TwisterStatus.NONE | ||
instance.reason = None | ||
else: | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this line need to removed to pass the test_report.py blackbox testing, as it expect to be no reason when NOTRUN
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In my opinion, the errors were caused by an assumption in the
test_footprint.py
. Proper usage of the NOTRUN status has broken that assumption, so I've fixed the test.