-
Notifications
You must be signed in to change notification settings - Fork 15
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #462 from telekom/feature/doc-update
Added Testerra status description to manual
- Loading branch information
Showing
10 changed files
with
61 additions
and
39 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,22 +1,60 @@ | ||
= Dashboard View | ||
|
||
== Overview | ||
|
||
The dashboard view gives an overview of the last test execution. It shows the total number of executed tests and of passed, skipped or failed tests. | ||
|
||
image::report-ng-05.png[align="center", alt="Dashboard"] | ||
|
||
On the Breakdown panel is a pie chart that shows the proportions, displayed with different colors according to the test status. | ||
|
||
Test status colors: | ||
[cols="2,^1,4",options="header"] | ||
Right beside the Breakdown pie chart you find another panel with the Top 3 failure types (aspects). | ||
There are two more panels below, one displaying detailed information about test execution duration, start and end times and another panel with a list of test classes that reports how many tests did pass, fail or were skipped in the individual test classes. | ||
|
||
== Status definitions | ||
|
||
Testerra defines the following *final 4 statuses* which describe the final test result: | ||
|
||
[cols="1a,4",options="header"] | ||
|=== | ||
| Test Status | Description | ||
|
||
| image::status_passed.png[] | ||
| The test was successful. | ||
|
||
| image::status_failed.png[] | ||
| The test was failed. | ||
|
||
| image::status_skipped.png[] | ||
| The test was not executed because a precondition failed like a configuration method, a data provider or an another test. | ||
|
||
| image::status_expected_failed.png[] | ||
| The test failed as expected and the test method is annotated with <<#Fails, @Fails>> because of a known bug or problem. | ||
|
||
|=== | ||
| Color | Test Status | Description | ||
| Green | Passed | The test was passed. | ||
| Yellow | Skipped | The test was not executed, because a test it depends on failed. | ||
| Red | Failed | The test failed. | ||
| Dark Red | Expected Fail | The test failed, as expected e.g. because of a known bug or problem. | ||
|
||
Additional to the statuses above Testerra defines some more _sub statuses_: | ||
|
||
[cols="1a,4",options="header"] | ||
|=== | ||
| Test Status | Description | ||
|
||
| image::status_retried.png[] | ||
| The test failed at it's first try. Testerra executed that test again if the <<#_retry_analyzer, RetryAnalyzer>> was active. | ||
|
||
| image::status_recovered.png[] | ||
| The test was passed at it's second (third...) execution if the <<#_retry_analyzer, RetryAnalyzer>> was active. | ||
|
||
| image::status_repaired.png[] | ||
| The test was passed but it is still annotated with `@Fails`. The annotation can be removed now. | ||
|
||
|=== | ||
|
||
[NOTE] | ||
==== | ||
* The dashboard only shows the final statuses in the 'Breakdown' and 'Test Classes' charts. | ||
* The 'Tests' table summarize all executed tests inclusive retried tests. Recovered and repaired tests belong to passed tests. | ||
* The tests view shows all tests with their detailed status. | ||
==== | ||
|
||
Check out the <<#Fails, @Fails>> Annotation to see how to mark a test as expected to fail. | ||
|
||
Right beside the Breakdown pie chart you find another panel with the Top 3 failure types (aspects). | ||
There are two more panels below, one displaying detailed information about test execution duration, start and end times and another panel with a list of test classes that reports how many tests did pass, fail or were skipped in the individual test classes. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.