Skip to content

Commit

Permalink
Merge pull request #462 from telekom/feature/doc-update
Browse files Browse the repository at this point in the history
Added Testerra status description to manual
  • Loading branch information
martingrossmann authored Nov 8, 2024
2 parents dbda685 + 0bf786a commit c64dace
Show file tree
Hide file tree
Showing 10 changed files with 61 additions and 39 deletions.
40 changes: 12 additions & 28 deletions docs/src/docs/doc-versions.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -3,37 +3,21 @@
* Latest Testerra 1 version: https://docs.testerra.io/testerra/1-latest/index.html
* Latest Testerra 2 version: https://docs.testerra.io/testerra/2-latest/index.html
[cols="1,4"]
|===
|Version|Link

|1.10
|https://docs.testerra.io/testerra/1.10/index.html

|1.11
|https://docs.testerra.io/testerra/1.11/index.html

|1.12
|https://docs.testerra.io/testerra/1.12/index.html

|1.13
|https://docs.testerra.io/testerra/1.13/index.html

|2.0
|https://docs.testerra.io/testerra/2.0/index.html

|2.1
|https://docs.testerra.io/testerra/2.1/index.html

|2.2
|https://docs.testerra.io/testerra/2.2/index.html

|2.4
|https://docs.testerra.io/testerra/2.4/index.html

|2.7
|https://docs.testerra.io/testerra/2.7/index.html
|1.10 | https://docs.testerra.io/testerra/1.10/index.html
|1.11 | https://docs.testerra.io/testerra/1.11/index.html
|1.12 | https://docs.testerra.io/testerra/1.12/index.html
|1.13 | https://docs.testerra.io/testerra/1.13/index.html
|2.0 | https://docs.testerra.io/testerra/2.0/index.html
|2.1 | https://docs.testerra.io/testerra/2.1/index.html
|2.2 | https://docs.testerra.io/testerra/2.2/index.html
|2.4 | https://docs.testerra.io/testerra/2.4/index.html
|2.6 | https://docs.testerra.io/testerra/2.5/index.html
|2.7 | https://docs.testerra.io/testerra/2.7/index.html
|2.8 | https://docs.testerra.io/testerra/2.8/index.html
|2.9 | https://docs.testerra.io/testerra/2.9/index.html
|===

You will find detailed release notes at https://github.com/telekom/testerra/releases.
58 changes: 48 additions & 10 deletions docs/src/docs/reports/dashboard.adoc
Original file line number Diff line number Diff line change
@@ -1,22 +1,60 @@
= Dashboard View

== Overview

The dashboard view gives an overview of the last test execution. It shows the total number of executed tests and of passed, skipped or failed tests.

image::report-ng-05.png[align="center", alt="Dashboard"]

On the Breakdown panel is a pie chart that shows the proportions, displayed with different colors according to the test status.

Test status colors:
[cols="2,^1,4",options="header"]
Right beside the Breakdown pie chart you find another panel with the Top 3 failure types (aspects).
There are two more panels below, one displaying detailed information about test execution duration, start and end times and another panel with a list of test classes that reports how many tests did pass, fail or were skipped in the individual test classes.

== Status definitions

Testerra defines the following *final 4 statuses* which describe the final test result:

[cols="1a,4",options="header"]
|===
| Test Status | Description

| image::status_passed.png[]
| The test was successful.

| image::status_failed.png[]
| The test was failed.

| image::status_skipped.png[]
| The test was not executed because a precondition failed like a configuration method, a data provider or an another test.

| image::status_expected_failed.png[]
| The test failed as expected and the test method is annotated with <<#Fails, @Fails>> because of a known bug or problem.

|===
| Color | Test Status | Description
| Green | Passed | The test was passed.
| Yellow | Skipped | The test was not executed, because a test it depends on failed.
| Red | Failed | The test failed.
| Dark Red | Expected Fail | The test failed, as expected e.g. because of a known bug or problem.

Additional to the statuses above Testerra defines some more _sub statuses_:

[cols="1a,4",options="header"]
|===
| Test Status | Description

| image::status_retried.png[]
| The test failed at it's first try. Testerra executed that test again if the <<#_retry_analyzer, RetryAnalyzer>> was active.

| image::status_recovered.png[]
| The test was passed at it's second (third...) execution if the <<#_retry_analyzer, RetryAnalyzer>> was active.

| image::status_repaired.png[]
| The test was passed but it is still annotated with `@Fails`. The annotation can be removed now.

|===

[NOTE]
====
* The dashboard only shows the final statuses in the 'Breakdown' and 'Test Classes' charts.
* The 'Tests' table summarize all executed tests inclusive retried tests. Recovered and repaired tests belong to passed tests.
* The tests view shows all tests with their detailed status.
====

Check out the <<#Fails, @Fails>> Annotation to see how to mark a test as expected to fail.

Right beside the Breakdown pie chart you find another panel with the Top 3 failure types (aspects).
There are two more panels below, one displaying detailed information about test execution duration, start and end times and another panel with a list of test classes that reports how many tests did pass, fail or were skipped in the individual test classes.
2 changes: 1 addition & 1 deletion docs/src/docs/reports/print-preview.adoc
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
= Print Preview
= Print preview

If you want to consolidate all report information into a single PDF file or print it directly, the "Print Report" feature
can be accessed by clicking the button located at the bottom of the navigation drawer.
Expand Down
Binary file added docs/src/images/status_expected_failed.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/src/images/status_failed.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/src/images/status_passed.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/src/images/status_recovered.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/src/images/status_repaired.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/src/images/status_retried.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/src/images/status_skipped.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit c64dace

Please sign in to comment.