Skip to content

Commit

Permalink
feat: plugin subsystem (#15)
Browse files Browse the repository at this point in the history 
* fix: broken/failed status support
  • Loading branch information
@noomorph
noomorph authored Nov 10, 2023
1 parent 2fc8193 commit 77f4c5a
Show file tree
Hide file tree
Showing 65 changed files with 1,592 additions and 932 deletions.
1 change: 1 addition & 0 deletions .eslintrc.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ module.exports = {
"unicorn/no-null": "off",
"prefer-rest-params": "off",
"@typescript-eslint/ban-types": "off",
"@typescript-eslint/triple-slash-reference": "off",
"unicorn/no-this-assignment": "off",
"unicorn/prefer-event-target": "off",
"unicorn/prefer-module": "off",
Expand Down
262 changes: 188 additions & 74 deletions docs/docs/api/01-descriptions.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,72 +14,89 @@ Please use GitHub docs for the latest stable version, `1.x.x`.

:::

A well-written description can enhance the usefulness of a test by providing clear context and expected outcomes. It can also assist in deciphering test failures.
A well-written description can enhance the usefulness of a test by providing clear context and expected outcomes.
Anyone who sees your test for the first time may benefit from a rich description.

There are two ways to define a description for an executable block:
There are two ways to define a description for a test:

* declaratively, via `@desc`, `@description`, `@descriptionHtml` JSDoc annotations (or even without them);
* programmatically, via `$Description('')` or `$DescriptionHtml('')` annotation functions.
* declaratively, via `@desc`, `@description`, `@descriptionHtml` docblocks (or even without them);
* programmatically, via our DSL – `$Description` or `$DescriptionHtml` pseudo-decorators.

The **description** can be defined for the built-in `it`, `test`, `beforeAll`, `beforeEach`, `afterAll`, `afterEach` blocks and [custom steps](02-steps.mdx).
:::info Note

This article will focus on the built-in blocks.
Descriptions are **not supported** on a _suite_ (`describe`) or _test hooks_ (`beforeAll`, `beforeEach`, `afterAll`, `afterEach`) level due to limitations of Jest and Allure Framework.

Anyway, we have a few workarounds for you down below. :wink:

:::

## Test cases

Allure Framework supports rich text descriptions for tests. But don't worry if you don't provide a description. The reporter will still present the test neatly by displaying the source code of the test.
Allure Framework supports rich text descriptions for tests in Markdown and HTML formats.

To make your experience better, `jest-allure2-reporter` appends a _source code_ of every
test to its description, so you can always get value from this feature.

<Tabs groupId="approach">
<TabItem value="jsdoc" label="JSDoc">
<TabItem value="docblock" label="Docblocks">

:::info Note

Docblocks must be **inside** the test function to work.

:::


```js
it('should add two numbers', () => {
test('should add two numbers', () => {
/**
* _Implicit_ annotation without `@desc` or `@description`.
*/
const a = 1;
const b = 2;
const sum = a + b;
expect(sum).toBe(3);
* This test demonstrates the `+` operator.
*/
expect(1 + 2).toBe(3);
});

it('should subtract two numbers', () => {
test('should multiply two numbers', () => {
/**
* @descriptionHtml
* <i>Explicit</i> annotation with <code>@descriptionHtml</code>.
*/
const a = 1;
const b = 2;
const diff = b - a;
expect(diff).toBe(1);
* @description
* This test demonstrates the `*` operator.
*/
expect(3 * 2).toBe(6);
});

test('should subtract two numbers', () => {
/**
* @descriptionHtml
* This test demonstrates the <code>-</code> operator.
*/
expect(2 - 1).toBe(1);
});
```

</TabItem>
<TabItem value="dsl" label="Function">
<TabItem value="dsl" label="DSL">

:::info Note

Pseudo-annotations must be **before** the `test` statement to work.

:::

```js
import { $Description } from 'jest-allure2-reporter';

$Description('_Implicit_ annotation without `@desc` or `@description`.')
it('should add two numbers', () => {
const a = 1;
const b = 2;
const sum = a + b;
expect(sum).toBe(3);
$Description('This test demonstrates the `+` operator.')
test('should add two numbers', () => {
expect(1 + 2).toBe(3);
});

$DescriptionHtml('<i>Explicit</i> annotation with <code>@descriptionHtml</code>.')
it('should subtract two numbers', () => {
const a = 1;
const b = 2;
const diff = b - a;
expect(diff).toBe(1);
$DescriptionHtml('This test demonstrates the <code>-</code> operator.')
test('should subtract two numbers', () => {
expect(2 - 1).toBe(1);
});
```

</TabItem>
<TabItem value="demo" label="Demo">
<TabItem value="demo" label="Preview">

TODO: add screenshot

Expand All @@ -88,32 +105,27 @@ it('should subtract two numbers', () => {

## Test hooks

Test hooks such as `beforeAll`, `beforeEach`, `afterAll`, `afterEach` can also have descriptions since technically they are considered [steps](02-steps.mdx) in the Allure report.

However, they are limited to plain text descriptions only due to Allure Framework limitations.
Test hooks such as `beforeAll`, `beforeEach`, `afterAll`, `afterEach` are treated [as steps](02-steps.mdx) in Allure Framework. Therefore, they can have only a plain name, but no description.

<Tabs groupId="approach">
<TabItem value="jsdoc" label="JSDoc">
<TabItem value="docblock" label="Docblocks">

```js
beforeAll(() => {
/** This hook runs before all tests. */
});

beforeEach(() => {
/** @desc This hook runs before each test. */
/** This hook runs before each test. */
});

afterEach(() => {
/**
* @description
* This hook runs after each test.
*/
/** This hook runs after each test. */
});
```

</TabItem>
<TabItem value="dsl" label="Function">
<TabItem value="dsl" label="DSL">

```js
import { $Description } from 'jest-allure2-reporter';
Expand All @@ -135,7 +147,7 @@ afterEach(() => {
```

</TabItem>
<TabItem value="demo" label="Demo">
<TabItem value="demo" label="Preview">

TODO: add screenshot

Expand All @@ -145,47 +157,109 @@ afterEach(() => {

## Test suites

Unfortunately, it is not possible to define a description for the entire test suite or an individual test suite (`describe` block) due to Allure Framework limitations.
Allure Framework doesn't treat test suites as separate entities, so the best we can offer is to prepend their descriptions to every test within the suite.

However, if you add a description on top of the test suite, it will be prepended to every test description within the suite, e.g.:
Due to Jest limitations, you can't use docblocks on a suite level, so the only way to add a description is to use our DSL.

<Tabs groupId="approach">
<TabItem value="jsdoc" label="JSDoc">
<TabItem value="dsl" label="DSL">

```js
describe('Sanity: Login flow', () => {
/** This description will be prepended to every test description. */
import { $Description } from 'jest-allure2-reporter';

$Description('The test is operating on `/login` page.')
describe('Sanity: Login flow', () => {
it('should login with valid credentials', () => {
/** This test logs in with valid credentials. */
/** Testing the transition to the `/dashboard` page. */
// ...
});

it('should login with invalid credentials', () => {
/** This test logs in with invalid credentials. */
/** Testing the validation summary component. */
// ...
});
});
```

</TabItem>
<TabItem value="dsl" label="Function">
<TabItem value="demo" label="Preview">

TODO: add screenshot

</TabItem>
</Tabs>

## Test files

In many cases you may find it acceptable to describe the whole test file, which usually is equal to adding a description to the top-level `describe` block. More often than not you have a single top-level `describe` block, so you won't notice the difference:

<Tabs groupId="approach">
<TabItem value="docblock" label="Docblocks">

```js
/**
* @description
* The test is operating on `/login` page.
*/
import { $Description } from 'jest-allure2-reporter';

$Description('This description will be prepended to every test description.')
describe('Sanity: Login flow', () => {
$Description('This test logs in with valid credentials.')
it('should login with valid credentials', () => {
/** Testing the transition to the `/dashboard` page. */
// ...
});

it('should login with invalid credentials', () => {
/** Testing the validation summary component. */
// ...
});
});
```

:::info Note

You **must** use `@desc` or `@description` pragma due to Jest limitations regarding file-level docblocks.

:::

</TabItem>
<TabItem value="dsl" label="DSL">

```js
import { allure } from 'jest-allure2-reporter';

allure.description('The test is operating on `/login` page.')

describe('Sanity: Login flow', () => {
it('should login with valid credentials', () => {
/** Testing the transition to the `/dashboard` page. */
// ...
});

$Description('This test logs in with invalid credentials.')
it('should login with invalid credentials', () => {
/** Testing the validation summary component. */
// ...
});
});
```

:::info Note

We use `allure.description` to ensure that the metadata is added to exactly to the actual context, which is the **test file** itself.

To simulate the behavior of `$Description` pseudo-decorator, we'd have to put it inside the `describe` block:

```js
describe('Sanity: Login flow', () => {
allure.description('The test is operating on `/login` page.')
// ...
});
```

:::

</TabItem>
<TabItem value="demo" label="Demo">
<TabItem value="demo" label="Preview">

TODO: add screenshot

Expand All @@ -194,23 +268,63 @@ describe('Sanity: Login flow', () => {

## Configuration

:::caution Work in progress
:::
### Description template

You can configure whether the source code of the test is included or not, using the `includeSourceCode` and `omitJSDoc` options. By default, both are set to `true`.
As mentioned before, a test description is a sequence of user-defined paragraphs, followed by a source code of the test itself.

As a result, the following test will be reported without JSDoc annotations:
To customize the template, you can use `description` option in your `jest.config.js`. Below is a rough example of how you can do it:

```js
it('should add two numbers', () => {
/**
* This test adds two numbers.
*/
const a = 1;
const b = 2;
const sum = a + b;
expect(sum).toBe(3);
});
/** @type {import('@jest/types').Config.InitialOptions} */
module.exports = {
testEnvironment: 'jest-allure2-reporter/environment-node',
reporters: [
'default',
[
'jest-allure2-reporter',
/** @type {import('jest-allure2-reporter').Options} */
{
testCase: {
description: ({ testCaseMetadata }) => [
...testCaseMetadata.description,
'```js',
...(testCaseMetadata.code?.beforeAll ?? []),
...(testCaseMetadata.code?.beforeEach ?? []),
...(testCaseMetadata.code?.test ?? []),
...(testCaseMetadata.code?.afterEach ?? []),
...(testCaseMetadata.code?.afterAll ?? []),
'```',
].join('\n\n'),
}
}
],
};
```
To switch to a HTML template, reset the `description` customizer to
return undefined and use `descriptionHtml` option instead:
```js
/** @type {import('@jest/types').Config.InitialOptions} */
module.exports = {
testEnvironment: 'jest-allure2-reporter/environment-node',
reporters: [
'default',
[
'jest-allure2-reporter',
/** @type {import('jest-allure2-reporter').Options} */
{
testCase: {
description: () => {}, // suppress the default template
descriptionHtml: ({ testCaseMetadata }) => { /* ... */ },
}
}
],
};
```
TODO: See the configuration section for more details.
### Markdown support
By default, `jest-allure2-reporter` uses `remark` processor to render Markdown descriptions. It is not possible to customize it right now, but we're working on it.
You'll be able to define your own `remark` plugins and configure the processor in one of the next releases.
Loading

0 comments on commit 77f4c5a

Please sign in to comment.