Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add appendix b, gherkin suite #203

Merged
merged 4 commits into from
Aug 29, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion specification/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@ sidebar_position: 0
- [Evaluation Context](./sections/03-evaluation-context.md)
- [Hooks](./sections/04-hooks.md)
- [Events](./sections/05-events.md)
- [Appendix-A](./appendix-A.md)
- [Appendix A: Included Utilities](./appendix-a-included-utilities.md)
Kavindu-Dodan marked this conversation as resolved.
Show resolved Hide resolved
- [Appendix B: Gherkin Suites](./appendix-b-gherkin-suites.md)

## Conformance

Expand Down
16 changes: 16 additions & 0 deletions specification/appendix-b-gherkin-suites.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
---
id: appendix-b
title: "Appendix B: Gherkin Suites"
description: A Set of End-to-End Tests for Validating OpenFeature Implementations
sidebar_position: 5
---

# Appendix B: Gherkin Suites

This section contains a set of language-agnostic end-to-end tests (defined in gherkin).
These tests can be used to validate the behavior of an OpenFeature implementation.
"Features" (test suites) can be used in conjunction with a cucumber test-runner for the language in question.

## Evaluation Feature

The [evaluation feature](./assets/gherkin/evaluation.feature) contains tests for the basic functionality of the [Evaluation API](./sections/01-flag-evaluation.md).
67 changes: 67 additions & 0 deletions specification/assets/gherkin/evaluation.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
Feature: Flag evaluation

# This test suite contains scenarios to test the flag evaluation API.

Background:
Given a provider is registered with cache disabled

# basic evaluation
Scenario: Resolves boolean value
When a boolean flag with key "boolean-flag" is evaluated with default value "false"
Then the resolved boolean value should be "true"

Scenario: Resolves string value
When a string flag with key "string-flag" is evaluated with default value "bye"
Then the resolved string value should be "hi"

Scenario: Resolves integer value
When an integer flag with key "integer-flag" is evaluated with default value 1
Then the resolved integer value should be 10

Scenario: Resolves float value
When a float flag with key "float-flag" is evaluated with default value 0.1
Then the resolved float value should be 0.5

Scenario: Resolves object value
When an object flag with key "object-flag" is evaluated with a null default value
Then the resolved object value should be contain fields "showImages", "title", and "imagesPerPage", with values "true", "Check out these pics!" and 100, respectively

# detailed evaluation
Scenario: Resolves boolean details
When a boolean flag with key "boolean-flag" is evaluated with details and default value "false"
Then the resolved boolean details value should be "true", the variant should be "on", and the reason should be "STATIC"

Scenario: Resolves string details
When a string flag with key "string-flag" is evaluated with details and default value "bye"
Then the resolved string details value should be "hi", the variant should be "greeting", and the reason should be "STATIC"

Scenario: Resolves integer details
When an integer flag with key "integer-flag" is evaluated with details and default value 1
Then the resolved integer details value should be 10, the variant should be "ten", and the reason should be "STATIC"

Scenario: Resolves float details
When a float flag with key "float-flag" is evaluated with details and default value 0.1
Then the resolved float details value should be 0.5, the variant should be "half", and the reason should be "STATIC"

Scenario: Resolves object details
When an object flag with key "object-flag" is evaluated with details and a null default value
Then the resolved object details value should be contain fields "showImages", "title", and "imagesPerPage", with values "true", "Check out these pics!" and 100, respectively
And the variant should be "template", and the reason should be "STATIC"

# context-aware evaluation
Scenario: Resolves based on context
When context contains keys "fn", "ln", "age", "customer" with values "Sulisław", "Świętopełk", 29, "false"
And a flag with key "context-aware" is evaluated with default value "EXTERNAL"
Then the resolved string response should be "INTERNAL"
And the resolved flag value is "EXTERNAL" when the context is empty

# errors
Scenario: Flag not found
When a non-existent string flag with key "missing-flag" is evaluated with details and a default value "uh-oh"
Then the default string value should be returned
And the reason should indicate an error and the error code should indicate a missing flag with "FLAG_NOT_FOUND"

Scenario: Type error
When a string flag with key "wrong-flag" is evaluated as an integer, with details and a default value 13
Then the default integer value should be returned
And the reason should indicate an error and the error code should indicate a type mismatch with "TYPE_MISMATCH"
Loading