diff --git a/specification/README.md b/specification/README.md index 01e9c2a2..57d0d65f 100644 --- a/specification/README.md +++ b/specification/README.md @@ -16,7 +16,8 @@ sidebar_position: 0 - [Evaluation Context](./sections/03-evaluation-context.md) - [Hooks](./sections/04-hooks.md) - [Events](./sections/05-events.md) -- [Appendix-A](./appendix-A.md) +- [Appendix A: Included Utilities](./appendix-a-included-utilities.md) +- [Appendix B: Gherkin Suites](./appendix-b-gherkin-suites.md) ## Conformance diff --git a/specification/appendix-A.md b/specification/appendix-a-included-utilities.md similarity index 100% rename from specification/appendix-A.md rename to specification/appendix-a-included-utilities.md diff --git a/specification/appendix-b-gherkin-suites.md b/specification/appendix-b-gherkin-suites.md new file mode 100644 index 00000000..3aeb49ce --- /dev/null +++ b/specification/appendix-b-gherkin-suites.md @@ -0,0 +1,16 @@ +--- +id: appendix-b +title: "Appendix B: Gherkin Suites" +description: A Set of End-to-End Tests for Validating OpenFeature Implementations +sidebar_position: 5 +--- + +# Appendix B: Gherkin Suites + +This section contains a set of language-agnostic end-to-end tests (defined in gherkin). +These tests can be used to validate the behavior of an OpenFeature implementation. +"Features" (test suites) can be used in conjunction with a cucumber test-runner for the language in question. + +## Evaluation Feature + +The [evaluation feature](./assets/gherkin/evaluation.feature) contains tests for the basic functionality of the [Evaluation API](./sections/01-flag-evaluation.md). diff --git a/specification/assets/gherkin/evaluation.feature b/specification/assets/gherkin/evaluation.feature new file mode 100644 index 00000000..250f8b01 --- /dev/null +++ b/specification/assets/gherkin/evaluation.feature @@ -0,0 +1,67 @@ +Feature: Flag evaluation + +# This test suite contains scenarios to test the flag evaluation API. + + Background: + Given a provider is registered with cache disabled + + # basic evaluation + Scenario: Resolves boolean value + When a boolean flag with key "boolean-flag" is evaluated with default value "false" + Then the resolved boolean value should be "true" + + Scenario: Resolves string value + When a string flag with key "string-flag" is evaluated with default value "bye" + Then the resolved string value should be "hi" + + Scenario: Resolves integer value + When an integer flag with key "integer-flag" is evaluated with default value 1 + Then the resolved integer value should be 10 + + Scenario: Resolves float value + When a float flag with key "float-flag" is evaluated with default value 0.1 + Then the resolved float value should be 0.5 + + Scenario: Resolves object value + When an object flag with key "object-flag" is evaluated with a null default value + Then the resolved object value should be contain fields "showImages", "title", and "imagesPerPage", with values "true", "Check out these pics!" and 100, respectively + + # detailed evaluation + Scenario: Resolves boolean details + When a boolean flag with key "boolean-flag" is evaluated with details and default value "false" + Then the resolved boolean details value should be "true", the variant should be "on", and the reason should be "STATIC" + + Scenario: Resolves string details + When a string flag with key "string-flag" is evaluated with details and default value "bye" + Then the resolved string details value should be "hi", the variant should be "greeting", and the reason should be "STATIC" + + Scenario: Resolves integer details + When an integer flag with key "integer-flag" is evaluated with details and default value 1 + Then the resolved integer details value should be 10, the variant should be "ten", and the reason should be "STATIC" + + Scenario: Resolves float details + When a float flag with key "float-flag" is evaluated with details and default value 0.1 + Then the resolved float details value should be 0.5, the variant should be "half", and the reason should be "STATIC" + + Scenario: Resolves object details + When an object flag with key "object-flag" is evaluated with details and a null default value + Then the resolved object details value should be contain fields "showImages", "title", and "imagesPerPage", with values "true", "Check out these pics!" and 100, respectively + And the variant should be "template", and the reason should be "STATIC" + + # context-aware evaluation + Scenario: Resolves based on context + When context contains keys "fn", "ln", "age", "customer" with values "Sulisław", "Świętopełk", 29, "false" + And a flag with key "context-aware" is evaluated with default value "EXTERNAL" + Then the resolved string response should be "INTERNAL" + And the resolved flag value is "EXTERNAL" when the context is empty + + # errors + Scenario: Flag not found + When a non-existent string flag with key "missing-flag" is evaluated with details and a default value "uh-oh" + Then the default string value should be returned + And the reason should indicate an error and the error code should indicate a missing flag with "FLAG_NOT_FOUND" + + Scenario: Type error + When a string flag with key "wrong-flag" is evaluated as an integer, with details and a default value 13 + Then the default integer value should be returned + And the reason should indicate an error and the error code should indicate a type mismatch with "TYPE_MISMATCH" \ No newline at end of file