Skip to content

Commit

Permalink
feat: add appendix b, gherkin suite (#203)
Browse files Browse the repository at this point in the history
Signed-off-by: Todd Baert <[email protected]>
Co-authored-by: Kavindu Dodanduwa <[email protected]>
Co-authored-by: Liran M <[email protected]>
  • Loading branch information
3 people authored Aug 29, 2023
1 parent f45bb27 commit 230bad2
Show file tree
Hide file tree
Showing 4 changed files with 85 additions and 1 deletion.
3 changes: 2 additions & 1 deletion specification/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@ sidebar_position: 0
- [Evaluation Context](./sections/03-evaluation-context.md)
- [Hooks](./sections/04-hooks.md)
- [Events](./sections/05-events.md)
- [Appendix-A](./appendix-A.md)
- [Appendix A: Included Utilities](./appendix-a-included-utilities.md)
- [Appendix B: Gherkin Suites](./appendix-b-gherkin-suites.md)

## Conformance

Expand Down
File renamed without changes.
16 changes: 16 additions & 0 deletions specification/appendix-b-gherkin-suites.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
---
id: appendix-b
title: "Appendix B: Gherkin Suites"
description: A Set of End-to-End Tests for Validating OpenFeature Implementations
sidebar_position: 5
---

# Appendix B: Gherkin Suites

This section contains a set of language-agnostic end-to-end tests (defined in gherkin).
These tests can be used to validate the behavior of an OpenFeature implementation.
"Features" (test suites) can be used in conjunction with a cucumber test-runner for the language in question.

## Evaluation Feature

The [evaluation feature](./assets/gherkin/evaluation.feature) contains tests for the basic functionality of the [Evaluation API](./sections/01-flag-evaluation.md).
67 changes: 67 additions & 0 deletions specification/assets/gherkin/evaluation.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
Feature: Flag evaluation

# This test suite contains scenarios to test the flag evaluation API.

Background:
Given a provider is registered with cache disabled

# basic evaluation
Scenario: Resolves boolean value
When a boolean flag with key "boolean-flag" is evaluated with default value "false"
Then the resolved boolean value should be "true"

Scenario: Resolves string value
When a string flag with key "string-flag" is evaluated with default value "bye"
Then the resolved string value should be "hi"

Scenario: Resolves integer value
When an integer flag with key "integer-flag" is evaluated with default value 1
Then the resolved integer value should be 10

Scenario: Resolves float value
When a float flag with key "float-flag" is evaluated with default value 0.1
Then the resolved float value should be 0.5

Scenario: Resolves object value
When an object flag with key "object-flag" is evaluated with a null default value
Then the resolved object value should be contain fields "showImages", "title", and "imagesPerPage", with values "true", "Check out these pics!" and 100, respectively

# detailed evaluation
Scenario: Resolves boolean details
When a boolean flag with key "boolean-flag" is evaluated with details and default value "false"
Then the resolved boolean details value should be "true", the variant should be "on", and the reason should be "STATIC"

Scenario: Resolves string details
When a string flag with key "string-flag" is evaluated with details and default value "bye"
Then the resolved string details value should be "hi", the variant should be "greeting", and the reason should be "STATIC"

Scenario: Resolves integer details
When an integer flag with key "integer-flag" is evaluated with details and default value 1
Then the resolved integer details value should be 10, the variant should be "ten", and the reason should be "STATIC"

Scenario: Resolves float details
When a float flag with key "float-flag" is evaluated with details and default value 0.1
Then the resolved float details value should be 0.5, the variant should be "half", and the reason should be "STATIC"

Scenario: Resolves object details
When an object flag with key "object-flag" is evaluated with details and a null default value
Then the resolved object details value should be contain fields "showImages", "title", and "imagesPerPage", with values "true", "Check out these pics!" and 100, respectively
And the variant should be "template", and the reason should be "STATIC"

# context-aware evaluation
Scenario: Resolves based on context
When context contains keys "fn", "ln", "age", "customer" with values "Sulisław", "Świętopełk", 29, "false"
And a flag with key "context-aware" is evaluated with default value "EXTERNAL"
Then the resolved string response should be "INTERNAL"
And the resolved flag value is "EXTERNAL" when the context is empty

# errors
Scenario: Flag not found
When a non-existent string flag with key "missing-flag" is evaluated with details and a default value "uh-oh"
Then the default string value should be returned
And the reason should indicate an error and the error code should indicate a missing flag with "FLAG_NOT_FOUND"

Scenario: Type error
When a string flag with key "wrong-flag" is evaluated as an integer, with details and a default value 13
Then the default integer value should be returned
And the reason should indicate an error and the error code should indicate a type mismatch with "TYPE_MISMATCH"

0 comments on commit 230bad2

Please sign in to comment.