Update the beta distribution
parameters
#647
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Update the beta distribution parameters in the
_select_variation_index
method to avoid bias towards lower success probability.The current specification of the beta distribution:
treats every exposure as a failure, that is overstates the failures thus undervalues the success probabilities of the variations. The effect is pronounced for variations with very high baseline conversion rates but less severe for variations with extremely low conversion rates.
Issue #, if available:
Similar to: #643
Description of changes:
Traditionally, the Thompson Sampling Algorithm for the Bernoulli Bandit Thompson Sampling algorithm is:
Where α, β represent the parameters of each arm i.e. the success and failure counts, respectively OR the number of
conversions
andnon-conversions
, respectively.Description of testing performed to validate your changes (required if pull request includes CloudFormation or source code changes):
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.