Skip to content

Commit

Permalink
Update the beta distribution parameters in the `_select_variation_i…
Browse files Browse the repository at this point in the history
…ndex` method (#647)

Update the beta distribution parameters in the `_select_variation_index` method to avoid bias towards lower success probability.

The current specification of the beta distribution:

```
theta = np.random.beta(conversions + 1, exposures + 1)
``` 
treats every exposure as a failure, that is overstates the failures thus undervalues the success probabilities of the variations. The effect is pronounced for variations with very high baseline conversion rates but less severe for variations with extremely low conversion rates. 

#643

Co-authored-by: James Jory <[email protected]>
  • Loading branch information
MustaphaU and james-jory authored Oct 28, 2024
1 parent e5791de commit 9c17583
Showing 1 changed file with 1 addition and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ def _select_variation_index(self):

# Sample from posterior (this is the Thompson Sampling approach)
# This leads to more exploration because variations with > uncertainty can then be selected
theta = np.random.beta(conversions + 1, exposures + 1)
theta = np.random.beta(conversions + 1, exposures - conversions + 1)

# Select variation index with highest posterior p of converting
return np.argmax(theta)

0 comments on commit 9c17583

Please sign in to comment.