default suggestion check error in iscontained

[poudelankit]

code for ConstraintSuggestion Default
suggestions = ConstraintSuggestionRunner(spark).onData(df).addConstraintRule(DEFAULT()).run()
for suggestion in suggestions['constraint_suggestions']:
print(suggestion['code_for_constraint'])

Result of default suggestion
The default constraint suggestion will return the following as a value under 'code_for_constraint':
.isContainedIn("Category #1", ["Dental Surgery", "Laboratory"], lambda x: x >= 0.98, "It should be above 0.98!")
which when applied to addCheck would result in error.

Check Implementation:
check = Check(spark,CheckLevel.Error,"Manual Check")
verification_runner = VerificationSuite(spark).onData(df).addCheck(check.isContainedIn("Category #1", ["Dental Surgery", "Laboratory"], lambda x: x >= 0.98, "It should be above 0.98!"))
verification_result = verification_runner.run()
df_checked = VerificationResult.checkResultsAsDataFrame(spark,verification_result)
df_checked.show(truncate=False)

error
TypeError: isContainedIn() takes 3 positional arguments but 5 were given

[chenliu0831]

This should fix the error #157?

[poudelankit]

Implementation of scala in python worked for me::

Change was made on checks.py

def isContainedIn(self, column,allowed_values, assertion=None, hint=None):
"""
* Asserts that every non-null value in a column is contained in a set of predefined values
*
* @param column Column to run the assertion on
* @param allowedValues Allowed values for the column
* @param assertion Function that receives a double input parameter and returns a boolean
* @param hint A hint to provide additional context why a constraint could have failed
* @return
*/
"""
if(assertion):
assertion_func = (
ScalaFunction1(self._spark_session.sparkContext._gateway, assertion)
)
hint = self._jvm.scala.Option.apply(hint)
arr = self._spark_session.sparkContext._gateway.new_array(self._jvm.java.lang.String, len(allowed_values))
for i in range(len(allowed_values)):
arr[i] = allowed_values[i]
if assertion:
self._Check = self._Check.isContainedIn(column, arr,assertion_func, hint)
else:
self._Check = self._Check.isContainedIn(column, arr)
return self