Snowpark XGBRegressor Ignores Sample Weights, Producing Identical Predictions for Different Models

[robertlessmore]

1. What version of Python are you using?

Python 3.11.8 | packaged by Anaconda, Inc. | (main, Feb 26 2024, 21:34:05) [MSC v.1916 64 bit (AMD64)]

What operating system and processor architecture are you using?
Windows-10-10.0.22631-SP0

3. What are the component versions in the environment?

_py-xgboost-mutex 2.0 cpu_0
abseil-cpp 20220623.0 h0e60522_0 conda-forge
absl-py 1.4.0 py311haa95532_0
aiobotocore 2.7.0 py311haa95532_0
aiohttp 3.9.3 py311h2bbff1b_0
aioitertools 0.7.1 pyhd3eb1b0_0
aiosignal 1.2.0 pyhd3eb1b0_0
alembic 1.8.1 py311haa95532_0
anyio 3.5.0 py311haa95532_0
appdirs 1.4.4 pyhd3eb1b0_0
argon2-cffi 21.3.0 pyhd3eb1b0_0
argon2-cffi-bindings 21.2.0 py311h2bbff1b_0
arrow-cpp 10.0.1 h9c18f36_4_cpu conda-forge
asn1crypto 1.5.1 py311haa95532_0
asttokens 2.0.5 pyhd3eb1b0_0
async-lru 2.0.4 py311haa95532_0
attrs 23.1.0 py311haa95532_0
aws-c-auth 0.6.19 h2bbff1b_0
aws-c-cal 0.5.20 h2bbff1b_0
aws-c-common 0.8.5 h2bbff1b_0
aws-c-compression 0.2.16 h2bbff1b_0
aws-c-event-stream 0.2.15 hd77b12b_0
aws-c-http 0.6.25 h2bbff1b_0
aws-c-io 0.13.10 h2bbff1b_0
aws-c-mqtt 0.7.13 h2bbff1b_0
aws-c-s3 0.1.51 h2bbff1b_0
aws-c-sdkutils 0.1.6 h2bbff1b_0
aws-checksums 0.1.13 h2bbff1b_0
aws-crt-cpp 0.18.16 hd77b12b_0
aws-sdk-cpp 1.9.379 h2768dcf_5 conda-forge
babel 2.11.0 py311haa95532_0
beautifulsoup4 4.12.2 py311haa95532_0
blas 1.0 mkl
bleach 4.1.0 pyhd3eb1b0_0
blinker 1.6.2 py311haa95532_0
boost-cpp 1.82.0 h59b6b97_2
botocore 1.31.64 py311haa95532_0
bottleneck 1.3.7 py311hd7041d2_0
brotli 1.0.9 h2bbff1b_7
brotli-bin 1.0.9 h2bbff1b_7
brotli-python 1.0.9 py311hd77b12b_7
bzip2 1.0.8 h2bbff1b_5
c-ares 1.19.1 h2bbff1b_0
ca-certificates 2024.7.2 haa95532_0 https://repo.anaconda.com/pkgs/snowflake
cachetools 4.2.2 pyhd3eb1b0_0
certifi 2024.7.4 py311haa95532_0 https://repo.anaconda.com/pkgs/snowflake
cffi 1.16.0 py311h2bbff1b_0
charset-normalizer 2.0.4 pyhd3eb1b0_0
click 8.1.7 py311haa95532_0
cloudpickle 2.2.1 py311haa95532_0
colorama 0.4.6 py311haa95532_0
comm 0.2.1 py311haa95532_0
contourpy 1.2.0 py311h59b6b97_0
cryptography 41.0.7 py311h89fc84f_0
cycler 0.11.0 pyhd3eb1b0_0
databricks-cli 0.17.6 py311haa95532_1
debugpy 1.6.7 py311hd77b12b_0
decorator 5.1.1 pyhd3eb1b0_0
defusedxml 0.7.1 pyhd3eb1b0_0
docker-py 4.4.1 py311haa95532_5
docker-pycreds 0.4.0 pyhd3eb1b0_0
entrypoints 0.4 py311haa95532_0
executing 0.8.3 pyhd3eb1b0_0
filelock 3.13.1 py311haa95532_0
flask 2.2.5 py311haa95532_0
fonttools 4.25.0 pyhd3eb1b0_0
freetype 2.12.1 ha860e81_0
frozenlist 1.4.0 py311h2bbff1b_0
fsspec 2023.10.0 py311haa95532_0
gflags 2.2.2 hd77b12b_1
gitdb 4.0.7 pyhd3eb1b0_0
gitpython 3.1.37 py311haa95532_0
glog 0.6.0 h4797de2_0 conda-forge
greenlet 3.0.1 py311hd77b12b_0
grpc-cpp 1.51.1 h9c18f36_0 conda-forge
icc_rt 2022.1.0 h6049295_2
icu 73.1 h6c2663c_0
idna 3.4 py311haa95532_0
importlib-metadata 6.0.0 py311haa95532_0
importlib_resources 6.1.1 py311haa95532_1
intel-openmp 2023.1.0 h59b6b97_46320
ipykernel 6.28.0 py311haa95532_0
ipython 8.20.0 py311haa95532_0
ipywidgets 8.1.2 py311haa95532_0
itsdangerous 2.0.1 pyhd3eb1b0_0
jedi 0.18.1 py311haa95532_1
jinja2 3.1.3 py311haa95532_0
jmespath 1.0.1 py311haa95532_0
joblib 1.2.0 py311haa95532_0
jpeg 9e h2bbff1b_1
json5 0.9.6 pyhd3eb1b0_0
jsonschema 4.19.2 py311haa95532_0
jsonschema-specifications 2023.7.1 py311haa95532_0
jupyter 1.0.0 py311haa95532_9
jupyter-lsp 2.2.0 py311haa95532_0
jupyter_client 8.6.0 py311haa95532_0
jupyter_console 6.6.3 py311haa95532_0
jupyter_core 5.5.0 py311haa95532_0
jupyter_events 0.8.0 py311haa95532_0
jupyter_server 2.10.0 py311haa95532_0
jupyter_server_terminals 0.4.4 py311haa95532_1
jupyterlab 4.0.11 py311haa95532_0
jupyterlab_pygments 0.1.2 py_0
jupyterlab_server 2.25.1 py311haa95532_0
jupyterlab_widgets 3.0.10 py311haa95532_0
kiwisolver 1.4.4 py311hd77b12b_0
krb5 1.20.1 h5b6d351_0
lerc 3.0 hd77b12b_0
libabseil 20220623.0 cxx17_h1a56200_6 conda-forge
libarrow 10.0.1 h226723c_4_cpu conda-forge
libboost 1.82.0 h3399ecb_2
libbrotlicommon 1.0.9 h2bbff1b_7
libbrotlidec 1.0.9 h2bbff1b_7
libbrotlienc 1.0.9 h2bbff1b_7
libclang 14.0.6 default_hb5a9fac_1
libclang13 14.0.6 default_h8e68704_1
libcrc32c 1.1.2 hd77b12b_0
libcurl 8.5.0 h86230a5_0
libdeflate 1.17 h2bbff1b_1
libevent 2.1.12 h56d1f94_1
libffi 3.4.4 hd77b12b_0
libgoogle-cloud 2.5.0 h5fc25aa_1 conda-forge
libgrpc 1.51.1 h6a6baca_0 conda-forge
libpng 1.6.39 h8cc25b3_0
libpq 12.17 h906ac69_0
libprotobuf 3.21.12 h12be248_2 conda-forge
libsodium 1.0.18 h62dcd97_0
libssh2 1.10.0 he2ea4bf_2
libthrift 0.16.0 h9ce19ad_2 conda-forge
libtiff 4.5.1 hd77b12b_0
libutf8proc 2.8.0 h82a8f57_0 conda-forge
libwebp-base 1.3.2 h2bbff1b_0
libxgboost 1.7.3 hd77b12b_0
libzlib 1.2.13 hcfcfb64_5 conda-forge
lz4-c 1.9.4 h2bbff1b_0
mako 1.2.3 py311haa95532_0
markdown 3.4.1 py311haa95532_0
markupsafe 2.1.3 py311h2bbff1b_0
matplotlib-base 3.8.0 py311hf62ec03_0
matplotlib-inline 0.1.6 py311haa95532_0
mistune 2.0.4 py311haa95532_0
mkl 2023.1.0 h6b88ed4_46358
mkl-service 2.4.0 py311h2bbff1b_1
mkl_fft 1.3.8 py311h2bbff1b_0
mkl_random 1.2.4 py311h59b6b97_0
mlflow 2.3.1 py311hd1fac3c_0
multidict 6.0.4 py311h2bbff1b_0
munkres 1.1.4 py_0
nbclient 0.8.0 py311haa95532_0
nbconvert 7.10.0 py311haa95532_0
nbformat 5.9.2 py311haa95532_0
nest-asyncio 1.6.0 py311haa95532_0
notebook 7.0.8 py311haa95532_0
notebook-shim 0.2.3 py311haa95532_0
numexpr 2.8.7 py311h1fcbade_0
numpy 1.26.4 py311hdab7c0b_0
numpy-base 1.26.4 py311hd01c5d8_0
oauthlib 3.2.2 py311haa95532_0
openjpeg 2.4.0 h4fc8c34_0
openssl 3.3.0 hcfcfb64_0 conda-forge
orc 1.9.0 hada7b9e_1 conda-forge
overrides 7.4.0 py311haa95532_0
packaging 23.1 py311haa95532_0
pandas 1.5.3 py311heda8569_0
pandocfilters 1.5.0 pyhd3eb1b0_0
parso 0.8.3 pyhd3eb1b0_0
patsy 0.5.6 pyhd8ed1ab_0 conda-forge
pillow 10.2.0 py311h2bbff1b_0
pip 23.3.1 py311haa95532_0
platformdirs 3.10.0 py311haa95532_0
ply 3.11 py311haa95532_0
prometheus_client 0.14.1 py311haa95532_0
prompt-toolkit 3.0.43 py311haa95532_0
prompt_toolkit 3.0.43 hd3eb1b0_0
protobuf 4.21.12 py311h12c1d0e_0 conda-forge
psutil 5.9.0 py311h2bbff1b_0
pure_eval 0.2.2 pyhd3eb1b0_0
py-xgboost 1.7.3 py311haa95532_0
pyarrow 10.0.1 py311h8a3a540_0
pycparser 2.21 pyhd3eb1b0_0
pygments 2.15.1 py311haa95532_1
pyjwt 2.4.0 py311haa95532_0
pyopenssl 23.2.0 py311haa95532_0
pyparsing 3.0.9 py311haa95532_0
pyqt 5.15.10 py311hd77b12b_0
pyqt5-sip 12.13.0 py311h2bbff1b_0
pysocks 1.7.1 py311haa95532_0
python 3.11.8 he1021f5_0
python-dateutil 2.8.3+snowflake1 py311haa95532_1 https://repo.anaconda.com/pkgs/snowflake
python-fastjsonschema 2.16.2 py311haa95532_0
python-json-logger 2.0.7 py311haa95532_0
python_abi 3.11 2_cp311 conda-forge
pytimeparse 1.1.8 py311haa95532_0
pytz 2023.3.post1 py311haa95532_0
pywin32 305 py311h2bbff1b_0
pywinpty 2.0.10 py311h5da7b33_0
pyyaml 6.0.1 py311h2bbff1b_0
pyzmq 25.1.2 py311hd77b12b_0
qt-main 5.15.2 h19c9488_10
qtconsole 5.5.1 py311haa95532_0
qtpy 2.4.1 py311haa95532_0
querystring_parser 1.2.4 py311haa95532_0
re2 2022.06.01 h0e60522_1 conda-forge
referencing 0.30.2 py311haa95532_0
requests 2.31.0 py311haa95532_1
retrying 1.3.3 pyhd3eb1b0_2
rfc3339-validator 0.1.4 py311haa95532_0
rfc3986-validator 0.1.1 py311haa95532_0
rpds-py 0.10.6 py311h062c2fa_0
s3fs 2023.10.0 py311haa95532_0
scikit-learn 1.2.2 py311hd77b12b_1
scipy 1.11.4 py311hc1ccb85_0
seaborn 0.13.2 hd8ed1ab_2 conda-forge
seaborn-base 0.13.2 pyhd8ed1ab_2 conda-forge
send2trash 1.8.2 py311haa95532_0
setuptools 68.2.2 py311haa95532_0
sip 6.7.12 py311hd77b12b_0
six 1.16.0 pyhd3eb1b0_1
smmap 4.0.0 pyhd3eb1b0_0
snappy 1.1.10 h6c2663c_1
sniffio 1.3.0 py311haa95532_0
snowflake-connector-python 3.7.0 py311hd77b12b_0
snowflake-ml-python 1.4.0 pypy_0 https://raw.githubusercontent.com/snowflakedb/snowflake-ml-python/conda/releases
snowflake-snowpark-python 1.13.0 py311haa95532_0
sortedcontainers 2.4.0 pyhd3eb1b0_0
soupsieve 2.5 py311haa95532_0
sqlalchemy 2.0.25 py311h2bbff1b_0
sqlite 3.41.2 h2bbff1b_0
sqlparse 0.4.4 py311haa95532_0
stack_data 0.2.0 pyhd3eb1b0_0
statsmodels 0.14.2 py311h0a17f05_0 conda-forge
tabulate 0.9.0 py311haa95532_0
tbb 2021.8.0 h59b6b97_0
terminado 0.17.1 py311haa95532_0
threadpoolctl 2.2.0 pyh0d69192_0
tinycss2 1.2.1 py311haa95532_0
tk 8.6.12 h2bbff1b_0
tomlkit 0.11.1 py311haa95532_0
tornado 6.3.3 py311h2bbff1b_0
traitlets 5.7.1 py311haa95532_0
typing-extensions 4.9.0 py311haa95532_1
typing_extensions 4.9.0 py311haa95532_1
tzdata 2024a h04d1e81_0
ucrt 10.0.20348.0 haa95532_0
urllib3 2.0.7 py311haa95532_0
utf8proc 2.6.1 h2bbff1b_1
vc 14.2 h21ff451_1
vc14_runtime 14.38.33130 h82b7239_18 conda-forge
vs2015_runtime 14.38.33130 hcb4865c_18 conda-forge
waitress 2.0.0 pyhd3eb1b0_0
wcwidth 0.2.5 pyhd3eb1b0_0
webencodings 0.5.1 py311haa95532_1
websocket-client 0.58.0 py311haa95532_4
werkzeug 2.3.8 py311haa95532_0
wheel 0.41.2 py311haa95532_0
widgetsnbextension 4.0.10 py311haa95532_0
win_inet_pton 1.1.0 py311haa95532_0
winpty 0.4.3 4
wrapt 1.14.1 py311h2bbff1b_0
xgboost 1.7.3 py311haa95532_0
xz 5.4.6 h8cc25b3_0
yaml 0.2.5 he774522_0
yarl 1.9.3 py311h2bbff1b_0
zeromq 4.3.5 hd77b12b_0
zipp 3.17.0 py311haa95532_0
zlib 1.2.13 hcfcfb64_5 conda-forge
zstd 1.5.5 hd43e919_0

4. What did you do?
   from snowflake.ml.modeling.xgboost import XGBRegressor
   from snowflake.snowpark.functions import col, random, sin, when, lit
   from utils import get_session

session = get_session.session()

N = 105
_ONE_MILLION = 106

df = session.range(1, N).to_df("ind").with_column(
"x_0", ((random() % _ONE_MILLION)/_ONE_MILLION)
)

df = df

df = df.with_columns(["weights1","weights2","weights3"],[lit(1.0),when(col("ind") < lit(N / 10), 1.0).otherwise(0.0),when(col("ind") > lit(N / 10), 1.0).otherwise(0.0)])

df = df.with_column(
"target",
when(col("ind") < lit(N / 10), 1.0).otherwise(0.0) * col("x_0") +
when(col("ind") > lit(N / 10), 1.0).otherwise(0.0) * sin(10*col("x_0"))
)

parameters = {
"input_cols":["X_0"],
"label_cols":["TARGET"],
}

model1 = XGBRegressor(
**parameters,
sample_weight_col="weights1",
output_cols= ["PREDICTION1"],

)
model2 = XGBRegressor(
**parameters,
sample_weight_col="weights2",
output_cols= ["PREDICTION2"],

)
model3 = XGBRegressor(
**parameters,
sample_weight_col="weights3",
output_cols= ["PREDICTION3"],

)

models = [model1, model2, model3]
for m in models:
m.fit(df)

test = session.range(-1, 1,0.01).to_df("X_0").with_column(
"sinus",
sin(10*col("X_0"))
)

for m in models:
test = m.predict(test)

test_snow = test.toPandas()
print(test_snow)

---

output:
X_0 SINUS PREDICTION1 PREDICTION2 PREDICTION3
0 -1.00 0.544021 0.515664 0.515664 0.515664
1 -0.99 0.457536 0.405519 0.405519 0.405519
2 -0.98 0.366479 0.183660 0.183660 0.183660
3 -0.97 0.271761 0.211220 0.211220 0.211220
4 -0.96 0.174327 0.039056 0.039056 0.039056
.. ... ... ... ... ...
195 0.95 -0.075151 0.047328 0.047328 0.047328
196 0.96 -0.174327 -0.060364 -0.060364 -0.060364
197 0.97 -0.271761 0.034832 0.034832 0.034832
198 0.98 -0.366479 -0.278535 -0.278535 -0.278535
199 0.99 -0.457536 -0.390598 -0.390598 -0.390598

5. What did you expect to see?
   I expected different models to produce different predictions due to the varying sample weights (weights1, weights2, weights3). Specifically:

• PREDICTION1 should reflect a model trained on the entire dataset equally.
• PREDICTION2 should reflect a model influenced more by the first 10,000 samples, which follow a linear pattern.
• PREDICTION3 should reflect a model influenced more by the samples beyond 10,000, which follow a sinusoidal pattern.

However, the Snowflake Snowpark implementation of XGBRegressor seems to ignore the sample weights, resulting in identical predictions for all models. Running a similar experiment directly with the standard xgboost library outside of Snowflake results in distinct linear and sinusoidal predictions for model2 and model3, respectively.

[sfc-gh-afero]

Thank you for reporting this issue, I was able to use your example to reproduce it on my end. We will investigate this issue as a bug.