"Node array" Error when trying to run the simba classifiers trained on another computer.

[sara-xinyue-liang]

Describe the bug
I was using the models I have been trained on a linux machine (the python version of that is 3.6,using anaconda) and I try to run the model on my Mac (python version of the simba is 3.10,using anaconda) and I ran in to an error about "node array from the pickle has an incompatible type"

To Reproduce
Steps to reproduce the behavior:

1. using the models I have trained on another machine. the zip file of the model has been attached.
2. set model parameter and run model
3. The simba main terminal stopped at "Analyzing video 20240812_testvideos20240724_01_4_f8710_11330..." and in my laptop's terminal, it shows an error.

"

(simba) saraliang@dhcp-10-31-152-214 ~ % simba
warning||SimBA is not extensively tested beyond python 3.6. You are using python 3.10. If you encounter errors in python>3.6, please report them on GitHub or Gitter (links in the help toolbar) and we will work together to fix the issues!
2024-09-09 09:52:58.219 python[1727:69538] TSM AdjustCapsLockLEDForKeyTransitionHandling - _ISSetPhysicalKeyboardCapsLockLED Inhibit
Exception in Tkinter callback
Traceback (most recent call last):
File "/Users/saraliang/anaconda3/envs/simba/lib/python3.10/tkinter/init.py", line 1921, in call
return self.func(*args)
File "/Users/saraliang/anaconda3/envs/simba/lib/python3.10/site-packages/simba/ui/tkinter_functions.py", line 436, in execute_command
cmd(**evaluated_kwargs)
File "/Users/saraliang/anaconda3/envs/simba/lib/python3.10/site-packages/simba/ui/pop_ups/set_machine_model_parameters_pop_up.py", line 51, in set
check_file_exist_and_readable(model_settings["path"].file_path)
File "/Users/saraliang/anaconda3/envs/simba/lib/python3.10/site-packages/simba/utils/checks.py", line 41, in check_file_exist_and_readable
raise NoFilesFoundError(
simba.utils.errors.NoFilesFoundError: SIMBA NO FILES FOUND ERROR: No file selected is not a valid file path
Exception in thread Thread-1 (execute_command):
Traceback (most recent call last):
File "/Users/saraliang/anaconda3/envs/simba/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
self.run()
File "/Users/saraliang/anaconda3/envs/simba/lib/python3.10/threading.py", line 953, in run
self._target(*self._args, **self._kwargs)
File "/Users/saraliang/anaconda3/envs/simba/lib/python3.10/site-packages/simba/ui/tkinter_functions.py", line 436, in execute_command
cmd(**evaluated_kwargs)
File "/Users/saraliang/anaconda3/envs/simba/lib/python3.10/site-packages/simba/SimBA.py", line 711, in runrfmodel
rf_model_runner.run()
File "/Users/saraliang/anaconda3/envs/simba/lib/python3.10/site-packages/simba/model/inference_batch.py", line 60, in run
clf = self.read_pickle(file_path=m_hyp["model_path"])
File "/Users/saraliang/anaconda3/envs/simba/lib/python3.10/site-packages/simba/mixins/train_model_mixin.py", line 1137, in read_pickle
clf = pickle.load(open(file_path, "rb"))
File "sklearn/tree/_tree.pyx", line 865, in sklearn.tree._tree.Tree.setstate
File "sklearn/tree/_tree.pyx", line 1571, in sklearn.tree._tree._check_node_ndarray
ValueError: node array from the pickle has an incompatible dtype:

• expected: {'names': ['left_child', 'right_child', 'feature', 'threshold', 'impurity', 'n_node_samples', 'weighted_n_node_samples', 'missing_go_to_left'], 'formats': ['<i8', '<i8', '<i8', '<f8', '<f8', '<i8', '<f8', 'u1'], 'offsets': [0, 8, 16, 24, 32, 40, 48, 56], 'itemsize': 64}
• got : [('left_child', '<i8'), ('right_child', '<i8'), ('feature', '<i8'), ('threshold', '<f8'), ('impurity', '<f8'), ('n_node_samples', '<i8'), ('weighted_n_node_samples', '<f8')]

"

Additional context
I guess that this error is from the fact that I was using model from two different version of simba. But I wonder if there is anyway around it (if not it is fine but it will be a lot more convenient if I can train model in one machine and run it on another one.)

Sincerely,
Sara Liang

[sara-xinyue-liang]

The model sav file was too large that I can't upload it here, but let me know if I can provide any more information

[sronilsson]

Hi @sara-xinyue-liang - yes I think I've had this one before.. when you install SimBA, the installation process checks which version of python you have, and installs the appropriate joblib and scickit-learn versions (it's different versions for python 3.6 and python versions above 3.9), it's these lines in the requirements.txt file:

scikit-learn == 0.22.2; python_version=="3.6"
scikit-learn == 1.4.2; python_version>="3.9"
joblib == 1.1.1; python_version=="3.6"
joblib == 1.4.2; python_version>="3.9"
....

Is it possible that the other machine runs python 3.6, and therefore has joblib version 1.1.1 and scikit-learn version 0.22.2, while you are running python 3.10 on your mac and have joblib version 1.4.2 and scikit-learn version 1.4.2?

If you install SimBA on the Mac using python 3.6 rather than python 3.10, does that solve it?

[sara-xinyue-liang]

Hi @sronilsson ,

I have some difficulties installing python 3.6 on Mac. But I actually just solved the node array issue I mentioned by downgrading the scikit-learn from 1.4.2 to 1.2.2. The job to run the model is successful and that is what I need! (just in case someone else who wants to run training and inference on different machines have encountered similar issue as I did. )

Sara Liang