Using Different GGML

[Alshamaa13]

Hello there,

Can you please let me know if we can use different GGML model other than ggml-gpt4all-j-v1.3-groovy?
Please provide us the instructions for the necessary changes to make.

Thanks,

[feathered-arch]

The instructions are literally in the readme. :)

[hardymathieu]

I tried using ggml-mpt-7b-base but even thought it can find it, it apparently can't load it.
Any idea why that might be?
PS E:\PrivateGPT\privateGPT\privateGPT-main> python privateGPT.py Found model file. gptj_model_load: loading model from 'models/ggml-mpt-7b-base.bin' - please wait ... gptj_model_load: invalid model file 'models/ggml-mpt-7b-base.bin' (bad magic) GPT-J ERROR: failed to load model from models/ggml-mpt-7b-base.binTraceback (most recent call last): File "E:\PrivateGPT\privateGPT\privateGPT-main\privateGPT.py", line 76, in <module> main() File "E:\PrivateGPT\privateGPT\privateGPT-main\privateGPT.py", line 36, in main llm = GPT4All(model=model_path, n_ctx=model_n_ctx, backend='gptj', callbacks=callbacks, verbose=False) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "pydantic\main.py", line 339, in pydantic.main.BaseModel.__init__ File "pydantic\main.py", line 1102, in pydantic.main.validate_model File "C:\Python311\Lib\site-packages\langchain\llms\gpt4all.py", line 152, in validate_environment values["client"].model.set_thread_count(values["n_threads"]) File "C:\Users\mathieu\AppData\Roaming\Python\Python311\site-packages\gpt4all\pyllmodel.py", line 153, in set_thread_count raise Exception("Model not loaded") Exception: Model not loaded

[feathered-arch]

Hmmmm. What flavor of python are you running, and on what system? Looks like you're in powershell. Intel python?

[Alshamaa13]

I have faced the same issue :)

[hardymathieu]

Python 3.11 Windows 11 Was in Powershell indeed. (it worked just fine with the defaut model, it's when I tried smthg that else that it stopped)

…

On Thu, Jun 1, 2023, 21:48 J T ***@***.***> wrote: Hmmmm. What flavor of python are you running, and on what system? Looks like you're in powershell. Intel python? — Reply to this email directly, view it on GitHub <#585 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AANOMKSSDVVGTV6VJDKN7JTXJDW2PANCNFSM6AAAAAAYXH7274> . You are receiving this because you commented.Message ID: ***@***.***>

[8bitaby]

Make sure you are correctly mentioning the model type and args. In privategpt.py these lines represent model loading.

match model_type:
    case "LlamaCpp":
        llm = LlamaCpp(model_path=model_path, n_ctx=model_n_ctx, callbacks=callbacks, verbose=False)
    case "GPT4All":
        llm = GPT4All(model=model_path, n_ctx=model_n_ctx, backend='gptj', callbacks=callbacks, verbose=False)
    case _default:
        print(f"Model {model_type} not supported!")
        exit;

Recheck which type of model is mentioned and used. Also just make sure MODEL_TYPE and MODEL_PATH in the .env are correct.