Snowflake files seem to need a different pipeline

[hpssjellis]

Snowflake files seem to need a different pipeline

Every other MLC model seems to easily load using the normal methods, but the 4 snowflake models fail with pipeline issues. I have tried defining a different pipeline in the config options but that did not work. Any suggestions.

I have a draft demo at https://hpssjellis.github.io/my-examples-of-ai-agents/public/web-llm/web-llm00.html and the errors are:

The relevant part of the code that loads the models is here

IncorrectPipelineLoadedError: ChatCompletionRequest expects model to be loaded with LLMChatPipeline. However, snowflake-arctic-embed-s-q0f32-MLC-b32 is not loaded with this pipeline.
    at MB.getModelStates (engine.ts:1206:15)
    at MB.getLLMStates (engine.ts:1161:17)
    at MB.<anonymous> (engine.ts:775:12)
    at Generator.next (<anonymous>)
    at tslib.es6.mjs:121:69
    at new Promise (<anonymous>)
    at cg (tslib.es6.mjs:117:10)
    at MB.chatCompletion (+esm:13:4614077)
    at zg.create (chat_completion.ts:75:24)
    at streamingGenerating (web-llm00.html:137:54)

127 : snowflake-arctic-embed-m-q0f32-MLC-b32 14 files
128 : snowflake-arctic-embed-m-q0f32-MLC-b4 14 files
129 : snowflake-arctic-embed-s-q0f32-MLC-b32 9 files
130 : snowflake-arctic-embed-s-q0f32-MLC-b4 9 files

A related issue with some information at #648

@tqchen @CharlieFRuan

I read in the documents that transformers.js might be able to load different pipelines.
https://huggingface.co/docs/transformers.js/index

@xenova

[CharlieFRuan]

Hi, thanks for the question. The snowflake models are embedding models that are not used to chat. So instead of using engine.chat.completions.create(), you should use engine.embeddings.create(). See this example for more: https://github.com/mlc-ai/web-llm/blob/main/examples/embeddings/src/embeddings.ts

[hpssjellis]

@CharlieFRuan that makes sense now. I think for my example I will just state they are not for chat. Thanks for your help.