No chat template specified for llava models error #6
Comments
|
Don't worry, it shouldn't affect outputs. This warning appears because we are using the default I will patch them today to stop the warning :) |
|
But to be sure, could you share the full trace back and reproducible example? |
|
The code: Traceback from the request: Trace back from the server: |
|
I ran the same script you ran with all version but couldn't manage to replicate your issue: You should only see a warning like this: Responses:v0.0.1 {
'id': 'chatcmpl-d55bb013',
'object': 'chat.completion',
'created': 1720734350,
'model': 'mlx-community/llava-1.5-7b-4bit',
'choices': [
{
'index': 0,
'message': {
'role': 'assistant',
'content': '</s>\nTwo cats are sleeping on a couch.'
}, 'finish_reason': 'stop'
}
]
}v0.1.0 {
'id': 'chatcmpl-5fd3cde5',
'object': 'chat.completion',
'created': 1720734521,
'model': 'mlx-community/llava-1.5-7b-4bit',
'choices': [
{
'index': 0,
'message': {
'role': 'assistant',
'content': '</s>\nTwo cats are laying on a couch, one on the left side and the other on the right side.'
}, 'finish_reason': 'stop'
}
]
} |
|
Can you please share the output of |
|
|
I haven't been able to replicate this issue. I ran the script on 2 different machines and it runs normally. Could you create a new virtual environment and try again ? |
|
Tried again in a new vent. Same result. This is python 3.11 on an M1 MacBook Pro with 16 GB ram. Request: Server: PIP List: |
|
Could you delete and re-download this model: |
|
Done, but no luck. Request SERVER: pip list |
|
Is there debugging I could add to the fastmlx.py file that would be helpful? |
|
Yes, there is. You can print the processor and its attributes. And print the configuration. Also, as a sanity you can comment out the chat template part and pass the formatted text directly. Example: I can share the code later. I'm currently travelling, and I'm on my phone. |
|
Upon further investigation, I can confirm that this is an mlx-vlm bug, not a fastmlx bug. I get the same chat template errors with all of the following: models--mlx-community--llava-1.5-7b-4bit Moving this to Blaizzy/mlx-vlm#51 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Getting this message:
File "/anaconda3/lib/python3.11/site-packages/transformers/processing_utils.py", line 926, in apply_chat_template
raise ValueError(
ValueError: No chat template is set for this processor. Please either set the
chat_templateattribute, or provide a chat template as an argument.Happened with:
models--mlx-community--llava-1.5-7b-4bit
models--mlx-community--llava-llama-3-8b-v1_1-8bit
The text was updated successfully, but these errors were encountered: