-
Notifications
You must be signed in to change notification settings - Fork 20
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
No chat template specified for llava models error #6
Comments
Don't worry, it shouldn't affect outputs. This warning appears because we are using the default I will patch them today to stop the warning :) |
But to be sure, could you share the full trace back and reproducible example? |
The code:
Traceback from the request:
Trace back from the server:
|
I ran the same script you ran with all version but couldn't manage to replicate your issue: You should only see a warning like this: Responses:v0.0.1 {
'id': 'chatcmpl-d55bb013',
'object': 'chat.completion',
'created': 1720734350,
'model': 'mlx-community/llava-1.5-7b-4bit',
'choices': [
{
'index': 0,
'message': {
'role': 'assistant',
'content': '</s>\nTwo cats are sleeping on a couch.'
}, 'finish_reason': 'stop'
}
]
} v0.1.0 {
'id': 'chatcmpl-5fd3cde5',
'object': 'chat.completion',
'created': 1720734521,
'model': 'mlx-community/llava-1.5-7b-4bit',
'choices': [
{
'index': 0,
'message': {
'role': 'assistant',
'content': '</s>\nTwo cats are laying on a couch, one on the left side and the other on the right side.'
}, 'finish_reason': 'stop'
}
]
} |
Can you please share the output of |
|
I haven't been able to replicate this issue. I ran the script on 2 different machines and it runs normally. Could you create a new virtual environment and try again ? |
Tried again in a new vent. Same result. This is python 3.11 on an M1 MacBook Pro with 16 GB ram. Request:
Server:
PIP List:
|
Could you delete and re-download this model: |
Done, but no luck. Request
SERVER:
pip list
|
Is there debugging I could add to the fastmlx.py file that would be helpful? |
Yes, there is. You can print the processor and its attributes. And print the configuration. Also, as a sanity you can comment out the chat template part and pass the formatted text directly. Example:
I can share the code later. I'm currently travelling, and I'm on my phone. |
Upon further investigation, I can confirm that this is an mlx-vlm bug, not a fastmlx bug. I get the same chat template errors with all of the following: models--mlx-community--llava-1.5-7b-4bit Moving this to Blaizzy/mlx-vlm#51 |
Getting this message:
File "/anaconda3/lib/python3.11/site-packages/transformers/processing_utils.py", line 926, in apply_chat_template
raise ValueError(
ValueError: No chat template is set for this processor. Please either set the
chat_template
attribute, or provide a chat template as an argument.Happened with:
models--mlx-community--llava-1.5-7b-4bit
models--mlx-community--llava-llama-3-8b-v1_1-8bit
The text was updated successfully, but these errors were encountered: