Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make model type backwards compatible #1212

Merged
merged 13 commits into from
Feb 1, 2025

Conversation

kunal-vaishnavi
Copy link
Contributor

@kunal-vaishnavi kunal-vaishnavi commented Jan 31, 2025

Description

This PR makes accessing the model type possible by reading from the GenAI config if the model object does not contain the type attribute. It also adds the chat and system templates for Qwen models.

Motivation and Context

This PR allows the examples to be backwards compatible with the published RCs for v0.6.0.

The Qwen chat template and Qwen system template were obtained from the following information.

>>> from transformers import AutoTokenizer
>>> tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct", cache_dir="./cache_dir")
tokenizer_config.json: 100%|█████████████████████████████████████████████████████████████████████████████| 7.30k/7.30k [00:00<00:00, 9.64MB/s]
vocab.json: 100%|████████████████████████████████████████████████████████████████████████████████████████| 2.78M/2.78M [00:00<00:00, 16.5MB/s]
merges.txt: 100%|████████████████████████████████████████████████████████████████████████████████████████| 1.67M/1.67M [00:00<00:00, 15.6MB/s]
tokenizer.json: 100%|████████████████████████████████████████████████████████████████████████████████████| 7.03M/7.03M [00:00<00:00, 21.5MB/s]
>>> prompt = "Give me a short introduction to large language model."
>>> messages = [ {"role": "system", "content": "You are Qwen, created by Alibaba Cloud. You are a helpful assistant."}, {"role": "user", "content": prompt} ]
>>> text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
>>> text
'<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n<|im_start|>user\nGive me a short introduction to large language model.<|im_end|>\n<|im_start|>assistant\n'

baijumeswani
baijumeswani previously approved these changes Jan 31, 2025
MaanavD
MaanavD previously approved these changes Jan 31, 2025
Copy link

@MaanavD MaanavD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Tested and works!

@apsonawane
Copy link
Contributor

In model-chat.py in line 105 should we not have this line to modify prompt based on chat_template?
prompt = f'{chat_template.format(input=text)}'

baijumeswani
baijumeswani previously approved these changes Jan 31, 2025
RyanUnderhill pushed a commit that referenced this pull request Feb 1, 2025
### Description

This PR replaces the dummy Hugging Face authentication token with an
actual read-only token.

### Motivation and Context

This fixes the CI failures that are happening. This change is separated
from [this PR](#1212)
for clarity in the repo's commit history.
@kunal-vaishnavi kunal-vaishnavi merged commit dba39b4 into main Feb 1, 2025
14 checks passed
@kunal-vaishnavi kunal-vaishnavi deleted the kvaishnavi/model-type-in-examples branch February 1, 2025 00:19
baijumeswani pushed a commit that referenced this pull request Feb 3, 2025
### Description

This PR replaces the dummy Hugging Face authentication token with an
actual read-only token.

### Motivation and Context

This fixes the CI failures that are happening. This change is separated
from [this PR](#1212)
for clarity in the repo's commit history.
baijumeswani pushed a commit that referenced this pull request Feb 3, 2025
### Description

This PR makes accessing the model type possible by reading from the
GenAI config if the model object does not contain the type attribute. It
also adds the chat and system templates for Qwen models.

### Motivation and Context

This PR allows the examples to be backwards compatible with the
published RCs for v0.6.0.

The Qwen chat template and Qwen system template were obtained from the
following information.

```
>>> from transformers import AutoTokenizer
>>> tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct", cache_dir="./cache_dir")
tokenizer_config.json: 100%|█████████████████████████████████████████████████████████████████████████████| 7.30k/7.30k [00:00<00:00, 9.64MB/s]
vocab.json: 100%|████████████████████████████████████████████████████████████████████████████████████████| 2.78M/2.78M [00:00<00:00, 16.5MB/s]
merges.txt: 100%|████████████████████████████████████████████████████████████████████████████████████████| 1.67M/1.67M [00:00<00:00, 15.6MB/s]
tokenizer.json: 100%|████████████████████████████████████████████████████████████████████████████████████| 7.03M/7.03M [00:00<00:00, 21.5MB/s]
>>> prompt = "Give me a short introduction to large language model."
>>> messages = [ {"role": "system", "content": "You are Qwen, created by Alibaba Cloud. You are a helpful assistant."}, {"role": "user", "content": prompt} ]
>>> text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
>>> text
'<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n<|im_start|>user\nGive me a short introduction to large language model.<|im_end|>\n<|im_start|>assistant\n'
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants