Donut generate ONLY <s><s>...<s></s>

[SerenaTetart]

After training the model, I have a normal training and validation loss decreasing over each epoch, only to return during generation the same token:
text_sequence: <s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s>...<s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s></s>

Edit: I solved it by using transformer version 4.40.2

[svchostdotexe]

that solved it for you? i have the same issue, but when i upgrade to 4.40.2 i get the following error:
TypeError: prepare_inputs_for_inference() got an unexpected keyword argument 'cache_position'

could you be so kind and post all your installed lib versions? (pip list and conda list)

edit: introducing the "cache_position" as an argument fixes the issue, now i can train the model with transformer==4.40.2! sadly that does not fix the <s><s><s><s><s><s> issue for me. any help would be greatly appreciated!

[DriraYosr]

I had the same issue with the version of transformers 4.38.2. Dowgrading transformers to the version 4.25.1 solved the issue!

[SerenaTetart]

@svchostdotexe the 2nd reason why you're getting this error might be because you have too much text or token in your training set

[nromero125]

Im having the same issue, I tried with multiple versions of transformers and result in the same. I was following this article https://medium.com/@paul_lefevre/information-extraction-from-id-documents-with-donut-792ca80d2a93

[Ruxin124]

@nromero125 Have you solved the problem? i follow the same resource as you, and get the same error.

[ephron-canopy]

@nromero125 @Ruxin124
ive followed the same article and got the same error you

this comment mentions the exact issue i faced,
#132 (comment)

But basically after changing the transformer version i was able to resolve the issue,

- python-version        3.8.19
- transformers          4.24.0
- timm                  0.6.13
- torch                 2.0.1

• this issue mainly occurs during the inference, so try debugging with check_val_every_n_epoch: 1 in config
• After resolving the issue, training and validation went well, but during the inference,

from donut import DonutModel
model = DonutModel.from_pretrained("result_v3/train_id/20240826_131036")

i was getting this error
NotImplementedError: Make sure `_init_weights` is implemented for <class 'donut.model.DonutModel'>

i was able to do prediction using the test.py srcipt

Found the issue was due to pip install donut-python which install donut_python-1.0.9-py3-none-any.whl
after uninstalling and installing the donut-python directly from the repo pip install . , it was resolved.

not sure what exactly happened there, cause it install the same file, but it resolved the issue,
adviced to install the donut library from directly from the repo

[Ruxin124]

@ephron-canopy
Thanks a lot for you reply, i have found the same issue about donut-python, and solved it the same way as you did.