Update japan_dict.txt

[madmalkav]

Update japan_dict.txt to include missing jouyou kanji ( #12940 )

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[GreatV]

Will this affect the previous jp model?

[madmalkav]

It is my understanding this file is only used when training a new model, so I don't think it would have any effect until a new model is trained, but there is a chance I misunderstood everything wrong and it works in a different way 😅

[GreatV]

Since they use the same character dictionary, using the previous model may result in inconsistent output, and I think it would be better to use a different filename (e.g., ja_ext_dict.txt).

[madmalkav]

If this file can indeed affect the current model, I would propose to hold this PR until a new version of the japanese model is going to be trained. I don't like too much the idea of creating a new file and called it "extended" because this is not really extending the dictionary, is fixing a fault in it.

[GreatV]

@madmalkav, That makes sense.

[madmalkav]

Can I ask why this similar PR for another language was already merged? What is the difference with my PR? I want to understand to see if I can do something else to move this forward.

[GreatV]

I don't think that PR being merged will work properly with the previous model.

[GreatV]

Maybe we could try to keep the version information of the character dictionary (a new column in the model list is used to show the dictionary used by the model), so that new character dictionaries can be merged in, and old models use old character dictionaries.

model code	description	character dictionary	model size	download	Update Date
ch	Chinese and English	https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.7/ppocr/utils/ppocr_keys_v1.txt	3.71M	inference model/ trained model	2020.9.22
ch	Chinese and English	https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.8/ppocr/utils/ppocr_keys_v1.txt	3M	inference model/ trained model	2024.9.22