Translate file on local computer

[hugh-m]

What I would like to do with argos-translate is translate a local file on my local machine via a script called on the Linux command line with the output on stdout. Can you point me in the right direction to achieve this?

Thanks

[PJ-Finlay]

You should be able to do this with the command line interface:

$ cat my_file.txt | argos-translate-cli --from-lang en --to-lang es

[hugh-m]

I am having a problem with translation into English. I have installed en_fr, fr_en, en_es, es_en. Translations from English work fine, but translations into English get an error message (below).

I am using Debian unstable, the files are installed in ~/.local/lib/python3.9/site-packages/argostranslate/ and ~/.local/bin.

I do have an nVidia graphiccs card (low powered), GeForce GT 710.

Can you help?

---Error message---

2021-02-22 16:20:27 WARNING: Language fr package default expects mwt, which has been added
/home/base/.local/lib/python3.9/site-packages/torch/cuda/init.py:52: UserWarning: CUDA initialization: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx (Triggered internally at /pytorch/c10/cuda/CUDAFunctions.cpp:100.)
return torch._C._cuda_getDeviceCount() > 0
2021-02-22 16:20:27 ERROR: Cannot load model from /home/base/.local/share/argos-translate/packages/fr_en/stanza/fr/mwt/gsd.pt
Traceback (most recent call last):
File "/home/base/.local/lib/python3.9/site-packages/stanza/pipeline/core.py", line 128, in init
self.processors[processor_name] = NAME_TO_PROCESSOR_CLASS[processor_name](config=curr_processor_config,
File "/home/base/.local/lib/python3.9/site-packages/stanza/pipeline/processor.py", line 155, in init
self._set_up_model(config, use_gpu)
File "/home/base/.local/lib/python3.9/site-packages/stanza/pipeline/mwt_processor.py", line 21, in _set_up_model
self._trainer = Trainer(model_file=config['model_path'], use_cuda=use_gpu)
File "/home/base/.local/lib/python3.9/site-packages/stanza/models/mwt/trainer.py", line 36, in init
self.load(model_file, use_cuda)
File "/home/base/.local/lib/python3.9/site-packages/stanza/models/mwt/trainer.py", line 141, in load
checkpoint = torch.load(filename, lambda storage, loc: storage)
File "/home/base/.local/lib/python3.9/site-packages/torch/serialization.py", line 581, in load
with _open_file_like(f, 'rb') as opened_file:
File "/home/base/.local/lib/python3.9/site-packages/torch/serialization.py", line 230, in _open_file_like
return _open_file(name_or_buffer, mode)
File "/home/base/.local/lib/python3.9/site-packages/torch/serialization.py", line 211, in init
super(_open_file, self).init(open(name, mode))
FileNotFoundError: [Errno 2] No such file or directory: '/home/base/.local/share/argos-translate/packages/fr_en/stanza/fr/mwt/gsd.pt'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/base/.local/bin/argos-translate-cli", line 5, in
cli.main()
File "/home/base/.local/lib/python3.9/site-packages/argostranslate/cli.py", line 48, in main
print(translation.translate(text_to_translate))
File "/home/base/.local/lib/python3.9/site-packages/argostranslate/translate.py", line 189, in translate
translated_paragraph = self.underlying.translate(paragraph)
File "/home/base/.local/lib/python3.9/site-packages/argostranslate/translate.py", line 119, in translate
apply_packaged_translation(self.pkg, paragraph, self.translator))
File "/home/base/.local/lib/python3.9/site-packages/argostranslate/translate.py", line 210, in apply_packaged_translation
stanza_pipeline = stanza.Pipeline(lang=pkg.from_code,
File "/home/base/.local/lib/python3.9/site-packages/stanza/pipeline/core.py", line 155, in init
raise FileNotFoundError('Could not find model file %s, although there are other models downloaded for language %s. Perhaps you need to download a specific model. Try: stanza.download(lang="%s",package=None,processors={"%s":"%s"})' % (model_path, lang, lang, processor_name, model_name)) from e
FileNotFoundError: Could not find model file /home/base/.local/share/argos-translate/packages/fr_en/stanza/fr/mwt/gsd.pt, although there are other models downloaded for language fr. Perhaps you need to download a specific model. Try: stanza.download(lang="fr",package=None,processors={"mwt":"gsd"})

[PJ-Finlay]

It looks like Argos Translate is having a hard time finding the Stanza model for French. I'm not sure why this is happening but the package install dir was recently moved. It should upgrade automatically but the upgrade doesn't work with Python <= 3.8. The upgrade could have also failed somehow. I would recommend:

Delete the legacy package directory

rm -rf ~/.argos-translate

Delete all installed packages (make sure you still have the original model files)

rm -rf ~/.local/share/argos-translate/packages

Reinstall the latest version from PyPI

python3 -m pip install --upgrade pip
python3 -m pip install --upgrade argostranslate

Reinstall the French package

[hugh-m]

Still the same problem after complete removal and fresh installation.

Here is the start of the installation message.

$ python3 -m pip install argostranslate
Defaulting to user installation because normal site-packages is not writeable
Collecting argostranslate
Using cached argostranslate-1.1.2-py3-none-any.whl (15 kB)
Collecting ctranslate2==1.17.1
Using cached ctranslate2-1.17.1-cp39-cp39-manylinux2014_x86_64.whl (75.4 MB)
Collecting sentencepiece==0.1.91
Using cached sentencepiece-0.1.91.tar.gz (500 kB)
ERROR: Command errored out with exit status 1:
command: /usr/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-ri216yp5/sentencepiece_d641d3a136f347fd9689e6d3a95e7d66/setup.py'"'"'; file='"'"'/tmp/pip-install-ri216yp5/sentencepiece_d641d3a136f347fd9689e6d3a95e7d66/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-6nokqjza
cwd: /tmp/pip-install-ri216yp5/sentencepiece_d641d3a136f347fd9689e6d3a95e7d66/
Complete output (5 lines):
Package sentencepiece was not found in the pkg-config search path.
Perhaps you should add the directory containing `sentencepiece.pc'
to the PKG_CONFIG_PATH environment variable
No package 'sentencepiece' found
Failed to find sentencepiece pkgconfig
----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/ba/f6/520b56e5977f62aee48833da8b4ff2fdc2b10ebfa0dd78556b1d707d4086/sentencepiece-0.1.91.tar.gz#sha256=f9700cf607ea064d9fad34c751fbf49953dcc56fe68c54b277481aa0aec5c18f (from https://pypi.org/simple/sentencepiece/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
Collecting argostranslate
Using cached argostranslate-1.1.1-py3-none-any.whl (15 kB)
Requirement already satisfied: stanza==1.2 in ./lib/python3.9/site-packages (from argostranslate) (1.2)
Using cached argostranslate-1.1.0-py3-none-any.whl (15 kB)
Using cached argostranslate-1.0.6-py3-none-any.whl (14 kB)
Using cached argostranslate-1.0.5-py3-none-any.whl (13 kB)
Collecting ctranslate2==1.14.0
Downloading ctranslate2-1.14.0-cp39-cp39-manylinux2010_x86_64.whl (13.9 MB)
|████████████████████████████████| 13.9 MB 562 kB/s
Collecting argostranslate
Using cached argostranslate-1.0.3-py3-none-any.whl (12 kB)
Using cached argostranslate-1.0-py3-none-any.whl (12 kB)
...

[PJ-Finlay]

This looks like a different issue than before. Before Stanza, which we use for sentence boundary detection, was failing to find the model. Now SentencePiece, which we use for tokenization is failing to install. My guess is that is this is some interaction between Debian unstable and SentencePiece. I'm linking some related issues, based on the linked discussions things I would suggest:

• Make sure pip is up to date and >= 19.3 (I'm guessing Debian stable has the most recent release so this isn't the issue).
• Install sentencepiece alone to isolate the issue python3 -m pip install sentencepiece==0.1.91.
• Post an issue on SentencePiece, my guess is that this is related to their Python packaging and they can probably help you.

Related issues:

google/sentencepiece#378

google/sentencepiece#554

#36 (Your earlier issue)