Statistics weirdness

[tomascohen]

Hi all!

We are trying to take some numbers out of weblate for generating our project's release notes. We are coming from Pootle so we are adjusting our tool set.

We just found that this numbers:
https://translate.koha-community.org/projects/koha/master/#languages

Don't match what's in the actual translation project. Take Spanish as an example. It says it is 90% translated and that there are 1707 untranslated strings. Then, when you click on the 1707 the spanish page tells you there are no strings to translate. There are some strings with some checks failing. But I fixed a few of them and the number is not changing either.

Any clues? Some cache needing a cleanup? I couldn't find it

[joubu]

Maybe fixed by #10473

[joubu]

Tried with weblate:bleeding which has the commits from #10473 but the numbers didn't change.

[tomascohen]

This is what I see in the logs

celery-celery stderr | [2023-11-29 11:57:12,238: ERROR/460] Failure while executing task: KeyError: 'stats_timestamp' celery-celery stderr | [2023-11-29 11:57:12,238: ERROR/ForkPoolWorker-3] Failure while executing task: KeyError: 'stats_timestamp' celery-celery stderr | [2023-11-29 11:57:12,238: ERROR/460] Failure while executing task: KeyError: {'signal': <Signal: task_failure providing_args={'exception', 'task_id', 'traceback', 'args', 'kwargs', 'einfo'}>, 'sender': <@task: weblate.utils.tasks.update_translation_stats_parents of weblate at 0x7fc9e3c49210>, 'task_id': 'facceb74-2c0e-4fae-83d0-d5db7df69706', 'args': [6931], 'kwargs': {}, 'traceback': <traceback object at 0x7fc9ce1f8240>, 'einfo': <ExceptionInfo: ExceptionWithTraceback()>} celery-celery stderr | [2023-11-29 11:57:12,238: ERROR/ForkPoolWorker-3] Failure while executing task: KeyError: {'signal': <Signal: task_failure providing_args={'exception', 'task_id', 'traceback', 'args', 'kwargs', 'einfo'}>, 'sender': <@task: weblate.utils.tasks.update_translation_stats_parents of weblate at 0x7fc9e3c49210>, 'task_id': 'facceb74-2c0e-4fae-83d0-d5db7df69706', 'args': [6931], 'kwargs': {}, 'traceback': <traceback object at 0x7fc9ce1f8240>, 'einfo': <ExceptionInfo: ExceptionWithTraceback()>} celery-celery stderr | [2023-11-29 11:57:12,238: ERROR/460] Failure while executing task celery-celery stderr | Traceback (most recent call last): celery-celery stderr | File "/usr/local/lib/python3.11/site-packages/celery/app/trace.py", line 477, in trace_task celery-celery stderr | R = retval = fun(*args, **kwargs) celery-celery stderr | ^^^^^^^^^^^^^^^^^^^^ celery-celery stderr | File "/usr/local/lib/python3.11/site-packages/celery/app/trace.py", line 760, in __protected_call__ celery-celery stderr | return self.run(*args, **kwargs) celery-celery stderr | ^^^^^^^^^^^^^^^^^^^^^^^^^ celery-celery stderr | File "/usr/local/lib/python3.11/site-packages/weblate/utils/tasks.py", line 79, in update_translation_stats_parents celery-celery stderr | translation.stats.update_parents() celery-celery stderr | File "/usr/local/lib/python3.11/site-packages/weblate/utils/stats.py", line 312, in update_parents celery-celery stderr | stat.update_stats() celery-celery stderr | File "/usr/local/lib/python3.11/site-packages/weblate/utils/stats.py", line 329, in update_stats celery-celery stderr | self.calculate_basic() celery-celery stderr | File "/usr/local/lib/python3.11/site-packages/weblate/utils/stats.py", line 337, in calculate_basic celery-celery stderr | self._calculate_basic() celery-celery stderr | File "/usr/local/lib/python3.11/site-packages/weblate/utils/stats.py", line 827, in _calculate_basic celery-celery stderr | stats[item] = max(values, default=time.time()) celery-celery stderr | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ celery-celery stderr | File "/usr/local/lib/python3.11/site-packages/weblate/utils/stats.py", line 824, in <genexpr> celery-celery stderr | values = (stats_obj.aggregate_get(item) for stats_obj in all_stats) celery-celery stderr | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ celery-celery stderr | File "/usr/local/lib/python3.11/site-packages/weblate/utils/stats.py", line 225, in aggregate_get celery-celery stderr | return self._data[name] celery-celery stderr | ~~~~~~~~~~^^^^^^ celery-celery stderr | KeyError: 'stats_timestamp' celery-celery stderr | [2023-11-29 11:57:12,238: ERROR/ForkPoolWorker-3] Failure while executing task celery-celery stderr | Traceback (most recent call last): celery-celery stderr | File "/usr/local/lib/python3.11/site-packages/celery/app/trace.py", line 477, in trace_task celery-celery stderr | R = retval = fun(*args, **kwargs) celery-celery stderr | ^^^^^^^^^^^^^^^^^^^^ celery-celery stderr | File "/usr/local/lib/python3.11/site-packages/celery/app/trace.py", line 760, in __protected_call__ celery-celery stderr | return self.run(*args, **kwargs) celery-celery stderr | ^^^^^^^^^^^^^^^^^^^^^^^^^ celery-celery stderr | File "/usr/local/lib/python3.11/site-packages/weblate/utils/tasks.py", line 79, in update_translation_stats_parents celery-celery stderr | translation.stats.update_parents() celery-celery stderr | File "/usr/local/lib/python3.11/site-packages/weblate/utils/stats.py", line 312, in update_parents celery-celery stderr | stat.update_stats() celery-celery stderr | File "/usr/local/lib/python3.11/site-packages/weblate/utils/stats.py", line 329, in update_stats celery-celery stderr | self.calculate_basic() celery-celery stderr | File "/usr/local/lib/python3.11/site-packages/weblate/utils/stats.py", line 337, in calculate_basic celery-celery stderr | self._calculate_basic() celery-celery stderr | File "/usr/local/lib/python3.11/site-packages/weblate/utils/stats.py", line 827, in _calculate_basic celery-celery stderr | stats[item] = max(values, default=time.time()) celery-celery stderr | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ celery-celery stderr | File "/usr/local/lib/python3.11/site-packages/weblate/utils/stats.py", line 824, in <genexpr> celery-celery stderr | values = (stats_obj.aggregate_get(item) for stats_obj in all_stats) celery-celery stderr | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ celery-celery stderr | File "/usr/local/lib/python3.11/site-packages/weblate/utils/stats.py", line 225, in aggregate_get celery-celery stderr | return self._data[name] celery-celery stderr | ~~~~~~~~~~^^^^^^ celery-celery stderr | KeyError: 'stats_timestamp'

[joubu]

Escalated into an issue #10508

[tomascohen]

thanks