New valgrind warning in Python 3.12.5

[julian-smith-artifex-com]

Bug report

Bug description:

I’ve just started seeing a Valgrind (1:3.18.1-1ubuntu2) warning in a Linux (ubuntu-22.04) Github test that started happening today when the Python version changed from 3.12.4 to 3.12.5.

The test is the pytest test suite in PyMuPDF - see https://github.com/pymupdf/PyMuPDF/actions/workflows/test-valgrind.yml.

There were no changes to PyMuPDF or MuPDF when the Valgrind warning appeared and Valgrind itself and pytest versions have also not changed. So it looks like the only change is Python itself.

Conditional jump or move depends on uninitialised value(s)
   at 0x4A66154: tok_get_fstring_mode (/home/runner/work/_temp/SourceCode/Parser/tokenizer.c:2705)
   by 0x4A63925: tok_get (/home/runner/work/_temp/SourceCode/Parser/tokenizer.c:2857)
   by 0x4A63925: _PyTokenizer_Get (/home/runner/work/_temp/SourceCode/Parser/tokenizer.c:2862)
   by 0x4A5282D: _PyPegen_fill_token (/home/runner/work/_temp/SourceCode/Parser/pegen.c:298)
   by 0x4A62BAF: fstring_replacement_field_rule (/home/runner/work/_temp/SourceCode/Parser/parser.c:15914)
   by 0x4A58F7A: fstring_middle_rule (/home/runner/work/_temp/SourceCode/Parser/parser.c:15858)
   by 0x4A58F7A: _loop0_114_rule (/home/runner/work/_temp/SourceCode/Parser/parser.c:32096)
   by 0x4A58F7A: fstring_rule (/home/runner/work/_temp/SourceCode/Parser/parser.c:16193)
   by 0x4A58F7A: _tmp_259_rule (/home/runner/work/_temp/SourceCode/Parser/parser.c:40681)
   by 0x4A58F7A: _loop1_115_rule (/home/runner/work/_temp/SourceCode/Parser/parser.c:32163)
   by 0x4A58F7A: strings_rule (/home/runner/work/_temp/SourceCode/Parser/parser.c:16294)
   by 0x4A580A7: atom_rule (/home/runner/work/_temp/SourceCode/Parser/parser.c:14690)
   by 0x4A60048: primary_raw (/home/runner/work/_temp/SourceCode/Parser/parser.c:14328)
   by 0x4A5FCE8: primary_rule (/home/runner/work/_temp/SourceCode/Parser/parser.c:14126)
   by 0x4A5F731: await_primary_rule (/home/runner/work/_temp/SourceCode/Parser/parser.c:14080)
   by 0x4A5F731: power_rule (/home/runner/work/_temp/SourceCode/Parser/parser.c:13956)
   by 0x4A5F731: factor_rule (/home/runner/work/_temp/SourceCode/Parser/parser.c:13906)
   by 0x4A5ECDE: term_raw (/home/runner/work/_temp/SourceCode/Parser/parser.c:13747)
   by 0x4A5E9C0: term_rule (/home/runner/work/_temp/SourceCode/Parser/parser.c:13509)
   by 0x4A5E80A: sum_raw (/home/runner/work/_temp/SourceCode/Parser/parser.c:13391)
   by 0x4A5E80A: sum_rule (/home/runner/work/_temp/SourceCode/Parser/parser.c:13342)

I can look at creating a cut-down reproducer if required.

Thanks,

- Julian

CPython versions tested on:

3.12

Operating systems tested on:

Linux

Linked PRs

• gh-123229: Fix valgrind warning by initializing the f-string buffers to 0 in the tokenizer #123263
• [3.13] gh-123229: Fix valgrind warning by initializing the f-string buffers to 0 in the tokenizer (GH-123263) #123264
• [3.12] gh-123229: Fix valgrind warning by initializing the f-string buffers to 0 in the tokenizer (GH-123263) #123265

[ZeroIntensity]

Probably a false positive, we just need to update the suppressions file.

[pablogsal]

Hummm, it may be freaking out that the array of f-string buffers it's not 0-initialized. If you have a small reproducer I can try to test this theory. Alternatively, you can try to compile Python yourself and test with this patch against 3.12:

diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c
index 3118fb19846..9e0dee8cc38 100644
--- a/Parser/tokenizer.c
+++ b/Parser/tokenizer.c
@@ -65,7 +65,7 @@ static const char *type_comment_prefix = "# type: ";

 static struct tok_state *tok_new(void) {
   struct tok_state *tok =
-      (struct tok_state *)PyMem_Malloc(sizeof(struct tok_state));
+      (struct tok_state *)PyMem_Calloc(1, sizeof(struct tok_state));
   if (tok == NULL)
     return NULL;
   tok->buf = tok->cur = tok->inp = NULL;

[julian-smith-artifex-com]

Here's a simple reproducer:

wget https://raw.githubusercontent.com/pymupdf/PyMuPDF/main/src/__init__.py
valgrind python3 __init__.py

For me this generates the valgrind warnings before the expected error ImportError: attempted relative import with no known parent package.

@pablogsal, your patch fixes the problem :-)

[pablogsal]

Fixed! Thanks for the report @julian-smith-artifex-com

[hauntsaninja]

Thanks for reporting and for the fix! This issue affected the behaviour of Black, see #123821 :-)