About the invalid of Py_InitializeFromConfig

[rruuaanng]

Bug report

Bug description:

My test code.

#include <stdio.h>
#include <Python.h>
int main(int argc, char* argv[]) {
        PyStatus py_status;
        PyConfig py_config;

        PyConfig_InitPythonConfig(&py_config);

        py_config.module_search_paths_set = 0;
        PyWideStringList_Append(&py_config.module_search_paths, L"/home/a");
        py_status = Py_InitializeFromConfig(&py_config);
        if (PyStatus_Exception(py_status)) {
                return -1;
        }

        for (int i = 0; i < py_config.module_search_paths.length; ++i) {
                printf("%ls \n", py_config.module_search_paths.items[i]);
        }

        puts("===========================================");
        PyRun_SimpleString("import sys; print(sys.path)");

        return 0;
}

output

/home/a
===========================================
['/usr/local/lib/python314.zip', '/usr/local/lib/python3.14', '/usr/local/lib/python3.14/lib-dynload', '/usr/local/lib/python3.14/site-packages']

This issue is a continuation of issue #122963. Based on my tests, I discovered that sys.path is actually obtained through the config retrieved by _PyInterpreterState_GetConfig, rather than the config passed by the user. As a result, the config argument in the Py_InitializeFromConfig function does not actually take effect. I have completed my modifications regarding this. My solution is to save the config passed by the user and then append its content to sys.path.
In fact, by the time pyinit_main is executed, the config obtained through the thread context is no longer the one passed by the user.

My change diff.

Py_InitializeFromConfig(const PyConfig *config)
 {
+    const PyConfig *uconfig = config;
+
     if (config == NULL) {
         return _PyStatus_ERR("initialization config is NULL");
     }
@@ -1409,6 +1410,19 @@ Py_InitializeFromConfig(const PyConfig *config)
         }
     }

+    PyObject *sys_path = PySys_GetObject("path");
+    if (uconfig->module_search_paths_set == 0
+     && sys_path != NULL && PyList_Check(sys_path)) {
+        for (Py_ssize_t i = 0; i < uconfig->module_search_paths.length; i++) {
+            wchar_t *item = uconfig->module_search_paths.items[i];
+            PyObject *py_item = PyUnicode_FromWideChar(item, wcslen(item));
+            if (item == NULL) {
+                /* I'm still thinking about what error handling to use here */
+            }
+            PyList_Append(sys_path, py_item);
+        }
+    }
+
     return _PyStatus_OK();
 }

changed output

/home/a
===========================================
['/usr/local/lib/python314.zip', '/usr/local/lib/python3.14', '/usr/local/lib/python3.14/lib-dynload', '/usr/local/lib/python3.14/site-packages', '/home/a']

If I'm allowed to fix this, I'll look for a more suitable modification than this one.

CPython versions tested on:

CPython main branch

Operating systems tested on:

No response

[rruuaanng]

cc @vstinner

[vstinner]

If you want to use module_search_paths, you must set module_search_paths_set to 1: https://docs.python.org/dev/c-api/init_config.html#c.PyConfig.module_search_paths

[rruuaanng]

I'm sorry. What I want to express is that it seems that when he executed sys.path initialization, interp.config was not passed by the user at that time.

Edit:
That's to say, the config argument isn't actually added to the path.

[rruuaanng]

╭─root at RUANG in ~/cpython on gh84016✘✘✘
╰─± cat test.c
#include <stdio.h>
#include <Python.h>


int main(int argc, char* argv[]) {
        PyStatus py_status;
        PyConfig py_config;

        PyConfig_InitPythonConfig(&py_config);

        py_config.module_search_paths_set = 1;
        PyWideStringList_Append(&py_config.module_search_paths, L"/home/a");
        py_status = Py_InitializeFromConfig(&py_config);
        if (PyStatus_Exception(py_status)) {
                return -1;
        }

        for (Py_ssize_t i = 0; i < py_config.module_search_paths.length; ++i) {
                printf("%ls \n", py_config.module_search_paths.items[i]);
        }

        puts("===========================================");
        PyRun_SimpleString("import sys; print(sys.path)");

        return 0;
}
╭─root at RUANG in ~/cpython on gh84016✘✘✘
╰─± gcc test.c -I Include -I . libpython3.14.a -lm -oq -g
╭─root at RUANG in ~/cpython on gh84016✘✘✘
╰─± ./q
╭─root at RUANG in ~/cpython on gh84016✘✘✘
╰─±

[vstinner]

╭─root at RUANG in ~/cpython on gh84016✘✘✘
╰─± ./q

Your program fails silently because you should compute the whole sys.path (e.g. path to the stdlib), not just add your /home/a directory.

I suggest you to modify sys.path after Python initialization, during before/during the initialization.

For example, call this function after Py_InitializeFromConfig():

PyRun_SimpleString("import sys; sys.path.append('/home/a')");

Yeah sorry, the sys.path configuration is quite complicated :-( It also changed in Python 3.11 with the addition of Modules/getpath.py.