WIP: Fix various issues with config file loading, parsing & generation

[julianeisel]

Includes changes proposed in !124.

I'm throwing this here for initial discussion together with !124. I'm planning to submit each fix individually (or as multiple commits).

Summary by Sourcery

Improve handling of config files and XML parsing.

New Features:

• Add fallback name for undocumented inner items in XML.

Bug Fixes:

• Fix XML parsing errors caused by invalid characters.

Enhancements:

• Merge src-dirs with the "INPUT" paths from the doxy-cfg-file.
• Create directories for output files if they don't exist.
• Always override GENERATE_XML, GENERATE_HTML, and GENERATE_LATEX in the Doxygen config.
• Set the Doxygen working directory to the config file's directory if provided.

[sourcery-ai]

Reviewer's Guide by Sourcery

This pull request addresses several issues related to Doxygen configuration, XML parsing, and file handling. It improves the robustness of the config loading, parsing, and generation process by adding better error handling, input path merging, and output directory creation.

Sequence diagram for XML parsing with error handling

sequenceDiagram
    participant Node
    participant ElementTree
    participant File

    Node->>ElementTree: parse(xml_file)
    alt Successful parsing
        ElementTree-->>Node: Return parsed XML
    else ParseError
        ElementTree-->>Node: Throw ParseError
        Node->>File: Read file contents
        File-->>Node: Return file data
        Node->>ElementTree: fromstring(contents with UTF-8 replacement)
        ElementTree-->>Node: Return parsed XML
    end
    Node->>Node: find('compounddef')

Loading

Class diagram showing Node class changes

classDiagram
    class Node {
        -_dirname: str
        -_xml: Element
        -_kind: Kind
        -_visibility: Visibility
        +__init__(xml_file, xml, project, parser, parent)
        +_check_for_children()
        +_parse_inner_into_node(refid, fallback_name)
        +_check_attrs()
        note for Node "Added error handling for XML parsing
and new helper method for child nodes"
    }

    class DoxygenRunner {
        -doxygenSource: str
        -tempDoxyFolder: str
        -doxyConfigFile: str
        +setDoxyCfg(doxyCfgNew: dict)
        +merge_doxygen_input(doxyCfg: dict)
        +getDoxygenRunFolder()
        note for DoxygenRunner "Added input path merging
and working directory handling"
    }

Loading

Flow diagram for config file loading and merging

flowchart TD
    A[Start Config Loading] --> B{Has doxy-cfg-file?}
    B -->|Yes| C[Load base config]
    B -->|No| D[Use default config]
    C --> E[Apply mandatory overrides]
    D --> E
    E --> F{Has src-dirs?}
    F -->|Yes| G[Merge INPUT paths]
    F -->|No| H[Use existing INPUT]
    G --> I[Update config]
    H --> I
    I --> J[Set output directory]
    J --> K[End Config Loading]

Loading

File-Level Changes

Change	Details	Files
Improved XML parsing by handling invalid characters.	Added a try-except block to catch XML parsing errors. Replaced invalid characters with valid UTF-8 characters before parsing. Added a fallback to create a 'compounddef' element if the XML file does not exist.	mkdoxy/node.py
Refactored node creation to handle missing XML files.	Created a new _parse_inner_into_node method to handle node creation. Moved the logic for creating a default 'compounddef' element into the new method. Simplified the node creation logic in _check_for_children.	mkdoxy/node.py
Improved Doxygen configuration by merging input paths and overriding output settings.	Added a merge_doxygen_input method to merge src-dirs with the INPUT paths from the config file. Always override GENERATE_XML, GENERATE_HTML, and GENERATE_LATEX settings. Log the merged input paths. Set the Doxygen execution directory to the config file's directory.	mkdoxy/doxyrun.py
Fixed a bug where the output directory was not created.	Created the parent directory of the output file if it does not exist.	mkdoxy/generatorAuto.py
Fixed a bug where the hash was not correctly generated.	Use the merged input paths to generate the hash.	mkdoxy/doxyrun.py

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time. You can also use
  this command to specify where the summary should be inserted.

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.