Transition from precompiled header files to C++ modules

[stf976]

Problem Description

Traditionally, header files in C++ projects are compiled with every translation unit that includes them. As projects grow, the recompilation of header files adds up and slows down the build process. In addition, during development cycles, header files may be changed frequently. Each change triggers a recompilation of all dependent files, aggravating the issue.

Different compiler developers have long addressed this issue with the use of precompiled headers. Projects in Visual Studio can define one precompiled header (PCH), e.g. cg_stdafx.hpp in the case of Gears-Vk, that includes other headers which should be precompiled. Files that depend on these headers can include the PCH, which replaces the headers and significantly reduces compilation times, as it is only compiled once for each project.

However, this solution has never been standardized, resulting in varying implementation details and a lack of compatibility between different compilers. C++20 modules provides a standardized, performant, and safe replacement to C++ header files (and thus PCHs).

The main task for this issue is to replace header files in Gears-Vk with C++ modules. The introduction of C++ modules may also be beneficial to restructure the framework more efficiently.

Status Quo

As of now, in Gears-Vk, gvk.hpp includes all header files of the project and is intended to make the use of Gears-Vk simple and efficient. A user can include gvk.hpp and access all functionality without worrying about which headers to include. It also allows to easily add all headers of Gears-Vk to a user project's PCH to speed up compilation.

Internally, gvk.hpp is also included by all files in the Gears-Vk project and included in the project's PCH. This has the downside that any change to any header will trigger a recompilation of all source and header files. It is generally not recommended to include a project's header files in its PCH; instead, only a few select external headers should be included, that provide the greatest performance gain (see e.g. (1), (2)).

A related problem occurs if a user project does not have gvk.hpp in its PCH, or PCHs are not available or disabled. Then each include in every one of the user's source files might directly or indirectly include gvk.hpp, which will then be compiled multiple times. This also happens if the framework itself is built without PCH, e.g. when using a compiler without PCH support or porting to platforms for which PCHs must be enabled differently than in VS (e.g. Linux with cmake, GCC): if Gears-Vk assumes PCH is always available, these builds will be slowed down considerably. The pros and cons of PCHs are discussed in more detail e.g. in (4).

Proposed Solution

C++ modules provide a modern, performant, safe replacement for header files with a syntax that should look familiar to C++ programmers. Modules can co-exist with traditional header files, so it is possible to successively adapt modules in C++ projects.

Modules have several advantages over plain header files. They are compiled into binary files which avoids recompilation with every translation unit and provides a unique list of symbols to link against, enforcing the One Definition Rule in C++. Private definitions and macros are not leaked from modules, unlike header files, and the order of importing modules in source files has no effect on program semantics. Modules can be split into sub-modules, definition and implementation files, if needed, to better represent the software structure.

Definintion of done:

• Make yourself familiar with C++20 modules. This primer (5)) and tutorial (6)) should give you a good overview and starting point. Think about how they might be used to restructure the framework. Where possible, use modules instead of header files to solve the following issues.
• As a prerequisite for the transition to modules, adapt files in Gears-Vk such that they only include the minimum of header files required to compile, and do not include gvk.hpp. Rationale: modules must include other modules directly and cannot depend on a catch-all header such as gvk.hpp because of circular references; it reduces recompilation to a minimum when files in the project change; it provides a clearer view of the physical and logical structure of the framework, thus enabling further improvements and increasing maintainability.
• Move all declarations in gvk.hpp to other Gears-Vk files; since no file in Gears-Vk should include gvk.hpp, the declarations must be placed elsewhere.
• Transform headers one by one into modules, and import the modules where needed. Make sure to only transform public headers into modules, and do not expose implementation details to the user. Only expose external headers if necessary for a user program to make use of a modules' functionality. Rationale: private headers should never be exposed to the user; a user may never need the external headers directly, thus not including them decreases dependencies and adds fewer symbols to the user program; it also helps to prevent exposing external headers that are no longer used in the framework itself (e.g. because an implementation detail changed).
• Remove gvk.hpp when it is no longer needed, or replace it with a header that includes all Gears-Vk modules (if there are more than one) for convenience.

Example

To try the transition to modules first hand, you might want to experiment with a single Gears-Vk header, e.g. image_data.hpp.

Create a new module file Gvk.Example.ixx, with the content:

export module Gvk.Example; // keywords export module marks this as a primary module interface unit

export import :ImageData; // bring in the ImageData partition, and export it to consumers of this module

Move the content of image_data.hpp to a new module partition file Gvk.Example:ImageData.ixx. Prepend this line to the file:

export module Gvk.Example:ImageData; // defines a module partition, ImageData, that's part of the module Gvk.Example

Add export qualifiers to every declaration that should be user-visible (e.g. classes, functions).

Move the content of image_data.cpp to a new module unit implementation file Gvk.Example:ImageData.cpp. Prepend this line to the file:

module;

// global module fragment area. Put #include directives here 

module Gvk.Example:ImageData;

Import the module wherever an include of image_data.hpp was previously needed:

import Gvk.Example;

Try to build the project with your new module!

Notes:

• C++ modules are available in Visual Studio 2022 17.1.0 or later.
• To find the minimum dependencies of a file, follow the guidelines as described in (3).
• In Visual Studio, build timings can be enabled with Tools->Options->Projects->VC++ Build and setting Build Timing to Yes.
• Keep an eye on the physical and logical structure of the framework and take note of potential improvements. See e.g. (3) and (4) for an introduction to the topic.

[johannesugb]

After having worked on some parts of Gears-Vk lately, I agree that the compile times have become pretty high. This is not an optimal situation for anyone contributing to the frameworks. I'll promote this issue to "urgent".