Alternate lazy shader loading implementation

[illwieckz]

Right now we have 3 kinds of GLSL shader loading implementation (r_lazyShaders):

• 0: build all shaders at game start before reaching the main menu (old default):
  it builds everything before displaying something useful and the user even don't know what's happening,
  the main menu may not be reachable if an advanced shader doesn't work even if unused by the main menu.
• 1: build shaders on demand until map loads, then build everything else at map load (current default)
  make sure only the shaders required to display the main menu are built before displaying the main menu,
  makes fast game load, and fast preset selection in main menu,
  makes possible to select a lower preset in main menu a shader would fail later at map load,
• 2: build all shaders on demand,
  makes sure no unused shader is built,
  a shader may be built while playing the game the first time the texture requiring it is displayed, producing stuttering

Option 1 is a good enough trade-off, but we still build many shaders for nothing. For example we will build the relief mapping and the deluxe mapping permutations even when loading a Tremulous map without such feature.

With option 1 the engine will also build any shader permutation for unused features, for example if we merge #258 that makes possible to generate normal maps from height maps at render time if normal maps are missing, that feature and all the permutations (with or without deluxe map, with lightmap, with grid lighting…) will be built even if there is no asset missing a normal map in the currently played map.

With 0.55, the engine actually selects the rendering function at q3shader load time, meaning we may be able to just implement a loop that would render all the textures on some unseen framebuffer at map load time, triggering the GLSL shader build. That means such lazy shader loading would exhaustively build all required GLSL shaders at map load time, without missing used ones, and without building unused permutations.

[illwieckz]

This would allow us to worry less about adding more GLSL shader permutations, because the amount of shader permutations per feature is basically N^N^N^N… Our only way to avoid building crazy amounts of permutations is that we know some permutations combinations can't happen so we skip them, but we could simply skip all permutations that are not used for rendering the current map.

[illwieckz]

Actually the deluxe/lightmap permutation usage is only known at run time, so the suggestion would not fully work. But we may force the building of only a couple of them to avoid that.

Another implementation for such lazy loading would just be to render the whole world and all models to some unseen framebuffer in a frame at the end of map load.

[slipher]

Another implementation for such lazy loading would just be to render the whole world and all models to some unseen framebuffer in a frame at the end of map load.

That would get you fairly close I suppose. There's probably still some ways that additional q3shaders can be loaded though, which could trigger more GLSL. Like maybe that map entity shader remap thing

[illwieckz]

Like maybe that map entity shader remap thing.

Interesting.

Another example of thing I know to be loaded in game is some UI textures when some UI is rendered for the first time, but this one is the kind of thing we can force the build just to be sure.

[slipher]

When using /glsl_restart, which currently behaves like r_lazyShaders 2, I notice that it can restart very quickly. So it does seem that on a typical map we are not using the vast majority of shaders. We might get some big wins in loading time here.

Maybe the cgame should be tasked with rendering all the possible things. For example, rendering models with all possible states (e.g. powered vs. unpowered), since those can have different shaders. To handle shader remaps, the world could be rendered multiple times with each remap activated and deactivated. But you'd have to render not just the main BSP, but also any sub-models and embedded md3/iqm/etc. models. This would require a lot of knowledge about BSP entities to added to the cgame.

[VReaperV]

Models and all that are already loaded by the first frame in https://github.com/Unvanquished/Unvanquished/blob/cc472ba717bd0e2a9e4d2e19f4daab25d154a6c4/src/cgame/cg_main.cpp#L1317-L1361

Not sure as to the BSP entities though. Engine should have enough information to build all the relevant shaders for them though.