Upgrade Visual Studio to a supported version in Windows Docker

[ericoporto]

Windows Dockerfile: ci/windows/Dockerfile

First, VS2019 (or VS2022) C++ builds are binarily compatible with VS2015 builds. So there's no change here. This is to make it easier to modify and upgrade the Windows build environment in use.

VS2015 is out of support and we rely in an old Docker image I had in dockerhub for it. Ideally, we want to upgrade to the current stable Visual Studio, which is VS2019 (erh, or maybe VS2022). But this can be a problem in Cirrus CI.

• VS2019/VS2022 occupies a lot more space
• The Docker Image gets bigger
• Cirrus CI takes so long to download the docker it timeouts the Windows build

The solution is finding out what are the parts (workloads and individual components) of VS we need and install only those.

References:

• What are workloads: https://docs.microsoft.com/en-us/visualstudio/install/modify-visual-studio
• Workloads list: https://docs.microsoft.com/en-us/visualstudio/install/workload-component-id-vs-build-tools
• Installing build tools in a container: https://docs.microsoft.com/en-us/visualstudio/install/build-tools-container
• Container issues with Build Tools https://docs.microsoft.com/en-us/visualstudio/install/build-tools-container-issues
• Cirrus Windows Docker images: https://github.com/cirruslabs/docker-images-windows
• MS Windows Container repository: https://github.com/microsoft/Windows-Containers
  • Note issues are closed in 30 days even if they aren't solved so there's a ton of problems accumulating there... (example Intermittent DNS issues with Windows containers microsoft/Windows-Containers#217)

---

this is a requirement for #1462

I talked a bit on the problem of unsupported VS2015 in PR #1291

[morganwillcock]

Is it feasible to decide to always track the latest versions of the build tools at all times and try to build with the default containers that Microsoft supply (or maybe just run on the CI runners directly)?

Visual Studio is pretty much on a rolling upgrade so I think that aligns with the release model, but unfortunately I don't think everyone will be in the position to always upgrade or that the upgrades themselves are always trouble free. I think the latest version is now 2021 rather than 2019.

[ericoporto]

default containers that Microsoft supply

Erh, where? (the official list from mcr is unhelpful)

version is now 2021 rather than 2019.

The latest release is 2022, but until not long ago it was still marked as preview in vswhere.exe, I haven't checked if it has been moved to stable. Edit: to clarify, if you ask the preview release installer if VS is installed, it says "no", even though the real answer is yes, so tools that use this (some MS own tools, but also CMake - in CMake you can override this by telling it "trust me") fail to find VS. But yet, if we could move to 2022 it may ease the transition.

[morganwillcock]

Erh, where?

I thought they were uploaded, but it seems they only supply a reference Dockerfile and it isn't up to date:
https://github.com/microsoft/vs-dockerfiles

...so probably within CI runners is the only place they are pre-installed, but that probably means moving the CI setup (although that is probably not out of the question because all of the setup and build commands are hidden in the CI configuration and should be separate).

[ericoporto]

moving the CI setup

You mean switching to something else? (like GitHub Actions or something)

all of the setup and build commands are hidden in the CI configuration

I have been recently thinking about making a small tool/script to build AGS (not only editor, and the engine, but even placing the manual in place, building the templates, ...), nothing too different from that first AGS bat script you did once.

[morganwillcock]

You mean switching to something else? (like GitHub Actions or something)

Yes, if the current one times out trying to download then there might not be much possibility to fix it, outside of trying to optimise the container size.

I have been recently thinking about making a small tool/script to build AGS (not only editor, and the engine, but even placing the manual in place, building the templates, ...), nothing too different from that first AGS bat script you did once.

I wonder if CMake can just take those actions are part of the build process, and maybe that also deals with the difference in Visual Studio versions too if the project/solution files are generated rather than manually upgraded between versions?

[ericoporto]

I wonder if CMake can just take those actions are part of the build process, and maybe that also deals with the difference in Visual Studio versions too if the project/solution files are generated rather than manually upgraded between versions?

It can do the download stuff, the installer and the C# stuff. One thing I am not sure, meaning is the one thing I haven't experimented with, is the C++/CLR (AGSNative). Edit: it does support that too.

[AlanDrake]

So, is there a plan to make the jump to VS2022?
Is anyone stuck with a version of Windows that doesn't run the newer VSes?

I keep VS2015 only for AGS, since all other stuff I need to compile usually moves on within 1-2 years.

[ericoporto]

Let's recap a bit

• vswhere doesn't detect build tools by default (from 2017 and above) (Support the BuiltTools SKU microsoft/vswhere#8)
• CMake uses vswhere and doesn't detect build tools (https://gitlab.kitware.com/cmake/cmake/-/issues/22197)
• This forces to install the full VS in the Windows container
• The Windows container image becomes so big it times out when downloading to Google Compute used in CirrusCI

So we need to check if this is changed or not - we need CMake to build our dependencies. Possible solution too is to build dependencies separately in a different repositor, release them and then get them in ags main repository - this may be a bit boring to handle the different branches. The good thing though is we could just release the dependencies to NuGet and then simply consume in the VS projects. (something like this, but including every binaries)

In the github actions, the engine is built using cmake and vs2022 and it appears to run fine in Windows 7. But the engine is easy, we could even build it with mingw too from a Linux CI.

Of course, the issue is the Editor (and the C++/CLR compiler)... This locks us into MSVC. I am investigating building the editor with GitHub Actions too, just taking a little to figure how to build/cache dependencies.

I prioritized making a Linux container that can build our installer first, working on it here: https://github.com/ericoporto/innosetup-wine

---

In summary, if AGSNative wasn't dependant on C++/CLR, we could build everything on Linux and just run tests on Windows. :/

[ivan-mogilko]

Is anyone stuck with a version of Windows that doesn't run the newer VSes?

I tried to install a VS 2022 community edition on my Win 7, but apparently they dropped support for Win 7 in the latest version (17.7), and I could not find if it's possible to install an earlier version of CE (they have links for paid variants only).

The highest VS i had installed on Win 7 was VS 2019, this was couple of years ago.

[ericoporto]

I started to work on this

• I am going to keep the images ltsc2019 based because ltsc2022 doesn't run in Windows 10 so I can't test locally (apparently they require the Windows server 2022 to run?)
• Microsoft made vs2015 available again for download and install, so I plan to add it (at least during transition)
• I also plan to add vs2022 for the future vs2019 projects, vs2022 has some interesting things like address sanitizers that may be interesting to run in CI

I played with the docker image in my computer this weekend. MS made some improvements that the size of the docker image is back at roughly what it was again.

[ericoporto]

Playing with it here https://github.com/ericoporto/min-ags-dev-env/blob/main/Dockerfile

The issue it seems is despite the install of Windows 8.1 SDK apparently running, it doesn't show up under the Windows Kits directory in the Program Files. It's a bit hard to find information online. The Windows 8.1 SDK does appear to install correctly in my computer just not in the docker environment, I am not sure yet why.

Edit:

https://learn.microsoft.com/en-us/archive/msdn-technet-forums/0f11530a-d7a0-4c50-aeaa-094d054402b8.

Apparently it's possible to target older Windows with win10 sdk

[ericoporto]

ok, I see similar defines in SDL

https://github.com/libsdl-org/SDL/blob/451dc414278dc7b19a997ef7a6d1acf1f951162d/src/core/windows/SDL_windows.h#L42

https://github.com/libsdl-org/SDL/blob/451dc414278dc7b19a997ef7a6d1acf1f951162d/src/core/windows/SDL_windows.c#L42

I think we indeed can use Windows 10 SDK to build for a lower version like Windows 7

https://learn.microsoft.com/en-us/cpp/porting/modifying-winver-and-win32-winnt?view=msvc-170

I think any defines need to happen before any #include <Windows.h>.

https://learn.microsoft.com/en-gb/windows/win32/winprog/using-the-windows-headers?redirectedfrom=MSDN#macros-for-conditional-declarations

I am still trying to understand how it is that things work already - do we not define anything? Is there a default already and it suits us, so no need to set things? (Perhaps we get things set from SDL?)

[ericoporto]

Apparently Microsoft removed VS 2015, VS 2017 and VS 2019 from downloads, so a new proper docker image would have to use VS 2022

[ivan-mogilko]

Apparently Microsoft removed VS 2015, VS 2017 and VS 2019 from downloads, so a new proper docker image would have to use VS 2022

Do you know if it will be possible to have repo solutions for VS 2019 and still make docker work with VS 2022?

I would need to upgrade Windows to a newer version in order to use VS 2022. I probably should do this, since programs stop working on Win7, but I cannot do this at a random moment, got to prepare for this transition... I will try after the new year.

I might try making a PR for VS2019 (#2241) as a preliminary step in the meantime. I already have 2019 installed, but those who don't might use VS 2022 to work with it, I think?

[ericoporto]

Do you know if it will be possible to have repo solutions for VS 2019 and still make docker work with VS 2022?

Yes, if we upgrade the docker first, it should be able to build the current repository without changes and then we could upgrade the projects.

VS 2022 has compatibility with VS 2015, VS 2017 and VS 2019, C++ wise. I am using it on the desktop.

What we use of Windows through windows.h and the Windows SDK, I think there are flags one can use to force only the APIs that are backwards compatible, but if those are not set the issue is that you may accidentally use APIs that are not backwards compatible but at most that - currently we don't.

Then there is the C++/CLI, for the Native glue in the Editor, I think this specifically just uses the newest everything when building this part of the code, but I am not super sure. I don't see an option to install the old versions of this, so only assuming from this.

So, in theory, using only VS 2022, I should be able to build the editor. And this is true, I am able to do in my notebook that only has VS2022, without upgrading anything in the current AGS Projects. I tried an year ago to build the Editor using GitHub Actions, which only has VS 2022, but I failed. I need to try again - since I have been actually using it for the last two months maybe I should be able to navigate it better... Idnk, need to try again.