Dotnet 9 Crash on Linux only

[HugoVG]

Describe the bug

YARP on Dotnet 9 crashes on Linux-x64

YARP attempts to use either IIS or HttpSys on linux machine in net9, but just works as intended on net8.
same package version just different runtime, strange enough the application works just fine under Docker, but doesn't work under WSL and native Ubuntu Server(24).
I originally caught it because the CICD tests were all failing after a dotnet 9 update, but they all passed on Windows (tested multiple workstation).

To Reproduce

Further technical details

.NET SDK:
Version: 9.0.100
Commit: a2bc464e40
Workload version: 9.0.100-manifests.6bf02610
MSBuild version: 17.12.7+a2bc464e4

Runtime Environment:
OS Name: ubuntu
OS Version: 22.04
OS Platform: Linux
RID: ubuntu.22.04-x64
Base Path: /usr/lib/dotnet/sdk/9.0.100/

.NET workloads installed:
There are no installed workloads to display.
Configured to use loose manifests when installing new manifests.

Host:
Version: 9.0.0
Architecture: x64
Commit: a2bc464e40

.NET SDKs installed:
9.0.100 [/usr/lib/dotnet/sdk]

.NET runtimes installed:
Microsoft.AspNetCore.App 9.0.0 [/usr/lib/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 9.0.0 [/usr/lib/dotnet/shared/Microsoft.NETCore.App]

• Include the version of the packages you are using Version 2.2.0 (worked under net8 no longer under net9)

• The platform (Linux/macOS/Windows) Linux

• Settings json

"ReverseProxy": {
        "Routes": {
            "all": {
                "ClusterId": "cluster1",
                "AuthorizationPolicy": "anonymous",
                "Match": {
                    "Path": "{**catch-all}"
                },
                "Order": 999
            }
        },
        "Clusters": {
            "cluster1": {
                "Destinations": {
                    "destination1": {
                        "Address": "https://<<FallbackServer>>"
                    }
                }
            }
        }
    },

[MihaZupan]

I can't repro this on WSL.

YARP attempts to use either IIS or HttpSys on linux machine in net9

Are you able to share error messages / stack traces / crash dumps?

[HugoVG]

Oh I'm stupid i thought i added the stack trace.

For the time being I've wrapped Yarp

if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
{
    app.MapReverseProxy();
}

Running on Kestrel
info: Program[0]
      Building services took 6755ms
info: Program[0]
      Current environment: Development
info: Program[0]
      Adding common Middleware took 19ms
info: Program[0]
      Mapping swagger took 195ms
info: Program[0]
      Setting up routing, Auth and controllers took 53ms
Unhandled exception. System.TypeLoadException: Could not load type 'Microsoft.AspNetCore.Server.HttpSys.IServerDelegationF
eature' from assembly 'Microsoft.AspNetCore.Server.HttpSys, Version=9.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'.
   at Yarp.ReverseProxy.Delegation.HttpSysDelegator..ctor(IServer server, ILogger`1 logger)
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)      
   at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
   at System.Reflection.MethodBaseInvoker.InvokeWithFewArgs(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.CreateServiceAccessor(ServiceIdentifier serviceIdentifier) 
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(ServiceIdentifier serviceIdentifier, ServiceProviderEngineScope serviceProviderEngineScope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType)      
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
   at Yarp.ReverseProxy.Management.IReverseProxyBuilderExtensions.<>c.<AddHttpSysDelegation>b__8_1(IServiceProvider p)    
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitFactory(FactoryCallSite factoryCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitIEnumerable(IEnumerableCallSite enumerableCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.CreateServiceAccessor(ServiceIdentifier serviceIdentifier) 
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(ServiceIdentifier serviceIdentifier, ServiceProviderEngineScope serviceProviderEngineScope)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
   at Microsoft.AspNetCore.Builder.ReverseProxyIEndpointRouteBuilderExtensions.GetOrCreateDataSource(IEndpointRouteBuilder endpoints)
   at Microsoft.AspNetCore.Builder.ReverseProxyIEndpointRouteBuilderExtensions.MapReverseProxy(IEndpointRouteBuilder endpoints, Action`1 configureApp)
   at Microsoft.AspNetCore.Builder.ReverseProxyIEndpointRouteBuilderExtensions.MapReverseProxy(IEndpointRouteBuilder endpoints)
   at Program.<Main>$(String[] args) in /<pathToProgram>/Program.cs:line 189
   at Program.<Main>(String[] args)

[MihaZupan]

YARP will attempt to fetch the IServerDelegationFeature at startup, but it won't throw if one doesn't exist (exactly because you might be running on Linux / using Kestrel)

reverse-proxy/src/ReverseProxy/Delegation/HttpSysDelegator.cs

Line 36 in ed43822

_serverDelegationFeature = server.Features?.Get<IServerDelegationFeature>();

---

System.TypeLoadException

This indicates that something went wrong when loading the Microsoft.AspNetCore.Server.HttpSys assembly (sadly not super informative as to the reason), and is outside of YARP's control.

Since I can't repro the same issue, it's possibly affected by something in your environment (how you installed dotnet, how you're packaging the app (self-contained, trimming, ...), any special monitoring/AV software, ...).

Can you please file an issue in https://github.com/dotnet/aspnetcore?

[MihaZupan]

Are you publishing the app as self-contained/with trimming/native AOT?
Things like that could affect type loading.

[HugoVG]

No they might be published self contained but normal dotnet run without AOT or trimming (sadly app is not compatible)