enableDependencyTracking: false in host.json is ignored, leading to excessive debug logs in Application Insights

[gunzip]

I am using Azure Functions with the Node.js runtime. I would like to enable end-to-end tracing of requests using OpenTelemetry. I have enabled the default integration with Application Insights by setting the APPLICATIONINSIGHTS_CONNECTION_STRING variable. Additionally, I have enabled Node.js process instrumentation using the following code:

import * as ai from "applicationinsights";

import { UndiciInstrumentation } from "@opentelemetry/instrumentation-undici";
import { registerInstrumentations } from "@opentelemetry/instrumentation";
import { metrics, trace } from "@opentelemetry/api";
import { AzureFunctionsInstrumentation } from "@azure/functions-opentelemetry-instrumentation";

// defaults to APPLICATIONINSIGHTS_CONNECTION_STRING
ai.setup().start();

// instrument native node fetch
registerInstrumentations({
  tracerProvider: trace.getTracerProvider(),
  meterProvider: metrics.getMeterProvider(),
  instrumentations: [
    new UndiciInstrumentation(),
    new AzureFunctionsInstrumentation(),
  ],
});

So far, everything works fine, and calls are successfully traced end-to-end. The host traces incoming HTTP requests (which cannot be intercepted in the worker), while the Node.js process traces everything else (outgoing HTTP calls, connections to Redis, CosmosDB, etc.).

Investigative information

• Timestamp: 2025-01-26T10:32:03.2868466Z
• Function App version: v4 with NodeJS 20.x
• Function App name: ASP-dev-9f8a (Y1: 0)
• Function name(s) (as appropriate): tracing-test
• Invocation ID: 81f0a762891c548b61d2606bd9c59e8c
• Region: Westeurope

Repro steps

1. Enable Application Insights integration via APPLICATIONINSIGHTS_CONNECTION_STRING.
2. Use the provided OpenTelemetry instrumentation code.
3. Observe traces in Application Insights.

Expected behavior

I would expect that setting the enableDependencyTracking option to false in the host.json file would prevent the host from logging external dependencies (such as calls to Azure Blob Storage for azure-webjobs-hosts), reducing the noise in traces and optimizing costs.

Actual behavior

Despite setting the following configuration in the host.json file:

{
  "version": "2.0",
  "extensions": {
    "http": {
      "routePrefix": ""
    }
  },
  "logging": {
    "logLevel": {
      "default": "Warning",
      "Host.Aggregator": "Information",
      "Host.Results": "Information",
      "Function": "Information"
    },
    "applicationInsights": {
      "enableDependencyTracking": false
    }
  },
  "telemetryMode": "OpenTelemetry"
}

The host continues to log a large amount of debug information (screenshot attached), which we want to exclude. These logs are unnecessary, make trace analysis difficult, and increase costs. The enableDependencyTracking setting does not seem to have any effect and appears to be ignored.

Known workarounds

Suppressing all host logs. However, this is undesirable as it would also suppress HTTP request logs, which are valuable.

Additional concerns

Even if it were possible to suppress dependency logs, I am concerned that it might also remove useful dependency traces, such as those related to output bindings managed by the host.

[jviau]

@gunzip this is by design. There are two things here:

1. applicationInsights:enableDependencyTracking only applies to the application insights SDK for the host process. When using telemetryMode: OpenTelemetry, the app insights SDK is not used and the entire logging:applicationInsights section is no longer applicable.
2. host.json only applies to the host process. If these traces are coming from your worker (node) process, you will need to configure those yourself in whatever way the otel module there supports.

@RohitRanjanMS - do you know the otel equivalent for suppressing certain trace sources?

[gunzip]

@jviau thanks for the answer! Let me add some observations.

Regarding your point:

When using telemetryMode: OpenTelemetry, the app insights SDK is not used, and the entire logging:applicationInsights section is no longer applicable.

In the official documentation on Using OpenTelemetry with Azure Functions, the configuration still references the applicationInsights block:

Proper sampling settings (e.g., to track host incoming HTTP requests) are essential. If the documentation is incorrect, how should we configure sampling without relying on the applicationInsights block?

The traces I've previously shared originate from the host logs. The .NET runtime utilizes the Application Insights SDK and initiates OpenTelemetry here:

https://github.com/Azure/azure-functions-host/blob/dev/src/WebJobs.Script/Diagnostics/OpenTelemetry/OpenTelemetryConfigurationExtensions.cs#L63

Additionally, there is a TelemetryProcessor intended to filter out debug traces:

https://github.com/Azure/azure-functions-host/blob/dev/src/WebJobs.Script/Diagnostics/OpenTelemetry/TraceFilterProcessor.cs#L37

However, despite this, it appears that the debug traces are still being logged and they come from the dotnet runtime:

Could you please clarify if there's an alternative approach to ensure these debug traces are effectively filtered out? Thanks again for your assistance!

[jviau]

In the official documentation on Using OpenTelemetry with Azure Functions, the configuration still references the applicationInsights block:

That is a good point, we will need to remove all of that from the sample. @RohitRanjanMS - can you confirm my statement that the logging:applicationInsights section in host.json does not apply to otel mode? If so, can we track work to remove that block from otel samples?

@RohitRanjanMS, is there documentation on how to configure otel sampling for the host? Or is that not available yet?

[RohitRanjanMS]

Hi @gunzip , thanks for reaching out. I've submitted a PR to update the documentation. Additionally, I have another PR to remove all blob-related dependencies, as they've been quite frustrating.

I'm hoping opentelemetry-configuration will handle the configuration effectively. I'm trying to avoid creating a solution specific to Functions.

[gunzip]

Hi @RohitRanjanMS, I'm impressed by how quickly this issue -which is a showstopper for us- has been addressed!

Do you have an estimated timeline for when these changes will be available in the Azure Functions runtime?

Also, I’ve noticed additional debug logs (specifically for HTTP requests) that probably should be filtered as well. IDK if they come from the worker this time:

Regarding the configuration of sampling for the host (i.e., legitimate HTTP requests from triggers), how can we align it with the sampling settings we define programmatically using the SDK?

Thanks again for your help!

[RohitRanjanMS]

Hi @gunzip, you can check the SDK Version property in the logs. We set two different values for the host and the worker, and these seem to be from the host process.

Regarding sampling, are you referring to defining it programmatically using the SDK as configuring sampling on the node worker? The trace context is propagated from the host process to the worker, so you can utilize the sampling decision from the incoming request (traceparent and trace flag) to ensure consistency between the host and worker processes.

[RohitRanjanMS]

My PR is merged, I don't have a date, but you can tentatively expect this to be available sometime in March.

[gunzip]

Thanks again, @RohitRanjanMS. I confirm that these traces originate from the host (SDK version: azurefunctions:4.1036.0.0_dotnet8.0.8:otel1.8.0:ext1.3.0-beta.1). They should probably be filtered out as well.

Regarding sampling: since we need to trace HTTP requests within the host and the host.json Application Insights settings are ignored, we need a way to configure sampling for these traces. Is there any way to apply sampling settings when using OpenTelemetry mode in the host?

[RohitRanjanMS]

You can use the environment variables - https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/

[gunzip]

I've tried setting

OTEL_TRACES_SAMPLER=traceidratio
OTEL_TRACES_SAMPLER_ARG=0.1

but it looks like it does not affect sampling rate

Same for parentbased_traceidratio