CDK Collecting Additional Metadata (under feature flag)

[GavinZZ]

Proposal

This discussion aims to expand the scope of usage data collected by the AWS CDK to better inform CDK development and improve communication for customer-impacting topics. Currently, for those that opt in, the CDK collects usage data on your CDK version and which L2 constructs you use. For more information on current CDK behavior, see Version Reporting.

This proposal expands the scope of usage data collection to include the following from L2 constructs defined in CDK applications:

• CDK-defined property keys - When you use a built-in property of an L2 construct, the property key will be collected. This includes built-in property keys nested in dictionary objects.
• Boolean and enum property values from CDK-defined property keys- For CDK-defined property keys, property values of only Boolean and enum types will be collected. All other types, such as string values or construct references will be redacted.
• Method name, keys, and property values of Boolean and ENUM types - When you use an L2 construct method, we will collect the method name, property keys, and property values of Boolean and enum types.

For property keys and values that you uniquely create, the entire object will be redacted. For example, if you use InlineApiDefinition to define an OpenAPI specification and pass it into a RestApi construct, the entire InlineApiDefinition object will be redacted. For an example, see Appendix.

Scope

This change will be introduced in a future CDK version release targeted for February 15, 2025 with the following behavior:

• New and current apps that opt out of version reporting - Any new or existing app that opts out through the current method of using the versionReporting configuration file flag or --no-version-reporting command line option will be automatically opted out of the additional usage data collection. However, we encourage you to opt in if possible due to the benefits of reporting this data.
• New apps created by the new CDK version - These apps will be opted in by default. However, you can choose to opt out, using the current feature flag method if you prefer.
• Current apps using the new CDK version - Apps that are currently opted in to reporting will not be automatically opted in to increased data collection, even if you are using the new CDK version. Instead, we will provide a feature flag that you can use to opt in to increased data collection in your current apps.

Benefits

Collecting this additional usage data provides the following benefits:

• Improve the scope and speed of customer-impacting communication - The CDK team regularly communicates security concerns, emerging issues, deprecation notices, and other topics that could impact your applications. Additional L2 construct usage data collection helps us identify affected CDK applications more precisely and communicate with you faster.
• Better prioritize and plan CDK development - Additional usage data helps us understand how you use L2 constructs and guides our development priorities. This data can inform us on which constructs need more development resources and will provide insight when planning deprecation timelines and alternative solutions based on actual usage patterns.

Potential Concern

The following are potential areas of concerns that CDK users may have:

• CloudFormation template size quotas - This change will result in a slight size increase of the AWS::CDK::Metadata resource in the produced CloudFormation template. Our tests of CDK applications, ranging from 10 to 200 L2 constructs with 10 to 1,000 properties per construct, showed only a 1-2% increase in CloudFormation template size. Therefore, the size increase will not affect most people, unless your template is very close to a quota limit. During synthesis, the CDK CLI will alert you if your template approaches a quota that could affect how your template is deployed or whether you may need to consider splitting up the template using NestedStack or other soutions.
• CDK performance - Currently, usage data is prefix-encoded and compressed at synthesis. At deployment, this data is decompressed and decoded. The additional data collected will be included in these actions and will not affect synthesis and deployment speeds in any noticeable way.
• Privacy concerns - To ensure we are collecting only non-sensitive user data, we have intentionally chosen to only collect method names, property keys, and property values of enum and Boolean types for L2 constructs and their methods. All other user data, such as string values that could contain sensitive information, are redacted at synthesis before the AWS::CDK::Metadata resource is compressed.

Appendix

To illustrate this change, the following is an example of a CDK app:

import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as s3 from 'aws-cdk-lib/aws-s3';

class MyStack extends cdk.Stack {
  constructor(scope, id, props) {
    super(scope, id, props);

    // Create an S3 bucket (L2 construct)
    const myBucket = new s3.Bucket(this, 'MyBucket', {
      bucketName: 'my-cdk-example-bucket', // String type
      versioned: true,                    // Boolean type
      removalPolicy: cdk.RemovalPolicy.DESTROY, // ENUM type
      lifecycleRules: [{                  // Array of object type
        expirationDate: new Date('2019-10-01'),
        objectSizeLessThan: 600,
        objectSizeGreaterThan: 500,
      }],
    });

    // Use a method of the L2 construct to define additional properties
    myBucket.addLifecycleRule({
      id: 'ExpireOldObjects',
      enabled: true, // Boolean
      expiration: cdk.Duration.days(90), // Expire objects after 90 days
    });
  }
}

// Define the CDK app and stack
const app = new cdk.App();
new MyStack(app, 'MyStack');
app.synth();

At synthesis, usage data is collected, compressed, and stored in the AWS::CDK::Metadata resource. Based on current behavior, the following is an example of the usage data that will be collected from our example app:

{ 
    "fqn": "aws-cdk-lib.aws-s3.Bucket", 
    "version": "v2.170.0" 
}

With this proposal, the following usage data will be collected. The * value replaces property values that will be redacted from data collection:

{ 
    "fqn": "aws-cdk-lib.aws_s3.Bucket", 
    "version": "2.170.0", 
    "metadata": [ 
        { 
            "type": "aws:cdk:analytics:construct", 
            "data": { 
                "bucketName": "*", 
                "versioned": true, 
                "removalPolicy": "cdk.RemovalPolicy.DESTROY", 
                "lifecycleRules": [ 
                    { 
                        "expirationDate": "*", 
                        "objectSizeLessThan": "*", 
                        "objectSizeGreaterThan": "*" 
                    } 
                ] 
            } 
        }, 
        { 
            "type": "aws:cdk:analytics:method", 
            "data": { 
                "name": "addLifecycleRule", 
                "prop": { 
                    "id": "*", 
                    "enabled": true, 
                    "expiration": "*",
                } 
            } 
        } 
    ] 
}

[GavinZZ]

Hello CDK community, I’d really appreciate hearing your thoughts on this new behavior. I’m also more than happy to address any questions you might have! Looking forward to hearing from you!

[gshpychka]

Currently, for those that opt in, the CDK collects usage data on your CDK version and which L2 constructs you use. For more information on current CDK behavior, see Version Reporting.

Wanted to clarify: the current analytics reporting is opt-out, not opt-in as implied here.

Could you clarify a bit more how the opt-in/opt-out precedures will look? The new feature flag is to opt in to the new data collection, right? If users don't change the feature flags, they will never be opted in?

Since the current behavior is opt-out and we use the configuration option to opt out, what would be the behavior if both the old opt-out and the new opt-in flags are set?

[GavinZZ]

Absolutely. Let me clarify the usage behaviour.

The current way of tracking whether CDK collects analytics information (cosntruct version and name) is relying on the flag versionReporting as described in the official documentation here. You can choose to opt out by either defining it in cdk.json file or through CDK CLI like cdk synth --no-version-reporting.

This feature proposal will introduce a new feature flag that is specifically used to track the whether or not CDK will collect on top of the current information being collected.

Now let's consider the following scenarios:

1. If versionReporting is not enabled, CDK does not collect any information regardless. (Even if you enable new feature flag, CDK still do not collect any information).
2. If versionReporting is enabled, CDK will collect construct version and construct name.
3. If versionReporting is enabled and the new feature flag is enabled, CDK will collect construct version, name, and additional analytics metadata information as described in the proposal.