feat: adds allFlagMetadata to clients and provider interface

[maxveldink]

This PR

Adds an optional allFlagMetadata accessor to clients and the provider interface for retrieving a list of flag keys available in the provider.

• Adds requirements for allFlagMetadata accessor
• Defines Collection and Provider Metadata types in the Types page and adds key information to Flag Metadata.

Related Issues

Discussion that predated this proposal.

[beeme1mr]

Thanks for getting this started and sorry for the delayed review. Please review my comments when you have a moment and let me know if you have any questions. Thanks!

[maxveldink]

@luizgribeiro, I see what you're saying about the return type. I worry that we'd be adding too much complexity to the provider method if we introduced a new type to model potential error situations from the provider. I think allowing the provider method to return null, for now, would give enough information to the client to know if the flags returned are empty versus unavailable.

I added one more callout on the provider spec to clarify that more.

[maxveldink]

I believe I've addressed all feedback on this PR now. I'll wait until tomorrow morning and merge this if there is no other feedback!

[dferber90]

The method is planned to be named getAllFlagMetadata but the description mentions this returns "currently active" flags only. Is "currently active" defined further somewhere?

[weyert]

Good point, worth clarifying this. My personal understanding is that it refers to the enabled feature flags in the provider

[maxveldink]

Thanks for catching that! I know active is semantically significant for most vendors. I tried to clarify this by changing it to available, which can be interpreted different per provider.

[weyert]

@dferber90 I hope it clarifies it. What do you think of this enhancement?

[jonathannorris]

Hey @maxveldink, what's the motivation behind having the type in the data model?

I think we might have a better chance of constraining the usage of this to its intended use-case if the data model was simpler and more restrictive:

for example, instead of:
{ flags: [{"key": "featureA", "type": "boolean"}, {"key": "featureB", "type": "string"}] }

A simpler mapping of just flag key -> feature key would less likely be used beyond just as metadata:

{ "key_1": "featureA", "key_2": "featureB"}

[dferber90]

The way we're able to show Feature Flags of any provider in the toolbar is that we query the various providers and map their API responses into this shape

interface FlagOptionType {
  value: any;
  label?: string;
}

interface FlagDefinitionType {
  options?: FlagOptionType[];
  /**
   * The URL where the feature flag can be managed.
   */
  origin?: string;
  description?: string;
}
 
type ProviderData = {
  definitions: Record<string, FlagDefinitionType>;
};

Being able to see the available options & description lets us build a rich UI for overwriting flags, where we can display the available flags, their descriptions, their options and the label of each option if present.

[toddbaert]

Hey @maxveldink, what's the motivation behind having the type in the data model?

I think we might have a better chance of constraining the usage of this to its intended use-case if the data model was simpler and more restrictive:

for example, instead of: { flags: [{"key": "featureA", "type": "boolean"}, {"key": "featureB", "type": "string"}] }

A simpler mapping of just flag key -> feature key would less likely be used beyond just as metadata:

{ "key_1": "featureA", "key_2": "featureB"}

My take is generally "less is more" but I'm not sure if anyone's envisioned use-case requires the type info. If not, I think we should drop it, but I'm wondering it if could be helpful. Some backends don't strictly type flags. cc @maxveldink @jonathannorris @dferber90

[toddbaert]

The way our Feature Flag toolbar integration works is that we query the various providers and map their API responses into this shape

interface FlagOptionType {
  value: any;
  label?: string;
}

interface FlagDefinitionType {
  options?: FlagOptionType[];
  /**
   * The URL where the feature flag can be managed.
   */
  origin?: string;
  description?: string;
}
 
type ProviderData = {
  definitions: Record<string, FlagDefinitionType>;;
};

Being able to see the available options & description lets us build a rich UI for overwriting flags, where we can display the available flags, their descriptions, their options and the label of each option if present.

I guess "label" would correspond to our key prop? description could be something supplied loosely in the metadata if I'm understanding correctly.

Is the proposed API sufficient then? Or is there anything we're missing? What is value? We don't evaluate these flags in our case so if it's the flag value I'm not sure we can satisfy that.

@dferber90

[maxveldink]

Hey @maxveldink, what's the motivation behind having the type in the data model?

I think we might have a better chance of constraining the usage of this to its intended use-case if the data model was simpler and more restrictive:

for example, instead of: { flags: [{"key": "featureA", "type": "boolean"}, {"key": "featureB", "type": "string"}] }

A simpler mapping of just flag key -> feature key would less likely be used beyond just as metadata:

{ "key_1": "featureA", "key_2": "featureB"}

I originally had a more straightforward collection like this, just really a list of keys. The need for a type comes from client evaluation. Callers need to know which client evaluation method to call for a given flag key, so we need some type information there to understand which client method to call.

@jonathannorris Is the current language too ambiguous for what should be in the type field? type is an optional property with a value of type string, indicating which client resolution method should be used to resolve the flag key.

[maxveldink]

@toddbaert @dferber90 Yep, description could be returned in the flag metadata type that we have defined. That can largely be left up to the provider/underyling vendor

[jonathannorris]

This PR

Adds an optional allFlagMetadata accessor to clients and the provider interface for retrieving a list of flag keys available in the provider.

• Adds requirements for allFlagMetadata accessor
• Defines Collection and Provider Metadata types in the Types page and adds key information to Flag Metadata.

Related Issues

Discussion that predated this proposal.

I'm weary about bringing this up and don't want to block this PR further, but it sounds like you would be better served by a bulk flag evaluation method. That is generally what our customers would use for this use case. I usually imagine "metadata" like this should primarily be used for analytics/tracking purposes, not really for having enough information to properly make a request for an individual flag evaluation.

[weyert]

Bulk evaluation costs money (e.g. 10x normal single flag evaluation) while fetching a list of flags available does not

[kinyoklion]

Bulk evaluation costs money (e.g. 10x normal single flag evaluation) while fetching a list of flags available does not

Currently the only way to provide this data for a LaunchDarkly provider would be via a bulk-evaluation, which is part of the reason I am dubious about use cases. For the LaunchDarkly case the primary cost will be in the time the evaluations takes. (Aside from more stateless ones like PHP, which will have a greater cost to execute bulk-evaluations.)

[toddbaert]

I think there's some value in, to use the language of the PR:

understanding which flag keys are currently available from a provider

I can imagine situations where this could be useful for debugging, administrative UIs, etc; generally for tangential use-cases not typical flag evaluation as we normally consider it. I appreciate how some SDKs/providers simply don't support an equivalent right now.

I'm still not sure if @dferber90 's use case is satisfied if we don't return actual evaluated flag values though.

[kinyoklion]

I think there's some value in, to use the language of the PR:

understanding which flag keys are currently available from a provider

I can imagine situations where this could be useful for debugging, administrative UIs, etc; generally for tangential use-cases not typical flag evaluation as we normally consider it. I appreciate how some SDKs/providers simply don't support an equivalent right now.

I'm still not sure if @dferber90 's use case is satisfied if we don't return actual evaluated flag values though.

I do see use of that information from a debugging perspective, but I don't know about this specific way of exposing it.

If your application is evaluating flags, then hooks provide the means to know what flags are evaluated, and what values they evaluated to.

The resolution details should contain information about flags that are evaluated that don't exist.

Provider configuration change events, when they can be supported, also provide that list of flags, but potentially in an incremental way.

I don't like bulk-evaluation, but it does solve these use cases, and I suspect flag meta data will just be used to make less efficient versions of bulk-evaluation (which is worse than just bulk-evaluation).

[toddbaert]

OK... I hate to throw more into the mix here, but I really don't want us to merge something we're not sure about.

I have a alternate path I'd like to propose which perhaps satisfies the original use-case brought up by @maxveldink and others, and dodges the valid pitfalls @kinyoklion mentions...

What if instead of adding a method to the client, we put a very similar payload in the READY event payload (and also enhance the PROVIDER_CONFIGURATION_CHANGED payload to feature the same). This means that if your provider supports it, you could attach a READY handler that would get all the flags and their metadata.

This has the advantage of being isomorphic with the PROVIDER_CONFIGURATION_CHANGED events, improving consistency. Plus, it makes it more clear that this set may change - and may need to be updated with change events; this can be easily done by adding a similar handler to those. The provider would have to be READY anyway in order to provide meaningful data via the proposed client.allFlagMetadata, so I don't think we're constraining ourselves in any way here. While it's true that users could still implement a dirty "bulk evaluation" this way, I think it's less likely to happen this way. EventDetails are already "best effort" so adding more optional data here feels idiomatic as well.

[maxveldink]

The current proposal seems to invite misuse, and we haven't reached an entire agreement. I'm cool with closing this attempt (with gratitude for the great conversation) so we can focus the discussion on using some other affordances in OpenFeature, such as hooks and events.

The Ruby SDK has yet to implement hooks and events, so I'm not as versed in them from an implementation perspective. Can we open a new issue/discussion for @toddbaert 's proposal?

[dferber90]

Seems our use case was misunderstood a bit. To us an API would be useful which answers the questions

• "Which flags does the provider offer, and what are their descriptions?"
• "For each flag the provider offers, which values can it resolve to, and what are those called if they have a name (label)?"

Imagine a payload like

{
  "myFlag": {
    "description": "A feature flag",
    "origin": "https://example.com/myFlag",
    "options": [
      { "value": false, "label": "off" },
      { "value": true, "label": "on" }
    ]
  }
}

What is value? We don't evaluate these flags in our case so if it's the flag value I'm not sure we can satisfy that.

Value is not the value the flag resolved to, but rather what it could resolve to. We turn this list of possible values into a dropdown where customers can select one of the possible values to override a feature flag.

---

Loading this kind of information usually requires a different API key, which needs to be kept secret. I'm not familiar enough with OpenFeature yet to know whether this would be in scope of OpenFeature, or whether OpenFeature is purely concerned with the "resolving feature flags" part.

[kinyoklion]

Seems our use case was misunderstood a bit. To us an API would be useful which answers the questions

• "Which flags does the provider offer, and what are their descriptions?"
• "For each flag the provider offers, which values can it resolve to, and what are those called if they have a name (label)?"

Imagine a payload like

{
  "myFlag": {
    "description": "A feature flag",
    "origin": "https://example.com/myFlag",
    "options": [
      { "value": false, "label": "off" },
      { "value": true, "label": "on" }
    ]
  }
}

What is value? We don't evaluate these flags in our case so if it's the flag value I'm not sure we can satisfy that.

Value is not the value the flag resolved to, but rather what it could resolve to. We turn this list of possible values into a dropdown where customers can select one of the possible values to override a feature flag.

Loading this kind of information usually requires a different API key, which needs to be kept secret. I'm not familiar enough with OpenFeature yet to know whether this would be in scope of OpenFeature, or whether OpenFeature is purely concerned with the "resolving feature flags" part.

This seems like a much greater level of detail than a provider would have in most cases. (Especially for single context paradigm where the available options are often not communicated to the client at all.)

I don't know what the requirements would be across many vendors, but for LaunchDarkly I know that it would require a different type of credential to get this kind of information than the credentials by SDKs.

It also seems like a debug level of feature, not a production feature? Something that maybe would lend itself to having some configuration to allow usage that could be easily disabled in production?

[maxveldink]

As mentioned above, closing for now. Thanks all for the conversation!