Exception when rendering card on a flow expecting an optional Config property.

[svpino]

A flow with an optional Config property raises an exception when trying to generate a card.

Code to reproduce the issue:

from metaflow import Config, FlowSpec, card, step

class Sample(FlowSpec):
    config = Config("config", default=None)

    @card
    @step
    def start(self):
        self.next(self.end)

    @step
    def end(self):
        pass


if __name__ == "__main__":
    Sample()

Pipeline output with the exception text:

2024-12-28 11:53:51.402 [1735404831395269/start/1 (pid 97376)] Task is starting.
2024-12-28 11:53:51.655 [1735404831395269/start/1 (pid 97376)] Card render failed with error :
2024-12-28 11:53:51.793 [1735404831395269/start/1 (pid 97376)]
2024-12-28 11:53:51.793 [1735404831395269/start/1 (pid 97376)] Usage: sample.py [OPTIONS] COMMAND [ARGS]...
2024-12-28 11:53:51.793 [1735404831395269/start/1 (pid 97376)]
2024-12-28 11:53:51.793 [1735404831395269/start/1 (pid 97376)] Error: Could not find configuration 'kv.config' in INFO file
2024-12-28 11:53:51.793 [1735404831395269/start/1 (pid 97376)]
2024-12-28 11:53:51.793 [1735404831395269/start/1 (pid 97376)] Card render failed with error :
2024-12-28 11:53:51.811 [1735404831395269/start/1 (pid 97376)]
2024-12-28 11:53:51.811 [1735404831395269/start/1 (pid 97376)] Usage: sample.py [OPTIONS] COMMAND [ARGS]...
2024-12-28 11:53:51.811 [1735404831395269/start/1 (pid 97376)]
2024-12-28 11:53:51.811 [1735404831395269/start/1 (pid 97376)] Error: Could not find configuration 'kv.config' in INFO file
2024-12-28 11:53:51.811 [1735404831395269/start/1 (pid 97376)]
2024-12-28 11:53:51.812 [1735404831395269/start/1 (pid 97376)] Task finished successfully.
2024-12-28 11:53:51.815 [1735404831395269/end/2 (pid 97382)] Task is starting.
2024-12-28 11:53:51.949 [1735404831395269/end/2 (pid 97382)] Task finished successfully.
2024-12-28 11:53:51.949 Done!

This error doesn't happen if there's a default value specified for the Config. Unfortunately, that's not always possible so there's no obvious workaround at this point.

[savingoyal]

Thanks for the bug report. We are looking into it.

[romain-intel]

Apologies for the delay -- I will fix it (I think I know what the issue is). I am curious, however, of your use case of a config with no default and not being required (ie: it can be None). We did discuss this internally (I have to go back and check) and am curious as to how you are planning to use it that way.

[svpino]

In this case, we have a flow capable of reading the configuration values from environment variables, but we want to add the ability to use a configuration file as well.

If a file is specified, we’ll read the settings from it. If not, we’ll use environment variables and default values.

Let me know if this answers your question and thanks for looking into this!