Dropdown with dynamic dataframe has selection issues when the data changes underneath

[reggian]

Setup

I have a static dict of test metadata which I currently update manually from time to time.

tests = {id_a: {...}, ..., id_n: {...}}.
tests_df = pd.DataFrame({"test_ids": tests.keys()})

I have a dynamic dropdown (test_id) using the above dataframe (tests_df).

I query the data based on the selected test_id.
The results contain among other things a date (date_time) for each entry.
I do some filtering and then I want to analyse a specific window of time. I do this by selecting a start date_time and a duration.

date_times_df = pd.DataFrame({"date_time": data['date_time'].astype(str)})

I here have a dynamic dropdown (window_start) using the above dataframe (date_times_df) and a text input for window_duration.

Using window_start and window_duration I then filter the original data to do some analysis.

Issue

Live mode

When I change the test_id dropdown and Run all, the data and its dates change. But the window_start dropdown for some reason does not update its dynamic list of values.

Edit mode

When I change the test_id dropdown and Run onwards, the data and its dates change.
The window_start dropdown's values change but the previously selected value is not present in the new set. Sometimes this just results in no analysis (as the window is out of data bounds) but a lot of the times this results in a grayed out unselectable dropdown and I have to unset and reset the date_times_df as its dataframe to fix it.

Possible solutions

One possible solution would be to provide a default value for the dropdown.
E.g. if the dropdown modifies the window_start variable, it would be nice to check if the value exists before and use it for the default selected value.

date_times_df = pd.DataFrame({"date_time": data['date_time'].astype(str)})
window_start = date_times_df[0]

Another possible solution would be to default to the first value of the dataframe.
Another possible solution would be to use the Date input method, but then again, I would want to limit the min and max dates that can be chosen.
I may be using this wrong and the solution is elsewhere. In this case I would be grateful if someone can point me in the right direction.

[lucasfcosta]

This is an amazingly written issue, thanks @reggian!

Indeed we don't have a way of detecting changes in dependant dataframes for these inputs. In fact, this will happen with any changes to the underlying DF for a dynamic dropdown.

IMO, both of these things should happen:

1. We should select the first value in the data frame if the currently selected one is not available anymore
2. We should probably refresh the dropdown's options whenever the underlying dataframe changes (we can't detect that precisely right now as we're not doing any static analysis atm, but we could simply re-eval the options whenever any python code runs)

Wrt to (2), I think that should probably be a "page option" though.

Do you have thoughts on this @reggian and @vieiralucas ?