RNNModel is not working

[Pedklin]

Никак не могу добиться, чтобы модель дала результат...
Код скопировал из вашей документации: https://docs.etna.ai/stable/tutorials/202-NN_examples.html.
Небольшие отличия из-за особенностей датасета (файл с датасетом приложил):
original_df = pd.read_csv("salary2.csv")
salary2.csv

original_df["timestamp"] = pd.to_datetime(original_df["week_key"])
original_df["target"] = original_df["sal_median"]
original_df["segment"] = original_df["segm"]
original_df.drop(columns=["week_key", "sal_median", "sal_diff", "segm"], inplace=True)
df = TSDataset.to_dataset(original_df)
ts = TSDataset(df, freq="W-SUN") // freq отличается от примера в документации, там D
horizon = 52
num_lags = 5

Насколько я понял embedding_sizes, поскольку в данном примере 2 сегмента, заменил определение так:
embedding_sizes = {"dateflag_day_number_in_week": (7, 7), "segment_code": (2, 7)}

model = RNNModel(
input_size = num_lags + 1,
encoder_length = 2 * horizon,
decoder_length = horizon,
embedding_sizes = embedding_sizes,
trainer_params = dict(max_epochs = 5), 
lr=1e-3)
transforms=[scaler, transform_lag, transform_date, segment_encoder, label_encoder]
pipe = Pipeline(model=model, transforms=transforms, horizon=horizon)
metric_tools = []
metric_tools.append(SMAPE())
metric_tools.append(MSE())
metrics, forecast, info = pipe.backtest(ts, metrics=metric_tools, n_folds=1)

При таких условиях pipe.backtest падает. Если уменьшить horizon до 45, не падает, но forecast даже близко не похож.
Я пробовал, поскольку здесь freq="W-SUN", заменить на
transform_date = DateFlagsTransform(day_number_in_week=False, day_number_in_month=True, out_column="dateflag")
label_encoder = LabelEncoderTransform(
in_column="dateflag_day_number_in_month", strategy="none", out_column="dateflag_day_number_in_month_label")
embedding_sizes = {"dateflag_day_number_in_month": (4, 4), "segment_code": (2, 4)}
В этом случае падает.

[egoriyaa]

Hi!
Can you share your full code snippet and error output?

[egoriyaa]

1. Error raises because number of timestamps is less then needed for your RNN config (encoder_length + decoder_length)
2. As to bad results, most probably you have not enough data (returning to the first answer) or not enough epochs to train good model

[Pedklin]

1. In this example number of timestamps = 228 > encoder_length + decoder_length = 156...
2. Well, ok, i reduce horizon = 20.
   I tryed to make following enhancement: if our frequency is W-SUN so i changed

transform_date = DateFlagsTransform(day_number_in_week=False, day_number_in_month=True, out_column="dateflag")
label_encoder = LabelEncoderTransform(in_column="dateflag_day_number_in_month", strategy="none",
out_column="dateflag_day_number_in_month_label")
embedding_sizes = {"dateflag_day_number_in_month": (31, 31), "segment_code": (2, 31)}

And also i set num_lags = 52. Should num_lags correlates with the seasonality?
So i got the following result:

May be it is not too bad....

[egoriyaa]

Okey, to be more precise, your data is splitted in the following way:

• last horizon steps are test part, because you set n_folds=1
• first horizon + num_lags steps are deleted because lag_{horizon+num_lags} feature is NaN and model can't be trained
• so there are 228 - horizon - (horizon + num_lags) + 1 = 115 remaining steps, but min number of timestamps is encoder_length + decoder_length = 156

[egoriyaa]

As to number of lags:
Ideally, the number of lags is selected using a backtest, but if there are few rows, it is easy to overfit. As an initial approximation, I would consider the reference points: the length of the seasonality, the horizon
The distance between two "peaks" looks like seasonality.

[Pedklin]

Next problem with this model:
at the row
metrics, forecast, info = pipe.backtest(ts, metrics=metric_tools, n_folds=1)
I get the error
input.size(-1) must be equal to input_size. Expected 35, got 36
This error rises only if i run model in environment created by PyInstaller. When the same script runs in VSCode it is ok.
PyInstaller:
ETHNA.zip

[egoriyaa]

Hello! Provide a full error trace, please.

[Pedklin]

Data file:
114-124.csv
Frankly i do not know how to get trace from exe....

[egoriyaa]

1. We found a small bug in example notebook with NN models (see [BUG] Fix working with embedding_sizes in 202-NN_examples notebook #488) (maybe it will help)
2. It is hard to understand the core problem without understanding what library/function raises error. Try to found previous output before input.size(-1) must be equal to input_size. Expected 35, got 36

[egoriyaa]

I found out that this error raises when the total number of features is not equal to expected number by model layers.
In your case there is one more feature then expected. In a quick glance I can't understand how many features there should be because there are many ifs.

[Pedklin]

Yes, thank you, the problem was that ts = TSDataset(df, freq=freq) contained extra columns that i did not drop.
When i dropped them then now all is ok.