Problem when training using one-image-token (live1 mode)

[yeungchenwa]

Hi, thanks for your work! When I trained the model under the live1 mode (just using [CLS] token), it will occur the problems that

Thanks again!

[chenjoya]

Hi, many thanks for your interests! Could you share with me the details of your extracted frame embedding? Is it 1x1024 for one frame (not 10x1024) right?

[yeungchenwa]

Hi, many thanks for your interests! Could you share with me the details of your extracted frame embedding? Is it 1x1024 for one frame (not 10x1024) right?

Thanks for your feedback! Yes, the extracted frame embedding is 1x1024 for one frame.

[zhangyl4]

I ran into the same issue and I guess it was caused by the LiveOneTrainingArguments setting frame_token_interval to '' instand of ','. I changed it and it worked, but I'm not sure it solved the root cause.