Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

context_shift怎么使用 #5

Open
hhzzlll opened this issue May 23, 2024 · 2 comments
Open

context_shift怎么使用 #5

hhzzlll opened this issue May 23, 2024 · 2 comments

Comments

@hhzzlll
Copy link

hhzzlll commented May 23, 2024

在ContextDiff_finetune.py文件中虽然传入了context_shift为True
QQ图片20240524005015
但在video_diffusion.pipelines.ddim_spatial_temporal.DDIMSpatioTemporalStableDiffusionPipeline的prepare_latents_ddim_inverted函数中却将context_shift注释了,这是什么意思,有使用到context_shift吗
QQ图片20240524005328
以及ddim_clean2noisy_loop函数也将context_shift注释了
QQ截图20240524005903

@ZhaochenYu0201
Copy link
Collaborator

In our paper, during the process of one-shot tuning for video editing, we utilize an adapter that has been trained on the LAION dataset during the text-to-image stage. We then fine-tune this pretrained adapter with the newly introduced spatio-temporal information from the original video, leading to further enhancement of the desired effects. However, for the sake of convenience in open-sourcing, the current version available is trained on a new adapter based on the original video. Due to the fact that one-shot tuning only involves the original video and its corresponding prompt data, the retrained adapter has a minimal impact on the semantic aspects. Therefore, in the previously open-sourced version, we did not enable it. We have updated our repo, you can directly use it now.

@hhzzlll
Copy link
Author

hhzzlll commented May 31, 2024

很感谢您能回答我的问题,我还有一些问题想请教一下您,论文里的公式13在代码中哪里体现出来,您有空的时候麻烦帮我指出来一下,谢谢!
QQ图片20240531221049

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants