Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

如何将多组token转换成单一token #9

Open
fanpengustc opened this issue Jan 22, 2024 · 4 comments
Open

如何将多组token转换成单一token #9

fanpengustc opened this issue Jan 22, 2024 · 4 comments

Comments

@fanpengustc
Copy link

看了你们其他的文章，有介绍将DAC提的token用来训unicats ，但是dac这样的token有8组每组1024个，但是unicat输入的事单一一组token，这个怎么转换呢？或者是有新的其他架构？
@cantabile-kwok
Copy link
Member

您好，请问具体指的是哪篇文章呢，一下子有点想不起来了orz。不过如果硬要训的话也可以像valle那样带一个NAR模型，只用unicats去建模第一层codec。但我们自己的实验里面还没有做过这种。

@fanpengustc
Copy link
Author

您好，请问具体指的是哪篇文章呢，一下子有点想不起来了orz。不过如果硬要训的话也可以像valle那样带一个NAR模型，只用unicats去建模第一层codec。但我们自己的实验里面还没有做过这种。

TOWARDS UNIVERSAL SPEECH DISCRETE TOKENS: A CASE STUDY FOR ASR AND TTS 这篇文章，里面有介绍dac等多组token

@cantabile-kwok
Copy link
Member

这篇文章里用到的Encodec和DAC主要是直接拿来重构，跟从vq-wav2vec一类的semantic token+CTX-vec2wav重构做比较的，所以其实没有用unicats来直接生成这样的多组token

@fanpengustc
Copy link
Author

这篇文章里用到的Encodec和DAC主要是直接拿来重构，跟从vq-wav2vec一类的semantic token+CTX-vec2wav重构做比较的，所以其实没有用unicats来直接生成这样的多组token

谢谢解答

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@fanpengustc @cantabile-kwok