Any way to integrate Riffusion into this audio/music UI?

[severian42]

Hey! First off, this interface and functionality is INCREDIBLE! Thank you so much for putting this together! I've been able to do some incredible stuff with my Mac Studio. I was wondering if there might be some way to incorporate Riffusion and it's various capabilities into this platform? Multiple attempts have been made by me on my local server to combine the Automatic111 extensions and such into this platform, but truth be told I am a total beginner and keep coming up short!

I am a Professional Audio Engineer who's just been learning all of this within the past 6 months. Essentially, I've been trying to build and piece together a true and dedicated Gradio training platform for pre-trained and custom audio/music models. Any thoughts on the possibility of this or maybe a better way to make it happen? I'm willing to help out in any way I can and even throw some money at the problem if it helps!

Thanks for taking the time to read this : )

[rsxdalv]

Ok, I will update this bit by bit. It might get noisy so don't worry about unsubscribing.

First stage is done - installation is possible and the project appears to be compatible. Now I'd like to add a lot more models in the future, so I'm thinking about making it more like a "plugin" (in case we have incompatible A/B models).

I need to find some code to make this run, probably the next stage will be a jupyter notebook.

[severian42]

Awesome! Thanks for being open to making updates! Your idea sounds great and right on point.

Let me know if there is anything I can do to help and support

[rsxdalv]

Ok, so here's the current status:

You can install riffusion using cmd_windows.bat:

pip install git+https://github.com/riffusion/riffusion@a861b48232ffb2e5a1cb358a68141d8af2a2d93f#egg=riffusion

and then launch it with the same cmd_windows.bat:

python -m riffusion.streamlit.playground

However, when I tried to to text-to-audio, I got this error:
riffusion/riffusion-hobby#148

To fix it, I needed to manually edit a file. Once I can have something that works out of the box I could add it to the repository.

[rsxdalv]

Ok, looking at this from 2 month perspective - riffusion is kind of abandoned on the official repository, honestly. I was hoping they would just notice and fix the bugs that popped up but that might take a really long time. Once I make progress with the new React UI I will check if there are usable third-party riffusion versions that are maintained. Meanwhile, the riffusion website seems to be getting upgrades.

[rsxdalv]

Took me a year but now there's an extension to the webui that allows riffusion inference. Although I do not know if it really competes with the newer models, I will say, as a SD 1.5 model it is really quite fast and approachable.