Stable diffusion for real-time music generation (web app)
Go to file
Hayk Martiros 6ca2ad6833 Add optional compressor 2022-12-23 15:49:11 -08:00
components Add optional compressor 2022-12-23 15:49:11 -08:00
external moving unmute to external 2022-12-14 00:54:26 -08:00
pages Add optional compressor 2022-12-23 15:49:11 -08:00
public Shrink a couple static assets 2022-12-16 09:25:43 -08:00
styles Add first half of about page 2022-11-27 18:23:53 -08:00
.eslintrc.json ignore weird apostrophe escape role 2022-12-05 20:40:05 -08:00
.gitignore unmute tone context on ios 2022-12-14 00:10:23 -08:00
LICENSE format license 2022-12-12 18:44:23 -08:00
README.md Tweak about page 2022-12-22 20:40:59 -08:00
next.config.js Disable strict mode 2022-11-24 14:53:51 -08:00
package-lock.json tweak 2022-12-12 11:33:42 -08:00
package.json Merge branch 'main' into draft-settingsViewDaisy 2022-12-12 11:29:32 -08:00
postcss.config.js Set up tailwind and get basic layout 2022-11-20 13:42:23 -08:00
samplePrompts.ts Add more prompts and fix flask 2022-12-22 20:27:35 -08:00
shaders.js Random seed and little tweaks 2022-11-24 23:26:44 -08:00
tailwind.config.js initial daisy implementation for settings 2022-12-11 18:45:48 -08:00
tsconfig.json Set up tailwind and get basic layout 2022-11-20 13:42:23 -08:00
types.ts hacky but functional panel shifting with alpha changes 2022-11-30 16:30:52 -08:00

README.md

Riffusion App

Riffusion is an app for real-time music generation with stable diffusion.

Read about it at https://www.riffusion.com/about and try it at https://www.riffusion.com/.

This repository contains the interactive web app that powers the website.

It is built with Next.js, React, Typescript, three.js, Tailwind, and Vercel.

Run

This is a Next.js project bootstrapped with create-next-app.

First, make sure you have Node v18 or greater installed using node --version.

Install packages:

npm install

Run the development server:

npm run dev
# or
yarn dev

Open http://localhost:3000 with your browser to see the app.

The app home is at pages/index.js. The page auto-updates as you edit the file. The about page is at pages/about.tsx.

The pages/api directory is mapped to /api/*. Files in this directory are treated as API routes instead of React pages.

Inference Server

To actually generate model outputs, we need a model backend that responds to inference requests via API. If you have a large GPU that can run stable diffusion in under five seconds, clone and run the instructions in the inference server to run the Flask app.

You will need to add a .env.local file in the root of this repository specifying the URL of the inference server:

RIFFUSION_FLASK_URL=http://127.0.0.1:3013/run_inference/

Citation

If you build on this work, please cite it as follows:

@article{Forsgren_Martiros_2022,
  author = {Forsgren, Seth* and Martiros, Hayk*},
  title = {{Riffusion - Stable diffusion for real-time music generation}},
  url = {https://riffusion.com/about},
  year = {2022}
}