Stable diffusion for real-time music generation (web app)
Go to file
Hayk Martiros b17ce408ff Always use compressor 2022-12-23 16:38:05 -08:00
components Add optional compressor 2022-12-23 15:49:11 -08:00
external moving unmute to external 2022-12-14 00:54:26 -08:00
pages Always use compressor 2022-12-23 16:38:05 -08:00
public Shrink a couple static assets 2022-12-16 09:25:43 -08:00
styles Add first half of about page 2022-11-27 18:23:53 -08:00
.eslintrc.json ignore weird apostrophe escape role 2022-12-05 20:40:05 -08:00
.gitignore unmute tone context on ios 2022-12-14 00:10:23 -08:00
LICENSE format license 2022-12-12 18:44:23 -08:00 Tweak about page 2022-12-22 20:40:59 -08:00
next.config.js Disable strict mode 2022-11-24 14:53:51 -08:00
package-lock.json tweak 2022-12-12 11:33:42 -08:00
package.json Merge branch 'main' into draft-settingsViewDaisy 2022-12-12 11:29:32 -08:00
postcss.config.js Set up tailwind and get basic layout 2022-11-20 13:42:23 -08:00
samplePrompts.ts Add more prompts and fix flask 2022-12-22 20:27:35 -08:00
shaders.js Random seed and little tweaks 2022-11-24 23:26:44 -08:00
tailwind.config.js initial daisy implementation for settings 2022-12-11 18:45:48 -08:00
tsconfig.json Set up tailwind and get basic layout 2022-11-20 13:42:23 -08:00
types.ts hacky but functional panel shifting with alpha changes 2022-11-30 16:30:52 -08:00

Riffusion App

Riffusion is an app for real-time music generation with stable diffusion.

Read about it at and try it at

This repository contains the interactive web app that powers the website.

It is built with Next.js, React, Typescript, three.js, Tailwind, and Vercel.


This is a Next.js project bootstrapped with create-next-app.

First, make sure you have Node v18 or greater installed using node --version.

Install packages:

npm install

Run the development server:

npm run dev
# or
yarn dev

Open http://localhost:3000 with your browser to see the app.

The app home is at pages/index.js. The page auto-updates as you edit the file. The about page is at pages/about.tsx.

The pages/api directory is mapped to /api/*. Files in this directory are treated as API routes instead of React pages.

Inference Server

To actually generate model outputs, we need a model backend that responds to inference requests via API. If you have a large GPU that can run stable diffusion in under five seconds, clone and run the instructions in the inference server to run the Flask app.

You will need to add a .env.local file in the root of this repository specifying the URL of the inference server:



If you build on this work, please cite it as follows:

  author = {Forsgren, Seth* and Martiros, Hayk*},
  title = {{Riffusion - Stable diffusion for real-time music generation}},
  url = {},
  year = {2022}