Riffusion

Riffusion

Developer(s)	Seth Forsgren Hayk Martiros
Initial release	December 15, 2022
Repository	github.com/hmartiro/riffusion-inference
Written in	Python
Type	Text-to-image model
License	MIT License
Website	riffusion.com

Generated spectrogram from the prompt "bossa nova with electric guitar" (top), and the resulting audio after conversion (bottom)

Riffusion is a neural network, designed by Seth Forsgren and Hayk Martiros, that generates music using images of sound rather than audio. ^[1]

The resulting music has been described as "de otro mundo" (otherworldly), ^[2] although unlikely to replace man-made music. ^[2] The model was made available on December 15, 2022, with the code also freely available on GitHub. ^[3] It was one of many models derived from Stable Diffusion. ^[4]

Riffusion is classified within a subset of AI text-to-music generators. In December 2022, Mubert ^[5] similarly used Stable Diffusion to turn descriptive text into music loops. In January 2023, Google published a paper on their own text-to-music generator called MusicLM. ^{[6] [7]}

Seth Forsgren and Hayk Martiros formed a startup, also called Riffusion, and raised $4 million in venture capital funding in October 2023. ^{[8] [9]}

References

1. Coldewey, Devin (December 15, 2022). "Try 'Riffusion,' an AI model that composes music by visualizing it".
2. Llano, Eutropio (December 15, 2022). "El generador de imágenes AI también puede producir música (con resultados de otro mundo)".
3. Nasi, Michele (December 15, 2022). "Riffusion: creare tracce audio con l'intelligenza artificiale". IlSoftware.it.
4. "文章に沿った楽曲を自動生成してくれるAI「Riffusion」登場、画像生成AI「Stable Diffusion」ベースで誰でも自由に利用可能". GIGAZINE. 16 December 2022.
5. "Mubert launches Text-to-Music interface – a completely new way to generate music from a single text prompt". December 21, 2022.
6. "MusicLM: Generating Music From Text". January 26, 2023.
7. "5 Reasons Google's MusicLM AI Text-to-Music App is Different". January 27, 2023.
8. Gal, Dr. Itay (February 10, 2025). "Free A.I. music creation platform launches, competing with Suno". The Jerusalem Post. Retrieved February 16, 2025.
9. Nuñez, Michael (January 30, 2025). "Riffusion's free AI music platform could be the Spotify of the future". VentureBeat. Retrieved February 16, 2025.