Adobe previews new genAI tools for custom audio

Adobe Research's Project Music GenAI Control allows creators to generate music from text prompts, and then have fine-grained control to edit the audio for their precise needs.

The new tools begin with a text prompt fed into a generative AI model, a method that Adobe already uses in Firefly. A user inputs a text prompt, such as “powerful rock,” “happy dance,” or “sad jazz” to generate music. Once the tools generate music, fine grained editing is integrated directly into the workflow.

Using a simple user interface, this allows users to transform their generated audio based on a reference melody; adjust the tempo, structure, and repeating patterns of a piece of music; choose when to increase and decrease the audio’s intensity; extend the length of a clip; re-mix a section; or generate a seamlessly repeatable loop.

Adobe says that Project Music GenAI Control has been trained in line with its AI ethics principles of accountability, responsibility, and transparency. We don't have any further details as yet,  but presumably that means it has been pointed at content that Adobe already licenses rather than the open internet and, y'know, all of music.

Certainly, the text prompt creation of new content is perhaps a bit harder than some of what the company has done recently in the AI field to reconcile with its concept of an AI co-pilot. Rather than simply making editors' jobs easier, it is effectively taking human composition and composers out of the equation entirely.

Where it is on more typical Adobe ground is in the editing tools it brings to the music. Instead of manually cutting existing music to make intros, outros, and background audio, Project Music GenAI Control can help users to create exactly the pieces they need—solving workflow pain points end-to-end. This sort of thing already features in Premiere Pro's audio tools, a simple slider allowing you to adjust music to the length of a clip, and is a huge time saver.

“One of the exciting things about these new tools is that they aren’t just about generating audio—they’re taking it to the level of Photoshop by giving creatives the same kind of deep control to shape, tweak, and edit their audio. It’s a kind of pixel-level control for music,” explains Nicholas Bryan, Senior Research Scientist at Adobe Research and one of the creators of the technologies.

Project Music GenAI Control is being developed in collaboration with colleagues at the University of California, San Diego (Zachary Novack, Julian McAuley, Taylor Berg-Kirkpatrick), and at the School of Computer Science, Carnegie Mellon University (Shih-Lun Wu, Chris Donahue, Shinji Watanabe).