Imagine a trumpet that meows or a saxophone that howls like a wolf. That’s the kind of creativity NVIDIA is touting with its latest AI audio tool, Fugatto. This innovative system is designed to generate completely unique sounds, music, and speech from simple text and audio prompts, even ones it hasn’t been specifically trained on. It’s an exciting leap for audio synthesis, combining artistry with deep learning.

Fugatto can create surreal soundscapes: think “a cello shouting in anger” or “deep bass pulses interspersed with high-pitched digital chirps, evoking a sentient machine awakening.” According to NVIDIA, this isn’t just remixing existing sounds—it’s forging entirely new ones. For example, Fugatto can take a dog’s bark and layer it into an electronic dance track or turn spoken words into operatic melodies. The AI can even alter vocal characteristics, such as changing accents or tones to convey emotions like calmness or rage.

The foundation of Fugatto lies in NVIDIA’s vast dataset and advanced AI models (PDF). The system was trained on millions of audio samples, encompassing speech, environmental sounds, and instruments, including some unusual inputs like BBC’s sound effects library. NVIDIA claims this training allows Fugatto to perform “zero-shot” tasks—creating outputs it wasn’t directly trained to handle—by blending its understanding of different audio domains.

While Fugatto represents a groundbreaking step in generative audio, it raises important questions. Competing AI audio tools from companies like Stability AI, Google DeepMind, and Adobe have already faced scrutiny over copyright concerns, particularly in music generation. NVIDIA seems aware of this landscape, emphasizing its model’s ability to synthesize new sounds rather than simply repurposing existing audio.

However, NVIDIA has not yet announced plans for a public release of Fugatto, leaving its potential applications—ranging from entertainment to industrial design—largely theoretical for now.

With Fugatto, NVIDIA envisions a future where sound design can be as imaginative as storytelling. Whether it’s creating a whimsical soundtrack for a video game or crafting entirely new genres of music, Fugatto might just redefine the boundaries of audio creativity. Yet, as with all AI advancements, its true impact will depend on how it’s used—and whether it can navigate the complex ethical and legal challenges of creative industries.