The landscape of music creation is undergoing a seismic shift, thanks to the rapid advancements in AI-driven music generation. While this technological leap opens up new creative horizons, it also raises significant concerns about artistic authenticity and copyright infringement. The challenge lies in developing detection methods that can keep pace with these end-to-end AI music generators. Current models, such as SpecTTTra, struggle with the diverse and evolving ecosystem of new generators, showing notable performance drops when faced with out-of-distribution (OOD) content. This highlights a critical gap: the need for more robust detection architectures and more challenging benchmarks.
Enter Melody or Machine (MoM), a new large-scale benchmark introduced by researchers Arnesh Batra, Dev Sharma, Krish Thukral, Ruhani Bhatia, Naman Batra, and Aditya Gautam. MoM comprises over 130,000 songs, totaling 6,665 hours of audio. This dataset is the most diverse to date, constructed using a mix of open and closed-source models. It also includes a curated OOD test set designed to foster the development of truly generalizable detectors. Alongside this benchmark, the researchers introduce CLAM, a novel dual-stream detection architecture. The hypothesis behind CLAM is that subtle, machine-induced inconsistencies between vocal and instrumental elements, often imperceptible in a mixed signal, can serve as powerful indicators of synthetic music.
CLAM operates by employing two distinct pre-trained audio encoders, MERT and Wave2Vec2, to create parallel representations of the audio. These representations are then fused by a learnable cross-aggregation module that models their inter-dependencies. The model is trained with a dual-loss objective: a standard binary cross-entropy loss for classification, complemented by a contrastive triplet loss. This triplet loss trains the model to distinguish between coherent and artificially mismatched stream pairings, enhancing its sensitivity to synthetic artifacts without presuming a simple feature alignment.
The results are impressive. CLAM achieves an F1 score of 0.925 on the challenging MoM benchmark, setting a new state-of-the-art in synthetic music forensics. This breakthrough is significant for the music and audio tech industry. As AI-generated music becomes more prevalent, the ability to detect synthetic content accurately is crucial for maintaining artistic integrity and protecting copyrights. The introduction of MoM and CLAM not only addresses these concerns but also paves the way for further advancements in the field. By providing a robust benchmark and a sophisticated detection architecture, researchers and developers now have the tools to create more reliable and generalizable detectors. This development is a step forward in ensuring that the rapidly evolving world of AI music generation is met with equally robust detection capabilities.
