Content moderation is a daunting task, especially in the fast-paced world of livestreaming where unwanted content can appear and spread rapidly. Researchers from a leading tech company have developed a hybrid moderation framework that aims to tackle this challenge head-on. This framework combines supervised classification for known violations with reference-based similarity matching for novel or subtle cases. The goal is to create a robust system that can detect both explicit violations and novel edge cases that might evade traditional classifiers.
The hybrid design processes multimodal inputs—text, audio, and visual—through both pipelines. A multimodal large language model (MLLM) is used to distill knowledge into each pipeline, boosting accuracy while keeping the inference process lightweight. In production, the classification pipeline achieves a recall rate of 67% at 80% precision, while the similarity pipeline achieves a recall rate of 76% at 80% precision. These metrics indicate a significant improvement in detecting unwanted content.
Large-scale A/B tests have shown a 6-8% reduction in user views of unwanted livestreams. This reduction is a testament to the effectiveness of the hybrid moderation framework. The framework’s ability to adapt to new and evolving forms of unwanted content makes it a scalable and adaptable approach to multimodal content governance. It addresses both explicit violations and emerging adversarial behaviors, providing a more secure and enjoyable experience for users.
The research highlights the importance of combining different approaches to content moderation. By leveraging the strengths of both supervised classification and similarity matching, the framework offers a comprehensive solution. The use of MLLMs further enhances the system’s accuracy and efficiency, ensuring that it can handle the complexities of livestreaming environments. As user-generated content continues to grow, such innovative solutions will be crucial in maintaining the integrity and safety of digital platforms.
