From e20c9cdf6c0dd5dc1f12565ebe2f2d648b9dc15f Mon Sep 17 00:00:00 2001
From: ZeyueT <ztianad@connect.ust.hk>
Date: Thu, 13 Mar 2025 19:38:22 +0800
Subject: [PATCH] update

---
 .gitignore | 2 ++
 README.md  | 6 +++---
 2 files changed, 5 insertions(+), 3 deletions(-)
 create mode 100644 .gitignore

diff --git a/.gitignore b/.gitignore
new file mode 100644
index 0000000..5be0c09
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1,2 @@
+/static/videos/*.mp4
+/static/videos/*.mov
\ No newline at end of file
diff --git a/README.md b/README.md
index 124dda4..22428a3 100644
--- a/README.md
+++ b/README.md
@@ -9,15 +9,15 @@
 
 [![Watch the video](https://github.com/user-attachments/assets/498f7d3f-cb7c-4f32-92f0-4b71fa0712e9)]
 
-<video width="720" controls>
-  <source src="https://github.com/user-attachments/assets/498f7d3f-cb7c-4f32-92f0-4b71fa0712e9.mp4" type="video/mp4">
-  Your browser does not support the video tag.
+<video width="100%" controls>
+  <source src="./static/videos/AudioX_DEMO.mov" type="video/mp4">
 </video>
 
 
 
 
 
+
 ## ✨ Abstract
 
 Audio and music generation have emerged as crucial tasks in many applications, yet existing approaches face significant limitations: they operate in isolation without unified capabilities across modalities, suffer from scarce high-quality, multi-modal training data, and struggle to effectively integrate diverse inputs. In this work, we propose AudioX, a unified Diffusion Transformer model for Anything-to-Audio and Music Generation. Unlike previous domain-specific models, AudioX can generate both general audio and music with high quality, while offering flexible natural language control and seamless processing of various modalities including text, video, image, music, and audio. Its key innovation is a multi-modal masked training strategy that masks inputs across modalities and forces the model to learn from masked inputs, yielding robust and unified cross-modal representations. To address data scarcity, we curate two comprehensive datasets: vggsound-caps with 190K audio captions based on the VGGSound dataset, and V2M-caps with 6 million music captions derived from the V2M dataset. Extensive experiments demonstrate that AudioX not only matches or outperforms state-of-the-art specialized models, but also offers remarkable versatility in handling diverse input modalities and generation tasks within a unified architecture.