XAI-Lyricist: Improving the Singability of AI-Generated Lyrics with Prosody Explanations

XAI-Lyricist: Improving the Singability of AI-Generated Lyrics with Prosody Explanations

Qihao Liang, Xichu Ma, Finale Doshi-Velez, Brian Lim, Ye Wang

Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence
Human-Centred AI. Pages 7877-7885. https://doi.org/10.24963/ijcai.2024/872

Explaining the singability of lyrics is an important but missing ability of language models (LMs) in song lyrics generation. This ability allows songwriters to quickly assess if LM-generated lyrics can be sung harmoniously with melodies and helps singers align lyrics with melodies during practice. This paper presents XAI-Lyricist, leveraging musical prosody to guide LMs in generating singable lyrics and providing human-understandable singability explanations. We employ a Transformer model to generate lyrics under musical prosody constraints and provide demonstrations of the lyrics' prosody patterns as singability explanations. XAI-Lyricist is evaluated by computational metrics (perplexity, prosody-BLEU) and a human-grounded study (human ratings, average time and number of attempts for singing). Experimental results show that musical prosody can significantly improve the singability of LM-generated lyrics. A controlled study with 14 singers also confirms the usefulness of the provided explanations in helping them to interpret lyrical singability faster than reading plain text lyrics.
Keywords:
Multidisciplinary Topics and Applications: MTA: Arts and creativity
Humans and AI: HAI: Human-AI collaboration
AI Ethics, Trust, Fairness: ETF: Explainability and interpretability