EmoGen: Eliminating Subjective Bias in Emotional Music Generation

Abstract

Music is used to convey emotions, and thus generating emotional music is important in automatic music generation. Previous work on emotional music generation directly uses annotated emotion labels as control signals, which suffers from subjective bias: different people may annotate different emotions on the same music, and one person may feel different emotions under different situations. Therefore, directly mapping emotion labels to music sequences in an end-to-end way would confuse the learning process and hinder the model from generating music with precise emotions. In this paper, we propose EmoGen, an emotional music generation system that leverages a set of emotion-related music attributes as the bridge between emotion and music, and divides the generation into two stages: emotion-to-attribute mapping with supervised clustering, and attribute-to-music generation with self-supervised learning. Both stages are beneficial: in the first stage, the attribute values around the clustering center represent the general emotions of these samples, which help eliminate the impacts of the subjective bias of emotion labels; in the second stage, the generation is completely disentangled from emotion labels and thus free from the subjective bias. Both subjective and objective evaluations show that EmoGen outperforms previous methods on emotion control accuracy and music quality respectively, which demonstrate our superiority in generating emotional music.

1. Comparison with Previous Methods

The emotion space is divided by arousal and valence into four quadrants based on Russell's 4Q model (paper link), as shown in the following Figure 1. "Qi" represents the i-th quadrant.

Figure.1 Russell's 4Q model(https://psu.pb.unizin.org/psych425/chapter/circumplex-models/). According to the valence (x-axis) and arousal (y-axis), human emotions can be divided into four quadrants.

We compare EmoGen with Conditional Sampling (CS, paper link) and Predictor Upper Confidence for Trees (PUCT, paper link). In order to conduct thorough evaluations, we design two training settings: 1) Setting 1: To ensure the music quality of generated music, following previous work, we pre-train the models on Pop1k7+LMD-Piano before fine-tuning on EMOPIA. 2) Setting 2: To demonstrate that EmoGen does not rely on labeled datasets and can leverage arbitrary datasets for emotional music generation, we train the generative model on Pop1k7+LMD-Piano+EMOPIA in the attribute-to-music generation stage. Since CS and PUCT require only labeled data in training, they cannot work in this setting. We present music pieces of these methods as follows.

1. Given an emotion quadrant as the target, although CS and PUCT can generate samples in the corresponding quadrant, they also generate samples with vague emotion (even in other emotional quadrants). Therefore, the proportion of correct samples for CS and PUCT is low.
2. In contrast, EmoGen generates samples with more explicit emotion and the proportion of correct samples is higher. Therefore, the emotional control accuracy of EmoGen is higher.
3. Besides, subjective evaluation results show that the music quality of EmoGen's samples is also higher than that of CS and PUCT (please refer to Subsection 4.2 in the paper for details).

2. Subjective Bias in Ground Truth

Compared with the pieces from the center, the emotion of the boundary pieces is more ambiguous although their labels are the same. Therefore, there exists subjective bias in ground truth, which will reduce the control accuracy of the end-to-end methods (e.g. CS and PUCT).

3. Experiment on TopMAGD

To evaluate EmoGen’s ability to generate emotional mu- sic on the arbitrary dataset, we conduct experiments of EmoGen on TopMAGD, which is a multi-instrument dataset with no emotion annotation. Here are music pieces generated by EmoGen in this experiment.

Q1

Q2

Q3

Q4

4. Selected Attributes

Figure.2 Control accuracy of different attributes.

The selected 100 attributes are shown in the following table. For the specific meaning of these attributes, please refer to jSymbolic Feature Explanations.