Music Recommendation System Being Used for Actual Empathy Concerts

The Full Recommendation System

Building a Song Recommendation System for Patient Sing-Along Response

Introduction

Music-based interventions often rely on familiar songs to encourage engagement, memory recall, and emotional response. In the setting this project targets, patients attend empathy recitals where pianists play cover versions of songs. Some songs lead patients to sing along, while others produce little or no visible reaction. The goal of this system is to use those observed reactions to recommend new songs that patients are more likely to sing along to in future sessions.

The proposed product is a recommendation system that takes two sets of songs as input:

• songs patients liked, meaning they sang along to them
• songs patients did not like, meaning they had no clear reaction to them

Given those examples, the system ranks new candidate songs by how likely they are to produce a sing-along response.

Core Hypothesis

The central hypothesis behind the system is that patient reaction depends more on the memory of the original recording than on the cover version played during the recital. A pianist's performance may trigger the reaction, but the underlying memory is likely connected to the song as the patient first encountered it: through the original or most familiar commercial recording.

This assumption is imperfect. Many factors that affect recall and engagement cannot be controlled, including the patient's mood, health, personal history, and the broader context in which they originally heard the song. However, when building a computational recommender, original recordings provide a more stable and reproducible signal than live cover performances. For that reason, this system evaluates songs using audio embeddings from recordings rather than from recital performances.

Candidate Song Generation

Before ranking songs, the system needs a candidate pool. Candidate generation is based on the idea that songs are more likely to be remembered if they were popular during a meaningful period of the patient's life. Two useful signals are:

• release year
• popularity

The repository includes fetch_billboard_candidates.py, which builds candidate song manifests from Billboard Hot 100 data. The script can filter songs by first charting year, peak chart position, weeks on chart, and output size. For example, it can generate a set of popular songs from the late 1950s and 1960s, which may be appropriate for a specific patient population depending on age and background.

Spotify metadata could also be useful for release year and popularity, but Spotify's recommendation API is no longer reliable enough to serve as the main ranking engine for this pipeline. This project therefore uses metadata only for candidate selection and relies on audio representation learning for evaluation.

Audio Representation with MuQ

Once candidate songs are collected, each recording is converted into an embedding vector. An embedding is a numerical representation of the audio that captures musical information learned by a model.

This project uses Tencent's MuQ model, specifically:

OpenMuQ/MuQ-large-msd-iter

The scripts load each audio file, convert it to mono audio at the target sample rate, divide longer tracks into chunks, embed each chunk with MuQ, and average the chunk embeddings into one vector for the full track. The result is one L2-normalized embedding vector per song.

At this point, the system has three groups of vectors:

• liked song embeddings
• disliked song embeddings
• candidate song embeddings

Why Simple Similarity Is Not Enough

A natural first approach is to compare each candidate song to the liked songs using cosine similarity. Since the embeddings are normalized, this is equivalent to a dot product. A candidate that is close to liked songs in embedding space would receive a high score.

However, this ignores the disliked set. Another possible approach is to compute two scores:

similarity_to_liked
similarity_to_disliked

The problem is deciding how to combine those scores. A candidate might be similar to both liked and disliked songs. Another candidate might be moderately similar to liked songs but very dissimilar to disliked songs. A simple similarity method does not clearly define how those tradeoffs should be ranked.

This motivates a pairwise ranking approach.

Bayesian Personalized Ranking

The system uses Bayesian Personalized Ranking, or BPR, to learn a preference vector from liked and disliked examples.

Instead of treating the task as classification, BPR treats it as a ranking problem. The model does not need to decide whether an individual song is absolutely good or bad. It only needs to learn that, for this patient response setting, liked songs should rank above disliked songs.

Let:

P = set of liked song embeddings
N = set of disliked song embeddings
w = learned preference vector

For a liked embedding p and a disliked embedding n, the system computes:

score(p) = p · w
score(n) = n · w

BPR trains w so that:

score(p) > score(n)

The loss function for a pair is:

loss = -log(sigmoid(score(p) - score(n)))

or equivalently:

loss = -log(sigmoid((p · w) - (n · w)))

If the liked song already scores much higher than the disliked song, the loss is small. If the disliked song scores higher, or if the scores are too close, the loss is larger. During training, the system samples liked-disliked pairs and uses back propagation to adjust w.

Importantly, MuQ itself is not retrained. The song embeddings remain fixed. The only learned object is the preference vector w.

What Back Propagation Changes

Back propagation determines how each dimension of w contributed to the ranking error. If a liked song does not score sufficiently higher than a disliked song, the optimizer changes w in the direction that reduces the BPR loss.

Intuitively, this update makes the learned vector point more toward the liked song embeddings and away from the disliked song embeddings. Over many sampled pairs, w becomes a compact representation of the musical characteristics that separate songs patients sang along to from songs that produced no reaction.

Candidate Ranking

After training, the system evaluates new songs by embedding each candidate recording with MuQ and scoring it against the learned preference vector:

candidate_score = candidate_embedding · w

Candidates are sorted from highest score to lowest score. The highest-ranked songs are the system's best predictions for songs that patients may sing along to.

Limitations

This system should be understood as a ranking aid, not a definitive model of patient memory or musical preference. There are several limitations.

Ones more relating to the premise of the hypothesis include:

• Patient response is affected by mood, health, context, and personal history.
• Original recordings may not perfectly match the version a patient remembers.
• Audio similarity does not capture lyrical familiarity, cultural meaning, or autobiographical associations directly.

More technical issues include:

• A small liked/disliked dataset may produce an unstable preference vector.
• Candidate generation strongly affects the final recommendations.

The system is therefore most useful when combined with human judgment. It can produce a ranked shortlist, but caregivers, musicians, and researchers should evaluate the recommendations in context.

Conclusion

This project uses music representation learning to support a practical recommendation problem: predicting which songs may lead patients to sing along. By embedding original recordings with MuQ and training a BPR preference vector from observed liked and disliked responses, the system turns qualitative recital observations into a repeatable ranking pipeline.

The result is not a general-purpose music recommender. It is a focused prototype for ranking candidate songs according to a specific patient response: whether a song is likely to trigger singing along.