Great clue. I had to try a few things using the hint but ultimately figured out where to attach the correct script and what to write. It really is just one line of code:
void Start () {
Object.DontDestroyOnLoad(this);
}
Attach it to the Music Player game object, and only the component will be destroyed but not the game object itself, so the music will keep on playing.