Awesome project! As a professional audio developer, I was really blown away that this was the author's first project working with audio.
For anyone interested, I'd recommend checking out "The Infinite Jukebox", which has a similar goal, but perhaps a more robust approach: http://infinitejukebox.playlistmachinery.com
If I had to guess at why your approach didn't work well on recorded music, it's probably because most of the time, there is more than a single event happening at a time, so picking out just the highest FFT bin is probably not a very robust "fingerprint" of that part of the music. The infinite jukebox uses timbral features as the fingerprint, rather than just a single note.
Thanks for the pointer to The Infinite Jukebox! It's impressive how well it does. It's biggest point of failure seems to be jumping to different parts of a verse or chorus and mixing up bits of lyrics. It makes that this would happen, but it would be really interesting to see what could be done to avoid this.
It's a shame you can no longer upload music to the InfiniteJukebox. Apparently it relied on a Spotify API that was shut down October 1st. Best hope going forward is an open source library that can offer the same track analysis that used to be provided by that API.
The API was an Echo Nest API. Spotify acquired Echo Nest and shut down all their APIs in 2016.
I'd love to see the app live on too! Spotify's API has similar functionality[1] to the old Echo Nest API, for now at least. (But I don't know if it returns all the same data.) Or, if you don't want to rely on Spotify, I bet Essentia[2][3] could do the job just as well. Essentia is the open-source brains behind AcousticBrainz[4]. So you could either use Essentia directly, or grab the data from the AcousticBrainz API.
For anyone interested, I'd recommend checking out "The Infinite Jukebox", which has a similar goal, but perhaps a more robust approach: http://infinitejukebox.playlistmachinery.com
If I had to guess at why your approach didn't work well on recorded music, it's probably because most of the time, there is more than a single event happening at a time, so picking out just the highest FFT bin is probably not a very robust "fingerprint" of that part of the music. The infinite jukebox uses timbral features as the fingerprint, rather than just a single note.