Ever had a tune stuck in your head but couldn’t quite place the name? Now, you can simply hum, whistle, or sing into your phone and ask “What Is This Song?” and Google Search can identify it. This might seem like magic, but it’s actually powered by sophisticated machine learning. Let’s dive into how this fascinating technology works.
At its core, a song’s melody acts like a unique identifier, much like a fingerprint. Each melody possesses its own distinct pattern. Our engineers have developed machine learning models specifically designed to recognize these melodic “fingerprints.” These models are the key to matching your hummed, whistled, or sung input to the correct song.
When you hum a melody into the search bar, the machine learning models spring into action. They transform the audio you provide into a numerical sequence. This sequence isn’t just a random set of numbers; it’s a representation of the song’s melody. Crucially, these models are trained on a vast dataset of music from diverse sources. This includes recordings of people singing, whistling, and humming, alongside studio-quality tracks. The algorithms are also clever enough to filter out extraneous details like background instruments, vocal timbre, and tone. By stripping away these layers, the system isolates the pure melodic essence of the song, leaving behind that crucial number-based sequence – the melodic fingerprint.
This melodic fingerprint is then compared against a massive library containing thousands upon thousands of songs from across the globe. The system performs this comparison in real-time, rapidly identifying potential matches. Think about the popular song “Dance Monkey” by Tones and I. You can recognize it instantly whether it’s the original studio recording, a live vocal performance, or even just someone humming the tune. Similarly, our machine learning models can recognize the underlying melody from the studio version and connect it to the audio of someone humming the same tune.
This innovative feature builds upon previous breakthroughs in music recognition technology from our Research team. You might recall the “Now Playing” feature on Pixel 2 launched in 2017. This used deep neural networks to bring low-power music recognition directly to mobile devices. In 2018, this technology was further integrated into the SoundSearch feature within the Google app, expanding its reach to a catalog of millions of songs. However, the “hum to search” functionality represents a significant leap forward. Now, identifying “what is this song” is possible even without lyrics or an original recording – all it takes is a hum.
