MediaMined: Jay LeBoeuf's Search Engine for Sounds

When Jay LeBoeuf was 10, he begged his parents to buy him an electric keyboard so he could plug it into his Apple IIGS computer and play ’80s synth rock. Now LeBoeuf, an electrical engineer with a background in classical piano and a master’s degree from Stanford’s Center for Computer Research in Music and Acoustics, wants to turn computers into virtual sound engineers. “A big motivation for me is that computers and mobile devices right now, they’re deaf,” he says. “They have no idea what’s going on.”

In 2008, LeBoeuf, 34, founded Imagine Research, a startup that on Nov. 9 unveiled its MediaMined software, which enables music and film producers to search, classify, and retrieve all manner of audio files. The software currently recognizes 400 distinct sounds, including instruments, voices, and noises like a dog’s bark and the crack of a baseball bat. LeBoeuf is working on expanding that vocabulary and early next year plans to roll out a feature that marks where in the audio file each distinct sound occurs. On an episode of, say, American Idol, “We can determine where the judges are speaking, when there’s music, and when there’s applause,” he says. “[We] use that to do intelligent summaries of a performance or allow you to skip to sections that have music.” Once audio files have been indexed by the software, then sound engineers can browse through digital movie archives and find, say, all the gunshots and explosions. MediaMined also allows recording artists to find a wide range of bass lines or drum tracks that might complement a guitar riff for their songs.