Andreas Sennheiser interview: How Magic Leap’s AR glasses will enable ‘transparent audio’

Without sound, visuals are nothing.
This week, German audio company Sennheiser reinforced that point as it launched its Ambeo AR One headphones for use with Magic Leap‘s augmented reality glasses. The new technology is aimed at making AR more immersive, with “transparent audio” effects that let you mix sound from the real world with music and sound coming in from your headphones.
The German company revealed the in-ear headphones at Magic Leap’s L.E.A.P. developer conference in Los Angeles. The Ambeo was the first major accessory to get the “Works with Magic Leap” certification. I spoke with Andreas Sennheiser, who, along with is brother Daniel, is part of the third generation of the family to run the company.
Sennheiser also launched the accompanying Ambeo Augmented Audio Lab app. The headphones will let developers and creators craft their own spatial computing experiences in which real sounds blend with virtual audio. With the Ambeo AR One, developers can mix it up, deciding how much of the outside world’s sound, captured by the headset’s built-in microphones, should blend into the spatial audio experience.
Here’s an edited transcript of our interview.

GamesBeat: And this is your headset?
Andreas Sennheiser: Yes, that’s the product we announced four hours ago. It’s been in the making for more than a year. We basically followed the development of Magic Leap, what they’ve done, to make sure that when they’re ready, we’re ready with the best possible AR audio solution. The outcome is this product.
It has microphones on the outside, so the transparent hearing functionality will allow you to have all the spatial information going on around you. After a minute or two you forget you’re in there. You’re just passing through the world. With the glasses you can look through, you get the sonic overlay of all the virtual elements. You get an augmented audio experience using this product.
GamesBeat: To what degree can you make it interactive? If you look at an object, can it interpret sounds from there?
Sennheiser: This is coming through the Magic Leap device. When objects are detected, you get the audio cues. Because this has a perfect binaural 3D representation, if something happens behind you, you get the right cues from behind. You can turn around and see the source of the sound. It’s fully integrated with the visual part of the Magic Leap glasses.

GamesBeat: I did the Star Wars Porg demo. They had a real speaker in the room, playing the music.
Sennheiser: Right, you probably didn’t wear any headset in that demo. That’s more just for the demo. You probably won’t install speakers in your home exclusively for Magic Leap. The other thing for that, the audio position in the room has to be right. A speaker has a fixed position. But if you have the head-mounted visual device and a head-mounted audio device, because of their interaction, the visuals and the audio are fully combined. There’s no discrepancy between the two.
GamesBeat: What do you call the transparent audio?
Sennheiser: The transparent hearing? We call that spatial audio. In a very simplistic way, it’s just making note of what you hear and bridging this one-centimeter gap, giving you the perfect 3D audio. But there’s more intelligence than that, just passing it through. You can also set it to active noise cancellation mode, so it cancels out more of the environment, or other different settings depending on how much of the environment you want to hear.
If you decide to watch a movie using your AR goggles, for instance, of course you don’t want to hear everything else around you. You can shut that off using the ANC. But if you’re playing a game or doing something that interacts with the environment, you want to hear all of the physical objects around you as well. Then you can put it to the highest level of transparency.
GamesBeat: If somebody is programming with this to do something coordinated around a visual apps, what might they do?
Sennheiser: We only have some suggestions and first ideas of what is possible. The universe will be opening up as soon as creators start to use it. We have launched a demo app, an audio app, called the Ambeo Lab. It’s available in the Magic Leap store. It allows you to experiment a bit with how the sound is being changed by positioning it behind physical objects, for instance.
If someone creates something, they respect three different levels of the audio. One is the audio providing a cue, somewhere to look at — it provides information about what happens around you. The second is it can create an atmosphere. The atmosphere, of course, needs to somehow reflect what you see. If you’re in a room like this, this zen-like atmosphere needs to sound different from if you’re in a railway station, say. Also, the integration between the Magic Leap product and this product ensures that all of the reflective behaviors of the walls are being correctly represented through the device.
If you take a sound source and place it behind this sofa, for instance, then you’ll hear the damping effect of that physical object in the sound. That’s part of what the creators need to experience — what it does with the sound when they begin to interact with the real world.
GamesBeat: Have you done anything like this before in other products, where you have the real world and virtual audio blending together?
Sennheiser: We have one product positioned in a similar way, our Ambeo smart headset. In addition to what this can do, it’s also a binaural recording device. If you plug it into your smartphone, you can have perfect 3D sound recordings. You can also set it to transparency mode, so if you’re riding your bike, you can hear cars and other things around you while you’re listening to your music. Likewise, there are different levels of transparency that you can use depending on the situation.
The other product we launched more than three years ago was the VR microphone. It pretty quickly became the standard for single-point VR production. It comes with software that does the AB conversion to allow the perfect 3D sound, if you have a 360-degree movie. A lot of these things are happening at Sennheiser.
I’d say for the last 15 years we’ve been researching the field. At the time it was just because we liked the idea. There wasn’t any commercial interest in it. But now, with the rise of AR and VR, we’ve seen how relevant this technology will be, blending and interacting with real and virtual objects.
GamesBeat: What’s in that part that you’re holding?

Above: Sennheiser and Magic Leap One

Image Credit: Sennheiser

Sennheiser: This includes all the electronics for the ANC and the transparency. This is basically the brain. These are just microphones and transducers. All the algorithms are running in here. Of course, this carries a battery. You slide it open this way. The other product I mentioned, the Ambeo smart headset, is powered by your smartphone.
GamesBeat: It’s an interesting combination of possibilities. Sometimes you want the sound from the real world and sometimes you want to shut it out.
Sennheiser: When I met Magic Leap and Rony for the first time, they were a small company, only 40 people. It was almost five years ago. The only thing we were sharing at the time was the vision to create something that allowed people to perfectly blend reality and reproduction. In other words, we wanted the brain to not know or feel that there’s a wall between the reproduction and reality.
What he was trying to achieve on the visual part, we’ve been trying to achieve on the audio part. We saw that this was the perfect match. We bring visuals and audio together. It creates an experience, a real, lasting memory. If you see something that feels real, and you hear something that sounds real, it becomes part of your memory. That’s the intention behind both of our companies, to make a lasting impression.
Source: VentureBeat
To Read Our Daily News Updates, Please visit Inventiva or Subscribe Our Newsletter & Push.