Varjo has announced a partnership with Volvo to enable safe driving with an augmented reality headset that you could eventually wear while driving. It can give you visual alerts, using computer graphics to highlight hazards ahead of you on the road.
Volvo is using Varjo’s XR-1 developer edition headset, which is like a VR headset but with pass-through video viewing so you can see what is going on in the outside world through cameras attached to the headset. Gothenburg, Sweden-based Volvo is also investing in Varjo.
It may sound like nuts to wear a headset on your face when you’re driving. But Volvo’s Casper Wikman, technical leader for R&D, said in an interview that the company has already done tests where human drivers have hit the road in Scandinavia while wearing the headsets. The company is showing the tech at the Augmented World Expo in Santa Clara, California, this week.
The developer edition is aimed at engineers, researchers, and designers who can use it to experiment with AR and mixed reality, or when you combine digital headsets with physical locations. The goal is to create mixed reality applications that are indistinguishable from the real world, so you can’t tell apart what is real and what is virtual.
“We have been in close collaboration with Volvo for over a year,” said Urho Konttori, founder and chief product officer at Helsinki-based Varjo, in an interview with VentureBeat. “We press a button and you see the wire frame. Then we press another button and it becomes a real car. You can see a semi-transparent heads-up display that Volvo is testing.”
Above: Urho Konttori (left) of Varjo and Caspar Wikman of Volvo.
Image Credit: Dean Takahashi
I tried it out yesterday and it was pretty amazing. I sat in a chair in an office, put on the headset, and then watched an augmented reality car form around me. At first, it was just the wire frame, or blueprints, of the car that came into my view within the headset. Then the Varjo folks flipped a switch and the full realistic version of the car materialized. I felt like I was sitting in a car, only I knew it couldn’t have been a real one because my head was bumping the virtual ceiling of the car.
I looked out the window and saw desks in the office. (You can see what that experience is like in this video and this video and this video). But then they switched the view again. All of a sudden, my car was in the middle of a street in Venice. I looked around and saw brick buildings. I could see a heads-up display on the windshield of the car that the car maker Volvo was only testing in digital form. And then a moose walked across the street, and my AR glasses identified it as a road hazard that required my attention.
“‘We can create a scene that is fully immersive and realistic with a full field of view,” Konttori said.
It was pretty realistic, without many of the artifacts you typically see in AR demos. But I did not get a chance to drive a car with headsets on it, and it may be a long time before that can happen for a lot of people.
With this kind of technology, it will eventually be difficult for you to tell if you are sitting in a faux car in an office, or sitting in a real car in a parking lot, or really driving a mountain road in the woods. And that’s a pretty cool idea that inspired the engineers at the Swedish carmaker to go to work on the technology.
Above: Varjo has teamed with Volvo to investigate AR in cars.
Image Credit: Varjo
The VR headset started shipping earlier this year. The add-on camera module and the headset will ship to developers in the second half of 2019. Pricing will be disclosed later, but it is one more example of a AR/VR company attacking the enterprise market first, where precision technology is more important than a low price for an initial product.
The resolution of the headset displays is much better than what is on the market today in the form of Oculus or HTC Vive VR headsets. It has two 12-megapixel cameras, which can create images with a resolution of more than 4K per eye, Konttori said. That enables the photorealistic image quality. The headset has integrated eye tracking so it can properly render the images from the correct point of view.
It can handle this kind of resolution because the headset is connected via wire to a powerful PC with an Nvidia GeForce RTX 2080 graphics card. If you put this headset in the car, it would have to be wired via a Thunderbolt 3 cable to a PC in the backseat or the trunk. The headset weighs about 2.6 pounds.
The XR-1 upgrades Varjo’s human-eye resolution headset (VR-1) with a front plate featuring the cameras with video pass-through mode. The device uses cameras to digitize the world in real-time, and then it multiplexes it inside the graphics processing unit (GPU) with the virtual content to show the combined result to the user. The Varjo VR-1 is the only device with the high resolution to make the seamless visual
blending possible, and the XR-1 has the only camera technology capable of producing high resolution with unperceivable latency (less than 15 milliseconds), according to Konttori.
Unlike competitor devices that are delivering mixed reality with ghost-like, semi-transparent renderings in a limited view, the XR-1 enables examining photorealistic mixed reality with a 90-degree field of view. That’s not bad, but we actually see the world with a 220-degree field of view. That means your peripheral vision is cut off in this headset.
To me, that’s not so cool for driving conditions. I was in a car wreck once because I didn’t see a car coming straight at me from the side. But Wikman said that the field of view is good for seeing the road ahead of you, and Volvo is investigating to see what can be done to make the driving experience safer by showing you alerts and other computer overlays in digital form.
With the XR-1 virtual objects appear as real as anything in the physical world and can themselves cast shadows or even illuminate reality. Black objects appear truly black, opaque objects indeed block real or virtual light, and semi-transparent objects will refract the light from the real world behind it. With the industry’s most advanced color matching technology, the colors are perfectly rendered for product trials and showcases.
Above: For safety, a passenger in the back seat can warn a driver in the front seat during Varjo’s testing.
Image Credit: Varjo
For the first time, users can also switch seamlessly between mixed reality and full virtual reality modes. The depth sensors in XR-1 allow mapping real-life objects and environments for building natural occlusion. Examples of industrial applications for the device include UX design, collaboration with photorealistic 3D models, training and simulation, as well as unique research and data analytics across real and virtual environments with XR-1’s 20/20 Eye Tracker.
Volvo is using the device to test-drive virtual car designs on the road, which is something that you can’t do in a car simulation, Wikman said. Volvo can test these ideas for things like the heads-up display many years before they exist in physical form. Volvo has used XR-1 prototypes for this new workflow since the summer of 2018.
By adding virtual elements to the interior of the car, Volvo designers are able to perform design-studies of future cars before they are even built. Volvo Cars engineers have also been test-driving a real car with the driver wearing an XR-1 headset. Adding photorealistic virtual elements or complete features to the test scenarios while driving (world first) enables UX concepts to be quickly iterated. The car designers can add virtual automobiles or unexpected road hazards (such as a virtual moose) to the road for realism, resulting in faster turnaround times and ultimately safer cars.
Volvo has been sufficiently impressed by Varjo’s technology that it is no longer just a partner. As of May 2019, Varjo will receive an investment by the Volvo Cars Tech Fund, the Swedish car maker’s venture capital fund that invests in high-potential technology start-ups. Founded in 2016, Varjo has about 145 employees.
“From the very beginning, our vision has been to create a product that can seamlessly merge the real and the virtual together,” said Niko Eiden, founder and CEO of Varjo, in a statement. “The incredibly advanced ways in which Volvo Cars uses the XR-1 show that Varjo’s technology enables things that have been previously impossible. Together with Volvo, we have started a new era in professional mixed reality.”
Source: VentureBeat
