This week, I had the exciting opportunity to test the mixed reality driving simulator developed by the Austrian startup, NXTR.io.
I met with the company’s CEO, Lukas Stranger, in the ARENA2036, where they proudly showcased their innovative product.
The Mixed Reality Simulator Setup:
The simulator setup involves a platform positioned under the front wheels of the vehicle, along with pedal sensors attached to the brake and acceleration pedals inside the car. In the image below, you can observe the NXRT platform beneath the car’s wheels. Each wheel is mounted on a “turntable,” with sensors on the right side detecting wheel movement, while the left one lacks sensors. This setup enables smooth steering, detecting movements transmitted to the control computer positioned between the two wheels. Pedal sensors allow for braking and acceleration, mimicking a car racing game. The sensor data is then transmitted to the central computer, calculating visuals for VR glasses based on the current configuration and sensor data.
The mixed reality driving simulator supports nearly any type of vehicle. In my trial, we used a Mercedes EQS SUV, equipped with an HTC Vive and a high-resolution camera developed by NXRT. This camera, attached to the VR goggles, contributes to creating augmented reality visuals.
The Driving Experience:
Upon entering the driver’s seat and donning the HTC Vive, the experience unfolded. The camera attached to the goggles projected the car interior, while the windshield and side windows were virtually generated, immersing me in a different environment. I was prompted to drive a designated route, steering and accelerating just like in a regular driving simulator. Later, I navigated through a garage, where an augmented reality hologram introduced the driving assistant I was about to test. This visualization explained the functions and operations of the driving assistance systems effectively. Returning to the street, scenarios unfolded, such as a person suddenly crossing the road, prompting the driving assistant to brake promptly—a compelling demonstration of how such a simulator can elucidate driver assistance systems.
Conclusion:
In conclusion, my experience with the mixed reality simulator was highly enjoyable. The potential for this technology is vast, with NXRT positioning it as a valuable tool for OEMs and Tier-One companies to test and showcase their driving assistance systems. However, I believe its applications extend beyond, potentially benefiting driving schools, driving test centers, theme parks, and amusement arcades, enhancing the realism of driving games.
While the technology is still in its early stages, there’s room for improvement. As discussed with Lukas Stranger, replacing the headset with the Quest 3 or, in the near future, an Apple Vision Pro, could significantly enhance visual quality. The accuracy of the windshield and side window cutouts can be improved by integrating 3D models of specific vehicles into the simulation software. Additionally, addressing issues such as the occlusion of the steering wheel in my EQS setup and enabling interaction with the vehicle’s software through interfaces like Mercedes MBUX would contribute to a more realistic and immersive experience.
My call to OEMs is clear: provide better interfaces to your car software. This not only aids startups like NXRT but also unlocks numerous use cases, delivering tangible value to both businesses and consumers.
Wishing NXRT, Lukas, and the team continued success. Keep up the excellent work!