If you’ve ever experienced Mercedes-Benz’s latest MBUX infotainment system, chances are you’ve experienced the augmented reality navigation feature. It displays a video image of the route in front of you and includes your navigation instructions – complete with road signs and onscreen arrows – when you arrive at your next waypoint. It takes a little getting used to at first, but after a while it becomes difficult to do without.
Mercedes now offers augmented reality at the service area with Mercedes-Benz Virtual Remote Assistance.
What it does is use Microsoft’s HoloLens 2 hands-free mixed reality device – essentially a headset with a pair of attached goggles that act as screens – in concert with Microsoft Dynamics 365 software to help better streamline the service that Mercedes-Benz vehicles need.
According to Mercedes, the new feature allows for faster return of service and more vehicles through the service bay. It also means more service bays generally available.
Also according to Mercedes, its current crop of new vehicles uses more than 100 million lines of code – more than the space shuttle and an F-35 fighter combined. Essentially, this improves the customer and driving experience, but the sheer complexity of the vehicles requires more technical precision in their maintenance.
Often times that meant bringing in an offsite technical specialist to help with the case, which historically meant a long train of emails that would inevitably slow down the process. Additionally, once it was determined that outside help was needed, specialists would have to visit dealerships across the country, which, of course, is not good for Mercedes’ environmental footprint. -Benz.
With HoloLens 2, however, all you need is a single email to let the Tech Specialist know that it is needed to start the process.
At this point, the technical specialist sends diagrams, schematics, holograms, pictures, etc. to the foreman in real time, via HoloLens. The foreman then chooses an appropriate background on which to display the diagrams, and he can manipulate them on the spot. Then they move to the vehicle and whatever they see the offsite specialist sees. You can look at a car part, have a diagram of that part right in your HoloLens as you work, and use it to diagnose and repair a host of issues that occur in vehicles these days.
Mercedes demonstrated that to us, and it’s very neat technology. Essentially, we got to see everything the on-premise tech saw, while at the same time the specialist was there with us, dictating during a Microsoft Teams style call exactly what to do. It was completely transparent and without any lag or lag – it really felt like we were in the same room together, even though the three of us were in completely different geographic locations.
As of July 2021, all 59 Mercedes-Benz dealers in Canada had a technician trained on HoloLens, as did 383 US dealerships. Interestingly, Mercedes’ home country of Germany is just starting to test the technology right now, hoping to have it in all dealerships sometime in 2022.
The potential uses of the technology go far beyond under-the-hood repairs. Mercedes-Benz is using HoloLens technology for counselor training and is considering a way to use the technology to work on body repairs as well as the more serious stuff.
What’s really interesting, however, is that auto repair and maintenance – while incredibly necessary – is often not the most glamorous job. Not that all jobs require glamor, of course, but ask Mercedes and they will tell you that this new technology has the potential to attract people from different backgrounds to the profession with the cutting edge technology it employs. And of course nowadays the technological rules and the more you can take advantage of them, the more potential hires you will have.