Augmented Reality in Cars: AR in the Automotive Industry

Augmented reality automotive systems today: which cars ship an AR-HUD, how they are built, and the latency and safety limits on driver overlays.

Augmented Reality in Cars: AR in the Automotive Industry
Written by TechnoLynx Published on 15 Oct 2024

The augmented reality automotive displays documented in production cars are windshield AR head-up displays (AR-HUDs). They project navigation arrows and driver-assistance cues so the graphics appear to sit on the road about ten metres ahead, rather than on a screen inside the car. Audi’s Q4 e-tron, the Mercedes-Benz S-Class and Volkswagen’s ID. models (as an option) ship this kind of system. BMW describes an optional 3D head-up display in Panoramic iDrive that shows navigation and assisted-driving notifications. What Audi’s system overlays is specific: a navigation arrow, lane-departure lines and a marker on the car ahead.

If you want a general introduction to what augmented reality is and where it is applied, start there. This page stays with the car.

Which production cars have an AR HUD, and what do they actually show?

Car Image planes and distance Field of view What the AR layer overlays Inputs that place the overlay
Audi Q4 e-tron AR section at roughly 10 m (further in some situations); status section at about 3 m Equivalent to a diagonal of around 70 inches Floating “drone” navigation arrow, red line over a lane marking when drifting without indicating, coloured stripe on the car ahead when adaptive cruise is active Front camera, radar, GPS navigation
Mercedes-Benz S-Class Virtual image at 10 m 10° horizontal × 5° vertical, equivalent to a 77-inch monitor Driving-assistance and navigation content blended into the scene ahead Not specified in the Mercedes release
Volkswagen ID. (optional) Far plane at about 10 m for dynamic 3D content; near plane at about 3 m for static 2D Not specified Turning arrows and destinations drawn on the roadway, assist-system information; speed and road signs on the near plane Front camera, radar sensor, navigation map, processed by one of the car’s two central computers
BMW 3D Head-Up Display (optional) Not specified Not specified Navigation and assisted-driving notifications in the driver’s field of vision Not specified

In the Audi and Volkswagen systems, the near plane shows ordinary HUD content: speed, traffic signs and static symbols. The AR content sits on the far plane, superimposed in the corresponding place on the outside world. Volkswagen’s own wording is that warnings are projected “within the scope of system limits”.

Zhou et al. (Micromachines, 2024) estimate that a HUD needs a horizontal field of view of at least 20° to cover the driver’s lane plus half a lane each side, and they put a typical AR-HUD at 10°. Hyundai Mobis and ZEISS presented a full-windshield holographic display programme at CES 2025.

How is an automotive AR display built: windshield, cluster or centre screen?

Zhou et al. (2024) describe three generations of HUD. First came the combiner HUD, a separate panel above the dashboard with an image about 1 m away that pulls the driver’s focus inside the car. Next came the windshield HUD, which uses the windshield itself as the combiner. The AR-HUD keeps that windshield approach and adds a much larger field of view and image distance. Market analysts at Global Market Insights put windshield HUDs at 73% of 2025 HUD revenue and call the windshield format the natural architecture for AR programmes.

Camera-feed AR on a centre screen works differently. Graphics are drawn over a live front-camera image, which avoids the optics problem but places the information away from the road. The Mercedes S-Class 3D driver display is auto-stereoscopic depth for the instrument cluster, not world-registered AR.

Picture generation and the optical path

An AR-HUD has two parts: a picture generation unit (PGU) that creates the image, and optics that fold the light path, magnify the image, form the eyebox and correct the distortion added by the windshield. Commercial PGUs use TFT-LCD, DLP, LCOS or micro-LED projection. Audi and Volkswagen both send light from a bright LCD onto two flat mirrors that split the near and far portions, then onto a large concave mirror and the windshield. Mercedes uses a Texas Instruments digital micromirror device with 1.3 million individual mirrors.

The long image distance has a mechanical consequence. A HUD with a short virtual image distance needs a wedged windshield to suppress the double image reflected from the glass’s front and back surfaces. According to Texas Instruments, AR-HUDs with long image distances no longer need the wedge, which saves the OEM money.

The parameters that separate AR from a conventional HUD

Parameter Research or supplier target What production ships
Virtual image distance Above 10 m, ideally above 20 m (Zhou et al., 2024); 7.5–20 m (Texas Instruments) About 10 m on the far plane
Field of view At least 20° horizontal for two-lane coverage; more than 10° × 5° for a “future-proof” AR-HUD (Zhou et al., 2024); 10–15° (Texas Instruments) 10° × 5° (Mercedes S-Class)
Image planes At least two: near plane at 2–5 m for status, far plane at 10 m or more for navigation (Zhou et al., 2024) Two planes at roughly 3 m and 10 m (Audi, Volkswagen)
Eyebox Larger than 90 × 60 mm so both eyes stay inside it (Zhou et al., 2024) Not specified
Brightness Above 10,000 nit (Zhou et al., 2024); up to 15,000 cd/m² for sunlit scenes, a level TI’s white paper says no automotive HUD available at its writing achieved Not published

Brightness is difficult because a windshield reflects only about 25% of the light, and pushing the PGU harder runs into heat and power limits. The image distance matters for comfort as well as cost. With a longer image distance, the driver can see the road and the graphic without refocusing between them.

Zhou et al. (2024) argue that pushing free-space mirror optics further makes the unit too large, which is why waveguides are attractive. Global Market Insights reports a Continental waveguide demonstrator with a 15° × 5° augmentation area at 10 m in a 10-litre package. Researchers have also proposed 3D HUDs, which use eye tracking to create a separate eyebox for each eye. Whether the resulting vergence-accommodation conflict causes discomfort at driving speed is still under investigation. Tracking the driver’s eyes and head is covered in our piece on COCO keypoints and pose estimation for driver-facing AR.

The sensor and compute chain that places an overlay

Volkswagen states that one of the ID.’s two central computers calculates where each symbol goes, using raw data from the front camera, the radar sensor and the navigation map. Audi’s equivalent, the AR Creator, renders symbols at 60 frames per second, warps them to the geometry of the projection optics and calculates their position from front-camera, radar and GPS data. Audi says its software runs to roughly 600,000 lines of code.

Global Market Insights points out the dependency directly. AR content relies on timely camera, radar, map and vehicle-state inputs, and a poorly calibrated overlay can make the driver interface worse than having no overlay at all. Zhou et al. (2024) go further and say LiDAR has become imperative for AR-HUDs to perceive road conditions. The production systems above, however, document camera and radar without LiDAR. We read that as a difference between research aspiration and what OEMs have chosen to ship, not a contradiction to settle.

The detector has to find the lane, the vehicle or the pedestrian reliably, and the [email protected]:0.95 detection metric is one way that accuracy is scored. A highlight that must stay attached to a moving car relies on multiple object tracking for AR overlays to hold identity from frame to frame. The graphics stack that draws the overlay brings its own choices, such as the AMDVLK vs RADV Vulkan driver.

Training a perception model and running it inside the vehicle are different engineering problems. The Eagle supercomputer piece looks at the training-scale end. A model that performs well in a training environment is a different artefact from one that performs on an embedded automotive target, and closing that gap is engineering work, not a hand-off. TechnoLynx works on this side of the stack through its AI engineering for manufacturing and automotive: the perception models and GPU code that feed an overlay pipeline, not the HUD optics or the safety sign-off.

What latency budget does a driver-facing AR overlay need at road speed?

The latency evidence below comes from head-worn AR research, not from road-speed overlays. What the evidence does show is that latency drives registration error, and that Audi’s production system predicts motion.

Lincoln et al. (UNC, IEEE VR 2016) state that system latency is the largest single source of registration error in existing AR systems, outweighing all other sources combined. They also note that AR requirements are stricter than VR’s, because the virtual image has to stay aligned with the real world. End-to-end latency is the sum of tracking, application, rendering, scanout, display and synchronisation delays. Without compensation, their 60 Hz display lagged the real scene by an amount proportional to the speed of motion, and the image lined up again only once the motion stopped. That was a head-mounted display tracking head motion, not a car. The mechanism carries over, but the numbers do not.

Audi’s description of the Q4 AR Creator shows how one production car handles this delay. Audi acknowledges that “a few fractions of a second” pass between a sensor identifying an object and the graphic appearing, and that the car can move considerably in that time through braking or a pothole. The system therefore predicts the car’s forward movement from the most recent data, estimates vertical movement from the camera, the radar and the stability-control (ESC) sensors, and applies a “shake compensation” a few milliseconds before output so the display does not jump out of position. The design choice is the point. The system predicts the car’s movement and compensates for shake before output.

Worked example: how far the car moves per rendered frame

The frame rate is Audi’s published 60 frames per second.

  1. 100 km/h is 100,000 m ÷ 3,600 s ≈ 27.8 m per second.
  2. At 60 frames per second, one frame lasts 1 ÷ 60 ≈ 16.7 ms.
  3. In one frame the car travels 27.8 × 0.0167 ≈ 0.46 m. This is travel per rendered frame, not the overlay’s alignment error.

Audi describes the sensor-to-output gap as “a few fractions of a second”. Without prediction, the overlay would therefore be drawn for an out-of-date car position. Vertical movement from braking or a pothole matters too, which is why Audi estimates it from camera, radar and ESC sensor data. Perception delay adds to all of this, and the limits it puts on vehicles more broadly are the subject of autonomous car challenges: perception, latency, safety.

How do safety-review and regulatory constraints shape what an automotive AR overlay is allowed to display versus what it technically could?

JAMA’s guidelines limit content shown while moving to driving information, and ISO 15008 excludes head-up display content.

The Japan Automobile Manufacturers Association (JAMA) Guidelines for In-vehicle Display Systems, applied to systems developed or modified after 1 January 2007, set four principles. Preferably, a display minimises adverse effects on safe driving, does not obstruct the forward view, avoids distracting information (entertainment is named), and is operable without hurting the driving task. More specifically, information shown while the car is moving must relate exclusively to driving and must not require continuous gaze. Motion pictures such as TV and video are prohibited, although simplified traffic information may be allowed. Scrolling text and displays of 31 or more characters are also prohibited while moving. Total glance time for an operating task must not exceed 8 seconds, or 7.5 seconds of shutter-open time under the occlusion test method. The JAMA guidelines also state that devices for enhancing the driver’s visibility are not subject to them, and a programme would need to decide where an AR-HUD falls on that line.

ISO 15008:2017 covers in-vehicle visual presentation, and ISO confirmed it as current in 2025. It sets minimum image-quality and legibility requirements for dynamic information shown to a passenger-car driver while moving. Its scope explicitly excludes “superimposed information on the external field (e.g. head-up displays)”, along with maps and pictorial images such as rear-view camera feeds. ISO 15008 therefore does not cover superimposed head-up display content.

We would expect any production programme to map the HMI rules for each target market before freezing its content set.

The user-study evidence is positive but depends on design. A Chalmers University of Technology thesis study found that an AR-HUD reduced perceived workload, particularly mental demand, effort and frustration, and that eye-tracking showed less visual attention going to the centre stack display. The same study found that the effect depended on timing, spatial positioning and information density. A study of AR glasses for navigation found a preference for non-central icon placement, especially bottom-centre in the real world, as a balance between distraction and workload. Zhou et al. (2024) warn that inappropriate or overly complex virtual images can divert a driver’s attention. We see the UX principles for augmented reality development applying here at a much higher cost of error.

Which OEMs lead AR dashboard adoption, and who follows?

Among the sources reviewed, the documented brands are Audi with the Q4 e-tron, Mercedes-Benz with the S-Class, Volkswagen with the ID. family, and BMW, which offers an optional 3D head-up display within Panoramic iDrive, a system it says will be used in all segments from the end of 2025. Global Market Insights also reports that BMW confirmed Panoramic Vision for Neue Klasse series production from the end of 2025.

Global Market Insights gives figures for China. It reports about 884,300 AR-HUD deliveries in China in 2024, which it says was 24.81% of combined windshield and AR HUD deliveries. It also estimates that 2025 was the first year AR HUD revenue overtook conventional HUD revenue, at 44.25% of 2025 HUD revenue. These are market-research estimates, so treat them as direction rather than precise measurements.

As for the rest of the industry, our reading of the evidence points to factory-fitted, windshield-based AR spreading down from premium models, with conventional HUDs remaining on platforms that cannot carry the engineering cost. Global Market Insights makes the same two points: factory fitment lets the HUD be calibrated to the windshield, vehicle geometry and ADAS inputs, which aftermarket devices cannot readily do, and conventional hardware remains necessary where AR’s engineering burden is not justified. The next step in hardware is visible in waveguide and holographic programmes from Continental and Hyundai Mobis with ZEISS. A cheap universal AR module is unlikely in the near term. The interdependence of optics, windshield, thermal design and calibration argues against it, a point the market analysis also makes.

Where is augmented reality used in automotive beyond the driver’s seat?

Away from the cockpit, the sources reviewed here document AR in production and service; we found no credible source on showroom or configurator AR. Boboc et al. (Applied Sciences, 2020) reviewed 55 studies of automotive AR published between 2002 and 2019, covering head-up displays alongside manufacturing.

BMW Group describes three production uses. AR goggles guide trainees through engine assembly steps under voice control, so one trainer can supervise three people instead of one, and BMW reports no difference in quality compared with conventional training. Toolmaking staff in Munich overlay CAD data on a tablet camera image of a tool to check an average of 50 criteria, such as drill holes, against the specification. Specialists also overlay CAD data on pre-series vehicles to check that parts like fenders and exhaust systems have the right dimensions and installation positions.

In service, Porsche Cars North America announced “Tech Live Look” smart glasses in November 2017, for launch in its dealerships in 2018. The glasses let a dealership technician share exactly what they see with a remote technical support team. Porsche reported that a July 2017 pilot at eight US dealerships and one in Canada suggested service resolution time could fall by up to 40%. That result comes from Porsche’s own pilot.

FAQ

Can I install a HUD in my car?

Aftermarket and combiner HUDs exist and broaden access to basic head-up information. True AR overlays are another matter. Global Market Insights notes that an aftermarket device cannot readily access or validate the windshield, vehicle geometry and ADAS parameters needed for reliable road-referenced content.

What is a HUD for cars?

A head-up display projects driving information such as speed, navigation and driver-assistance data onto the scene beyond the windshield. An AR-HUD goes further and integrates that information with the real road.

Back See Blogs
arrow icon