AV testing ZF coPILOT and Nvidia Drive

After years of hype promising that autonomous vehicles will be here “soon,” they are finally starting to trickle into the market. To gauge what the automotive industry is thinking and planning, we surveyed more than 260 industry leaders including automakers (28% of respondents), Tier 1s (25%), Tier 2s (13%) industry experts (21%) and others (13%) to provide insight into the future of AI and communications for autonomous vehicles as they see it today.

Full Stack AV Technologies

Special permission from a local government authority is currently required to test or operate a L4 vehicle on public roads.  This is granted when the authority is convinced of the safety to passengers, other drivers, pedestrians and surrounding objects.  Achieving that level of safety is relatively straightforward for 90% of situations.  The last 10% is the most challenging, costly and time-consuming to solve and verify: rare edge cases that are difficult to model and test.

Achieving the ability to safely operate on public roads requires a full stack of finely-tuned and tested AV technology.  The system needs advanced hardware to sense and control the vehicle, and fast, high-powered central automotive computers capable of processing, interpreting and acting upon terabytes of sensor data in a few milliseconds. 

A complex set of AI algorithms is then required to take this data and decide how to plot a safe path to the destination while reacting to situations that arise. This AI is based on machine learning that is trained by using mountains of real world and simulation data that include a multitude of scenarios.

Important Factors to Make Level 4 Vehicles Safe Enough for Public Roads

Figure 1

We asked industry leaders to share the most critical elements of enabling and ensuring safe operation on public roads.  The answers covered a range of topics with relatively close agreement among respondents.  First for automakers and suppliers (64%) is the capability of recognizing objects in all weather and lighting conditions.  This indicates that reliable object recognition in all conditions is perhaps the most challenging technology to develop. Consultants rank this second after redundant failover systems.

What happens when a self-driving system fails? When equipment stops working, redundant hardware kicks in to take over.  If the OS or a software subroutine crashes, a low-level failover system takes control to safely bring the vehicle to a stop.  Consultants think these systems are of primary importance at 65%.  Suppliers choose failover systems as secondary (56%), while automakers choose them as tertiary (46%). 

Test test test

The ability of L4 systems to keep a vehicle within its lane is a real challenge, especially on sharp curves, intersections with separate left- or right-turn lanes and unmarked roads.  Automakers choose lane-keeping in all weather and lighting conditions as their second critical element at 50%.  Both suppliers (53%) and consultants (46%) rank this as of tertiary importance.  Lane-keeping on unmarked roads ranks fourth for automakers and suppliers at 30% each, while consultants rank this in fifth place at 22%.

Enabling a smooth transfer of control from the L4 system to the driver is a critical function for Level 3 and Level 4 vehicles.  This feature ranks in fourth place for consultants, while 20% of automakers and 28% of suppliers rank it in sixth place.  One assumes these latter two groups feel they have solved this potentially dangerous transfer.  Let’s hope they are right!

Test 2 Test 2 Test 2

As vehicles learn to drive themselves and become more connected to the internet, infrastructure and each other, cybersecurity becomes increasingly crucial to system integrity and safety.  Developers use numerous methods to help shield critical systems from attacks and data from being corrupted or stolen.  Automakers (24%), suppliers (29%) and consultants (22%) agree that enhanced cybersecurity ranks fifth in importance.

Communications, redundant actuation systems and 5G implementation rank last among the choices with less than 20% of respondents giving them priority.

AI and Machine Learning System Challenges for Government Approval

Figure 2

While the previous question relates to vehicle-level challenges, this question focuses on the subsystems and capabilities necessary for AVs to receive government approval to operate on public roads. 

Not surprisingly, systems and AI that can sense and perceive well in all weather and lighting conditions rank first, with 71% of automakers, 61% of suppliers and 56% of consultants.  This explains the plethora of companies offering sensor hardware, software, services and platforms to the burgeoning AV sector.

Systems to detect and avoid collisions rank as critical, and are a natural evolution from ADAS systems.  Various lighting and weather conditions can obscure or confound sensors and recognition systems.  This collision-prevention category ranks second overall with 57% of respondents.  Lane detection systems rank third as they did in the previous question with 45% of the total, followed by reliable failover systems at 36%.

Test 3 Test 3 Test 3

 

A track record of safety, a crucial factor in governmental approval, ranks fifth at 32%.  Developers and automakers are careful to test and verify thoroughly before implementing new versions of hardware or software.  In the U.S., NHTSA issued a Standing General Order in June 2021 requiring companies to report crashes that occur on public roads in the U.S. involving vehicles with L2 to L5 features.

AI and hardware capable of prediction and path planning rank sixth at 30%, followed by the ability to accelerate and brake safely and appropriately (27%).  Prediction and planning involve sensor fusion of terabytes of data in real time, which ranks seventh at 18%.  Steering normally and appropriately was followed by localization as the last two items at 10% and 9%, respectively.

AI Development Challenges

Figure 3

Numerous established companies and startups are offering a dizzying array of technologies and services.  The AV sector is beginning to consolidate but is a long way from a relatively steady state.  As a result, the top-ranked development difficulty for automakers (47%) is rapidly evolving technology that is not in a steady state.  For example, every time a new piece of hardware or software is added to a system, it affects other systems that can require much work and testing to rebalance and verify the overall technology stack. 

Related to this is the challenge of integrating subsystems and AI software from multiple suppliers, which ranks second at 38% overall.  Once a system is built it must be rigorously tested with real world and simulated data.  Acquiring and labeling of real data can be a challenge, especially for rare edge cases.  This factor ranks in third place at 36%.

Cybersecurity and deterministic behavior rank fourth (36%) and fifth (35%), related to the integrity and predictability of the AV system.  A related challenge is the accuracy of simulations to the real world, which ranks seventh at 29%. 

The cost of hardware components ranks sixth at 30%; this includes lidar sensors, high-powered central computers and the ultra-fast and adaptable in-vehicle networks required to connect everything.  This is associated with the challenge of balancing computing power and power consumption at the edge, as opposed to utilizing cloud resources. This ranks eighth at 28%.

V2X: Connecting the vehicle to the cloud, city infrastructure and other vehicles as well as performing OTA software updates require a lot of bandwidth which takes energy and money, ranking this item ninth at 26%.  Last on the list is more accurate localization systems at 20%, which would indicate this challenge is mostly solved.

In summary:

• Object sensing and recognition in all weather and lighting conditions remain the top challenges for AI systems.

• This is closely followed by lane keeping in all conditions, including unmarked roads.

• Reliable collision detection and avoidance is a major goal for both developers and government regulators.

• Redundant fail-safe and failover systems to safely handle a system failure or driver inattention are among the top challenges.

• A major development concern arises from the nascent state of the AV sector.Rapidly evolving offerings from numerous companies mean technology stacks have to be rebalanced and retested when new technology is adopted.

This is the third article exploring results from our Autonomous Vehicle Industry Survey.  See article one here, and article two here.  And check the Wards Intelligence website for future articles in this series.