Enhancing and evolving vehicle perception with deep learning

The dreams of autonomous cars are becoming a reality. The automotive industry’s quest for full autonomy is steadily progressing primarily through the implementation of multiple advanced driver assistance systems (ADAS) in vehicles. The average new car in today’s dealership is equipped with multiple cameras, as well as radar and ultrasonic sensors, that enable perception-based functions such as automatic parking assist, automatic emergency braking, lane keep assistance, driver drowsiness alert, etc.

Camera-based perception functions are vital in today’s vehicles. The functionality of perception systems strikes strong similarities with the human body. The camera, or image sensor, acts as the eye of the vehicle. Data is sent from the image sensor to the main processor, or brain, where it is understood and interpreted using various algorithms. Finally, the decision is acted upon by sending commands to control the steering, accelerator, and/or braking (the hands and feet, for the sake of the analogy). Over the past decade, automotive perception has evolved from a basic rear view camera to full 3D surround view with parking assist capabilities. Similar to how human capabilities advance as the brain develops, these advances in ADAS technology have been built on the foundation of innovative perception algorithms running on increasingly efficient hardware platforms.

Deep learning overview

One of the hottest topics in the autonomous driving space today is “deep learning,” a subset of machine learning. Deep learning is a computational method used to make accurate classifications and predictions based on neural networks that have been trained on vast amounts of data. Neural networks are sets of algorithms designed to recognize patterns in data. Many ADAS applications, such as front camera perception, use convolutional neural networks (CNNs), to perform tasks like object detection and classification more efficiently than traditional computer vision methodologies. In the example below (figure 1), deep learning is used to classify vehicles, roads, signs, pedestrians, and background, and distinguish them visually in the output.  TI’s deep learning expertise led to the development of an extensive collection of resources including the TI Deep Learning (TIDL) software framework, which simplifies the algorithm training, development, and porting process for developers. For more information about deep learning for automotive, read our blog “AI in Automotive: Practical Deep Learning.”

Figure 1: Example of object detection and classification using TIDL software framework on TDA2 processors

Evolving automotive perception systems with deep learning

TI has a long history supporting both automotive and computer vision applications. As the technology supporting these two areas has converged, it’s been especially important to develop chips with high levels of functional safety capability, power efficiency and performance. The Jacinto™ TDAx processor platform assists automotive OEMs and tier-1 suppliers to develop and implement deep learning algorithms for ADAS applications. One automotive software company, Momenta, recently leveraged TI’s heterogeneous TDAx processor architecture in their new perception system to enable SAE L2-L4 autonomous functionality. The combination of the TDAx processor architecture, TIDL software framework and Momenta’s deep learning expertise in one solution enables automakers and tier-1 suppliers to potentially increase network efficiency while maintaining accurate perception of lanes, vehicles, pedestrians and other objects.

To learn more about how TI supports deep learning for automotive applications and automotive processors, read "AI in Automotive: Practical deep learning" or see Jacinto™ TDAx ADAS SoCs