Semi-autonomous vehicles have already been developed by automotive engineers, and the possibility of fully self-driving vehicles becoming a reality is not far off. Research suggests that the advent of autonomous driving (AD) could generate revenue ranging from $300 billion to $400 billion by 2035.
The self-driving car is not only a testament to how advanced technology has become but also a topic of debate. There are legitimate worries regarding safety, malfunctioning technology, cybersecurity breaches, and the possibility of job losses in the driving industry. However, on the other hand, the opposite may hold true as well. Autonomous driving (AD) has the potential to offer safer transportation, greater convenience, and increased productivity or leisure time. Instead of spending hours stuck in traffic, the “drivers” of tomorrow could use their commute time to work, read, or catch up on their favorite TV series.
The sensor technology used in self-driving vehicles is a critical component, specifically heterogeneous sensors. These sensors collect data that is then trained using artificial intelligence (AI) and machine learning (ML) to analyze and respond to the vehicle’s surroundings. By utilizing AI and ML algorithms, the vehicle can use the sensor data to determine the optimal route, make decisions about where it can or cannot drive, detect nearby objects, pedestrians, or other vehicles to avoid collisions, and respond to unexpected scenarios.
The development of self-driving cars has primarily focused on two efforts:
- Utilizing cameras and computer vision to enable driving capabilities.
- Implementing sensor fusion, which involves using heterogeneous sensors to enable the vehicle to “see,” “listen,” and “sense” its environment.
According to most engineers, the success of autonomous driving (AD) depends not only on vehicle cameras and computer vision but also on sensor fusion. Sensor fusion is considered the safest and most reliable choice for enabling self-driving capabilities.
There are four major sensor technologies used in self-driving vehicles:
- Cameras
- LIDAR
- Radar
- Sonar
Self-driving vehicles are increasingly being recognized as a tangible future possibility, thanks to the advancement of sensor fusion technology and rapidly improving AI. It is estimated that by 2030, approximately 12% of vehicle registrations worldwide will consist of autonomous driving (AD) vehicles.
Just as concerns were raised about job losses when computers were introduced, the advent of self-driving vehicles has also sparked similar worries. However, we know that computer technology has actually created millions of jobs worldwide. It is likely that the emergence of self-driving vehicles will also lead to an increased demand for skill-based jobs in the automotive industry.
Let us delve into the sensor technologies that are driving the development of autonomous driving.
CAMERA
Cameras are currently utilized in vehicles for a variety of purposes such as reverse parking, reverse driving, adaptive cruise control, and lane departure warnings.
In order to obtain a 360-degree view of their surroundings, self-driving vehicles utilize high-resolution color image cameras. These cameras capture images in various forms such as multi-dimensional data, different angles, and video segments. Different methods for capturing images and videos are being tested, along with the use of AI technology to ensure safe on-the-road decision-making. However, these tasks are resource-intensive and require significant amounts of computing power.
High-resolution cameras, particularly when combined with advanced AI and ML, have demonstrated significant potential for autonomous driving. These cameras are capable of accurately detecting and recognizing objects, sensing the movement of other vehicles, identifying a suitable route, and creating a 3D visualization of the surroundings. The cameras simulate human eyes and enable the vehicle to navigate similarly to a human-operated vehicle.
However, there are also drawbacks to using cameras for autonomous driving. One major limitation is that a camera’s visibility is heavily dependent on environmental conditions. As a passive sensor, cameras are often unreliable in low-visibility scenarios. While infrared cameras can be used as an alternative, they require interpretation by AI and ML algorithms, which is still an area of ongoing research and development.
There are two types of camera sensors commonly used for autonomous driving: mono and stereo cameras. Mono cameras have a single lens and image sensor, and are capable of capturing two-dimensional images that can recognize objects, people, and traffic signals. However, 2D images are insufficient for determining the depth or distance of objects, which would require complex ML algorithms that may not always yield reliable results.
Stereo cameras, on the other hand, have two lenses and two image sensors, allowing them to capture two images simultaneously from different angles. By processing these images, the camera can accurately determine the depth and distance of objects, making it a better choice for AD. However, stereo cameras may also struggle in low-light conditions.
To address this issue, some developers are combining mono cameras with distance-measuring techniques such as LIDAR or radar, and sensor fusion to accurately predict traffic conditions.
While cameras play an important role in AD, they do have their limitations and will require additional support.
LIDAR
LIDAR is a prominent technology enabling self-driving vehicles and has been used for geospatial sensing since the 1980s. Self-driving cars often feature a rotating LIDAR sensor mounted on the roof.
Two types of LIDAR sensors are utilized in AD. One type is a mechanically rotating LIDAR system that is mounted on a vehicle’s roof. However, these systems are generally expensive and susceptible to vibrations. The other type is solid-state LIDAR, which does not require rotation and is the preferred choice for self-driving cars.
A LIDAR sensor is an active sensor that utilizes the time-of-flight principle. It emits numerous infrared laser beams to its surroundings and detects the reflected pulses using a photo-detector. By measuring the time taken between the emission of the laser beam and its detection by the photo-detector, the LIDAR system can determine the distance to objects and create a 3D map of the vehicle’s surroundings.
Based on the time-of-flight principle, the LIDAR sensor emits thousands of infrared laser beams to its surroundings and measures the time taken between emission and detection of the reflected pulses using a photo-detector. Using the speed of light, the LIDAR system calculates the distance to an object based on the time spent between emission and detection. A three-dimensional point cloud is created using the distance covered by different pulses, and the reflected pulses are recorded as point clouds, which represent a 3D object in space.
LIDAR systems are highly accurate and can detect extremely small objects. However, like visible-light cameras, LIDAR is unreliable in low-light visibility as the reflection of the laser pulses can be affected by weather conditions. Another drawback is the cost, which can be in the thousands.
Despite these limitations, LIDAR holds great promise for AD as new developments are continuously being tested and improved upon.
Radar
Radar sensors are already a common feature in many vehicles, providing functionalities such as adaptive cruise control, driver assistance, collision avoidance, and automatic braking. The two most commonly used radar frequencies are 77GHz for long-range detection and 24GHz for short-range detection. The short-range radar (24 GHz) can detect objects up to a distance of 30 meters, making it ideal for collision avoidance and parking assistance, while the long-range radar (77 GHz) can detect objects up to 250 meters away and is used for object detection, adaptive cruise control, and assisted braking. These radar systems are cost-effective and reliable for their intended purposes.
Radar is a useful tool for detecting the movement of surrounding vehicles and potential obstructions, especially when combined with cameras. However, its capability for self-driving is limited as it cannot classify objects, only detect them. Low-resolution radar can be used in conjunction with mono or stereo cameras and LIDAR to aid in low-visibility situations.
Sonar
Passive and active sonar technologies are being explored for their potential use in AD. Passive sonar technology listens to sounds from surrounding objects to estimate an object’s distance. On the other hand, active sonar technology emits sound waves and detects echoes to estimate the distance of nearby objects based on the time-of-flight principle.
Although sonar technology can operate in low-visibility conditions, it has several drawbacks that make it less advantageous for self-driving vehicles. The speed of sound limits the real-time operation of sonar, which is critical for safe AD. Additionally, sonar technology can produce false positives and cannot recognize or classify large objects at short range. Therefore, sonar is only useful for collision avoidance in unexpected conditions.
Inertial sensors
Inertial sensors, including accelerometers and gyroscopes, are crucial for self-driving vehicles. They enable the tracking of a vehicle’s movement and orientation, allowing it to stabilize on uneven surfaces and take evasive action to avoid potential accidents.
GPS
A reliable and accurate GPS system is a crucial component of self-driving technology. By using a satellite-based triangulation technique, GPS allows a vehicle to precisely determine its location in three-dimensional space.
However, there are instances when GPS signals are unavailable or disrupted due to obstacles or intentional interference. In such cases, self-driving vehicles must rely on a combination of inertial sensors and data from a local cellular network to accurately track the car’s position. This is why many self-driving car prototypes incorporate multiple redundant sensors and backup systems to ensure the vehicle can navigate safely and accurately even in the absence of GPS signals.
Conclusion
Self-driving vehicles rely on multiple heterogeneous sensors to operate effectively. One advantage of using many sensors is that if one fails, another one can compensate for it. A sensor-fusion technique is necessary for fully autonomous vehicles, using data from different sensors to determine the surroundings.
Currently, there are several approaches being tested in the development of AD. One approach uses stereo cameras to enable self-driving, while another uses mono cameras to provide 360-degree vision, incorporating LIDAR or radar technology to sense distance. A third approach uses stereo cameras with radar sensors.
Cameras with sensors will likely be required for AD to effectively classify and recognize objects. Radar and LIDAR technologies can assist in using sensor fusion to create a weather-resistant self-driving solution. They can add a 3D element, ensuring a better understanding of the driving environment.
Sonar or ultrasonic sensors can also play a key role as they’re weather-resistant and reasonably cost-effective, providing an effective collision avoidance and emergency handling solution. Ultimately, self-driving cars will rely on a combination of all these technologies.