Improving Deep Learning based Perception for Challenging Environments: A Project Overview
Machine learning methods have revolutionized the way we interact with technology, from voice assistants to self-driving cars. However, these methods still face challenges in certain environments, such as nighttime or inclement weather conditions. The goal of this project was to improve the robustness of today's machine learning methods, particularly those based on deep learning, in such challenging environments.
To achieve this objective, the project took a two-pronged approach. First, the team developed technologies for generating synthetic data, allowing them to control variables such as rain, snow, and dirt on sensors. Second, they developed new methods for processing this data and improving object detection in challenging conditions. In addition, the team worked on compacting these systems to reduce their resource usage and bring them closer to production.
Expected Results and Effects
The project yielded promising results. By using synthetic data, the team was able to significantly reduce the cost of annotating data. In fact, they found that up to 90% of annotated data could be replaced with synthetic data. This not only saves time and money but also allows for more data to be collected and used in training machine learning models.
Another exciting development was the team's ability to synthesize data from failed sensors in vehicles. By using machine learning and data from other sensors, they were able to create accurate representations of the environment that would have been missed otherwise. The team also explored the best way of sensor fusion for object detection and found promising results.
Perhaps most impressive was the team's ability to reduce the latency of the object detection system by over 60% using the methods developed in the project. This is a significant improvement that could have a major impact on real-world applications, particularly those that require quick and accurate object detection.
Planned Approach and Implementation
To achieve these results, the team designed an interface to the Carla Simulator with semi-automatic functionality. This allowed them to generate 3D worlds with challenging conditions, which in turn allowed them to collect large amounts of synthetic data for training machine learning models.
The team used the Kitty open database for the development of autonomous vehicles as the basis for their vehicle sensor setup. They also used Nvidias Jetson Xavier AGX as a hardware platform for accelerated deep learning. Monthly meetings and additional technical meetings were held as needed to ensure coordination and progress.
The project was coordinated by Volvo GTT initially, but Embedl was added as a partner during the course of the project. Together, the team was able to make significant strides in improving machine learning methods for challenging environments.
In conclusion, this project represents an important step forward in the development of machine learning methods that can operate effectively in a wide range of environments. By developing technologies for generating synthetic data and improving object detection in challenging conditions, the team has made significant contributions to the field. These developments could have important applications in fields such as autonomous vehicles, robotics, and more.
You can read the End of project report here
