Improving autonomous driving safety

Pavement markings serve as a highly effective traffic control device to communicate critical visual information to road users. While motorists become adept at interpreting pavement markings through driving experience and intuitively navigate their surroundings, autonomous vehicles continue to be challenged by sensing and perception issues. Carnegie Mellon University (CMU) partnered with PPG Industries (PPG) to investigate encoding information in traffic paint to provide a richer control on driving environments as seen by machine perception systems, thereby reducing computational overhead and leading to a safer driving experience.

Pavement markings convey basic information about the layout of streets and the lanes within while ensuring visibility under different environmental conditions. Every few years, road workers replenish faded pavement markings with a fresh layer of paint. Aswin Sankaranarayanan, professor of electrical and computer engineering at Carnegie Mellon University, sees an opportunity to leverage this existing replenishment process and enhance the functionality of traffic paint using infrared additives invisible to the human eye but observable by cameras.

“In our project, we use this idea that the paint that we use on streets for traffic marking is changed much more frequently than road signs or traffic signals,” says Sankaranarayanan. “Repainting pavement markings is something that's done all the time, which presents an opportunity to do something innovative without changing this existing system as far as humans see it.”

PPG (Pittsburgh, PA) approached CMU with a project concept, and PITA facilitated an introduction to Sankaranarayanan, an expert in the study of light and materials sensors. The collaboration resulted in manipulating the spectral properties of traffic paints to encode data observable by a spectrally sensitive camera, significantly enhancing the amount of information that infrastructure can present to assisted and autonomous driving systems. By encoding information on existing lane markings and structures, the team believes it can make it simpler for autonomous vehicles to understand their surroundings without adversely impacting the current system that works well for humans.

The project leverages a phenomenon known as metamerism, in which colors that appear similar present different properties when viewed under different lighting conditions. Using two copies of the same paint color, a coat of paint containing near infrared light (NIR) additives is layered on top of another coat without additives, with no obvious perceptual difference to human drivers.

With this idea of metamerism that we're using, we can have two completely different spectrums of light that appear identical to people because the human eye is a weak distinguisher of colors relative to machine vision technology,” says Sankaranarayanan. “The color combination will look the same to us but any camera that can detect infrared light will see the barcode or design that you've laid out.”

Recognizing the infrared paint, machine vision systems used in autonomous vehicles will have more input to identify the boundaries of individual traffic lanes and better anticipate upcoming stops and lane changes. Instead of relying on the camera to identity a specific marking, such as a stop sign, and reason where to stop the vehicle, the information can be reported in advance through paint on lane markers.

“We can help assisted and autonomous driving systems navigate potentially dangerous driving scenarios,” says Sankaranarayanan. For example, road workers can embed a message on the lane marker of a curved road that alerts the vehicle to an obscured stop sign. “Even a small amount of advance information could make a significant difference in safety, especially when factors like weather, poor visibility, and [human] driver distractibility are considered.”

Sankaranarayanan believes there is enormous potential for advancing the safety of autonomous driving using a low-cost, highly scalable technology that leverages existing infrastructure surrounding vehicles on roads and highways.

With the onset of autonomous vehicles, machine reading of traffic markings will be a required capability. This project will help us to understand what kind of changes and new technologies we need to develop in order to meet the need of both human interactions and interactions with modern cameras in the vehicle.

Gobinda Saha, Global Technical Director – Traffic Solutions , PPG Industries

“The potential of rapid scalability is what excites me most about this project,” says Sankaranarayanan. “If in two years’ time you make an improvement to the paint combination that requires a slightly different shade or hue, you can implement that change whenever those roads are due to have pavement markings replenished. Within a matter of years, nearly every street in Pennsylvania can benefit from the technology.”

Currently, the team is collecting spectral signatures associated with various paints to find a combination of paints which look the same to the human eye but appear maximally different to a spectral camera. When the team identifies the right paint combination and their wavelengths, they will build a spectral camera which can lock on to these two signals and enter testing with autonomous vehicles.

Sankaranarayanan credits the PITA program for identifying the potential synergy between his group’s expertise and PPG’s interests. Pioneered by CMU in the 1980s, Western Pennsylvania has a rich history of over 30 years in autonomous driving research from academia, industry, and startups. Sankaranarayanan believes there are natural partners around Pittsburgh who could deploy the team’s infrared paint technology in their test beds.

“I’m hopeful that there will be greater adoption of the technologies for assisting drivers in various ways because that will naturally lead to greater safety when it comes to navigating our streets,” says Sankaranarayanan. “If you can augment the infrastructure around a vehicle to make it safer and easier to drive, that benefits all of Pennsylvania. If successful here, the test that we're running in its [Pittsburgh] application then extends itself nationwide and potentially worldwide.”