The next generation of materials

We all know how powerful images are as a tool for communication, evoking responses and capturing information in a unique way. In materials science and engineering, images that capture a material’s microstructure are important to understanding the properties of materials. But there’s a need to improve methods to quickly and effectively extract that information from these images—a need which state-of-the-art breakthroughs in technology like artificial intelligence could meet.

As part of the Citrine Research NextGen Fellowship summer research program, five Carnegie Mellon undergraduate students developed approaches to predict the average grain size of metal microstructures using machine learning and computer vision. Materials science students Thomas Matson, Jingxi Cai, and Keith Kozlosky, math and statistics student Eric Li, and recent alumnus Toby Francis (MSE’18), were advised by Materials Science and Engineering (MSE) Professor Liz Holm, whose own work uses computer vision to analyze the microstructures of materials. Ph.D. student Andrew Kitahara and recent MSE alumna Ankita Mangal also helped advise the group.

The NextGen Fellowship program is sponsored by Citrine Informatics through an award from Schmidt Futures. Citrine, a materials company that uses artificial intelligence to develop materials, provided the support for students to use machine learning from day one, beginning with a weeklong boot camp held in the Bay Area. While most of the CMU participants had prior computational experience, only one had previously worked with machine learning.

Applying artificial intelligence technologies, such as machine learning and computer vision, in materials research and industry is becoming increasingly important. It could mean accelerating discovery processes, automating tedious tasks that scientists currently do by hand, and allowing materials scientists and engineers to make sense of enormous amounts of data. For example, knowing the grain size of a material, or the volume of the material that has the same crystal structure, can reveal a lot about its properties. Looking at images of the microstructure is the best way to study the grain size and grain boundaries, or where the crystal structure changes. Current methods include tracing the boundaries between the grains by hand then feeding it back into the computer, or drawing lines across an image and approximating how many times it crosses the boundary to estimate grain size. By training a neural network to solve the problem automatically, scientists could save a lot of time.

You don’t know if you’re going to succeed or fail, you don’t know what the answers are, and that’s part of what research is.

Liz Holm, Professor, Materials Science and Engineering, Carnegie Mellon University

The Carnegie Mellon research team set out to tackle this important research question by first creating simulated micrographs—images taken from a microscope—to approximate what real images of grain size would look like, even accounting for instances where the boundary between two grains isn’t clear. The students divided into three teams, each taking different approaches to solving the problem.

Matson and Cai built a neural network and trained it on 16,000 images to predict the average grain size. Li used a neural network that is normally used to classify objects in images, such as birds, trees, etc. He used it to extract features from the microstructure images, then put the output into a regression model to predict the average grain size. Kozlosky took a more creative approach. Instead of adapting a neural network to analyze and classify the micrographs, he put the images through an untrained neural network, which acted as a data extractor. With the output, he performed a normal regression to predict the average grain size.

“One of my favorite moments was when the neural network began working,” said Matson. “It was a situation where I was learning by doing, and on top of that trying to solve a problem where there’s not an answer. It was really rewarding to feel like I’d produced something that worked.”

The first approach had the highest correlation between real and predicted grain size, with the already existing neural network next, and the untrained system was the least correlated, though for each approach the error was within 10% and could be good enough for use in industry. At the end of the program, the students presented their findings at a final research symposium in Colorado.

“We got to see what it was like to do real work in computational materials science,” said Kozlosky, “and beyond that I had my first real experience using machine learning and seeing how it works. I realized that I really enjoyed that kind of work and want to pursue it.”

Each student had a different experience and takeaway from the program, but they all agreed that it was extremely beneficial, from getting to work with a company on a real-life problem where there is no clear answer, to experience working on a research team, to learning how machine learning can help answer questions in engineering first-hand.

“When I advise students about research over the summers, you don’t know if you’re going to succeed or fail, you don’t know what the answers are, and that’s part of what research is,” said Holm. “You may find out you love it or hate it, but either way that’s a good outcome.”