Automating engineering’s ideal manager

Engineering is a collaborative practice, but effective teamwork can take many different forms. For some projects, teamwork means accomplishing all tasks as a group; for other projects, it is better for everyone to work individually before connecting their pieces like a puzzle.

Jon Cagan, a professor of mechanical engineering, and Chris McComb, an associate professor of mechanical engineering, have long studied the engineering design process with Ken Kotovsky, a professor of psychology at Carnegie Mellon University. They have shown that artificial intelligence (AI) can be beneficial to design teams; their previous work found that having AI as an assistive tool can make human teams more efficient and effective.

Josh Gyory, who earned his Ph.D. in mechanical engineering from CMU in summer 2021, has studied if and how process managers improve team performance. He found that having these managers, who oversee the team’s progress instead of their output, makes the team more efficient.

To use AI as more than just an assistive tool, Gyory worked with Cagan, McComb, and Kotovsky to integrate AI as a true member of the team. They designed an AI agent to oversee the problem-solving processes of engineering teams in real-time and, in a new study, compared the performance of teams managed by the agent to those managed by humans.

Their results, recently published in the Journal of Mechanical Design, suggest that AI managers perform at least as well as and are even a bit more adaptable than human managers. That is, they are able to manage the design process both efficiently and effectively.

Gyory’s research began with a human study to understand the strategies of managers who focus on the problem solving process. He found that human managers tended to emphasize communication among team members, and he came up with a set of interventions that reflected those observations. Gyory then trained an AI agent to intake and analyze information about team behaviors and choose interventions based on the team’s communications (how they think) and actions (what they do) as needed.

Artificial intelligence is getting smarter and smarter and is able to track multiple team measures at once and over time. By letting the AI agent manage the process, humans can focus on the design of the solution.

Josh Gyory, Postdoctoral Researcher, MechE

“I think automating process management is beneficial for several reasons,” Gyory said. “Artificial intelligence is getting smarter and smarter and is able to track multiple team measures at once and over time. By letting the AI agent manage the process, humans can focus on the design of the solution.”

Gyory had teams, working remotely in two sessions, solve an engineering design problem—designing drones and delivery paths to maximize their teams’ profit. In the second session, the researchers changed the design problem to test the team and manager’s adaptability. Half of the teams were managed by humans, while the other half were managed by the AI agent.

Both the human and AI managers supervised their team’s communication and decision making during the design process. At predetermined time intervals, the managers had the opportunity to choose from the set of interventions, including not to intervene. By the end, AI managers intervened more often than the human managers, but the AI used only a subset of the interventions, whereas the humans used them all, though not necessarily in a beneficial way. When analyzing team performance, the researchers found cases where an intervention negatively impacted team performance; this was more common among human managers than AI managers. By comparing the output of teams managed by humans and AI, the researchers found that AI can be an effective manager.

“The artificial intelligence agent, in real time, is able to manage the process of a team of humans solving complex problems as well as a human manages that process,” Cagan said. “It can even do a bit better, especially during the time when the teams need to be adaptable.”

Further research would expand the range of interventions AI managers could use. The human managers in the study reported feeling constrained by having to choose from the given set of interventions. They also wanted to provide positive reinforcement to boost team morale.

The large-scale teams required to complete complex engineering projects are increasingly global in nature. Distributed teams are one of many applications for this study.

“We were able to run the experiment within constraints that are reflective of the way that engineers work,” McComb said. “In the world we live in today, with remote work and remote collaboration, that's how many teams function.”