Feet beyond fiction

Harry Potter’s enchanted Marauder’s Map allows the titular character to see every individual in Hogwarts, displaying their name and exact location. Carnegie Mellon’s Pei Zhang and Stanford University’s Hae Young Noh have demonstrated a new system that uses floor vibrations to match this magical feat—and much, much more.

In lieu of spells or charms, the pair’s “Marauder’s Map” relies on vibrations transmitted through the floor of a structure. Devices deployed on the ground pick up the vibrations, and machine learning algorithms filter out background noise created by other objects and machinery. This essentially turns the entire structure into one gigantic sensor platform.

With this information, Zhang and Noh are able to treat an individual’s footprint almost like a fingerprint, using their footsteps and gait to determine their identity and location. Based on the way they are walking, the researchers are also working to derive things like the individual’s size, health status, and even their state of mind—surpassing even their magical inspiration.

Zhang, an associate professor of electrical and computer engineering, has extensive experience in sensing and using machine learning to interpret sensor data. Noh, formerly an associate professor of civil and environmental engineering at Carnegie Mellon, is a structural engineer focusing on understanding how vibrations are transferred through the building. Their unique collaboration is the driving force bringing this idea from the pages of children’s fantasy into the modern world.

The possible applications for their work are multiplying with every new piece of information they’re able to pry from their test participants’ foot—or hoof—prints.

Agricultural implementation is already allowing farmers in Thailand to monitor the state of their livestock. Pig farmers testing the sensors are able to monitor the location and movement of their animals, increasing efficiency and allowing them to identify sick animals. This also reduces the need for widespread application of antibiotics, which is costly and promotes dangerous antibiotic resistance.

The greatest opportunities the team has identified so far lie in health and medicine. Their sensors are aiding doctors in tracking the progression of neurological disease in populations that have difficulty accessing medical care. Changes in a person’s movement or regular activities can provide an early warning of a heart attack, stroke, or whether they may be about to experience a fall. Vibrations passing from countertops into the floor can even indicate whether a nurse or doctor has washed their hands.

As their technology continues to improve, Zhang and Noh hope to be able to learn information like an individual’s mood or affect. Meanwhile, sensor platforms with a similar capability to determine location and identity are being tested in AR, VR, HVAC control, and other emerging fields.

Famed science fiction writer Arthur C. Clarke once said, “any sufficiently advanced technology is indistinguishable from magic.” Though their technology is at the cutting edge of innovation, in this case Noh and Zhang’s work bears one important distinction from magic: they’re going beyond.