Duke Engineers Developing Technology Underlying Brain-Machine Interfaces
Biomedical engineers at Duke's Pratt School of Engineering are developing much of the basic technology behind Duke experiments aiming to enable primates and ultimately humans to operate machines exclusively with their brain signals. Their efforts include custom engineering of interface devices, programming of "neural net" computer systems and extensive computer analysis.
"The issue here is really the challenge of developing technology and also understanding how the brain works," said Craig Henriquez, the Pratt School's W.H. Gardner Jr. Associate Professor of Biomedical Engineering, in an interview.
In October 2003, Henriquez was among a team of Duke researchers, led by neurobiology professor Miguel Nicolelis, that announced it had used advanced electrode and computer systems to allow rhesus monkeys to consciously control the movements of a robot arm using only signals from their brains and visual feedback from a video screen.
In a new report to be published in the July 2004 issue of the journal Neurosurgery, neurosurgeon Dennis Turner, Nicolelis and other Duke neurobiologists and neurosurgeons described how they recorded analogous neural activity in humans.
Those human volunteer Parkinson's disease patients gripped pressure bulbs to demonstrate that techniques found to work in the monkeys might also work in humans.
Henriquez said he expects that the analysis and models being developed by his group should also apply to humans. Henriquez, a computer modeler and computational electrobiologist who is also an associate professor of computer science, co-directs with Nicolelis Duke's Center for Neuroengineering, which aims to advance methods of recording and analyzing the signals emanating from the brain's nerve cells, called neurons.
Henriquez is also a co-principal investigator, along with Pratt School associate professor of biomedical engineering Patrick Wolf and other investigators at and outside Duke, in a $26 million project to develop such human-brain interfaces. That Defense Advanced Research Projects Agency (DARPA)- funded project is led by Nicolelis.
While Nicolelis led the landmark first study with rhesus monkeys, Henriquez, Wolf and their engineering students were working to advance the methods used to monitor, record and analyze the animals' complex neural signals.
An early challenge, said Henriquez, was devising a recording system that could handle the unprecedented number of microelectrodes connecting an unusually large number of nerve cells. Henriquez said members of his engineering center worked with Plexon, Inc., a Dallas company that builds data acquisition systems for the brain to create a custom recording system that could push the limits of microelectrode numbers from 128 to 528.
The experiments initially required the monkeys to learn to use and grasp a joystick to superimpose a cursor over a target on a video screen. At the same time, the researchers had to analyze the faint nerve cell outputs to match particular neural signals with particular movements of the animals' hands and arms.
After tying specific limb movements to specific neural signals, the researchers used those signals to guide the motor controls that directed a mechanical robot arm located in a different room than the monkeys. That way, the animals also inadvertently operated the robot arm when they worked the joystick.
When Nicolelis and his neurobiological collaborators then removed the joystick they found the animals continued to move their limbs in the air while nerve signals and the robot arm responded accordingly.
Most startlingly, the animals then learned to play with the robot without moving their limbs at all. Dropping their arms to their sides, they manipulated the robot with brain signals alone.
Accomplishing this feat provided a Herculean technical challenge of information processing and interpretation for Henriquez and fellow biomedical engineers. Various nerves and muscles in the animals' limbs were responding to various signals from the animals' brains. "But we didn't exactly know how," Henriquez said. "We had access to limited information.
"There is a technique in engineering called 'system identification' where you have an input and an output and you have to figure out what is in the middle," he continued. Using that system identification technique, the engineers created sets of computer rules -- called algorithms - that related inputs to outputs and mapped connections between the two.
The algorithms took the form of what are called neural networks that, analogous to the brain's own circuitry, "learn" to create associations between hard-to-decipher patterns.
In essence, neural networks "simulate brain activity," Henriquez said. "The monkey's arm is moving so you can keep track of the in-position or the velocity of the arm. You can relate the firing patterns of the neurons to that and then let the neural network figure out the relationship between input and output." Perhaps the most amazing finding, he added, was that the animal learned to use this relationship by modifying its own neuronal circuitry.
Analyzing weeks of recorded data required massive computing power. The engineers used a so-called Beowulf computer cluster which links numbers of desktop computers to achieve supercomputer-like performance. "We could get a month's worth of data analysis done in a day," he recalled.
Now a new medical team is gearing up to begin applying these innovations to humans in need, such as those with spinal cord injuries or lost limbs. The move beyond monkeys brings up a fresh set of engineering challenges, including moving to wireless operations and learning to understand the brain signaling patterns of a different species.
Investigators will also continue the monkey work as they move towards a DARPA study goal of being able to implant more than 1,000 tiny recording electrodes in brain nerve cells. Such a massive wiring effort aims at learning even more about how brain signaling works. But it "would not be amenable to human use," Henriquez said.
So the Pratt School's Wolf heads a group that is designing and building a miniaturized wireless device for humans "that mimics a lot of the capabilities of the system we use in monkeys," Henriquez said.
The advanced device would be implanted in patient's brain but be powered from outside the body. It would have to be robust enough to handle flows of wireless signals both exiting and entering the brain.
Signals leaving the brain would have to be "interpreted" by intermediate processing computers. The processed information would then go to the computers that drive a robot arm or other prosthetic devices on the other end. Signals representing something like "touch" would then have to flow back remotely to a patient's brain as they would if the patient's neural circuitry were undamaged.
Henriquez suspects one of the biggest ultimate challenges may be discerning how disabled patients who have lost their feeling can "sense" how much force to apply while using their brains to instruct a robot to grasp and lift a fork, a pen or a heavier object. A new father, Henriquez believes it would be helpful to study the way babies learn to do it.
Tasks like this are drawing in DARPA project participants not only from Duke but also from MIT, the State University of New York at Brooklyn, the University of Florida and the private Plexon, Inc. "We are not set up to build things that are robust forever," Henriquez cautioned about university research. "Eventually this is going to have to be taken over by a company."
As researchers advance from monkeys to people, they also face the shorter-term challenges of learning the differences between the ways primates and human brains process information. "Some parallel experiments will probably have to be done in monkeys as we learn what's possible and not possible with humans," he said.
Henriquez has moved to the brain after spending years applying his expertise in computational modeling and electrophysiology to the heart. He and collaborators in and outside Duke are still separately working to create a multi-scale computerized model of a mouse heart. The model allows researchers to simulate how changes at scales ranging from genes to the entire organ will affect the genesis of life-threatening heart arrhythmias.
"A lot of the heart research is mature, and a lot of the brain research, particularly in neural engineering, is immature," he said. "To be in an immature science that is just beginning to see the fruits is very exciting. We're getting so many requests for students to do research at the undergraduate and graduate level that we can't even manage it."
His own engineering group supervises around eight graduate students in a given year, one or two post doctoral investigators and anywhere between three and five undergraduates, he said.
"In the end, it's the students that really matter," he said. "They're the legacy of the program. If we're going to have a world-class research program we have to train world class students."