Being able to beam yourself into a robotic body has all kinds of applications, from the practical to the fanciful. Existing interfaces that could make this possible tend to be bulky, but a wireless electronic skin made by Chinese researchers promises far more natural control.
While intelligent robots may one day be able to match humans’ dexterity and adaptability, they still struggle to carry out many of the tasks we’d like them to be able to do. In the meantime, many believe that creating ways for humans to teleoperate robotic bodies could be a useful halfway house.
The approach could be particularly useful for scenarios that are hazardous for humans yet still beyond the capabilities of autonomous robots. For instance, bomb disposal or radioactive waste cleanup, or more topically, medical professionals treating highly infectious patients.
While remote-controlled robots already exist, being able to control them through natural body movements could make the experience far more intuitive. It could also be crucial for developing practical robotic exoskeletons and better prosthetics, and even make it possible to create immersive entertainment experiences where users take control of a robotic body.
While solutions exist for translating human movement into signals for robots, it typically involves the use of cumbersome equipment that the user has to wear or complicated computer vision systems.
Now, a team of researchers from China has created a flexible electronic skin packed with sensors, wireless transmitters, and tiny vibrating magnets that can provide haptic feedback to the user. By attaching these patches to various parts of the body like the hand, forearm, or knee, the system can record the user’s movements and transmit them to robotic devices.
The research, described in a paper published in Science Advances, builds on rapid advances in flexible electronics in recent years, but its major contribution is packing many components into a compact, powerful, and user-friendly package.
The system’s sensors rely on piezoresistive materials, whose electrical resistance changes when subjected to mechanical stress. This allows them to act as bending sensors, so when the patches are attached to a user’s joint the change in resistance corresponds to the angle at which it is bent.
These sensors are connected to a central microcontroller via wiggly copper wires that wave up and down in a snake-like fashion. This zigzag pattern allows the wires to easily expand when stretched or bent, preventing them from breaking under stress. The voltage signals from the sensors are then processed and transmitted via Bluetooth, either directly to a nearby robotic device or a computer, which can then pass them on via a local network or the internet.
Crucially, the researchers have also built in a feedback system. The same piezoresistive sensors can be attached to parts of the robotic device, for instance on the fingertips where they can act as pressure sensors.
Signals from these sensors are transmitted to the electronic skin, where they are used to control tiny magnets that vibrate at different frequencies depending on how much pressure was applied. The researchers showed that humans controlling a robotic hand could use the feedback to distinguish between cubes of rubber with varying levels of hardness.
Importantly, the response time for the feedback signals was as low as 4 microseconds while operating directly over Bluetooth and just 350 microseconds operating over a local Wi-Fi network, which is below the 550 microseconds it takes for humans to react to tactile stimuli. Transmitting the signals over the internet led to considerably longer response times, though—between 30 and 50 milliseconds.
Nonetheless, the researchers showed that by combining different configurations of patches with visual feedback from VR goggles, human users could control a remote-controlled car with their fingers, use a robotic arm to carry out a COVID swab test, and even get a basic humanoid robot to walk, squat, clean a room, and help nurse a patient.
The patches are powered by an onboard lithium-ion battery that provides enough juice for all of its haptic feedback devices to operate continuously at full power for more than an hour. In standby mode it can last for nearly two weeks, and the device’s copper wires can even act as an antenna to wirelessly recharge the battery.
Inevitably, the system will still require considerable finessing before it can be used in real-world settings. But its impressive capabilities and neat design suggest that unobtrusive flexible sensors that could let us remotely control robots might not be too far away.
