NJIT Expert Trains Robots to Use Their Hands, Earns NSF CAREER Grant

C-3PO walked upright and spoke six million languages, but never did much with his hands besides letting the wookie win at holochess.

Such is the state of modern robotics and cyberintelligence, where a Boston Dynamics droid does backflips and Apple Siri maintains natural-language conversations, while precision manipulation of physical objects remains elusive, explained Cong Wang, associate professor of electrical and computer engineering at New Jersey Institute of Technology.

Wang's plan to fix this involves a two-fingered robot, crowdsourcing, artificial intelligence and a National Science Foundation grant of $560,000 as part of the agency's prestigious early-career awards program.

"It's what I call robot physical intelligence," Wang said. "For robots to coexist with humans, they don't really need to be super smart. They don't need to have the intelligence of being the world champion in the 'Go' game."

Oxen, dogs and horses certainly aren't beating Jeopardy or becoming chess grand masters, but they're incredibly useful to humans — that's what we should aim for in robotics, Wang asserted. "To handle a lot of everyday work, you don't need to be an expert of all the truth in the universe. But instead you need to be able to move your body, use your hands in a very ideal manner."

Wang is focusing on theoretical work until the pandemic subsides and students settle back into his laboratory this fall, then he'll finish what he calls the chopstick robot. Two electromechanical fingers are enough to do many jobs — one finger to serve as a brace and platform, the other to slide forward/backward and move sideways or vertically.

"You would be amazed at how far people can go with chopsticks. Maybe a bad example, but I've seen videos of thieves picking pockets using chopsticks. That's proof of how the neuromotor system of human beings can make use of something so simple," he said. Software designed in a similar manner taught a previous robot of Wang's to solve a puzzle game, as shown in the video below.

Wang will connect his chopbot to the Amazon Mechanical Turk crowdsourced task system, which lets remote volunteers perform minor tasks. Their task will be to manipulate the robotic hand against simple games and object pick-ups.

Each finger on the hand will have sensors to record movement data such as depth, direction, pressure and speed. When the research is done, all movements will be fed into an artificial intelligence processor, in an attempt to teach the robot which manipulations worked and which didn't. Potentially, the robot will learn from these actions and even develop its own manipulations that the human volunteers hadn't tried. In that case a robot will teach us new tricks.

Wang's use of crowdsourcing is out of respect for evolution. He could just as easily recruit an army of NJIT students to teach the robot in person, but he's aiming for scale and scope.

"My philosophy on this topic is there's no way that one scholar can lead a few Ph.D. students, crowd around their computers for a few semesters or a couple of years, then create a physically intelligent being. I don't think so because humans evolved into humans over a very long time. I think we should have respect to Mother Nature to fairly estimate the relative capability between us and how good, how wild nature can be.

"That being said, I think it's going to take a longer shot. It's going to take a non-trivial evolution, a process for robots to gradually develop a physical intelligence ... That process I think can be accelerated by mentoring from humans."