NJIT Experts Lead $1.5M NSF Grant to Study Social Media Moderation

NJIT researchers received an $849,024 National Science Foundation grant to help invent ways of taming the wilderness that is online content moderation.

Anyone who interacts in online communities knows the pain, whether you commented on an article, calmed down friends arguing with each other on your Facebook wall, made a tough Yelp review or got censored by a forum manager. "Never read the comments" is a mantra of news writers in the 21st century, although we all secretly do.

Content moderation goes back to the dawn of the printing press, explained Yvette Wohn, director of the Social Interaction Lab in Ying Wu College of Computing’s informatics department. She said the modern problem is the vast scale and unpredictably dynamic nature of social content versus the small number of people who moderate it. Nobody knows how many moderators exist, but the figure is at least in the tens of thousands, she said.

"All of my research is related to people interacting through social media. Content moderation was kind of a byproduct of this research," Wohn said. Her first social media account was on a South Korean site called Cyworld in the late-1990s. Her NJIT colleague, professor Aritra Dasgupta who leads a data visualization research group, got his first taste of social media on a site called Orkut in India around the early 2000s. Wohn is principal investigator and Dasgupta is co-principal investigator for the project.

The new grant is part of a $1.5 million collaboration with the University of Michigan and University of California-Los Angeles. They'll use the funds primarily to pay tuition and stipends for doctoral students. Speaking for all parties involved in the collaboration, "First we want to build some tools to help them manage more things at once. Second we want to develop a way to reduce their stress and emotional fatigue," because moderators constantly see the worst sides of users, Wohn noted. "Third, in terms of developing these computational tools, that requires artificial intelligence and machine learning. We have to make sure the data are shown to moderators in a way they understand, which is where visualization will play a very important role."

They'll divide the work. Michigan researchers will develop the moderation software, UCLA will try to learn from professional moderators, and Wohn's team of graduate students will work with volunteer moderators. Dasgupta and his graduate students will develop visualization techniques that help moderators quickly understand and act upon recommendations of predictive models, and ultimately put malicious content and its creators in quarantine. Rice professor Elizabeth Petrick, formerly of NJIT, may contribute historical analysis.

It is vital to include people who are volunteer moderators from online communities because their ranks greatly outnumber professionals by many thousands, Wohn said. Also, even the best AI software only understands limited context, for example a human knows the difference between art and pornography but an algorithm only detects an image of someone nude. That leads to poorly designed software making absurd mistakes, such as blocking images from anatomy textbooks or Michelangelo's David.

Wohn and Dasgupta understand that no software or research can ever truly solve the problems of social media content moderation. So their goals are more realistic. "I think what is really meaningful about this grant, aside from all the research, is we are recognizing it's a problem and we are trying to do something about it. It shows where our values are as society. … There's going to be no complete solution, but we're putting in resources to think about how we can address this," Wohn explained. "Volunteer moderation allows for users to take more responsibility and we are building tools to help them take that responsibility," she observed.

Wohn also noted that the grant required applications to build teams from wide academic backgrounds. As such, her team includes an economist, a historian and a behavioral scientist, in addition to traditional computer scientists and informatics experts. A separate grant may enable research about how to facilitate better content rather than so much negative angles.

"It is an uphill battle," Wohn said, avoiding the word "solve" because of the gigantic size of the problem. "Just because you know you can't cure something doesn't mean you can't try."