Computing Professor Makes Artificial Intelligence More Reliable

Artificial intelligence software used in everything from cancer research to vehicle navigation sometimes relies on unreliable data-sorting algorithms that can lead to serious real-world problems, but an NJIT expert is working on new ways to identify the causes and find possible solutions.

Many of the popular algorithms for data sorting, also known as clustering, are vital to how AI operates but can yield responses that are quite inconsistent — so if the algorithms can't be trusted, then nor can the sorting and the final answer, explained Iulian Neamtiu, a professor of computer science at Ying Wu College of Computing.

In addition to consistency, clustering algorithms can be deterministic or non-deterministic, which refers to whether the algorithm is expected to produce the same answer every time it's asked the same question based on the same set of data. Neamtiu focuses his clustering research on the consistency of deterministic algorithms.

Neamtiu received a $400,000, three-year grant from the National Science Foundation to further study this dilemma. His team plans to build an automated online system where people can test their AI algorithms to locate sources of inconsistency before releasing their code in the wild, because the current testing methods are labor-intensive.

"Following the same recipe, so to speak, that's supposed to lead to the same results, actually in practice leads to widely different results," he explained.

He used the example of asking AI software to design cars. The software must establish that each car has consistent locations for the accelerator, brakes, steering wheel and turn signals, and that those controls always perform the same function when applied. Drivers would be imperiled without those two expectations, and similarly AI software would be unreliable, potentially putting lives at risk.

This is bad. You wouldn't expect two desktop calculators to give you different answers.

The team performed initial tests with 528 data sets in common toolkits including Matlab, MLpack, R, and TensorFlow. They used three deterministic algorithm types known as affinity propagation, DBSCAN and hierarchical agglomerative clustering. They also examined the deterministic portion of a fourth algorithm called K-means.

"We have three published papers and another one in the works that expose such issues and show how developers could fix the issues," Neamtiu said. "We devised a simple yet effective approach called differential clustering that compares an implementation with itself, or with other implementations, to make sure they give the same answer on basic clustering tasks."

Building on those techniques, they also developed solutions that include eliminating sources of non-determinism, such as the common addition of noise — small amounts of random data to help the algorithm cluster the results — which is dangerous, akin to "throwing some black ice onto the road to see if the driver is alert," he joked. Another possible solution pauses the algorithm after each round of cluster refinement and then essentially X-rays the results to look for signs that inconsistencies are evolving.

For now, Neamtiu's team is limiting their research to software-based AI systems. Later in the grant period, their research scope will expand to hardware-accelerated AI. Hardware versions of AI processors work much faster and more efficiently than their software counterparts because they're specifically designed for AI tasks, he noted.

"We're crashing programs and we're pounding algorithms," Neamtiu said. "This is bad. You wouldn't expect two desktop calculators to give you different answers."