AI, Relying on Hardware Support, Could Improve by Thinking for Itself

People keep finding novel uses for generative artificial intelligence, the latest being that it can learn to design specialized hardware to make itself work faster.

Generative AI applications such as large language models became mainstream when ChatGPT went viral in 2022, but they require copious, complicated hardware underneath their user-friendly skins, especially when asked to act on more than just interactive text.

“Specialized [hardware] accelerators are crucial for maximizing the potential of AI tools, but current design tools’ complexity and required hardware expertise hinder innovation,” explained Arnob Ghosh, assistant professor in New Jersey Institute of Technology’s Electrical and Computer Engineering department.

Ghosh, along with colleagues Shaahin Angizi and Abdallah Khreishah, had the meta-idea to tweak a large language model as their assistant. They thought of training it to learn the context of what’s needed when designing hardware acceleration, based on a user’s needs for accuracy, energy usage and speed.

“We are trying to provide the optimal context so that an LLM can generate the desired results. This is based on the idea that the LLM can indeed demonstrate in-context learning,” Ghosh said. “The challenging question is how can we do prompt optimization here. Some basic instructions might not work. The prompt must consist of some of the elements of the codes themselves so that we can provide the optimal context to the LLM."

Their ideas include providing some hand-crafted instructions for basic hardware designs so the LLM has a basis from which to extrapolate its own creations, fine-tuning the model’s parameters for specific tasks, and using Khreishah’s graphical neural network to simplify how much virtual thinking the model must perform.

The trio are each focusing on part of the problem. Ghosh is optimizing the prompts and writing code that lets a large language model think about how to develop circuits, Angizi is working on non-traditional computing architectures and Khreishah designs the representation learning, which refers to how AI decides the format for interpreting your commands.

Companies like AMD, IBM, Intel and Nvidia are all deeply involved in the AI business. IBM is developing its own hardware, while AMD and Nvidia are in the research stages, and secretive Apple is starting to send employees to relevant conferences, Angizi said.

The NJIT researchers already released their dataset and are now developing a prototype of the prompt optimization, funded by a faculty seed grant. They’re aiming to have their first framework complete by the end of the fall 2024 semester. It may require human intervention to know if the hardware design is good or bad.

Ultimately, “The world that we want to create is accelerator design with minimal human intervention. Perhaps some automated system can verify whether the codes or the designs are good or bad efficiently,” Ghosh stated. “These are exciting prospects, and perhaps we will see some new challenges that require innovations.”