NJIT Develops New Computer Memory Control to Speed Up Cloud Servers

Virtual machines are an important kind of software, invented in the 1960s but only popular since the 2000s, that enables servers to run several operating systems simultaneously — but lately they're facing an efficiency problem that NJIT professor Xiaoning Ding believes he can address.

The majority of servers are now virtualized, especially those in the cloud, because it saves companies money and reduces data center concerns such as power and heat. The problem is that new high-end servers have more than one type of memory, and the standard management techniques don't do well at arranging and assigning that memory to virtual machines, so anyone who improves that process can ultimately speed up computer networks everywhere.

"Memory interfaces in virtual machines are flat, uniform, and homogeneous. Thus, it cannot correctly reflect the real structure of memory devices in computers, which is hierarchical, non-uniform, and heterogeneous. Our research is to create a memory interface that can more faithfully reflect the real structure of memory devices," Ding explained. The research for Ding and his graduate students is funded by a $297,664 grant from the National Science Foundation.

Traditional servers use dynamic RAM. Modern high-servers also use non-volatile RAM and cache memory. Non-volatile RAM can read and write data as fast as traditional memory, but it can also remember data as a hard drive does when the computer is turned off, whereas regular DRAM is erased when power is removed. It's similar to flash memory, which is what's used in mobile devices and solid-state hard drives, but flash is usually slower than computer RAM. A compromise in NVRAM is the slow speed of writing data into it. Meanwhile, cache is a small-capacity memory device with extremely high speed. In new servers, cache capacity increases to a few hundred megabytes, as compared to just a few megabytes in older servers.

Tripling the types of memory that must be shared across virtual machines is a much steeper challenge for hypervisors, which is the class of software that administers virtual machines. Ding explained that there is constantly a balance between maximizing a server's use of memory resources and minimizing competition for those resources between virtual machines.

"Under-utilization happens when providing active virtual machines with only small partitions, and some resources do not belong to any partitions and are not utilized. Contention happens when active virtual machines need large partitions, the total size of which exceeds the resources on the memory devices, thus each virtual machine cannot get enough resources. Thus, both under-utilization and contention lead to low performance," Ding wrote.

Ding's focus is to improve the mechanisms by which virtual machines share computer memory. He wants to significantly extend existing memory-management mechanisms in ways such as accurately predicting sizes needed by virtual machines, adjusting partition sizes as needed, and activating virtual machines that can better share the memory resources. Another idea applies to partitions, which are the software walls between virtual machines on a hard drive. Two ways to do that are compression and data deduplication — the former a standard practice that makes chuncks of information smaller, the latter a common way of minimizing redundant data to avoid wasting resources — which in turn would reduce the required partition sizes, so they each need less memory in the first place.

The project is nearly finished, Ding said. His team plans to finish testing by summer 2020, after which a paper will be submitted to computer science conferences and possibly journals. The code will be published with an open-source license.