NJIT and New Jersey Symphony Orchestra To Bring Classical Music into the Digital Age

Music lovers know that there is nothing like a live performance, but the COVID-19 pandemic inspired Newark's New Jersey Symphony Orchestra to challenge Ying Wu College of Computing faculty and students to investigate better ways to present virtual concerts using mixed-reality technology.

The orchestra's goal is to find better solutions than early-pandemic attempts at virtual live concerts, where each musician simultaneously performed their role from a safe location through a virtual meeting system such as Zoom, leading to underwhelming results because of network latency and lack of acceptable fidelity.

Even beyond the pandemic, rather than view the limitations of existing technologies as an obstacle to the presentation of high-quality musical experiences, there could be new opportunities to deploy emerging technologies in the digital age. Such research could also create new opportunities for artist collaboration and new forms of audience engagement.

Ideally, NJIT researchers will determine the musical applications of augmented- and virtual-reality technology, largely under the direction of Associate Professor Jacob Chakareski. He specializes in applying AR/VR to real-world problems, such as a recent project of helping teach vision-impaired people to cope outside of controlled environments.

Chakareski said an example of the new musical applications could include audio streaming algorithms that better facilitate live, multi-location performances without the latency and quality issues common on current platforms. Another example could be interactive environments that change or move with the music and movements of the musicians. He's looking forward to seeing what other ideas NJIT researchers devise, and said he welcomes proposals by June 1 at jacobcha@njit.edu.

“This is not only a timely challenge we are looking to solve, but also one that New Jersey Symphony Orchestra hopes will reinvigorate its followers while also bringing its music to a younger audience by making it more appealing and easier to engage with,” he said. “To be able to connect musicians from around the world and give them a means to collaborate and perform together without the limitations of being in different locations is very exciting. I look forward to experiencing the creative output of this effort.”

Phil Neches, an orchestra trustee who is also the chair of the Ying Wu College of Computing Advisory Board, saw this as an ideal opportunity to bring the two organizations together and helped make the connection. "This initiative is a way to get NJIT to come up with clever and creative ways to make the digital delivery of music a better experience for the audience and more natural for the performers,” said Neches, who is also helping fund the projects.

“During the pandemic, the New Jersey Symphony Orchestra, like many organizations, has relied on offering digital experiences to keep our musicians connected with its audiences,” noted Gabriel van Aalst, president and CEO. “With the return of in-person live performances coming soon, we need to continue pushing the boundaries of what digital technology can do to share our musicians’ performances across New Jersey and beyond in new ways. We are excited to partner with NJIT and learn what its brilliant computing faculty and students can create. Exploring how live performances and digital experiences can potentially intersect is a chance for innovation and we look forward to seeing what’s possible."