EDUCAUSE 2024: Industry Partnerships Bring Large Language Models to Life

As universities turn to artificial intelligence to maximize efficiencies, large language models (LLMs) are pivotal tools for personalizing student experiences, optimizing administrative operations and advancing academic research.

In a “lunch and learn” session at the 2024 EDUCAUSE Annual Conference, experts from NVIDIA, NetApp and CDW discussed their areas of expertise and how the organizations can work together to create generative AI solutions in increasingly complex and sensitive higher education environments.

NVIDIA is building digital assistants to streamline processes on the front and back ends of operations. NVIDIA NeMo is a platform that enables organizations to develop custom generative AI to meet their needs. One example that Mark West, manager for higher education research business at NVIDIA, gave is ChipNeMo, a digital assistant created for semiconductor design.

“They’ve put all of our engineering documents and information into the AI model and created this chatbot,” he said. “Now, engineers can go quickly find the information that they need. But also, as personnel changes, you haven’t lost that institutional technology intelligence and that knowledge.” NVIDIA also creates bots that allow employees to get information about internal processes, troubleshoot IT issues and access public financial information.

West explained the concept of retrieval-augmented generation, commonly referred to as RAG, which is how large language models get their information. NVIDIA published a paper outlining its FACTS framework, which is the company’s holistic approach to building RAG-based chatbots. The framework addresses the freshness, architecture, cost, testing and security elements of building an enterprise-level chatbot.

This process can be replicated in higher ed, West said. The University of Michigan gives all students and staff access to U-M GPT, which has enterprise-specific data unique to the university. Another tool, U-M Maizey, allows users to create a custom GPT experience.

While NetApp is typically known as a data storage company, it also has a hand in generative AI, said Matt Lawson, director of solution engineering at the company.

“The new gold rush is data,” he said. “Data on your institutions is one of your most valuable assets that you have to manage. Your AI strategy is really a subset of your data strategy, because data is foundational to being able to do AI.”

NetApp helps organizations, including higher education institutions, organize and manage their data in an intelligent way.

“Once organizations kind of view intelligent data infrastructure as their approach to managing data, it helps them elevate their approach to data management beyond just storing data to unlocking data to its full potential to achieve the mission and objectives of the organization, including unlocking AI,” Lawson said.

Cost and security are major factors in how higher education institutions choose their data management strategies, and Lawson said NetApp helps colleges and universities select “the right technology for the right workload at the right cost,” all with a layered security approach.

“One of the key things about AI projects is the ability to move data seamlessly,” he said. “It’s also the ability to have traceability, so as data gets exposed to generative AI, you know what data was used to train the model.”

Creating an intelligent and secure generative AI platform requires a holistic approach, which is often made possible through industry partnerships. NVIDIA and NetApp work together: NetApp provides the infrastructure to deploy NVIDIA’s intelligent chatbots, which enhances data traceability. 

“You can actually see where that data is coming from when a user gets an answer back from the LLM, or you can ensure that when you’re incorporating your private data into the responses of the LLM, you’re not exposing your private data to the big, bad internet of the world,” Lawson said.

CDW can act as a partner to bring all parties together with additional expertise in a space that many IT professionals are still trying to figure out, said Bryan Scott, senior business development manager in CDW’s Digital Velocity group. Within this group, CDW has developed an AI Center of Excellence that combines expertise from CDW partners and internal experts and helps organizations develop their own AI environments.

In education, Scott said, scalability is vital to meeting the needs of ever-growing environments, and these partnerships can meet the complex needs of institutions of all sizes.

“It’s going to be critical to have the infrastructure to continue to support the complexity as it increases,” Scott said. “When we work together, we’re allowed to ensure secure and compliant data so that institutions, regardless of size, can benefit from AI innovation without prohibitive costs or variables.”

As generative AI evolves, its potential use cases in higher education continue to grow. Panelists were asked how they see the role of AI evolving over the next few years and what institutions can do now to prepare. 

All panelists agreed that generative AI will have an impact on teaching and learning, providing more personalized experience for students.

“I think the opportunity is huge in terms of student success,” Lawson said. “How do we embrace this as a tool to go further faster in education, and to further student success and help them achieve?”

Scott said that achieving the personalized educational experiences that AI can provide is incumbent on having the right infrastructure in place to support the technology.

“Institutions really need to invest in the infrastructure necessary to support the AI applications now,” he said. “There are hardware needs and there’s going to be some software needs. When we’re able to work together, we can put together the full set of solutions — whether it be infrastructure needs or data management — to ensure that institutions maximize their investments. But it needs to start now.”