AWS re:Invent: 3 Vital Themes from the 2024 Event

With AWS re:invent 2024, the Amazon Web Services show for developers, analysts, journalists and others, in the rearview mirror, one thing is clear: Artificial intelligence, especially generative AI, is the tech on the minds of virtually all of the Las Vegas event’s 60,000 attendees and was the topic of most of its informational sessions.

For AWS, which is the largest cloud platform provider in the world by market share, the question is not whether AI is important but how it can help its customers use AI to innovate on its platform. Some organizations are considering returning to owned data centers as a way of finding the computing power needed to run AI workloads. A few themes from the show stand out:

The time it takes a large language model such as Chat GPT to answer user prompts is called its “time to token,” and keeping that time short requires massive computing power, said David Wharton, CDW’s chief architect for the AWS platform. One big question for cloud platforms is whether they can offer the combination of power and services that businesses need to run such workloads cost-effectively.

AWS CEO Matt Garman insisted that his company can. “Today, by far the biggest compute problem out there is generative AI, and the vast majority of generative AI workloads today run on NVIDIA,” Garman said during his keynote address, referring to the microprocessor maker know for graphics processing units that are powerful enough to handle AI workloads. “AWS is by far the best place to run GPU workloads, and part of the reason is that AWS and NVIDIA have been partnering for 14 years.”

Garman announced that AWS will release new virtual servers, or “instances,” that run NVIDIA GPUs as well as those produced by its chief rival, AMD. He also said that the latest version of AWS’s own AI microchip, Trainium 2, and said it was already working on Trainium 3. AWS started developing its own AI chips several years ago, with the long-term goal of providing everything a business, no matter its size, needs to run all of its AI workloads without it having to build its own data centers.

It launched its first Trainium chips in 2022. “We knew it was early, the software was early and it wasn’t going to be perfect for every workload,” Garman said. “But we saw enough interest that we knew we were on the right track.”

Companies have spent a lot of time trying to think up novel ways to make use of generative AI’s capabilities, Wharton said, but the best use case in these early days is to simply improve on existing processes. 

“It’s really about saving time,” he said. “We find that the best uses are to solve existing problems or to streamline old processes. If you have a workflow that takes hours, days or even weeks, you can cut that down by implementing AI into that workflow.”

There are many examples. AI-infused call centers can help agents quickly fill out paperwork or can provide access to resources that quickly answer customers’ questions. AI can rapidly produce documentation, write product descriptions, or summarize key points from meetings and create next-step action items. These are all tasks that take humans much more time to perform on their own.

“There are a lot of active use cases for your current problems,” Wharton said he tells businesses. “Let’s solve those first, instead of necessarily thinking of net-new ideas for what to do with AI. People are getting a little more realistic about what they can do with AI. The hype curve is starting to crash, but the reality curve is starting to plateau, and that’s where the opportunity is.”

AWS speakers touted the company’s new Amazon Q AI tool, which helps organizations with just those kinds of tasks. Dilip Kumar, vice president of Amazon Q Business, predicted a “not so distant” future in which “80 % of the work that you’re doing is automated, and the other 20% is 10x quality.” Asana used Amazon Q to build AI into its well-known project management platform, allowing users to access the unstructured data that lives in users’ emails, instant messaging and other communication tools, according to Chief Product Officer Alex Hood.

While the promise of AI is clear, the challenge is twofold. First, how can companies cost-effectively access the necessary GPUs to power AI workloads? Second, how can they go about getting their data into a state where it can make use of AI?

Wharton said that data hygiene is probably the biggest obstacle that most organizations have to AI success. “You need good data, and you need compliant data,” he said. “It’s not enough that you have the data. What if you accidentally drop payment card information into your AI model, for example? That would not be good.” In addition, he said, “you have to have your data lakes structured properly so as to control costs. You need data to be accessible and integrated.”

That’s why businesses need the help of an experienced guide as they plot their AI course: to help them set achievable objectives with their AI projects, select the right technologies to support those objectives (be they on-premises, in the cloud or both), and form and act upon a data strategy that allows them to cost-effectively capitalize on AI’s promise.