Amazon reveals updated smart devices and software powered by Alexa LLM

For Amazon’s Fire TV, Alexa LLM will help improve how people search for movies and shows — or offer suggestions based what they’ve seen or want to see. The improved Alexa will also be in the new version of Amazon’s smart glasses, the Echo Frames. It’ll also power the newly unveiled Echo Hub, an 8-inch mountable touchscreen wall panel that can be used for controlling all the devices in a smart home. Other features will help translate calls, assist people with mobility and speech disabilities, set mood lighting, activate cameras, control air conditioning, and operate vacuum cleaners.

To personalize answers for each user, Amazon’s updated LLM also lets Alexa remember conversations and recall that information later and provide additional context for actions. Alexa also is getting improvements to its voice later this year so its timing and tone sound more dynamic. Although it still doesn’t sound human, it’s evolving to be less robot-like than when it debuted in 2014.

“What makes our LLM special is that it doesn’t tell you things, it actually does things and that’s incredibly hard,” Rohit Prasad, Amazon’s vp and head scientist for AI, said during a press briefing. “The LLM is tuned for voice, but it has thousands of devices and services connected to it and your personal context as in what do you watch, what do you listen to and what do you eat. And on top of that, a suite of responsible AI techniques that makes Alexa that trusted AI assistant and yet a fun personality.”

As part of the updates, Amazon will soon allow developers to integrate their custom-built LLMs with Alexa. For example, Character.AI will integrate with Alexa so people can choose to interact with dozens of virtual characters ranging from fitness coaches and fictional characters to AI versions of historic figures including Socrates and Grace Hopper. Other example Amazon mentioned is the AI music platform Splash, which will let users create songs with their own voice.

Amazon’s updates showcase how generative AI can help form “the connective tissue” across various technologies and AI applications, according to Forrester senior analyst Rowan Curran. By using an LLM to better understand user intent, it seems Amazon will be able to help Alexa navigate between and across tasks.

“A crucial aspect of this being more than a novelty is that the LLM isn’t just linked to other systems for information retrieval; it’s connected to outside systems and applications, allowing users to execute actions and therefore accomplish meaningful tasks,” Curran said. 

According to Curran, linking together speech-to-text improvements and connecting Amazon’s API architecture to outside systems will help Alexa LLM to understand user intent. To be able to “wrap everything in natural speech is where we can begin to see the future of how we will use this technology near-ubiquitously in our everyday lives,” he added.

Amazon’s moves are likely to raise big questions around data privacy. The updates also come just months after Amazon settled two privacy-related lawsuits with the Federal Trade Commission. Earlier this year, the company agreed to pay $5.8 million over violations related to its Ring doorbell and paid the FTC another $25 million to settle a separate lawsuit related to allegations that Alexa’s voice AI violated U.S. child privacy laws.

During Wednesday’s press event, Dave Limp, Amazon’s svp of devices and services, said “we believe there is absolutely no trade off between trust and performance.” New features like Map View will be opt-in and allow people to delete their floor plans. However, the potential use of data for advertising is likely going to be followed closely by marketers and scrutinized by privacy experts. (A recent report from Forrester mentioned how brands might use genAI in the near future to better anticipate what customers want through “watching and listening.”)

“As you might imagine, I have one of the most Alexa-fied homes out there and I would not bring anything into that environment to compromise my family’s privacy,” Limp said. “That’s why we’re gonna continue to give customers information and controls so that they can be in charge of their experience, even with generative AI powering this in the background.”

It’s too soon to tell if the updates will help accelerate smart home adoption — smart speaker sales have been slowing according to recent estimates — but updated LLMs could help make smart devices more useful and interactive.

“The smart speaker revolution that began a decade ago didn’t live up to the hype,” said Greg Kahn, president & CEO of GK Digital Ventures. “But things [are] suddenly different with the rise of generative artificial intelligence, which is becoming embedded in all industries and, to a certain extent, in most consumers’ daily lives … The question comes down to how quickly it can invigorate Alexa and its Echo products with genAI. This is the moment when we’ll see Amazon solidify its position there — or this could be the moment when another rival (Google) or an upstart races ahead.”