Sears department stores have largely disappeared across the United States, but the brand and its appliance repair service are still in business, complete with a modern twist: an AI chatbot and phone assistant named Samantha.
As the historic retailer steps into the future, though, new research shows that conversations people had with the chatbot were publicly exposed online.
Fowler says that when he disclosed the finding to Transformco, he received a reply from someone who claimed that they were connecting him directly with a Samantha AI Chatbot manager.
He says that individual never replied to him, though, even after a follow -up message.
The second shock came from the fact that a surprising number of the audio calls captured hours of ambient audio after customers apparently thought a call had ended.
Some of the recordings were up to four hours long.
It is unclear why customers left the calls running once they were done speaking to the Sears AI agent, but these extended recording sessions may have captured private conversations and sensitive details that Sears customers thought they were discussing privately as they went about their days.
“You could hear the TV playing, you could hear people having conversations, and this recorded all of it,” Fowler says.
The files also show people getting frustrated with glitchy chatbots, which sometimes failed to answer questions or also pushed people toward human customer service agents.
Just two minutes into one 76-minute audio call observed by Fowler, the person trying to get help from the company asks to speak to a human.
The AI voice bot responds: “I am fully equipped to address your needs efficiently and can resolve your issue right away.
Whereas connecting with a live agent may involve a short wait.” Just a few minutes later, the bot struggles to complete the task it is asked about.
“I am facing some errors while assisting you with your plan.
Can I transfer your call to our live agent who will help with your request?”
In one text transcript, which begins near 11 am and ends at 1:30 pm, a person speaking with the Samantha “AI virtual voice” grows increasingly frustrated with the replies.
“Where's my technician?” they repeat 28 times in a row.
After getting some more responses they were unhappy with, the transcript shows, the person repeats: “You’re a computer.
You’re a computer.
You’re a computer.”
The situation comes as companies continue to scramble to integrate generative AI into their technology stacks, and the exposures highlight the privacy, trust , and reputational risks of using bots to directly interact with customers.
Carissa Véliz, an author and associate professor at the University of Oxford, says that in some circumstances people may feel safer when talking to a machine.
“The machine, after all, will not want to rob your house,” she points out.
But she adds that people often have little choice about trusting companies with their sensitive information.
“They should also give people more choices: the choice to talk with a human being if they prefer it and the choice to not have their conversation recorded,” Véliz says.
“In the long run you want your customers to be safe and feel comfortable, not alienated and exploited.”
Related Stories
Source: This article was originally published by Wired
Read Full Original Article →
Comments (0)
No comments yet. Be the first to comment!
Leave a Comment