Despite its rapid integration in nearly every facet of daily life, AI technology is not perfect. While it might seem like generative AI knows everything, it can make mistakes, or make up information entirely. That’s why it’s concerning to tech reporters like myself that companies are adding AI tech to tool so many people rely on and take for granted.
Search is one of those tools. Since the late 90s, we’ve been conditioned to rely on search results to find the information we’re looking for. For many of us, that means loading up Google, entering a search, and accepting the first page of results, sometimes the first result or two alone. Now, with Google’s AI Overviews taking over the top of most search results, many of us simply glance at the AI-generated result and take it at face value.
There are many issues with this new approach, but there’s one key issue to focus on today: As reported by Digital Trends, when you search for a company’s phone number, Google’s AI Overviews and even AI Mode might recommend a scammer’s phone number instead.
Scammers are “hacking” Google’s AI
Digital Trends highlights four examples of this situation in action. First, there’s Alex Rivlin, who posted on Facebook about his experience of trying to contact Royal Caribbean’s customer experience. Rivlin wanted to book a shuttle through the service, but couldn’t find the company’s support number on their website. So, like many of us, Rivlin googled “royal caribbean customer service phone number 24 hours usa,” and called the number that appeared in the AI Overview.
When Rivlin called, the “customer service” experience seemed above board, and the “rep” was very knowledgeable. Rivlin provided his credit card information to pay for the shuttle, but was concerned once the rep started asking for his date of birth. Since Royal Caribbean already had that info, it seemed suspicious, so after hanging up the phone, Rivlin checked his credit card statement, and noticed a charge from a foreign company he’d never dealt with before. After that, he noticed a small charge to the American Cancer Society, and called the credit card company to cancel the card.
As of this article, if you ask Google who that spam number belongs to, the automated result (not even the AI Overview, mind you) will say Royal Caribbean. That’s pulling from a website that appears to be impersonating the official simpler.grants.gov site. If you click the link, the page is dead, but Google is still pulling the information that existed while the site was still up. Based on this, it appears scammers are listing fake numbers on fraudulent websites, and tricking Google into sourcing that data. Google’s AI then sees “Royal Caribbean” next to this phone number on a .gov site, thinks it’s legit, and surfaces it in an AI-generated result.
Digital Trends then highlighted this example from a Redditor posting to r/ScamNumbers. The Redditor was googling “how to fix a misspelled name on Southwest,” which lead them to an AI Overview result with a scam number. This user appears to have either already known the legitimate Southwest number, or perhaps located the real number to compare to the one from this result, and saw through the scam without calling.
If you google the phony phone number, you can see a link to “Document360,” as well as the following snippet: “To correct a passenger’s name or Change Name on an Southwest Airlines Flight Ticket, reach out to Southwest Airlines customer support. Call +1-855-234-9795.” This is, again, a tactic to trick Google into presenting the scam number in its results. This time, the scammers are phishing for users searching for this specific issue, which increases the chances Google will deliver this result for this specific search.
What do you think so far?
Digital Trends also highlights a since-deleted post on Reddit, one user came close to getting scammed when googling the customer support number for a food delivery service. Similarly, in a final example, one man lost over $3,400 when he called the number that appeared for the food delivery service Swiggy’s.
Don’t assume the AI answer is correct
Google’s AI Overviews isn’t malicious; rather, it’s flawed. The underlying tech can struggle to distinguish between legitimate and false information. It lacks the awareness to understand that a site impersonating a government page can host a number and claim it to be a certain business: To the AI, that means that’s the business’ number, and, as such, it includes it in the results. It’s the same flaw that lead to Google’s disastrous rollout for AI Overviews last year. Back then, the model was even worse, pulling jokes from Reddit as legitimate sources. (No, glue does not actually make your cheese stick to your pizza.)
That’s not to say that the AI answer will always be wrong. The technology can still pull from high-quality sources and return results that are accurate. But there are flaws here that make AI answers too risky to count on. If you’d rather not scroll down to check out the traditional list of links yourself, at least click through the AI answer’s sources to see where it’s pulling the information from. If the source is sketchy, assume the answer is too.
When it comes to company contact information, I’d recommend always going directly to the source. If the company doesn’t list the phone number on their website, assume it doesn’t exist, and try to find a different contact method directly through the company. Scammers are too clever to rely on the open web for this information.