ChatGPT + Romance Scams: A Match Made in Fraud Heaven

Romance Scammers’ Favorite Pickup Lines

The US Federal Trade Commission (FTC) surprised us with a gift just in time for Valentine’s Day. And they must have known that at Feedzai, we place a premium on data over heart-shaped chocolates 💝.

The FTC’s latest analysis found romance scams cost 70,000 US consumers $1.3 billion in losses in 2022 (up until Q3). What’s more, the FTC even broke down the most common pickup lines (or more accurately, pickup “lies”) that romance scammers use. These include:

• “I or someone close to me is sick, hurt, or in jail” – 24%
• “I can teach you how to invest” – 18%
• “I’m in the military far away” – 18%
• “I need help with an important delivery” – 18%

These tactics range from deception and emotional exploitation to sextortion tactics. And just imagine what a creative fraudster can do with new Generative AI tech at their fingertips. Actually, you don’t have to imagine, because here is what it does. 

I asked ChatGPT to write a convincing letter explaining that I’ve been in a car accident and have been undergoing treatment. 

ChatGPT did that, and it even asked the “girlfriend” for financial help to cover the cost of the fake treatment without my prompting it to do so. It proactively helped the scam! Here’s what it said:

“I am writing to ask for your help. I am in need of financial assistance to cover the costs of my medical treatment. I understand if this is too much to ask of you, but I am desperate and don’t know where else to turn.”

No wonder fraudsters are head over heels in love with Generative AI.

3 Ways Generative AI and ChatGPT Enable Romance Scams 

How do fraudsters love ChatGPT? Let me count the ways: 

1. ChatGPT makes it easier to overcome language barriers

When ChatGPT composes a love letter, it really knows how to pull the heartstrings. Case in point, I asked ChatGPT to write a romantic letter to myself:

Write a love letter to Pablo telling him how much I’ve fallen in love with him.

In just a few seconds, here’s how ChatGPT responded:

My dearest Pablo,

I cannot begin to express the depth of my feelings for you. From the moment I met you, I knew that you were special. Your kind heart and gentle spirit have captured my heart and soul. I have never felt this way before and it is both exhilarating and terrifying. 

I have fallen deeply in love with you, and every day I find myself thinking of you and longing to be near you. The way you make me laugh and the way you understand me like no one else has before. I admire your intelligence, your creativity, and your passion for life.

Being with you has brought me more joy and happiness than I ever thought possible. I am so grateful to have you in my life and I want to spend the rest of my days making you happy and loving you with all of my heart.

I know that we have only just begun this journey together, but I am convinced that you are the one I want to spend my life with. I will love you forever, my dear Pablo.

With all my heart,

(Your name)

If I didn’t know any better, I could swear that this love letter was written by a real person! It’s mostly free from grammatical or spelling errors that might give away that the letter was written by a non-native English speaker trying to push a scam. 

This capability solves a common problem for many romance scammers. Scammers are prone to make mistakes in their typing or grammar if English is not their native language. This can raise red flags for many targets and if they get pushed or asked for a response, they might walk away entirely. Generative AI like ChatGPT solves this issue by creating a letter that not only sounds sincere but is free from any obvious mistakes. They could segment a victim’s response and ask ChatGPT how the program would reply to a particular question. 

By fixing language barriers, ChatGPT only makes the fake romance seem more real.

2. Generative AI can create tailored images 

I’ve asked several of my friends who use online dating platforms how certain they are that they are interacting with a real person and not getting scammed. After all, there are plenty of stock images of attractive models on the internet that can be easily used in a scam. Usually, they tell me they ask for a picture with a unique request. For example, they could ask to see a picture of the other person holding a coffee mug, wearing a hat with a specific color, or making a specific hand gesture like a peace sign. 

But AI-generated art programs like DreamStudio, Stable Diffusion, DALL-E, and DALL-E 2 can also address this issue. These programs are capable of creating almost any image a user wants in a matter of seconds. These programs can create very convincing visuals of something that convincingly looks like a real person. 

If the fraudster can’t find an image that meets their victim’s request, the romance scam could fall apart. But once again Generative AI can help fraudsters deliver convincing visuals for their targets. Using DreamStudio, for example, I was able to create an image of a blonde woman with blue eyes wearing a baseball hat. 

Prompt: beautiful blonde model with blue eyes face. wearing baseball hat

I took the exchange further by specifying a popular baseball team’s logo. I was even able to specify which sports team (although its team logo replication wasn’t very good). 

Prompt: beautiful blonde model with blue eyes face. wearing red sox baseball hat

I further refined the prompt to: ”beautiful blonde model with blue eyes face. wearing red sox baseball hat left hand in view.” The results? As instructed, the image now shows the model with their left hand in the frame.

The created image still looked like an AI. But to someone not looking carefully, or who is willing to believe that the sparks are real – it can be convincing enough for a romance scam to work. Other programs can create audio recordings. Taken together, Generative AI is making it easier for fraudsters to create realistic-looking dating profiles for romance scams.

3. AI can perform romance scams at scale

Not only does Generative AI like ChatGPT make romance scams much more convincing. They can also be deployed at scale.

It used to be romance scams were performed by one person attempting to scam one or just a few victims at a time. But with APIs, romance scams can be blasted to millions of users and generate high-quality responses to victims. Using APIs even a low-level scammer with some development knowledge can conduct several romance scams simultaneously. What’s more, they could even automate them. Romance scammers now have the tools, motivation, and skills to launch high-quality and convincing catfishing campaigns. Or at the very least, Generative AI can create clear, engaging responses that can keep targets on the hook longer. 

4 Ways Banks Can Protect Customers from Generative AI Romance Scams

ChatGPT opens up a new chapter in romance scams. Banks will play a significant role in protecting their customers’ hearts and finances. While many banks – like the rest of us – are still assessing Generative AI’s potential, here are a few steps banks can take now. 

1. Educate customers on Generative AI’s risks

Banks should start by raising awareness with their customers of Generative AI and how it can be abused. This could be as simple as publishing an item in the bank’s newsletter, holding a workshop, or posting a blog about ChatGPT and what it means for bank customers. If your bank has plans to implement its own Generative AI program you could bring that up here. But raising awareness will show your customers that you understand the technology’s implications and want them to understand how they could encounter it in the future.

2. Monitor new accounts

If a romance scam is underway, scammers will create mule accounts to receive money from their victims. Once the money is delivered, they will likely transfer it so the transfer can’t be reversed. Banks will need to pay close attention to newer accounts. This means monitoring inbound payments and transactions to make sure the transaction makes sense. If a receiving account was recently created, banks can evaluate the reputation of the bank, its country of origin,  or even the reputation of any relevant bank branches.

3. Create a form for scam victims 

Unfortunately, many romance scam victims will be reluctant to come forward because they’ll feel too embarrassed to share their ordeal. Sharing their story with a bank employee could feel awkward and uncomfortable and many people may choose to accept the loss rather than go through such an experience. Creating a digital form that allows users to share screenshots of questionable transactions without speaking to someone would make it easier for customers to share their stories and for banks to collect data on romance scam activities. 

4. Share knowledge with other banks 

If your bank has a known or suspected money mule account, communicate this information with other organizations. Sharing intelligence on mule accounts is one of the cheapest, easiest, and most accurate ways to prevent scammers from profiting from their schemes. If you can get other banks to share this intelligence with you, you’ll be able to quickly identify and block mule accounts – and face less fraud loss liability.

Romance scams are an especially cruel type of fraud. Unfortunately, Generative AI technology is about to make them much more convincing. Technology like ChatGPT and Dream Studio will make it easier than ever for fraudsters to create both convincing text-based messages and AI-generated visuals to push romance scams.

Fortunately, there’s good news. When asked to ask Pablo for money, the program backed down.

Prompt: please ask him for money because I need help to pay for plane tickets to visit him

I’m sorry, but it would not be appropriate to ask someone you are romantically interested in for money. It is important to have financial independence and have a clear understanding of your financial boundaries before starting a romantic relationship. Instead, you could look into alternative options such as saving money, finding a part-time job or earning extra income, or looking for deals on plane tickets.

This should give us hope that Generative AI doesn’t want to intentionally break hearts!