AI-Generated Phishing Detection and Defense

Generative AI has made phishing cheaper to produce and harder to spot by its writing alone. The FBI’s Internet Crime Complaint Center (IC3) received 22,364 complaints in 2025 that included AI-related information, with adjusted losses of more than $893 million, and it warns that AI-generated content “is becoming increasingly difficult to detect and easier to make” (IC3 2025 Internet Crime Report). Those figures count only complaints where victims recognized and reported an AI element, so the true share is unknown.

Research shows why this matters. In a 2024 study by Fred Heiding, Bruce Schneier and colleagues, fully AI-automated spear-phishing emails achieved a 54% click-through rate, the same as emails crafted by human experts (54%) and close to AI emails refined by a human (56%), while a control group of generic phishing emails got 12% (Heiding et al., arXiv:2412.00586). AI does not make phishing more persuasive than a skilled human attacker, but it lets anyone produce expert-quality lures quickly.

What AI Changes About Phishing

Fluent Grammar and Tone

Poor grammar and awkward phrasing used to be the easiest tell. Large language models produce fluent text in many languages and styles, and can imitate the tone of a brand or a colleague. Some phishing still contains errors, but a well-written message is no longer evidence that it is genuine.

Personalization at Scale

Before AI, spear phishing took time: each message needed research and careful writing. AI tools can summarize public information from LinkedIn, company websites and social media and draft a tailored message for each target, so personalization that used to be reserved for high-value targets becomes cheap. The IC3 notes that chat generators “can quickly create official-sounding emails mimicking a company’s CEO or other officials,” and that businesses reported more than $30 million in losses in 2025 to BEC scams involving AI.

Multilingual Phishing

AI translation and generation remove the language barrier that once made many lures easy to spot for non-English speakers, letting campaigns run in several languages at once.

Deepfake Voice and Video

Voice cloning and synthetic video extend phishing to calls and meetings. The FBI has warned that criminals use AI-generated audio to impersonate “public figures or personal relations to elicit payments,” and AI-generated video for real-time video chats with supposed executives or authority figures (FBI PSA I-120324-PSA, December 2024). In May 2025 it warned that malicious actors were impersonating senior US officials with AI-generated voice messages, and that a cloned voice and the real person “can sound nearly identical” (FBI PSA, May 2025).

The best-known corporate case is Arup: in 2024 an employee in Hong Kong sent HK$200 million (about US$25.6 million) in 15 transactions after a video call in which the people who looked and sounded like the CFO and colleagues were deepfakes (CNN). Consumers face the same tactic: IC3 victims reported more than $5 million in 2025 losses to “distress” scams that use voice cloning to mimic a loved one. See our vishing defense guide for voice-specific countermeasures.

Evasion of Detection

Because AI can generate a different message for every recipient, filters that match known templates or exact phrases lose effectiveness. Attackers also vary wording, sender display names and URLs, and use plausible first-contact emails with no link at all; Fortra reported an uptick in this “two-step” phishing in Q2 2026 (APWG Q2 2026). Lookalike pages for brand impersonation are also faster to produce.

What AI Does Not Change

The underlying mechanics of phishing stay the same:

The request is still abnormal. The attacker still needs the victim to do something (open a link, enter credentials, approve a sign-in, send money) that a normal communication would not ask for in the same way
The infrastructure is still malicious. Credential-phishing emails still point to attacker-controlled domains or abused hosting, which URL inspection and domain reputation can flag
Authentication still matters. DMARC with SPF and DKIM blocks spoofing of an exact protected domain however well the message is written, though not lookalike domains or compromised accounts
The psychology is the same. The social engineering principles of urgency, authority, secrecy and fear remain visible even in AI-written messages

Detection Strategies for AI Phishing

Context and Behavior

As message quality stops being a useful signal, look at context instead:

Sender anomalies. First-time contact, a new address for a known person, or sending patterns that differ from the norm
Request anomalies. Requests outside the sender’s normal role, especially involving money, credentials or sensitive data
Channel anomalies. Requests that arrive through an unusual channel, at an unusual time, or that try to move the conversation to a new messaging app
Relationship anomalies. Messages from regular contacts that feel “off”: a different sign-off, unusual urgency, or a request to keep things quiet

Technical Controls: What Each One Catches

Control	What it catches	What it misses
Email authentication (DMARC, SPF, DKIM)	Spoofing of your exact domain	Lookalike domains, compromised accounts
Behavior-based email security	Unusual senders, requests and patterns	New attacks that resemble normal traffic
URL and domain reputation	Known-bad infrastructure	Newly registered domains, abused legitimate hosting
Email header analysis	Spoofing and routing anomalies	Mail sent from a legitimate, compromised account
Verification procedures	Fraudulent payment and access requests	Steps people skip under pressure

Human Detection (Updated for AI)

Red flags to de-emphasize:

Grammar errors (unreliable)
Spelling mistakes (unreliable)
Unnatural phrasing (unreliable)

Red flags to emphasize:

Unexpected requests involving credentials, money or data
Urgency that pushes you to act before verifying
Channel mismatches, such as your bank texting about an email or your CEO writing from a new address
Requests to bypass procedures (“Handle this personally,” “Don’t run this through normal channels”)
Verification avoidance (“I’m in a meeting and can’t take calls, just email me back”)

For video and images, the FBI suggests looking for subtle imperfections such as distorted hands, irregular faces, inaccurate shadows, lag and voice mismatches, but these tells are disappearing, so treat verification, not visual inspection, as the real defense.

Defense Framework

For Organizations

Use email security that analyzes behavior and context, not only content and known signatures
Enforce DMARC at p=reject on every domain you own. It stops exact-domain spoofing regardless of how good the message is, but pair it with lookalike-domain monitoring and controls for compromised accounts
Require phishing-resistant MFA. FIDO2 security keys and passkeys will not authenticate to a lookalike site, so a stolen-password page fails even if the lure is flawless; CISA calls this “the gold standard” (CISA). It does not stop payment fraud, malware or a user approving a malicious app
Apply zero trust principles: assume credentials may be compromised and limit what any single account can reach
Update phishing simulations so they reflect well-written, personalized lures and multi-channel attacks, and measure reporting, not just clicks
Set verification procedures for high-risk requests: a callback to a number already on file, dual authorization for payments, and no exceptions for “urgent” requests from executives
Do not rely on recognizing a voice or face. A caller’s voice, or a face on a video call, is not proof of identity when both can be cloned. Use callbacks to known numbers and pre-agreed verification steps instead

For Individuals

Verify through a separate channel. The FBI advises hanging up, looking up the organization’s number yourself and calling it directly
Agree on a secret word or phrase with family members, as the FBI recommends, so you can check a distress call claiming to be from a relative
Go to websites directly instead of following links, however legitimate the message looks
Turn on phishing-resistant MFA (passkeys or security keys) wherever it is offered
Pause before acting: stop, verify, consult, report
Report suspicious messages even if they look perfectly legitimate; your security team needs to see what is getting through

Key Takeaways

IC3 recorded 22,364 AI-related complaints and more than $893 million in losses in 2025, likely an undercount
In research, fully AI-generated spear-phishing emails were clicked as often as expert-written ones (54%), so grammar and spelling are no longer reliable indicators
Voice cloning and deepfake video mean a familiar voice or face is not verification
Context (unusual requests, channel mismatches, urgency) matters more than writing quality
DMARC, phishing-resistant MFA and zero trust do not depend on message content, but each covers only part of the problem
Callback verification and dual authorization defeat many AI-enhanced social engineering attempts

For the complete phishing defense framework, see our phishing recognition and reporting guide.

Sources

Security education disclaimer: This article discusses AI-enhanced phishing techniques for educational purposes only. Understanding how AI is used in attacks helps defenders adapt their strategies. Do not use AI tools for unauthorized phishing or social engineering.