Phishing has been around for decades. The basic playbook has always been the same: send someone a fake message, make it look real enough to fool them, and wait for them to hand over credentials, click a bad link, or wire funds to the wrong account.

For a long time, phishing attacks were easy to spot if you knew what to look for. Awkward grammar, misspelled domain names, and generic greetings like "Dear Customer" were tell-tale signs. Security training taught people to pause before clicking, and most organizations felt reasonably confident their employees could sniff out a scam.

Then generative AI changed everything.

AI phishing, sometimes called generative AI phishing or AI-powered phishing, is what happens when attackers take the same old playbook and supercharge it with large language models (LLMs), voice cloning tools, and AI-generated imagery. The result? Messages that read like they came from a trusted colleague. Voicemails that sound exactly like your CEO. Emails that reference your name, your company, your recent projects, and your actual relationships — because AI scraped that information from LinkedIn, your company website, or a data breach.

See how easy it was for the Huntress team to create a deepfake of CEO, Kyle Hanslovan during a Tradecraft Tuesday. 

The volume has exploded, too. Generative AI allows cybercriminals to produce thousands of highly customized phishing messages in the time it used to take to write one. That means more attacks, targeting more people, with far less effort on the attacker's end.

The mechanics of an AI phishing attack aren't magic they're a combination of data, automation, and language models working together.

• Step 1: Reconnaissance Before any message is sent,threat actors gather information about their target. They pull data from social media profiles, LinkedIn pages, company websites, press releases, and leaked databases. AI tools can process and synthesize this information quickly, building a detailed profile of the person or organization being targeted.

• Step 2: Content Generation With a target profile in hand, threat actors feed that data into an LLM (ChatGPT, Claude, Gemini) and instruct it to write a convincing, personalized message. The AI handles grammar, tone, and context automatically. It can mimic the writing style of someone the target knows and trusts if it has enough sample text to work from. The more information the threat actors have the more convincing the phishing attempt can be. 

• Step 3: Delivery at Scale Because AI can generate and send messages automatically, attackers can launch campaigns targeting hundreds or thousands of people simultaneously. Each message can be uniquely tailored to its recipient, which is something that would have required an enormous amount of manual labor in the past.

• Step 4: Exploitation When the target takes the bait, which includes clicking a link, entering credentials on a fake login page, responding to a fraudulent wire transfer request, or downloading a malicious attachment, the cybercriminals get what they were after.

AI and phishing overlap in several distinct attack formats. Here are the most common:

AI-Generated Spear Phishing Traditional spear phishing involves targeting a specific individual with a carefully crafted message. AI makes this faster and more convincing. Attackers can generate highly personalized emails referencing real names, departments, recent events, or internal jargon pulled from publicly available information. What used to take hours now takes seconds.

Vishing with Voice Cloning Voice phishing (vishing) has gone to a new level with AI-powered voice cloning. Attackers can use as little as a few seconds of publicly available audio such as  a podcast, a conference recording, a social media video  to clone someone's voice. Targets receive calls that sound indistinguishable from their manager, a bank representative, or an IT support technician. These calls are used to extract passwords, approve fraudulent transactions, or gain remote access.

AI-Enhanced Deepfake Attacks Deepfake technology can generate realistic video or audio impersonating a real person. In some cases, attackers have used deepfake video calls to impersonate executives in virtual meetings, convincing employees to take actions like initiating wire transfers. 

Business Email Compromise (BEC) with AI Assist Business email compromise is one of the most financially damaging forms of cybercrime. AI has made BEC attacks more convincing by helping attackers write urgent, plausible-sounding requests for wire transfers, invoice changes, or credential resets all while matching the tone and style of the person they're impersonating.

The warning signs employees learned to look for in traditional phishing don't always apply to AI-generated attacks. This is a significant problem for organizations that rely heavily on outdated security awareness training as their primary defense.

Here's what's changed:

No more obvious grammar mistakes. AI-generated text is fluent, polished, and contextually accurate. It doesn't make the kinds of grammatical errors that used to flag a phishing email immediately.

Personalization erases generic tells. When an email references your actual name, your actual manager, and a project you actually worked on last week, it no longer feels like a mass-blast scam. That familiarity makes people less suspicious.

Volume overwhelms manual review. When attackers can launch thousands of unique, targeted campaigns simultaneously, the sheer number of attempts increases the odds that some will succeed even with well-trained employees.

AI can make it easier for attackers to slip past content filters. Many email security tools look for patterns in known phishing messages. AI-generated content is unique each time, making it harder for signature-based filters to catch.

The Cybersecurity and Infrastructure Security Agency (CISA) notes that phishing remains one of the most common attack vectors, and that AI is actively raising the bar for what "suspicious" looks like.

AI phishing isn't theoretical, it's already happening across industries.

In one widely reported case, a finance employee at a multinational company was deceived by a deepfake video call that appeared to show their CFO and other colleagues. The employee was convinced to transfer approximately $25 million before the fraud was discovered.

Voice cloning attacks have also been used to impersonate executives over the phone, with employees approving fraudulent wire transfers after receiving calls that sounded identical to their CEO.

AI-generated spear phishing campaigns have been used to target healthcare organizations, law firms, and government contractors — sectors where access to sensitive data makes them high-value targets.

These incidents point to a reality that cybersecurity professionals are dealing with every day: the tools attackers use are improving faster than many defenses can keep up with. Understanding the role of dark AI and the tools cybercriminals use is an important part of staying ahead of these threats.

Defending against AI phishing attacks requires more than a single tool or policy. It takes a layered approach to cybersecurity often called "defense in depth."

1. Use phishing-resistant multi-factor authentication (MFA) Standard MFA adds a layer of protection, but attackers have developed techniques to bypass it. Phishing-resistant forms of MFA  like hardware security keys are harder to defeat even when credentials are compromised. CISA strongly recommends phishing-resistant MFA as a priority control.

2. Use modern security awareness training. Employees need to know that AI phishing attacks don't look like the scams they learned about years ago. Moder training should include examples of AI-generated email, voice cloning, and deepfake attacks and not just the classic "Nigerian prince" email of years past. Check out Huntress'comprehensive phishing guide for practical guidance.

3. Verify unusual requests out-of-band. If someone calls or emails asking for a wire transfer, credential reset, or sensitive data — especially with urgency — verify through a different channel. Call the person back at a known number. Don't rely on the contact information in the suspicious message.

4. Use layered email and identity security. Combine email filtering with identity-focused tools like Managed ITDR and endpoint protection like Managed EDR, backed by a 24/7 AI-Centric SOC, so you can catch activity that slips past the gateway.

5. Monitor for AI-generated threats at the endpoint. Even when a phishing attempt succeeds, catching the attacker's activity after the fact can contain the damage. Endpoint detection and response (EDR) tools can identify suspicious behavior that follows a successful compromise. Make sure your team can handle the volume of alerts they produce, otherwise you can opt for a Managed EDR. 

6. Establish internal verification protocols. Set up clear policies for how your team handles requests involving money, credentials, or sensitive data, especially requests that arrive urgently, out of the ordinary, or from leadership. A quick phone verification step can stop a lot of AI phishing attacks cold.

AI phishing represents a genuine shift in the threat landscape. It takes an already effective attack method and makes it faster, more convincing, more personal, and harder to catch with the tools and training that used to work.

The core of a good defense hasn't changed: verify before you trust, layer your security controls, keep training current, and make sure your technology can keep pace with how attackers are evolving. What has changed is the urgency. AI phishing attacks aren't a future problem — they're hitting organizations right now, across every industry.

Understanding what AI phishing is and how it works is the first step. Building defenses that account for it is the next one. Try Huntress for free and see how our Managed Security Platform helps you defend against AI-powered phishing.