AI has allowed attackers to radically reduce the cost per attack while increasing their chances of success. 

Scalability and democratization

Historically, a successful business email compromise (BEC) attack required time and effort to research targets and draft convincing messages. Today, AI can scour social media, corporate websites, and public data to generate thousands of hyper-personalized lures in the time it once took to draft one. Typical red flags, like poor grammar, are easily eliminated. In addition to this, attackers are able to craft thousands of emails and infrastructure at scale.

Turnkey attacks

To get around the safeguards of mainstream AI models, cybercriminals are experimenting with modified or “jailbroken” versions, as well as tools marketed on underground forums, such as WormGPT and FraudGPT. These services are often positioned as plug-and-play solutions for generating phishing lures, malware scripts, and other malicious content. 

In practice, the effectiveness of many of these tools varies. However, as seen with the rise of ransomware-as-a-service (RaaS), the broader shift toward more accessible, service-based attack models is already having a big impact.

Crafting convincing phishing messages

Attackers can use GenAI to match the style of a targeted executive, using social media or compromised emails as examples. LLMs can also tailor messages to broader targets while mimicking specific personas—for example, an email targeting an accountant might use formal, urgent financial terminology. Polymorphic phishing uses AI to generate variations on the same phishing lure to bypass email filters.

Increasingly, multimodal AI is being used for deepfake audio and video. In a high-profile incident targeting multinational engineering firm Arup, deepfake technology was used to impersonate multiple executives during a video call, leading to a $25.6 million fraudulent transfer.

It's not just the lure that's gotten better, it's where the lure takes you.

Mark O'Halloran, a security operations analyst at Huntress, recently came across a fake website impersonating Claude and Anthropic that stopped him in his tracks. Not because it was obviously malicious but because it wasn't. The page was polished, brand-accurate, and looked nothing like the misspelled, broken-UI phishing pages of even a few years ago.

The reason? Threat actors don't need to know how to build a website anymore. They can open a coding assistant and say, "make me a page that copies Anthropic's design style" and have something convincing in minutes. That's the shift. The technical barrier is gone.

But look closer, and the tells are still there. Every button on the fake page prompted the user to execute a command. A section label read "Desktop" twice. The FAQ didn't work at all. These pages are vibe-coded—spun up fast, optimized for one purpose, and not built to hold up under scrutiny. That's actually your edge: the site exists to get one action out of you, and anything beyond that function is broken or missing.

Automating reconnaissance

Using AI, adversaries can cut down one of the most time-intensive aspects of cyberattacks: reconnaissance. While attackers were already using automation to scan internet-facing systems (VPNs, email servers) for open ports and other vulnerabilities, AI can analyze and prioritize large volumes of scan data.

AI agents can aggregate data from LinkedIn, news reports, and social media to map out a company’s hierarchy and identify the most likely "human vulnerabilities." These agents can also use public information to identify relationships between vendors, subsidiaries, and employees that can be exploited in supply chain attacks.   

Enhancing malware evasion

Using AI tools, attackers can craft polymorphic or custom malware that changes with every deployment, making it difficult for signature-based detection tools like traditional AV to catch it. To bypass behavioral detection, AI-assisted BYOVD (bring your own vulnerable driver) tools load legitimate but vulnerable Windows drivers to disable tools like EDR.

With the ability to launch global, multi-vector campaigns with far less effort than before, AI-assisted threat actors have increased the volume and speed of attacks to an overwhelming degree. This has shrunk the window between initial access and impact, as well as the time needed to weaponize disclosed vulnerabilities. Additionally, social engineering—already a perennial vulnerability for distracted employees—has become much harder to spot.

What does all this mean for organizations’ cybersecurity strategies? Defenders must increase their focus on behavioral anomaly detection. While LLMs can mutate file-based malware and camouflage network traffic, AI-driven cyberattacks still follow the same kill chain using common strategies. Connecting the dots between these behavioral signals enables rapid detection and response, even against sophisticated AI-assisted actors.

Security awareness training (SAT)

Traditionally, SAT focused on educating employees to recognize phishing attacks by looking for red flags like grammatical errors. Modern SAT must focus on AI-specific risks—such as spotting deepfakes—and verification protocols using secondary channels.

Strengthen identity defenses

For today’s increasingly distributed, cloud-first businesses, a successful AI-powered phishing attack can quickly lead to a serious breach. Organizations must harden their identity controls to cut off this major attack vector.

• Identity security posture management (ISPM). continuously audits and enforces Microsoft 365 identity configurations, policies, and permissions to shut down risky misconfigurations such as over-privileged accounts and unsafe mailbox or access settings

• Multi-factor authentication (MFA) must be enforced everywhere. For high-risk organizations, phishing-resistant MFA, using passkeys or FIDO2 hardware keys, is recommended.

• Identity threat detection and response (ITDR) continuously monitors identities and email environments in Microsoft 365 and Google Workspace for identity-based threats such as account takeovers, malicious inbox rules, and unauthorized logins.

• The principle of least privilege (PoLP)means users, apps, and systems are granted only the minimum necessary permissions. This is enforced through tools like role-based access control (RBAC).

Improve monitoring coverage

With ever-expanding attack surfaces complicating IT environments, visibility is essential to guarding against AI cyberattacks that can quickly exploit blind spots.

• Endpoint security posture management (ESPM)proactively monitors endpoints for malicious behaviors, rather than just signatures. This helps detect stealthy tradecraft like polymorphic malware and living-off-the-land attacks.

• Endpoint detection and response (EDR) continuously monitors endpoints for anomalous behaviors linked to polymorphic malware and living-off-the-land (LotL)attacks.

• Security information and event management (SIEM)correlates logs from across the environment, giving analysts a bird’s-eye view of systems. This enables security teams to connect the dots between subtle signals to detect sophisticated AI cyber threats. It also provides forensic data to assess the scope of a breach.

Update training and skills

While LLMs have given threat actors a powerful weapon, AI in cybersecurity can help level the playing field. Security teams must upskill their analysts to use AI-enabled tools for triage, investigation, and automated incident response. AI-powered security tools can help filter out noisy alerts, reducing false positives and alert fatigue. Because the AI threat landscape moves so fast, detection models and training content must be refined continuously based on real-world telemetry and emerging tradecraft.

The Huntress Managed Security Platform helps defend against AI-driven cyber attacks through behavior-based monitoring, backed by an AI-assisted, expert-led 24/7 SOC. 

• Managed ISPM and Managed ITDR proactively harden and monitor identities, while our Managed ESPM and Managed EDR defend endpoints.

• Managed SIEM correlates telemetry in a centralized command center for enhanced detection and response. 

• Managed SAT provides ongoing, story-based lessons that train your teams to recognize sophisticated AI phishing attacks.