Background
AI is here, and we're hearing about all the high-profile security risks that come with it. AI security discussions often focus on attacks against AI companies and models (such as the recent OpenAI-HuggingFace intrusion). However, the Huntress Security Operations Center (SOC) is seeing a more prolific, day-to-day threat: threat actors abusing the ways users interact with AI platforms.
Over the past nine months, we have seen threat actors abuse legitimate AI features and services, including:
Claude Artifacts: content that Claude generates and displays as a separate preview pane within a chat; Claude Artifacts can be "published" and made publicly available via a link.
Claude.ai/sharelinks: shareable URLs generated when a user publishes a Claude conversation. These links can appear in search results when they are posted in locations that search engines can crawl, such as forums or social media.ChatGPT and Grok conversations: shared conversations that are hosted on legitimate AI platforms that can be indexed and surfaced in search results, including for troubleshooting queries.
There's a common thread here: these are all part of an AI attack surface, which involves a trust boundary around shareable AI content, public mini-apps, and sponsored search placement.
Threat actors are using these because they combine multiple layers of trust. AI users recognize the platform, branding, and surrounding content, which can make malicious instructions or downloads appear legitimate. These campaigns may remain active for only hours or days before the provider removes the content, but they can still attract significant attention during that window.
Here's what we've seen, and what defenders need to know.
FakeAgent: a Claude Artifact malvertising attack
In July, we saw a campaign that hit more than 29 organizations that started with a malicious Claude Artifact publicly hosted on the legitimate Claude.ai domain.
Because public artifacts can be created and shared by any user and are meant for lightweight things like demos, charts, or one-page sites, Anthropic doesn't vet their content beyond a generic disclaimer shown on every artifact page stating: "Content is user-generated and unverified."
Figure 1: Claude Desktop/Cowork phishing page hosted as a Claude Artifact.
Users searched for the Claude desktop app on Bing, which took them to this malicious public Claude Artifact on the actual Claude AI domain. The artifact visually impersonated a Claude Desktop / Claude Cowork download page. This appeared to be a legitimate download link for the desktop app (ClaudeDesktop.exe) -- but in reality, targets were redirected to an external domain (downloading-api[.]it[.]com), which ultimately led to the download of malware (SectopRAT).
We reported the malicious Claude Artifact to Anthropic; as of publication on July 22, the public Claude Artifact had been removed. However, we continue to see incidents that come from the malicious redirect domain downloading-api[.]it[.]com, with several incidents continuing to occur through August with the same hallmarks of this initial campaign.
Fake Claude install guide via claude.ai/share
Huntress investigated an incident where a victim searching Google for Claude on Mac clicked a sponsored result and followed a claude.ai/share conversation posing as an Apple Support guide.
Here, attackers were weaponizing a shared Claude conversation at claude.ai/share/<id>. Anthropic allows users to publish conversations through public share links, and the attacker exploited that legitimate feature to host the lure directly on Claude's own website.
This attack was the culmination of a few different tricks: it involves a sponsored search result, which can imply trust to a user as it's a domain that shows up at the top of the Google search. Because the link used the real claude.ai domain, it also appeared trustworthy. The victim didn't encounter a misspelled or lookalike domain, and they didn't see a certificate warning, making the deception significantly harder to detect.
The "Apple Support install guide" told the victim to paste a curl command into Terminal. This launched a six-stage kill chain for the MacSync stealer that resulted in the comprehensive theft of cookies, credentials, keychain secrets, Telegram sessions, SSH and cloud keys.
AI poisoning: ChatGPT and Grok conversations
In December, an ordinary search for "Clear disk space on macOS" led a Mac user into high-ranked ChatGPT and Grok conversations, hosted on their respective legitimate platforms. Both conversations gave "advice" on how to clear disk space – but in reality, these were delivering ClickFix-like instructions.
The attack vector here was legitimate ChatGPT and Grok conversations – these shared conversation links (on chatgpt.com and grok.com) are crawlable/indexable pages. Attackers first crafted a conversation with an AI assistant that ends in a malicious "fix," and then hit "share," generating a public URL on the platform's own trusted domain. SEO poisoning was then used to push that link to the top of Google results for common troubleshooting queries.
Figure 2: Top search results via Google Search for this attack
Because the link lives on the actual ChatGPT and Grok domains, which people already trust, victims didn't suspect it was fake. The "advice" in the conversation seemed plausible and recommended the targets run Terminal commands to "clean up" the system. These commands actually led to the download of the AMOS stealer.
This type of attack vector has continued since December, with Cato Networks security researchers in August reporting other variants of this attack that specifically involve a fake OpenAI Codex download. Like the one that we saw, this other variant of the attack involved sponsored results via Google Search, convinced macOS victims to paste a malicious command in Terminal, and ended with AMOS malware.
Conclusion
AI security isn't limited to the models themselves. In each of these cases, attackers used legitimate AI platforms, shared conversations, public artifacts, and search results to make malware and malicious instructions look trustworthy. Things like a familiar brand, a real domain, or a convincing troubleshooting guide shouldn't be treated as proof that something is safe, especially when it asks users to paste commands into Terminal or PowerShell.
Defenders should treat clipboard-driven execution and AI-assisted troubleshooting as security concerns. Restrict script execution from the clipboard, enforce application allow-listing, monitor for new scheduled tasks and antivirus exclusion changes, and train users to recognize ClickFix-style lures. Individuals can use ad blockers to prevent or reduce their exposure to the sponsored search risks that are described in these campaigns.
If you come across malicious content, it's important to report it to platform vendors quickly. These campaigns may be short-lived, but layered controls and rapid reporting can reduce the window of opportunity and help keep trusted AI experiences from becoming trusted malware delivery channels.
