Over the last year, we’ve worked our asses off to prepare for an inbound tidal wave of identity-focused tradecraft. Considering our partners’ significant investments in the Microsoft 365 ecosystem, we’ve specifically targeted emerging business email compromise (BEC) and account takeover tactics, techniques, and procedures (TTPs). Since releasing our new product MDR for Microsoft 365 a few months ago, we’ve onboarded hundreds of thousands of identities and reported 1,000+ incidents before shit could hit the fan. However, we think there’s a handful of things we probably got wrong in these first few releases and want to lay it all on the table.

Before we get into the good stuff, let me start with the same mantra I used to found this company eight years ago: At Huntress, we deliver ruthless transparency, hold ourselves most accountable, and always give back more than we take. Today is no different, and we’ll keep communicating openly as long as it’s needed. Said more directly, we will own/address any fuck-ups or missteps promptly.

Considering our obsession with no-BS transparency, we’ve put together the following sections to address the most popular themes, answer questions, and kickoff dialog:

• Current capabilities and our R&D approach
• What we likely missed or got wrong
• What we are working on next

Have something urgent? Hit me up at kyle[squiggly a symbol]huntress.com.

Current Capabilities and Our R&D Approach

Every product built at Huntress consists of multiple “capabilities” (think Persistent Footholds, Ransomware Canaries, etc.). We select these capabilities based on “what will have the biggest impact?” and “how will these capabilities layer to provide compensating coverage?”. Once selected, these capabilities are perpetually maintained by product engineers, researchers, detection engineers, and SOC analysts.

As of today, we’ve selected and prioritized the following MDR for Microsoft 365 capabilities based on hacker tradecraft and risk probability for SMBs:

Unwanted Logins

Few things are more brutal than an account takeover. As a result, we invested significant research into fully understanding this technique and the massive number of offensive sub-techniques that can lead to an unwanted login. Our tech behind this capability starts by parsing Microsoft 365 audit logs and enriching these events with information on “where” and “how.” For our first line of detection, we compare this data against known bad locations and shady IP sources (e.g., data centers known for harboring adversary workloads and uncommon providers with high percentages of malicious activity) to identify threat actors accessing accounts early and lock them out using Huntress’ identity isolation. We’ve had solid success finding malicious activity and isolating affected identities with this method, but we've also observed the fragility of location-based detection (geofencing) and created multiple bypasses using token theft, faux-device registration, etc.

Take a peek at our Product Lab webinar episodes one and two for our commentary on these. Needless to say, the perpetual improvement cycle for Unwanted Logins is just getting started.

Shadow Workflows

Another tactic exploited by threat actors is manipulating mail delivery using inbox rules and mail forwarding techniques. After an unwanted login occurs, these shadow workflows are often leveraged to exfiltrate sensitive data and to obfuscate emails from the intended recipient by moving or deleting incoming mail. Our initial detection capabilities parsed and stored relevant new workflow events for algorithmic and human analysis. Shortly afterward, we expanded our data collection capability to gather and analyze pre-existing workflows (i.e., historical inbox rules) to uncover past indicators of compromise. These techniques follow a formulaic pattern that Huntress uses to determine if they are illegitimate and queue up remediations. We’ll continue to mature our detection routines throughout the foreseeable future to maximize discovery efficacy and minimize false positives.

‍

Evasive Behaviors

There isn’t just a single bridge across the castle moat when it comes to Microsoft 365. Threat actors may ignore the standard user login portal and use their ill-gotten credentials with different access methods, including the Microsoft Graph API, the Azure command line, and the Azure SDK. Each approach varies greatly in observability, and similar shady actions produce different indicators based on the access method. Recently released offensive security frameworks like GraphRunner show the power of these alternative access methods. We have prioritized expanding observability for threat actor activity when they leverage these alternative access methods to catch their shadiness.

Rogue Apps

Within SMBs, we’ve observed threat actors installing malicious applications or weaponizing legitimate Microsoft 365 applications to exfiltrate data and establish persistence within a Microsoft 365 environment. These apps will often have extensive permissions to access the desired Microsoft 365 data while avoiding administrator and defender scrutiny. We’ve also witnessed threat actors using the permissions of installed applications to act on their behalf, bypassing the need to elevate a compromised identity’s privileges. The rogue apps detector is still in its early stages but will see rapid R&D investments to continue to combat this emerging tradecraft.

Although there’s plenty of additional offensive tradecraft lurking out there, we feel very confident that continuing to develop these four capabilities before adding more will deliver the most impactful value to our customers and partners today.

What We Likely Missed or Got Wrong

Since our initial launch, we have gotten tons of solid feedback from our customers, partners, and prospects that largely aligns with our own positive, but sometimes spicy, assessments. In this section, we’re going to dive into what we think we would have done differently or better.

Unwanted Logins

When we first approached the account takeover and BEC problem, we prioritized many harder problems first. One of these problems was “impossible travel” detection. Based on our red team assessments, we feel other vendors' geolocation approach to impossible travel is fairly brittle and easily bypassed by junior threat actors. Furthermore, depending on inaccurate and unreliable IP data can create a ton of false positives. We prioritized innovating in this area rather than starting with the basic geofiltering functionality that many expect. This was a mistake as our long-term approach is still maturing, which can allow low-hanging fruit to slip by.

What we’ve already done to address it: In the first week of December, we released detectors focused on anonymous VPN usage and anomalous logins. In the short time since we rolled out the new anonymous VPN and anomalous user location detections, we’ve seen a huge improvement in our efficacy. Approximately two-thirds of the malicious activity we see comes from an anonymizing service and almost half from VPNs.  To be clear, while our end goal is the same, our approach is not what most people think of when they hear “impossible travel.”  Instead, our solution utilizes anonymous VPN and anomalous user location data to determine suspicious logins. We have even more improvements coming in December, and we will be releasing geo-allowlisting functionality for more granular control (see the “What We Are Working On Next” section to learn more).

Balancing true detections and false positives

Historically, we’ve been extremely committed to minimizing false positives in our Managed EDR product. It’s been a core element of our mission, which is why we took the same approach with MDR for Microsoft 365. In some cases, we over-adjusted this ratio because we were concerned we would pass along too many false positives to our partners. As a result, we missed some activity that we should have caught. Nothing pisses us off more than missing threat actor activity. Nothing.

What we’ve already done to address it: Getting the detection tuning right for any security product is a task that’s never truly finished, but we have made significant improvements. We are now generating more false positives than we did before, but our SOC can filter most of those out before our users see them. As a result, we’re detecting even more threats.

Onboarding

We had some significant issues with onboarding in the product's initial release that caused pain for our partners and customers.

What we've already done to address it: We have streamlined the onboarding process, removing lengthy steps and quickly releasing fixes for the most common errors associated with the process. These include escalations to detect integration issues, establish direct integration as default, and introduce integration repair functionality. Integrating a Microsoft 365 tenant now takes less than 90 seconds. As a result of these improvements, over 90% of onboarding processes are now completed without any issues. We know that isn’t 100% yet, but when issues do arise, we solve them in less than 48 hours on average.

What We Are Working On Next

I mentioned above that we have already released updates to detect anonymized VPN usage and anomalous VPN detection. We have many more updates scheduled to be live by the end of December. This includes over a dozen new Microsoft 365 detections based on locations and threat intelligence data, which should deliver on our promise to detect suspicious activity with a more innovative and effective approach. We are also working on additional detections and geo-allowlisting that will more closely resemble what most people think of when they refer to impossible travel.

Other updates we are working on include:

• “User behavior fingerprints” to provide additional improvements to our recent anomaly detections.
• Improved inbox rule creation and modification detection to make it harder for adversaries to evade detection.
• Weekly cadence of reports for users with compromised credentials, detected as successful username/password, new and suspect location, with or without VPN, but failed MFA. Higher risk detections will still be actioned and reported on immediately.
• Proactive communication to partners when Huntress receives Microsoft 365 events for any newly onboard tenants, providing proof of life for the product.

• • •

Hopefully, this gives you some valuable insight into our MDR for Microsoft 365 product, where we’ve come from, and where we are going. But we aren’t going to stop here. We are committed to delivering regular updates on our progress on our blog and through release notes to our partners.

If you want to dive deeper into the world of account takeover threats, Matt Kiely and Dray Agha will discuss Microsoft 365 attack and defense in next week’s (December 12) holiday-themed Tradecraft Tuesday. It should be a fun one!

As always, you can reach out to the team at Huntress or to me directly, and we will be happy to answer any questions you may have.