When it comes to breaches, it’s hard to find a silver lining when the end result is customer down time, data theft, or damaged reputations. For Managed Service Providers with tens to hundreds (or even thousands) of clients, the stakes are even higher. In this MSP Moment, we’re highlighting how NTConnections, a Washington DC based MSP, responded to a database outage which quickly escalated into an incident response effort.
It all started when NTConnections was called into an organization who reported a SQL Server failure. Like all jobs, their technicians performed some initial discovery work, quickly restored the server, and then analyzed the root cause. After inspecting the SQL server event logs, verifying antivirus was up-to-date, and reviewing the firewall’s logging configuration, they were left with an uneasy feeling that the database may have been compromised. NTConnections deployed a free trial of Huntress on the client’s server to uncover what was lurking. In just a few minutes, Huntress discovered hackers created multiple footholds that provided backdoor access to the server.
With the Huntress remediation recommendations in-hand, NTConnections CEO Scott Ostergard informed the client of the continuing threat and proposed an incident response effort to quickly assess the impact and contain the compromise. After receiving CIO approval, the Huntress staff teamed with Scott to help discover the initial access vector, establish an intrusion time-line, determine what (if any) data exfiltration occurred, analyze the malware, and validate his team’s countermeasures.
This level of partnership returned immediate dividends. Together, we discovered the attackers gained access by brute-forcing the database’s administrator account (SA) password, and NTConnections has a new client. See the some of the commands we discovered and remediated below:
The hackers were creative and took the following actions:
- Disabled a firewall-affiliated service to ensure outbound internet access
- Downloaded obfuscated backdoors over FTP
- Created persistence mechanisms for long term access
- Deleted MSSQL and Windows event logs to cover their footprints
In addition to backdoors, the attackers downloaded several post-exploitation tools. These included software to mine cryptocurrency and capabilities to prevent common antivirus products from running.
Thankfully, NTConnections’ monitoring and our threat hunting was able to discover the customer’s breach before the situation escalated. Using our remediation recommendations and NTConnections’ backup/restoration services, the customer was able to focus on business without interruption.
Huntress is a simple decision. This is a tool that immediately alerts us to potential problems; providing us the steps to remediation and preventing a small issue from becoming a major problem. With just the time savings we gain from this partnership, it far exceeds the cost.” -Scott Ostergard
This moment highlights why we’re so proud to call NTConnections our partner and are honored to help complement their existing cybersecurity investments. With that said, it didn’t stop here. Have your staff check out our detailed technical write up to learn exactly where things went wrong.
If you’re a Managed Service Provider looking to minimize customer downtime while protecting your profit margins, we’d love to partner with you. Sign up for our free trial and we’ll show you what’s already slipped past your preventative security stack.
