The security team at Microsoft had their New Year’s Eve festivities ruined as they worked to patch a massive breach of 250 million customer service and support records.
The records date back as far as 2005 and are as recent as December 2019, and consist of online chat records between customers and Microsoft support personnel. Microsoft says that personal information was scrubbed from the customer service records before they were stored, but they did at minimum contain email and IP addresses stored in plain text. Security researchers believe that the exposure goes beyond that.
The mass of Microsoft customer service records
In a statement posted to the company blog on January 22, Microsoft revealed that they uncovered misconfigured security rules in a database on December 29. It was an internal company database used for analytics, apparently not normally accessible to the outside world. The company says that the error was made on December 5, leaving the customer service records exposed for most of the month. The issue was fixed on December 31.
The company conducted an internal investigation and claims that there was no sign of malicious use, nor did most of the customer service records contain personal identifiable information.
However, the company did admit that some email addresses that were entered in non-standard formats may have survived the automatic scrubbing of the logs and remained in plaintext.
Microsoft says it is personally notifying anyone whose email address was exposed in this way.
The company has announced some security policy revisions in response to the incident.
Microsoft will be auditing its internal security policies and putting additional tools in place to ensure that stored customer service records of this type have all sensitive personal information redacted from them. A new internal alert system will also be put into place to better monitor misconfigurations that can lead to potential breaches.
Is there any risk to Microsoft end users?
That’s the story from Microsoft’s end. Further details from other sources indicate that there is more to worry about than the company is indicating.
In a report published by Threatpost, security researchers with Comparitech claim that they found five Elasticsearch servers indexed by the IoT vulnerability search engine BinaryEdge. Each server contained a full copy of the database containing the customer service records.
Comparitech applauded Microsoft for their quick response to notification of the issue, but also noted the presence of sensitive information that was not mentioned in Microsoft’s public statement.
Microsoft does redact certain key personal information from these logs: email addresses, payment information and contract numbers. However, there is other personally identifiable information that remains behind and may have been exposed online. The Comparitech team found case numbers, details of cases, resolutions, remarks and internal notes that were marked as “confidential” in the customer service records.
According to Ekaterina Khrustaleva, COO of web security company ImmuniWeb:
“Assuming the data was not exploited by malicious actors as per the official statement, there is not much practical risk so far. However, it is impossible to say whether the information from this server, or other presumably existing servers, has ever been detected and stolen by cybercriminals.
“The absence of PII in the dump is irrelevant here, given that technical support logs frequently expose VIP clients, their internal systems and network configurations, and even passwords. The data is a gold mine for patient criminals aiming to breach large organizations and governments.
“Worse, many large companies and not only Microsoft have lost visibility of their external attack surface, exposing their clients and partners to significant risks. We will likely see a multitude of similar incidents in 2020.”
The primary danger is that this information could be used in technical support scams directed at Microsoft customers. Scammers very frequently identify themselves as Microsoft support agents, cold-calling targets and trying to convince them that something is wrong with their computer. The most common variety of the scam is to try to sell the target an overpriced piece of unnecessary “virus scanning” software (and potentially steal their credit card number in the process), but the bolder scammers may attempt to get the target to grant them remote control of their computer. Scammers might also simply email targets and try to get them to visit malware links under the ruse of providing some sort of technical support.
If your email address was exposed in the breach, you should get a legitimate message to that address from Microsoft at some point. Outside of breach notifications such as these, however, the company generally does not contact customers first about technical support matters. A phone call from “Microsoft” that comes out of the blue is almost certainly a scam attempt. And the company will never ask users to install remote desktop viewing or access software.
Paul Ducklin, Principal Research Scientist at Sophos, points out that breach victims should also be on the lookout for fake “notifications” from hackers who impersonate the Microsoft support team:
“Hundreds millions of records were exposed, but it sounds as though comparatively few people actually had recognizable email addresses in the leaked database. In other words, most people won’t actually receive warnings from Microsoft – but might well receive “warnings” from crooks claiming to be Microsoft.
“Remember: don’t click on links in security warnings, even if you think they’re real. That way you will avoid end up on phishing sites by mistake, and you won’t put in your password where you shouldn’t. Find your own way to any login pages you use, and never let yourself be frightened or cajoled into relying on contact data provided in an email.”
It is unclear if anyone outside of Comparitech and Microsoft accessed the customer service records during the breach window, but anyone who has conducted an online technical support chat with Microsoft in the past 14 years should be wary of potential scam attempts that make use of that information.
A rough stretch for Microsoft
2020 has gotten off to a rough start for Microsoft in terms of security issues. Two weeks ago, the company was forced to push an emergency security update to all users after the NSA found a critical vulnerability in its cryptographic systems. Only days later, a serious vulnerability was discovered in Internet Explorer that the company has not opted to patch despite declaring ongoing support for the browser for at least the rest of the active life of Windows 10.
Microsoft also suffered a similar internal data breach last year. In April, hackers were able to breach the company’s internal customer support network and may have had access to the contents of the email accounts of some Outlook users.
Roger Grimes, Data-Driven Defense Evangelist for KnowBe4, shared some thoughts on how Microsoft should recover:
“This is a fairly common type of hack. Overly permissive permissions abound on servers and cloud products all over the Internet. Having worked for Microsoft for 15 years, 11 years as a full-time employee, I’ve seen firsthand how much they try to fight scenarios like this. There are multiple layers of controls and education designed to stop it from happening. And it shows you how hard it is to prevent it 100% of the time. Nothing is perfect. Mistakes and leaks happen. Every organization has overly permissive permissions. Every! It’s just a matter of if someone outside the organization discovers it or if someone takes advantage of it. In this case, as bad as it is, it was discovered by someone who didn’t do malicious things with it. Sure, the data, sitting unprotected, could have also been used by the bad guys, but so far no one has made that case or provided evidence that it has been used maliciously. So far, all that is known is that a security researcher found it and reported it. That’s a pretty good outcome if that’s all that happens. The question is how any organization treats a report like this. I know Microsoft is treating it seriously and looking at how it happened. Because that’s the most important part. Anyone can have a mistake. The most important question is how the mistake happened and how to prevent it from happening next time, and if there are any others that could have happened from the same set of circumstances. As long as the organization uses the report to aggressively figure out what happened and fix it, it’s not necessarily the worst thing to happen…to the organization or the customer’s data they protect. It makes it more likely that customers’ data will be better protected now and in the future. I know Microsoft is going to do that. But if another organization got the same report, fixed the permissions, and went on with life without finding out HOW it happened…well, that’s a different story.”
CTO and Co-Founder Chris DeRamus of DivvyCloud had a similar perspective on how both Microsoft and organizations at large can improve network security and prevent similar incidents from happening:
“Misconfiguring a cloud server can have massive consequences, especially when the server contains hundreds of millions of customers’ records. Aside from this incident with Microsoft, we have seen misconfigured Elasticsearch servers become an increasingly common culprit that recently caused data leaks at companies including Rubrik, Voipo, Gearbest, Meditab, and Dow Jones.
“What sticks out about this incident is the fact that in early November 2019, Microsoft announced that it will honor CCPA throughout the U.S., and it was the first company to extend GDPR rights to customers around the world. This shows that even a forward-thinking company like Microsoft, who is unrelentingly dedicated to protecting their customers, can suffer a data breach due to misconfigurations. If they can be affected, anyone can. This illustrates that being compliant does not guarantee that you are secure, especially for companies that have adopted cloud and multi-cloud environments. The software-defined nature of the cloud leads to frequent changes and it is important that organizations implement a continuous and automated cloud security strategy in order to detect and remediate threats such as misconfigurations and compliance violations in real-time.
“Additionally, organizations must be cognizant of their cloud service providers’ storage access policies and use these policies to define access. Microsoft must ensure that their security team understands that incorrectly configured policies can result in costly damages. In this instance, because the records exposed include customer email and IP addresses, affected customers should be on high alert for phishing scams.”