Latest posts by Paul Schnackenburg (see all)
- Office 365 Secure Score – Securing Exchange Online - Thu, Aug 3 2017
- Office 365 Secure Score - Reporting and monitoring - Tue, Aug 1 2017
- Office 365 Secure Score - MFA for users and auditing - Mon, Jul 31 2017
Reporting in Office 365 ^
The first report is successful sign-ins after multiple failures. This could be a sign of a successful brute-force password-hacking attempt or a very forgetful user. To qualify for the 15-point score increase, you'll need to review this report weekly. Log in at https://securescore.office.com, pick Learn more for the signs-ins after multiple failures report, and then click the Review button. This takes you (at the time of writing) to the old Azure portal and a list of reports.
Another report that'll give you 10 points if you look at it weekly is the sign-ins from unknown sources report. This one looks at logins from anonymous proxy IP addresses, such as the Tor network. It's highly unlikely that your users have a valid reason to access Office 365 anonymously, so this could give you an indication of a breached account.
Next on the list for 10 points is the signs-ins from multiple geographies report. This one covers "impossible travel" situations. In these, logins from a single user account occur from two separate locations such that it would be impossible for a user to travel between the locations in that time. Note that if a user has shared a password with a co-worker, is using a VPN, or if the IP geography assignment isn't accurate, this could provide false positives in the report. But in general, it's a good sign that one of the logins is by an attacker.
Continuing with the theme of reports, this time for 5 points, have a look at mailbox access by non-owners. This one is good for spotting malicious access to user mailboxes. Note however that users often have legitimate access to other people's mailboxes, so make sure you know what's legitimate and what's not.
Another 5-point report is the malware detections report. This shows how many times Microsoft blocked a malware attachment before it got to your users. The thinking here is that if you see an increase in attacks, you might want to warn your users about a particular type of malware.
Note that by default you probably don't have access to this report. Head over to https://outlook.office365.com/ecp, click on permissions, admin roles, and add your account to the Hygiene Management role.
Tracking user accounts and devices in Office 365 ^
Another thing to review weekly for 10 points is check role group changes. You could do this by checking the membership manually in the portal Groups. You can also filter the audit log search for role administration activity. The point is to catch suspect role group changes where attackers may be elevating their privileges.
For 10 points, you should also have a look at the device sign-in report weekly. This tracks devices possibly infected with malware or new device sign-ins from a possibly compromised account.
For 5 points, have a look at the account provisioning activity report. If you use a third-party application or service for user provisioning, this report will show expected traffic. But if you don’t, consider any records in this report malicious.
Note that for Secure Score to record the review of a report, you have to open it from the Secure Score portal. If you just go to the AAD portal and open them, it won't record the views.
These are lots of reports to look at, especially on a weekly basis. But Microsoft's point here I think is to show the line between your responsibility for security as a consumer of a cloud service and Microsoft's responsibility. There's a misconception that "if it's in the cloud, security is the provider's responsibility."
Yes, Microsoft manages some areas such as physical access to data centers, server and network protection, and governance over their IT management engineers. But endpoint security, good configuration, and most importantly, monitoring of what's going on in your tenant is your responsibility. And these reports are a good way to start. Another option for monitoring is to use the System Center Operations Manager (SCOM) Office 365 management pack, which is a bit limited in the value it provides; there's more information here.
There's also an Operations Management Suite (OMS) solution for Office 365—read more here. This solution provides similar information to the audit logs we looked at in part 2, but the power of OMS of course is the log analytics part. This enables you to correlate SharePoint and Exchange usage data along with user and group changes in AAD.
In the fourth and final piece of this series, we'll look at other Secure Score actions along with what's required if you want to do all possible actions. In addition, we'll explore email tracking and some other recommendations around security.