- Azure Sentinel—A real-world example - Tue, Oct 12 2021
- Deploying Windows Hello for Business - Wed, Aug 4 2021
- Azure Purview: Data governance for on-premises, multicloud, and SaaS data - Wed, Feb 17 2021
This article will look at Azure Sentinel as it turns one and a half years old, how it works, and why you should consider it if you don't have an SIEM today or if you're looking to replace an existing solution. Note that the foundations of Sentinel are a lot older than this—it's built on mature Azure technology building blocks (Log Analytics, Logic Apps, etc.) and it's the direct descendant of Microsoft's internal security technology that is used to protect Azure, Microsoft 365, Bing, Xbox, etc., every day.
Sentinel Basics ^
A SIEM is a central storage location for all your security and event logs from (ideally) all nodes on your network. Endpoints, switches, routers, firewalls, proxies, VMs, cloud apps, etc. all forward their logs to this central location where (again, ideally) the data is analyzed, events correlated, and alerts raised as suspicious activity is detected. It's the tool that gives you the visibility that a "zero trust/assume breach" approach demands.
"Zero trust" replaces the traditional "if you're on the corporate network, you are trusted" approach with one that evaluates each device connection, each login for each application/sensitive data for trustworthiness, and applies policies accordingly. "Assume breach," on the other hand, moves the focus from just protecting the network and its resources toward adding planning, monitoring, and network segmentation for the time when you are compromised.
I think it's fair to say that on-premises SIEMs have their challenges, often requiring a lot of care and feeding to maintain the infrastructure, taking away from the actual work of spotting intrusions. Furthermore, as licensing costs are often based on the amount of data stored, choices are made to limit sources or the length of time log data is stored, which decreases visibility.
It's no coincidence that both Google (Chronicle) and Microsoft (Sentinel) announced their cloud-based SIEMs at nearly the same time, both promoting the benefits of a cloud-based solution as requiring less infrastructure maintenance.
Once you have created a Sentinel workspace, you need to connect your data sources. As mentioned above, if you're using Office 365, its log data can be ingested and analyzed at no extra cost. This is an excellent way to get started, as is the 31-day free trial.
With the data in place, you can use built-in or custom alerts to find out when something fishy is going on as well as trawl through the data with advanced hunting.
Connecting security data sources ^
The process for connecting sources varies. In general, cloud/API-based solutions are straightforward, whereas on-premises locations involve more work. The number of supported sources is too long to list here and is getting longer all the time, but some highlights include AWS (CloudTrail), Azure AD, Azure Defender (formerly Azure Security Center), Barracuda, F5, Forcepoint, and Zimperium. Using Common Event Format (CEF), you can import Check Point, Cisco, Fortinet, Palo Alto, ZScaler, and many others.
Many data sources also come with preconfigured workbooks, query samples, and analytic templates, helping you to get started with understanding the data.
Workbooks and notebooks ^
The Office 365 Connector, for example, provides Exchange, SharePoint, and overall Microsoft 365 workbooks, essentially interactive dashboards that allow you to visualize and drill into the activity data.
Microsoft 365 workbook
The query language for Sentinel (and the underlying Log Analytics platform in Azure) is Kusto Query Language (KQL), which has similarities to SQL (somewhat easing the learning curve). You can run simple queries directly in the Sentinel UI, and most connectors provide a set of sample queries. Here, I am running a query to identify new admin account activity.
If your queries grow and you need to add in more code and explanations for others to follow, you should look at notebooks along with visualizations. Notebooks are built on new machine learning (ML) functionality in Azure, which in turn is built on Jupyter notebooks.
Hunting and threat intelligence ^
As you become more familiar with digging through the log data, looking for signs of intrusions, you'll start hunting, customizing the built-in queries, but eventually branching out to build your own. Each query matches one or more Mitre Att&ck tactics and techniques. If you're not familiar, Mitre's framework is a way to categorize different attack methods, as well as link these to different groups of attackers. It's all free data; you can explore more here.
If you're seeing something happening that's of interest, you can add bookmarks and comments. You can also switch to Livestream mode, which will query the data as it's being streamed into Sentinel to catch attacks in real time.
Another essential part of any SIEM is the ability to bring in Threat Intelligence (TI) feeds, with up-to-date indicators of risk (v4/v6 IP addresses, domain names, file hashes for malicious content, and URLs) so that if any of those show up in your logs, they can be flagged. You can now manually add indicators on the new TI blade as well.
For many years, the default behavior of most IT security products was "spot something potentially malicious, alert a human." This has led to several problems. One is that as the number of alerts increases, you need to add more humans to scale. Second—and this is especially noticeable in environments with multiple disparate security products—alerts in separate products are not easily correlated, even though they may be related to the same attack. Sentinel uses ML to correlate low-fidelity alerts that may not be noteworthy by themselves, but when looked at in aggregate across machines and networks, produce high-fidelity incidents with a low rate of false positives (Microsoft calls this Fusion). Here you can see a low-severity incident involving rare Exchange admin activity.
When you investigate incidents, you get a graph that shows you the relationship between different devices, user accounts, and processes, helping visual creatures understand the issue quicker. Here's an example of the investigation graph for an anomalous login. This screenshot also shows the timeline view, which correlates activities over time.
Watchlist and Playbooks ^
A recent addition is the ability to bring in CSV files of non-security data, such as lists of assets, high-value accounts, recently terminated employees, etc. and use this data in queries to provide additional context.
Sentinel also provides Security Orchestration, Automation, and Response (SOAR) capabilities through logic apps. The idea here is again to automate the low-level work and free up SOC analysts to focus on actual incident management.
As an example, if an unusual travel incident (I'd think most travel at the moment is unusual) is raised, a playbook can automatically send an email or a text message to the user and/or their manager to verify if it indeed was them logging in from this unusual country. If it was, the alert is closed without any human having to be involved; if not, it's flagged as a serious incident. Playbooks can also be used to block attacks automatically, force a user to perform MFA, or enrich the information in an incident before it's shown to an analyst, and much more.
Entity behavior and community ^
User and Entity Behavior Analytics (UEBA), in public preview since Ignite 2020, only takes a week or two to build up a pattern of each user's and device's routine activity, which can then be used to flag abnormal behavior in alerts. The data that Sentinel uses ML to build up is also stored in Sentinel so an analyst can use these artifacts when building custom queries.
Another feature now in preview is the ability to build your own ML models on top of Sentinel data.
The real strength of Sentinel, however, and why I think it's growing into a serious contender as a SIEM for enterprises, as well as a very attractive option for SMBs, is the community around it. There's a thriving GitHub repository with Microsoft and community contributed hunting queries, workbooks, notebooks, playbooks, and analytic rules.
Subscribe to 4sysops newsletter!
If you're interested in learning how to use Sentinel, this page provides training videos, etc. This page, on the other hand, lets you deploy a lab environment with VMs, Sentinel, and simulated data (taken from this site), with a single click.