You can follow him on Twitter @Ruben8Z.
- Monitoring Microsoft 365 with SCOM and the NiCE Active 365 Management Pack - Tue, Feb 7 2023
- SCOM.Addons.MailIn: Monitor anything that can send email with SCOM - Mon, May 25 2020
- Display a user’s logged-on computer in Active Directory Users and Computers (ADUC) - Mon, Jan 21 2019
This review starts by explaining the solution architecture of NiCE Active 365 MP 4.1, followed by installation and configuration. Next, we cover my favorite key functions for monitoring SharePoint Online and OneDrive, Teams, Exchange Online, and Exchange hybrid. Finally, Azure AD Connect monitoring is reviewed, ending with license and Azure AD service principal monitoring.
Monitoring with SCOM and the NiCE Active 365 Management Pack
Comparing NiCE with Microsoft's approach
The table below compares Microsoft's Management Pack for Microsoft 365 with the NiCE Active 365 Management Pack.
| Microsoft M365 MP | NiCE Active 365 MP | |
| Service Coverage | SharePoint Online and OneDrive combined (single site) Teams Exchange Online Exchange hybrid M365 licensing M365 portal health |
SharePoint Online (5 sites) OneDrive (5 accounts) Teams Exchange Online Exchange hybrid M365 licensing M365 portal health M365 Secure Score AAD Connect (AD Sync) AAD service principals (enterprise apps/app registrations) |
| Distributed Monitoring | Watcher nodes (agent process) | Collector stations (dedicated process) Multiple proxy–server configuration |
| Support | Microsoft Support Forum | Individual dedicated support |
| Customer Care | Microsoft Support Forum Community events |
Personal contact with architects/developers Community events |
| Feature Requests | Irregular, based on the popularity of suggestions in the SCOM Forum | Ensured, either as part of a subsequent release or as a custom development |
| Costs | Free | Based on tenant size (ranges) |
Solution architecture
Different setups allow for a tailored monitoring experience. NiCE Active 365 Collector Server resides on a SCOM management server or gateway server. It runs in its own context, retrieves performance data from Microsoft 365 and Azure AD, and runs the tests.
Online Only Mode is the choice if no on-prem Exchange Server exists.
High level architecture used when in online only mode
Hybrid Mode considers Exchange on-prem together with Exchange Online. Hybrid workloads come into play. Optionally, geo-proxies can be centrally configured to monitor Microsoft via proxy servers in remote locations. Geo-proxies can also work in online-only mode.
High level architecture for hybrid environments with proxies
Collector stations have been added in the current release. They perform all tests on commonly monitored clients, offering individual monitoring directly from remote locations. Azure Cache for Redis consolidates and optimizes data before it is loaded back into SCOM.
High level architecture used when CollectorStations are configured
Installation and configuration
Before starting, ensure that Microsoft 365 and Azure services can be contacted directly or via a proxy server in your network. Install .NET Framework 4.7.2, .NET Runtime 6, Visual C++, Azure AD MSOnline Module, and the Azure AD PowerShell Graph Module on the server that will run the NiCE Active 365 Collector Server.
User accounts are required for synthetic transactions on Exchange Server, Exchange Online, SharePoint Online, OneDrive, Teams, and AD Connect Server.
Additionally, one enterprise application is needed to retrieve information about Microsoft 365 components. It is worth knowing that no highly privileged accounts are required. The least privilege is implemented all the way through.
Service principal in Azure Active Directory used for secure data retrieval via Graph API
The tenant ID, user accounts, and client ID of all resources that were created up until now need to be noted in XML configuration files that already exist as templates.
Running the installation wizard will copy the management pack files and templates into distinct directories. After customizing the parameters in the configuration files, management pack files are imported as usual.
The documentation files describe all steps in detail.
Monitors and rules
SharePoint Online
Setting quotas on SharePoint Online sites mitigates the risk of running out of capacity for the whole tenant. Monitors for Storage Consumed Percentage and Least Storage Available (GB) provide a list of sites that are about to run out of space.
SharePointOnline Monitoring Least Storage Available GB for all sites
Up to five individual SharePoint sites can be monitored for availability and file transfer speed. This is even more useful when geo-proxies or collector stations are used to check from different locations or sites in the network.
SharePointOnline Monitoring Specified Site File Upload Latency
The Number of Active Files and Number of Files Stored metrics are meant to understand the usage of SharePoint Online. Knowing these metrics allows consultation with the business for better usage or consideration of the data lifecycle.
SharePointOnline Collecting number of active files on top N sites
Exchange Online
Like SharePoint Online, synthetic transactions are used to monitor mailflow availability and mailflow latency. In addition, one dedicated monitor tests the free/busy functionality and helps track health.
Exchange Online Free:Busy Check
Among the ways of measuring mailflow latency, Autodiscover Retrieval Duration and Mailbox Logon Duration are good indicators for measuring how email works for your organization.
Exchange Online Tracking Mailbox Logon Duration
Exchange hybrid
If there are still Exchange Servers in a local datacenter, end-to-end tests ensure that mailflow works and is performing as expected.
Exchange Hybrid Measuring Mailflow Receive Latency between On Prem and ExO
Eleven configurable tests allow testing the key functionality of the messaging infrastructure. Within the tests, both Mailflow Send & Receive capabilities and latency values are checked against thresholds.
Exchange Hybrid End to end tests for FreeBusy Mailflow Mail Queues and Mail Submission
OneDrive
Service availability and performance, measured from up to five different accounts, are possible here. When using geo-proxies or collector stations, speed and functionality can be simulated from different sites.
OneDrive Measuring file download in milliseconds
Counters for used storage, active files, and synced files help determine how accounts are utilized. If OneDrive is used for service accounts, these counters offer ideal monitoring.
Teams
Starting with monitors for service availability and time lag in chat messages, various underlying network parameters are measured and tested. Additionally, statistical information about the total duration and total calls is traced.
Teams Available performance counter
Networks that do not follow Microsoft's recommendations regarding name resolution or local outbreaks usually tend to have reduced quality in video or voice calls. Jitter, package loss, and round-trip time provide good indications about networks that might need improvement.
When using geo-proxies or collector stations, it becomes easier to spot differences between sites as measurements are recorded.
To keep SCOM databases well utilized, NiCE utilizes caching services in Azure. Only aggregated values are then stored locally.
Teams Performance counter for Average Package Loss Rate
Azure AD Connect
Azure AD Connect synchronizes Users, Groups, and Computer accounts between local Active Directory and Azure Active Directory. It can only run actively on one Windows Server and runs on a changeable schedule. Connectors, stages, and profiles allow detailed configuration of the service itself.
The management pack actively monitors the service state, connectors, and run profiles.
AAD Connect Monitoring Run Profiles
Performance rules track failures, transferred objects, and duration, and bring visibility and awareness of the synchronization.
AAD Connect Tracking Export Updates per connector
AAD service principals
Service principals are used as trust-binding objects, such as allowing a third-party service to use Azure Active Directory as an identity provider for single sign-on.
Often, a shared secret is used between Azure AD and a third-party service. This secret has a maximum lifetime of two years and must be replaced before expiration.
Azure Portal showing expiration on app registrations
The Service Principal view exposes all discovered service principals and their health state. Secret expiration and assignment compliance are considered for determining the status.
AAD service principals Showing discovered objects and their health state
A monitor for secret expiration alerts runs with a configurable threshold to avoid service outages.
Assignment Compliance is a monitor (disabled here) that helps identify which app registrations can be used without explicitly granting a user to it.
AAD service principals Monitoring Secret Expirations days
Side note: Azure AD Service Principal Monitoring was added as part of version 4.1 via a customer feature request.
Summary
NiCE has been providing IT monitoring solutions for more than two decades. With their recent Active M365 MP version 4.1, they offer a variety of performance rules and monitors that help track, verify, report, and allow the mitigation of risks when using M365.
Customers receive a management pack that is steadily evolving. Security is always considered, with least privilege access as a main principle.
Subscribe to 4sysops newsletter!
Feature requests are happily accepted, evaluated, and usually implemented individually. They can be accommodated either as part of the product so that they are available for all customers, as a private custom extension, or as an additional customer-sponsored public feature. A free trial is available on request.
