Tag Archives: Monitoring

ExoPrise Cloud Monitoring solution

When you are running an on-premises IT environment you most likely have some kind of monitoring solution. If something goes wrong in the infrastructure you are notified almost immediately, and you can take appropriate action.

Things are different when using cloud services. Your services are running in a datacenter somewhere else, controlled by another organization and with an internet connection (that sometimes can be unreliable).

For a customer I checked out ExoPrise, a SaaS (Software as a Service) based cloud monitoring solution. With surprising results.

ExoPrise private site

Exoprise is a cloud monitoring solution which is offered as a SaaS solution. This means Exoprise is running in a datacenter somewhere, and you have a subscription for using the services. Because it is a SaaS solution, installing and configuring the monitoring solution is just a matter of minutes.

Exoprise is running in a datacenter somewhere, but when using Exoprise you configure your own private site. A private site is a WIN32 service running on a Windows server in your own environment. From this private site, cloud services are monitored. From an end-user perspective, the private site has the same user experience as your local users have.

You can install and configure multiple sensors in your private site. Each service has its own sensor, there are sensors for Exchange Online, Microsoft Teams (Messages and A/V), Skype for Business, Free/Busy, ADFS, OneDrive, SSLCheck, Amazon, Google, Salesforce …. Tons of sensors are available.

When configured, a sensor performs synthetic transactions against the cloud service. For example, the Exchange Online sensor looks at the average logon time, message transfer speed and network latency. The results are shown graphically for your own sensor, but because it is a SaaS solution Exoprise can compare your results against the results of the rest of the world which they call ‘crowd’. This is shown in the following screenshot, the lower line is my own sensor, the upper (thinner) line is the crowd average.

With all this working from home due to the Covid-19 crisis a lot of organization have been implementing Microsoft Teams rapidly. As an admin you want to know you Teams is performing from your local environment. When configuring a Teams AV sensor it performs all kinds or synthetic transactions, very similar to the transactions a regular user is performing. The sensor is using a test identity in your Teams environment and it uses a bot in Exoprise to communicate with. This way it can measure the call quality of Teams and it measure logon time, A/V streams (audio jitter, packet loss, bitrate), frames per second etc.

When configuring a sensor, an alarm is automatically created. This alarm can be configured during creation:

When the sensor is triggered because of a transaction, an alarm is sent to the email address that is configured and this gives you an immediate overview that something is wrong with the service. And to be honest, I was a bit surprised how often alarms are generated and thus how often you will receive an email. The email will show which sensor is generating the alarm, some analysis information and alarm details as shown in the following screenshot.

Besides the alarm message you’ll also get a weekly overview with real user performance data that is gathered from ‘the crowd’ so it averages over all Exoprise sensors that are deployed. This will show overall trends for all cloud services, for example:

Summary

There are multiple cloud monitoring solutions available and I had the opportunity to have a look at the ExoPrise solution. I was surprised by the ease of configuration and ease of use. It is a cloud based service, pull your credit card and you’ll be working within 20 minutes. It is a great tool for monitoring your environment, but at the same time it is a great tool for troubleshooting purposes (when you are a consultant).

I was also surprised by the data that was gathered by the ExoPrise sensors. It shows immediately when something is wrong, and you are notified before your users start calling the servicedesk that something is not working. And that happened more often than I thought before.

 

Basic Authentication in Office 365 Part II

Update. Microsoft has changed their plans due to the Covid-19 crisis going on at the moment. Support for Basic Authentication in Exchange Online has been postponed to the second half of 2021 according to their blogpost on Basic Authentication and Exchange Online – April 2020 Update.

There are a few things to be aware of. For new tenants, Basic Authentication is already turned off, for older tenants it is still turned on. However, if Basic Authentication has not been used in a tenant it will be turned off as well. This will start upcoming October.

In my previous blogpost I explained more about basic and modern authentication, how they work and how to identify which method your outlook client is using. In this blogpost I will explain more about monitoring basic authentication to find out which clients are currently still using basic authentication in your Office 365 environment. I will continue with how to disable basic authentication and how to test what might happen.

Monitoring Basic Authentication

In my previous blogpost I explained a bit more about basic authentication and how to identify it, and the working of modern authentication.

The next step is to identify how many users and application are actually using basic authentication in your Office 365 environment. After all, these are the users that are impacted when Microsoft stops basic authentication.

To identify this, logon to the Azure Active Directory Portal (https://aad.portal.azure.com) and select sign-ins (under Monitoring). There you will see an overview of all sign-ins in Azure AD, successful and failed, for all clients, all services and all locations. An example is shown in the following screenshot (click to enlarge):

This shows all logins in Azure AD, for all aplications and services, failed and successful. You can use the Add Filters button to narrow down the information, in this blogpost to show only information regarding Basic Authentication.

To do this, click Add Filter | Select Client App | Click Apply

Click on “Client App: None Selected” and select all options except Browser and Mobile Apps and Desktop Clients as shown in the following screenshot (click to enlarge):

Modern Authentication Clients

Note. Updated the screenshot on April 6, 2020. Microsoft made a nice GUI enhancement here to easily identify different clients (modern vs legacy).

Now an overview will be shown of all basic authentication attempts in your environment. When you select one entry it will show additional details, including the client application, the username and the the user agent (which identifies the client app) as shown in the following screenshot (click to enlarge):

Another interesting thing is that you can identify where all failed basic authentication attempts are coming from. Add a filter Status | Failure and you will see only failed attempts. Some are legitimate (typo when entering password) but most of them are just brute force attacks. The following screenshot shows attempts coming from Russia, Thailand and New Caledonia, located where we don’t have offices. You can also see that the attempt is coming from a script (User agent CBAInPROD) and that it’s using IMAP4 (which is disabled for all mailboxes). This is one reason why you want to disable basic authentication in your tenant. Click to enlarge:

This is an easy way to identify mobile clients that use ActiveSync as a protocol and thus are using basic authentication. Apple iOS native mail client support OAuth2 since iOS11, so all recent iPhones are using modern authentication. For the Android native mail client things are different. The native Gmail client support OAuth2 but cannot be used of course with Office 365. Most other mail clients do not support OAuth2 yet, so these are using basic authentication and will run into issues when Microsoft stops basic authentication. In other words, these clients will stop working. Change the Client App filter to Exchange ActiveSync only and remove the Status | Failure filter. It will show a list of mobile users that use basic authentication as shown in the following screenshot (username is removed for privacy reasons) (click to enlarge):

Note. Outlook for iOS and Outlook for Android are using OAth2 so these will continue to work.

So, using the filtering options on the sign-in page in the Azure AD portal you can identify which clients are still using basic authentication when accessing Office 365 services (and thus which clients are impacted when basic authentication is stopped).

Disabling basic authentication

It is possible to disable basic authentication in your Office 365 by creating an Authentication Policy and apply this policy to users. Once applied they can no longer use basic authentication to logon to any Office 365 service. To create a new Authentication Policy use the following command in Exchange Online PowerShell:

[PS] C:\> New-AuthenticationPolicy -Name “Block Basic Authentication”

To add a user to the policy and effectively block basic authentication for this user you can use the following command in Exchange Online PowerShell:

[PS] C:\> Set-User -Identity j.wesselius@exchangelabs.nl -AuthenticationPolicy “Block Basic Authentication”

It will take up to 24 hours before this policy is effective. To take the policy effect (almost immediately, or at least within 30 minutes) you can use the following command:

[PS] C:\> Set-User -Identity j.wesselius@exchangelabs.nl -AuthenticationPolicy “Block Basic Authentication” -STSRefreshTokensValidFrom $([System.DateTime]::UtcNow)

To remove a user from an authentication policy you can use $Null for the authentication policy:

[PS] C:\> Set-User -Identity j.wesselius@exchangelabs.nl -AuthenticationPolicy $Null

When you have a number of users added to this authentication policy you can start testing with various clients and create a table with clients and scenarios, like the table below:

Client Results
Office 2010 Stops working (keeps asking for password)
Office 2013/2016 Continues to work (was already using Modern Authentication)
Outlook 2010 on-premises mailbox, cross-premises free/busy Continues to work, but need further investigation (note 1)
Outlook 2013/2016 on-premises mailbox, cross-premises free/busy Continues to work
iPhone 8, iOS13, native mailclient Continues to work
iPhone 8, iOS13, Outlook for iOS Continues to work
Samsung A10, Android 9, native Email client 6.1.11.6 Stops working
Samsung A10, Android 9, AquaMail (by MobiSystems, supports OAuth) Continues to work (note 2)
Samsung A10, Android 9, Outlook for Android Continues to work
Exchange Online PowerShell New-PSSession Stops working (note 3)
Exchange Online PowerShell module Continues to work
Exchange PowerShell V2 Continues to work
POP3 Clients TBD
IMAP4 Clients TBD

Note 1. In this scenario an Outlook client is using an on-premises mailbox but tries to retrieve free/busy information from a mailbox that’s in Exchange Online. Both accounts have basic authentication disabled in Azure AD.

Note 2. The native mailclient in Android 9 (on my Samsung A10) only supports basic authentication. This is not a device limitation but an application limitation. AquaMail (from MobiSystems) for example does support OAuth and keeps working when basic authentication is disabled. AquaMail however is not a free application but a subscription based application.

Note 3. It is possible to connect to Exchange Online as shown in line 9 of the table using the following method:

$ExCred = Get-Credential TenantAdminAccount
$Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://ps.outlook.com/powershell/ -Credential $ExCred -Authentication Basic -AllowRedirection
Import-PSSession $Session

This is using basic authentication and will stop working. However, you should not use this way of working anyway because it does not support MFA, which is a recommended best practice for admin accounts! For more information please check Multi Factor Authentication MFA in Office 365 for admin accounts.

Summary

In the previous two blogposts I tried to explain a bit more about basic and modern authentication, and what might happen when Microsoft ends support for basic authentication in Exchange Online next October.

For sure, things will break when connecting to Exchange Online. The most obvious is Outlook 2010 which won’t connect anymore. Native mobile clients that do not support oAuth2 (common in Android mail apps, but also older iPhones) stop working too. If you don’t act now you will be in a lot of trouble when Microsoft makes the change.

For now, start testing using the options I explained in this second blogpost. Create your own list of apps and services that use basic authentication and start testing with an authentication policy that blocks basic authentication. That’s the only way to prepare for this major (mega major) upcoming change. But in the end, we will all benefit from a security point of view.

More information

Database MDB02 is low on log volume space

My Exchange 2013 (CU7) servers are logging warning approx. every hour regarding the free available disk space on the Mailbox database volumes in the

Application and Services | Microsoft | Exchange | Managed Availability | Monitoring crimson channel in the eventlog. The threshold for this is set to 175GB, and especially for lab environment this can be too low.

The following event is logged:

Log Name: Microsoft-Exchange-ManagedAvailability/Monitoring
Source: Microsoft-Exchange-ManagedAvailability
Date: 1/22/2015 12:07:46 PM
Event ID: 4
Task Category: Monitoring
Level: Error
Keywords:
User: SYSTEM
Computer: EXCH01.contoso.com
Description:
Database ‘MDB02’ is low on log volume space. ‘MDB02’ is low on log volume space [F:\]. Current=86,97 GB, Threshold=175,78 GB
Continue reading Database MDB02 is low on log volume space