All posts by jaapwesselius

Exchange 2016 CU15 and Exchange 2019 CU4 released

On December 17, 2019 Microsoft has released its quarterly updates for Exchange Server:

  • Exchange 2016 CU15.
  • Exchange 2019 CU4 (only available via Volume License).

There are a couple of things that are worth noting:

  1. Both Exchange server versions need the .NET Framework 4.8. If you are running an older version of Exchange (much older) consult Michel de Rooij’s blogpost Upgrade Paths for CU’s and .NET.
  2. If you are running an Exchange Hybrid version there’s the n-1 policy. This means your on-premises versions of Exchange should be Exchange 2016 CU14 or Exchange 2019 CU3 at minimum.
  3. There’s an update on the Exchange calculator which is now version 10.3.
  4. There are no schema changes compared to the previous version so there’s no need to run Setup.exe /PrepareSchema. I always recommend running Setup.exe /PrepareAD to make sure any additional features or changes like (for example) RBAC are applied correctly.

So now real new features which is in line with Microsoft’s strategy. If you need the latest and greatest then Exchange Online is the way to go. If you need a stable on-premises environment you’re good with Exchange 2016 or Exchange 2019.

More information

Released: December 2019 Quarterly Exchange Updates
Exchange 2016 CU15 Download: https://www.microsoft.com/en-us/download/details.aspx?id=100780
Exchange 2016 CU15 UM Pack: https://www.microsoft.com/en-us/download/details.aspx?id=100781

 

Outlook 2010 stays offline with Exchange Online

One of my clients is running Exchange 2010 in hybrid mode, and they have Outlook 2010 and Outlook 365 ProPlus client. For testing purposes, I have two VMs, one with Windows 7 and Office 2010 and one with Windows 10 and Office 365 ProPlus. And every Monday morning I run the Windows 7 VM for an hour or so to see if everything is working fine 😊

This morning my Outlook 2010 was working offline, and it didn’t want to go online (OWA and Outlook 365 ProPlus were working fine). Remove the Outlook profile but creating a new Outlook profile didn’t work. After a minute the dreaded an encrypted connection to your mail server is not available error message appeared:

An encrypted connection to your mail server is not available

Mostly this is caused by Autodiscover that goes wrong somewhere, the Remote Connectivity Analyzer shows that Autodiscover to the on-premises Exchange 2010 goes well, but that the redirect to Exchange Online goes wrong and it generates the following error message:

An HTTP 456 Unauthorized response was received from the remote Unknown server. This indicates that the user may not have logged on for the first time, or the account may be locked. To logon, go to http://portal.microsoftonline.com.

And further down more details are revealed:

X-AutoDiscovery-Error: LiveIdBasicAuth:AppPasswordRequired:<RequestId=8a51c25b-9213-4873-aff8-ebc1da40544f>;

An HTTP 456 Unauthorized response was received from the remote Unknown server

The AppPasswordRequired explains more. Last week I changed the MFA settings (see previous authenticator app for Office 365 blogpost). This works fine for OWA and Office 365 ProPlus, but not for Outlook 2010. Since Outlook 2010 does not work with Office 365 MFA, especially not in a hybrid environment (not even with an App Password).

The only workaround here was to temporarily disable MFA for my user account, create a new Outlook profile (which worked fine without MFA) and re-enable MFA. Again, Outlook 2010 does not recognize the MFA and still works with Exchange Online using basic authentication, but all other Office 365 services work fine with Office 365 MFA (both SMS and Authenticator authentication).

Authenticator app for Office 365

I have been running MFA for Office 365 user accounts up-and-running for quite some time now and very satisfied with it. But as you may have seen in the blogpost, I have been running SMS only, and with a 30 days renewal that works fine. But I was also interested in the Authenticator app, especially when running multiple clients on mobile devices.

Changing the authentication can be done on a per-user basis. Logon to the Microsoft portal (portal.office.com) using your regular work account. Select My Account (under your thumbnail profile picture) and select Security and Privacy and click Additional security verification as shown in the following screenshot:

Select Update your phone numbers used for account security, check the Authenticator app or token checkbox and click Setup authenticator app button.

Scan the QR code on your mobile device in the authenticator app, confirm the registration, click Save and you’re all set. The next time you logon to Office 365 you’ll see the following Approve Sign in Request window:

But instead of entering a verification code received via SMS you must approve the sign in on the Authenticator app.

SPF and DMARC when domain is not used for email

Just a quick post on SPF and DMARC when you have a domain that’s not used for email. In this scenario mail will never be sent out by any mailserver. If someone does send out email, it is most likely malicious email and can be ignored.

You can add the following records to your DNS:

SPF:

V=spf1 -all

DMARC:

v=DMARC1;p=reject;sp=reject;pct=100

Receiving mail servers that check for SPF and DMARC will see that it’s not valid and will reject the message.

 

Free/busy not working from Exchange Online to Exchange on-premises

Recently users started to complain that free/busy information was not available, more specifically users that had their mailbox in Exchange Online were not able to retrieve availability information from their colleagues or meeting rooms that were still in Exchange 2010 on-premises.

Complaints came from multiple users from multiple countries, there are multiple sites with multiple Exchange 2010 servers with multiple breakout points to the Internet, so the issue was consistent and not really related to only one Exchange 2010 server. And it was only happening cross-premises, and only from Exchange Online to Exchange 2010. From Exchange 2010 to Exchange Online it was working flawlessly, so was between mailboxes in Exchange 2010.

You see this happen often right after configuring an Exchange hybrid configuration with the HCW, but in my case it had been working fine for quite some time, so it had to be related to something we had changed recently. We had WLAN changes (new provider), Windows Update, Exchange 2010 rollup updates, SSL certificates, new Send and Receive Connectors, but nothing that immediately pointed in the right direction. To make things worse, the Remote Connectivity Analyzer (your first stop when troubleshooting) didn’t see any issues, everything worked well. Autodiscover returned the correct information from mailboxes in Exchange Online and Exchange 2010, and the free/busy test in RCA worked well.

Note. A lot of people don’t know this feature, but in the Remote Connectivity Analyzer you can check free/busy in a hybrid environment. Just select the Free/Busy radio button under the Office 365 tab as shown in the following screenshot:

Remote Connectivity Analyzer FreeBusy

My mailbox was in Exchange Online and I also did experience the issue, but at the same time I was able to open the calendar cross-premises. Ok, this is Outlook Anywhere and not Exchange Web Services (which is used for free/busy) but at least it ruled out a firewall issue.

When running the Get-OrganizationRelationship command, I could verify the TargetAutodiscoverEpr property, which was set to the correct Autodiscover URL. Using the TargetSharingEpr property instead of the TargetAutodiscoverEpr property didn’t help.

Get-FederationInformation command with the -DomainName switch… all looks good.
Security on the Virtual Directory? Running these commands often solve unexpected issues:

Get-AutodiscoverVirtualDirectory | Set-AutodiscoverVirtualDirectory -WSSecurity $true
Get-WebServicesVirtualDirectory -Server WPGREXC01 | Set-WebServicesVirtualDirectory -WSSecurity $True

Again, no success… Time for deeper troubleshooting. At this moment Microsoft support was already involved as well….

When testing I could see the request entering the Exchange 2010 server in the IIS logs, but the servers returned a 500 Error, so something in the request was causing issues on the Exchange server:


2019-10-14 06:45:06 192.168.25.119 POST /ews/exchange.asmx/WSSecurity – 443 – 52.97.155.157 ASProxy/CrossForest/Directory/https/15.20.2347.020/MailTips – 500 0 0 31


2019-10-14 06:45:33 192.168.25.119 POST /ews/exchange.asmx/WSSecurity – 443 – 52.97.140.165 ASProxy/CrossForest/Directory/https/15.20.2347.020/MailTips – 500 0 0 31


2019-10-14 06:45:51 192.168.25.119 POST /ews/exchange.asmx/WSSecurity – 443 – 52.97.139.125 ASProxy/CrossForest/Directory/https/15.20.2347.020/Freebusy – 500 0 0 15


2019-10-14 06:45:51 192.168.25.119 POST /ews/exchange.asmx/WSSecurity – 443 – 52.97.158.5 ASProxy/CrossForest/Directory/https/15.20.2347.020/MailTips – 500 0 0 31


2019-10-14 06:45:51 192.168.25.119 POST /ews/exchange.asmx/WSSecurity – 443 – 52.97.139.125 ASProxy/CrossForest/Directory/https/15.20.2347.020/MailTips – 500 0 0 31

The next step to try was to recycle the AutodiscoverAppPool and the ExchangeServicesAppPool in the IIS Manager, but unfortunately this didn’t help.
After looking out of the window what might be the issue I had another look at the eventlog on the Exchange server, and found the following certificate warning:

EventID 403 Certificate is expired


Log Name: Application
Source: MSExchange Common
Date: 10/16/2019 9:28:59 AM
Event ID: 403
Task Category: Configuration
Level: Error
Keywords: Classic
User: N/A
Computer: Exchange.contoso.com
Description:
The certificate named ‘791BC6AD9893AA570DF03452B4F8069C8A743C29’ in the Federation Trust ‘Microsoft Federation Gateway’ is expired. Please review the Federation Trust properties and the certificates installed in the certificate store of the server.

This rings a bell. The certificate with the thumbprint mentioned in the error message is not on the Exchange server, but it’s in the Microsoft Federation Gateway. I didn’t see this earlier, but when checking the federation with Get-FederationTrust | FL you can see certificate information, and one certificate expired some time ago. July 2019 to be precise.

You can also run the Test-FederationTrust on the Exchange server. If you ran into this issue, you should see an error message like Failed to validate delegation token in the TokenValidation section.

Fixing this is easy, just run the following command:

Get-FederationTrust | Set-FederationTrust -RefreshMetadata

After running this command it works like a charm.

This only happens in Exchange 2010 (in Exchange 2013 and higher it is fixed automatically), and looking at the support chances are you are not running Exchange 2010 anymore when other certificates are renewed. You can running the command mentioned before on a regular basis, or you can use a scheduled task to perform this automatically on a regular basis.

Schtasks /create /sc Daily /tn FedRefresh /tr “C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
-version 2.0 -command Add-PSSnapIn Microsoft.Exchange.Management.PowerShell.E2010;
$fedTrust = Get-FederationTrust;Set-FederationTrust -Identity $fedTrust.Name -RefreshMetadata” /ru System

So why didn’t we notice this in the first place? The certificate change was announced by Microsoft and other community blogs, but this was summer holiday time. Also lack of resources in IT Staff didn’t help either. Too bad, but in the end it was fixed (with help of Microsoft support 😊)

More information