Exchange

Hotfix Update for Exchange 2016 and Exchange 2019

Leave a comment

Wait, what? On April 23, 2024 Microsoft has released a hotfix update for Exchange 2016 and Exchange 2019 and as MVP’s we only learned about this last week.

A hotfix update or HU contains fixes for issues that might arise with a security update in Exchange server. For example, the March 2024 SU for Exchange server introduced a number of issues, and these are fixed with this HU. Besided hotfixes, a HU can also contain new features that did not make it in the last security update (SU) or Cumulative Update (CU). In this HU for example, Hybrid Modern Authentication for OWA and ECP is introduced as a new feature. Another new feature introduced in this HU is the support for ECC (Elliptic Curve Cryptography) certificates. ECC certificates however are not supported for the federation trust certificate, the Exchange server OAuth certificate and ECC certificates cannot be used when ADFS claims-based authentication is used.

The following issues are fixed in this HU:

  • “We can’t open this document” error in OWA after installing March 2024 SU
  • Search error in Outlook cached mode after installing March 2024 SU
  • OwaDeepTestProbe and EacBackEndLogonProbe fail after installing March 2024 SU
  • Edit permissions option in the ECP can’t be edited
  • Outlook doesn’t display unread message icon after installing Exchange Server March 2024 SU
  • My Templates add-in doesn’t work after installing Exchange Server March 2024 SU
  • Download domains not working after installing the March 2024 SU

You can download this hotfix update for Exchange server here:

Exchange 2019 CU14 HU2 – https://www.microsoft.com/en-us/download/details.aspx?id=106021
Exchange 2019 CU13 HU6 – https://www.microsoft.com/en-us/download/details.aspx?id=106022
Exchange 2016 CU23 HU13 – https://www.microsoft.com/en-us/download/details.aspx?id=106023

Be aware that the filename for all versions of this HU is the same (Exchange2019-KB5037224-x64-en.exe) so when downloading multiple versions make sure you store them at different locations.

A hotfix update is cumulative and includes all security features and fixes from the previous security updates. When running Exchange 2019 CU14 and you have not installed the March 2024 security update then there’s no need to install this first. Just continue with the immediate installation of this HU.

More information

Exchange

Exchange 2019 CU14

Leave a comment

On February 13, 2024 Microsoft has released a new Cumulative Update for Exchange 2019, the 2024 H1 Cumulative Update (or CU14). One major ‘new’ feature and some minor features.

Extended Protection

One of the interesting things in this update is that it by default enables Extended Protection on all virtual directories of your Exchange 2019 server.

Extended protection is not new, it was introduced in August 2022 Security Updates. At the time I already wrote about, which you can read on https://jaapwesselius.com/2022/08/09/exchange-security-updates-august-2022/.

Extended Protection is an enhancement on the existing Windows Authentication. Extended Protection mitigates authentication relay or ‘man in the middle’ attacks. It is implemented by using channel binding information using a Channel Binding Token (CBT). There’s no need to configure anything, Extended Protection is automatically configured by the CU14 setup. It is a per server setting, so it is only enabled on the server you have installed CU14 on.

There are some prerequisites that need to be met before you can enable Extended Protection. Run the healthchecker.ps1 script and check the prequisites page for Extended Protection on the Microsoft page.

If you want to install CU14, but you environment does not meet the prerequisites, you can install CU14 unattended with the /DoNotEnableEP or /DoNotEnableEPFEEWS (EPFEEWS is Extended Protection FrontEnd EWS) switch.

Please be aware that this is not the recommended approach. The recommended approach is to prepare your environment for Extended Protection, enable it on older server and then install CU14 with automatic enabling Extended Protection.

TLS 1.3

Unfortunately TLS 1.3 (which is supported on Windows 2022) did not make it in this update, and is now scheduled to be included in CU15 (2024 H2 Cumulative Update, so by the end of this year).

CVE-2024-21410

CVE-2024-21410 was also released today, and this CVE is addressed in CU14 by Extended Protection. This is also true for Exchange 2016 CU23 (EP was introduced in an earlier Security Update for Exchange 2016).

Installing Exchange 2019 CU14

CU14 does not come with an Active Directory Schema update; the Schema is still at version 17003. The Exchange configuration version is increased by one (now 16762) and the Domain version has not changed (still 13243).

Use the following commands to upgrade the Configuration and Domain Partition in your environment:

Z:\>Setup.exe /PrepareAD /IAcceptExchangeServerLicenseTerms_DiagnosticDataOn

And:

Z:\>Setup.exe /PrepareDomain /IAcceptExchangeServerLicenseTerms_DiagnosticDataOn

You’ll notice the following output on the console:

Exchange Setup has enabled Extended Protection on all the virtual directories on this machine. For more information visit: https://aka.ms/EnableEPviaSetup. 

We recommend running Exchange HealthChecker script to evaluate if there are any configuration issues which can cause feature breakdowns. HealthChecker script can be downloaded from https://aka.ms/ExchangeSetupHC.

At this point, nothing has changed on the virtual directories of your Exchange server so the information is not needed at this point. Running the HealthChecker script is always a good thing before running any upgrade. When using the unattended upgrade, use the following command:

Z:\>Setup.EXE /Mode:Upgrade /IAcceptExchangeServerLicenseTerms_DiagnosticDataOn

And when needed, you can add the /DoNotEnableEP switch here. Or you can also use the GUI setup of course if you prefer to do that.

Some remarks about CU14

  • Make sure you fully understand the impact of Extended Protection. For more information check the Configure Extended Protection in Exchange server page on the Microsoft site.
  • CU14 supports the .NET Framework 4.8.1 on Windows Server 2022 (and only on Windows Server 2022)
  • This Cumulative Update contains all binaries from previous Cumulative Updates, including all previously released Security Updates.
  • Download CU14 at https://www.microsoft.com/en-us/download/details.aspx?id=105878
  • Knowledgebase article KB5035606 contains more information, including known issues with this release.
  • Before bringing in production, please test thoroughly in your test environment.
  • The only supported versions of Exchange 2019 are Exchange 2019 Cumulative Update 14 and Cumulative Update 13. Earlier versions are no longer supported (and no longer receive Security Updates)
  • Exchange 2013 is old, and out of extended support. No more Security Updates will be released for Exchange 2013. If you are still running Exchange 2013, make sure you migrate to Exchange 2019 or Exchange Online anytime soon!
  • Exchange 2016 is out of mainstream support, which means that Microsoft will no longer release new Cumulative Updates for Exchange 2016. Security Updates will be released until Extended Support for Exchange 2016 ends.
Exchange, Exchange Hybrid, Office365

Special characters in Active Directory and Exchange Online

Leave a comment

During migrations to Exchange Online I get the question regarding special characters in the User Principal Name (UPN) and e-mail address. Every time I have to check this again and again, so it’s time to do a write-up.

The UserPrincipalName (UPN)

The UserPrincipalName (UPN)

The UPN is the user’s identifier in Active Directory, and it is formatted like j.wesselius@exchangelabs.nl. It is a Microsoft recommendation to keep the user’s email address and UPN identical, but that’s not a hard requirement.

The UserPrincipalName attribute has the following characteristics and/or requirements:

  • The @ character is required.
  • The @ character cannot be the first or the last character of a UPN.
  • The total length cannot exceed the 113 characters limit. 64 characters in front of the @ character (i.e. username) and 48 characters after the @ character (i.e. domain name).
  • Allowed characters are A – Z, a – z, 0 – 9, ‘ . – _ ! # ^ ~
  • Invalid characters are \ % & * + / = ? { } | < > ( ) ; : , [ ] “
  • An Umlaut, tilde and accents are also invalid characters.
  • The UPN cannot end with a dot.
  • The UPN cannot contain spaces.
  • With directory synchronization in mind:
    • a routable domain must be used for the UPN (for a stand-alone AD this is not the case).
    • UPN must be unique and cannot contain any duplicated value in the directory (like UPN of user A is the same as e-mail address of user B).

The last bullet is something I see a lot in hybrid scenarios. In Exchange 2019 it is possible to have a user with a UPN like J.Doe@exchangelabs.nl, and another user with an identical email address J.Doe@exchangelabs.nl. Although it is confusing, it is possible on-premises.

In Exchange Online this is not possible, and when you have Entra ID in place, it will generate error messages, and strip the email address from the second user. Needless to say, you must fix this inconsistency (which can be problematic since you must remove an email address from a mailbox).

A little bit related is the samAccountName attribute of a user. This has the following limitations:

  • The maximum length is 20 characters.
  • It must be unique in the entire organization.
  • The following characters are invalid: [ ] \ | / , : < > + = ; ? * ‘ and the double quoute character “

Email Addresses

In Exchange and Exchange Online there are four e-mail address related attributes:

  • The mail attribute. The mail attribute of a recipient must be unique in the entire organization.
  • mailNickName (or alias). Must be unique in the entire organization and it cannot start with a period.
  • ProxyAddresses. This is a multi-valued attribute and has the following restrictions:
    • The maximum length of an entry is 256 characters.
    • It cannot contain a space character.
    • It must be unique in the entire organization.
    • It cannot contain any of the following characters: < > ( ) ; , [ ] and a double quote character “. The colon character : is allowed, but only after an identifier like SMTP: or X500:
    • Special characters with an umlaut, accent or tilde are invalid.
    • TargetAddress. This is used for forwarding email messages, in a hybrid environment this is the remote routing address. It is a singe value attribute, and has the same limitations as the proxyAddresses attribute.

Most likely there are more related attributes that need attention, but these are the most interesting I see when working with customers.

Exchange Hybrid

HCW8078 – Migration Endpoint could not be created

Leave a comment

When running the hybrid configuration wizard on an Exchange 2016 server (in an Exchange 2010 to Exchange 2016 migration) to create a (classic) hybrid configuration, the wizard failed with several error messages as shown in the following screenshot:

Or in plaint text:

Microsoft.Exchange.Migration.MigrationServerConnectionFailedException. The connection to the server ‘mail.contoso.com’ could not be completed.

Microsoft.Exchange.MailboxReplicationService.MRSRemoteTransientException
The call to ‘https://mail.contoso.com/EWS/mrsproxy.svc&#8217; failed. Error details: The HTTP request was forbidden with client authentication scheme ‘Negotiate’. –> The remote server returned an error: (403) Forbidden..

Microsoft.Exchange.MailboxReplicationService.MRSRemotePermanentException
The HTTP request was forbidden with client authentication scheme ‘Negotiate’.

Microsoft.Exchange.MailboxReplicationService.MRSRemotePermanentException. The remote server returned an error: (403) Forbidden. 

I did the following troubleshooting steps:

  • Manually creating a migration endpoint using the Exchange Online Admin console. This failed as well.
  • Firewall restrictions were in place, but the Microsoft IP ranges were all configured correctly
  • Checked the MRS. It was enabled (well, that’s wat the Exchange PowerShell said) and authentication was set correctly

Eventually it turned out that the MRS was the problem. Although Exchange Management Shell returned that the MRS Proxy was enabled, it was not functioning correctly. I disabled the MRS proxy, enabled it again and restarted IIS using the following PowerShell commands:

[PS] C:\> Set-WebServicesVirtualDirectory -Identity EXCH01\EWS -MRSProxyEnabled $False
[PS] C:\> Set-WebServicesVirtualDirectory -Identity EXCH01\EWS -MRSProxyEnabled $True
[PS] C:\> IISRESET

When finished, using the Get-WebServicesVirtualDirectory command still returned that MRS Proxy was enabled, as shown here:

[PS] C:\>Get-WebServicesVirtualDirectory -server exch01 | select Name,MRSProxyEnabled
 
Name                   MRSProxyEnabled
----                   ---------------
EWS (Default Web Site)            True

But this time it was working correctly and the Hybrid Configuration Wizard finished successfully.

Exchange, Exchange Hybrid

Hybrid configuration wizard hangs when adding federated domains

Leave a comment

I am working on an Exchange 2010 project (no typo!) where all mailboxes need to be moved to Exchange Online. Since Exchange 2010 is out-of-support for years, creating an Exchange 2010 hybrid environment is not a good idea. So, instead I installed Exchange 2016 and created a hybrid Exchange 2016 environment. But that brings it challenges as well.

The first issue we ran into when creating the hybrid configuration was the TLS 1.2 issue. Exchange 2016 (on Windows 2016) supports TLS 1.2, but it needs to be enabled. To enable TLS 1.2 on Exchange 2016, follow the instructions on the Exchange Server TLS configuration best practices article on the Microsoft website.

When running the Hybrid Configuration Wizard for the first time you must add the domains for federation with Office 365. DNS TXT records were added for all four domains but validating failed after validating the first domain. The wizard hangs on validating subsequent domains.

Removing the other three domains from the wizard and thus validating with only one domain solved our problem and the wizard successfully finished. Later on we were able to add the other domains, but only one at a time.