Exchange

Exchange 2019 CU14

Leave a comment

On February 13, 2024 Microsoft has released a new Cumulative Update for Exchange 2019, the 2024 H1 Cumulative Update (or CU14). One major ‘new’ feature and some minor features.

Extended Protection

One of the interesting things in this update is that it by default enables Extended Protection on all virtual directories of your Exchange 2019 server.

Extended protection is not new, it was introduced in August 2022 Security Updates. At the time I already wrote about, which you can read on https://jaapwesselius.com/2022/08/09/exchange-security-updates-august-2022/.

Extended Protection is an enhancement on the existing Windows Authentication. Extended Protection mitigates authentication relay or ‘man in the middle’ attacks. It is implemented by using channel binding information using a Channel Binding Token (CBT). There’s no need to configure anything, Extended Protection is automatically configured by the CU14 setup. It is a per server setting, so it is only enabled on the server you have installed CU14 on.

There are some prerequisites that need to be met before you can enable Extended Protection. Run the healthchecker.ps1 script and check the prequisites page for Extended Protection on the Microsoft page.

If you want to install CU14, but you environment does not meet the prerequisites, you can install CU14 unattended with the /DoNotEnableEP or /DoNotEnableEPFEEWS (EPFEEWS is Extended Protection FrontEnd EWS) switch.

Please be aware that this is not the recommended approach. The recommended approach is to prepare your environment for Extended Protection, enable it on older server and then install CU14 with automatic enabling Extended Protection.

TLS 1.3

Unfortunately TLS 1.3 (which is supported on Windows 2022) did not make it in this update, and is now scheduled to be included in CU15 (2024 H2 Cumulative Update, so by the end of this year).

CVE-2024-21410

CVE-2024-21410 was also released today, and this CVE is addressed in CU14 by Extended Protection. This is also true for Exchange 2016 CU23 (EP was introduced in an earlier Security Update for Exchange 2016).

Installing Exchange 2019 CU14

CU14 does not come with an Active Directory Schema update; the Schema is still at version 17003. The Exchange configuration version is increased by one (now 16762) and the Domain version has not changed (still 13243).

Use the following commands to upgrade the Configuration and Domain Partition in your environment:

Z:\>Setup.exe /PrepareAD /IAcceptExchangeServerLicenseTerms_DiagnosticDataOn

And:

Z:\>Setup.exe /PrepareDomain /IAcceptExchangeServerLicenseTerms_DiagnosticDataOn

You’ll notice the following output on the console:

Exchange Setup has enabled Extended Protection on all the virtual directories on this machine. For more information visit: https://aka.ms/EnableEPviaSetup. 

We recommend running Exchange HealthChecker script to evaluate if there are any configuration issues which can cause feature breakdowns. HealthChecker script can be downloaded from https://aka.ms/ExchangeSetupHC.

At this point, nothing has changed on the virtual directories of your Exchange server so the information is not needed at this point. Running the HealthChecker script is always a good thing before running any upgrade. When using the unattended upgrade, use the following command:

Z:\>Setup.EXE /Mode:Upgrade /IAcceptExchangeServerLicenseTerms_DiagnosticDataOn

And when needed, you can add the /DoNotEnableEP switch here. Or you can also use the GUI setup of course if you prefer to do that.

Some remarks about CU14

  • Make sure you fully understand the impact of Extended Protection. For more information check the Configure Extended Protection in Exchange server page on the Microsoft site.
  • CU14 supports the .NET Framework 4.8.1 on Windows Server 2022 (and only on Windows Server 2022)
  • This Cumulative Update contains all binaries from previous Cumulative Updates, including all previously released Security Updates.
  • Download CU14 at https://www.microsoft.com/en-us/download/details.aspx?id=105878
  • Knowledgebase article KB5035606 contains more information, including known issues with this release.
  • Before bringing in production, please test thoroughly in your test environment.
  • The only supported versions of Exchange 2019 are Exchange 2019 Cumulative Update 14 and Cumulative Update 13. Earlier versions are no longer supported (and no longer receive Security Updates)
  • Exchange 2013 is old, and out of extended support. No more Security Updates will be released for Exchange 2013. If you are still running Exchange 2013, make sure you migrate to Exchange 2019 or Exchange Online anytime soon!
  • Exchange 2016 is out of mainstream support, which means that Microsoft will no longer release new Cumulative Updates for Exchange 2016. Security Updates will be released until Extended Support for Exchange 2016 ends.
Exchange, Exchange Hybrid, Office365

Special characters in Active Directory and Exchange Online

Leave a comment

During migrations to Exchange Online I get the question regarding special characters in the User Principal Name (UPN) and e-mail address. Every time I have to check this again and again, so it’s time to do a write-up.

The UserPrincipalName (UPN)

The UserPrincipalName (UPN)

The UPN is the user’s identifier in Active Directory, and it is formatted like j.wesselius@exchangelabs.nl. It is a Microsoft recommendation to keep the user’s email address and UPN identical, but that’s not a hard requirement.

The UserPrincipalName attribute has the following characteristics and/or requirements:

  • The @ character is required.
  • The @ character cannot be the first or the last character of a UPN.
  • The total length cannot exceed the 113 characters limit. 64 characters in front of the @ character (i.e. username) and 48 characters after the @ character (i.e. domain name).
  • Allowed characters are A – Z, a – z, 0 – 9, ‘ . – _ ! # ^ ~
  • Invalid characters are \ % & * + / = ? { } | < > ( ) ; : , [ ] “
  • An Umlaut, tilde and accents are also invalid characters.
  • The UPN cannot end with a dot.
  • The UPN cannot contain spaces.
  • With directory synchronization in mind:
    • a routable domain must be used for the UPN (for a stand-alone AD this is not the case).
    • UPN must be unique and cannot contain any duplicated value in the directory (like UPN of user A is the same as e-mail address of user B).

The last bullet is something I see a lot in hybrid scenarios. In Exchange 2019 it is possible to have a user with a UPN like J.Doe@exchangelabs.nl, and another user with an identical email address J.Doe@exchangelabs.nl. Although it is confusing, it is possible on-premises.

In Exchange Online this is not possible, and when you have Entra ID in place, it will generate error messages, and strip the email address from the second user. Needless to say, you must fix this inconsistency (which can be problematic since you must remove an email address from a mailbox).

A little bit related is the samAccountName attribute of a user. This has the following limitations:

  • The maximum length is 20 characters.
  • It must be unique in the entire organization.
  • The following characters are invalid: [ ] \ | / , : < > + = ; ? * ‘ and the double quoute character “

Email Addresses

In Exchange and Exchange Online there are four e-mail address related attributes:

  • The mail attribute. The mail attribute of a recipient must be unique in the entire organization.
  • mailNickName (or alias). Must be unique in the entire organization and it cannot start with a period.
  • ProxyAddresses. This is a multi-valued attribute and has the following restrictions:
    • The maximum length of an entry is 256 characters.
    • It cannot contain a space character.
    • It must be unique in the entire organization.
    • It cannot contain any of the following characters: < > ( ) ; , [ ] and a double quote character “. The colon character : is allowed, but only after an identifier like SMTP: or X500:
    • Special characters with an umlaut, accent or tilde are invalid.
    • TargetAddress. This is used for forwarding email messages, in a hybrid environment this is the remote routing address. It is a singe value attribute, and has the same limitations as the proxyAddresses attribute.

Most likely there are more related attributes that need attention, but these are the most interesting I see when working with customers.

Exchange Hybrid

HCW8078 – Migration Endpoint could not be created

Leave a comment

When running the hybrid configuration wizard on an Exchange 2016 server (in an Exchange 2010 to Exchange 2016 migration) to create a (classic) hybrid configuration, the wizard failed with several error messages as shown in the following screenshot:

Or in plaint text:

Microsoft.Exchange.Migration.MigrationServerConnectionFailedException. The connection to the server ‘mail.contoso.com’ could not be completed.

Microsoft.Exchange.MailboxReplicationService.MRSRemoteTransientException
The call to ‘https://mail.contoso.com/EWS/mrsproxy.svc&#8217; failed. Error details: The HTTP request was forbidden with client authentication scheme ‘Negotiate’. –> The remote server returned an error: (403) Forbidden..

Microsoft.Exchange.MailboxReplicationService.MRSRemotePermanentException
The HTTP request was forbidden with client authentication scheme ‘Negotiate’.

Microsoft.Exchange.MailboxReplicationService.MRSRemotePermanentException. The remote server returned an error: (403) Forbidden. 

I did the following troubleshooting steps:

  • Manually creating a migration endpoint using the Exchange Online Admin console. This failed as well.
  • Firewall restrictions were in place, but the Microsoft IP ranges were all configured correctly
  • Checked the MRS. It was enabled (well, that’s wat the Exchange PowerShell said) and authentication was set correctly

Eventually it turned out that the MRS was the problem. Although Exchange Management Shell returned that the MRS Proxy was enabled, it was not functioning correctly. I disabled the MRS proxy, enabled it again and restarted IIS using the following PowerShell commands:

[PS] C:\> Set-WebServicesVirtualDirectory -Identity EXCH01\EWS -MRSProxyEnabled $False
[PS] C:\> Set-WebServicesVirtualDirectory -Identity EXCH01\EWS -MRSProxyEnabled $True
[PS] C:\> IISRESET

When finished, using the Get-WebServicesVirtualDirectory command still returned that MRS Proxy was enabled, as shown here:

[PS] C:\>Get-WebServicesVirtualDirectory -server exch01 | select Name,MRSProxyEnabled
 
Name                   MRSProxyEnabled
----                   ---------------
EWS (Default Web Site)            True

But this time it was working correctly and the Hybrid Configuration Wizard finished successfully.

Exchange, Exchange Hybrid

Hybrid configuration wizard hangs when adding federated domains

Leave a comment

I am working on an Exchange 2010 project (no typo!) where all mailboxes need to be moved to Exchange Online. Since Exchange 2010 is out-of-support for years, creating an Exchange 2010 hybrid environment is not a good idea. So, instead I installed Exchange 2016 and created a hybrid Exchange 2016 environment. But that brings it challenges as well.

The first issue we ran into when creating the hybrid configuration was the TLS 1.2 issue. Exchange 2016 (on Windows 2016) supports TLS 1.2, but it needs to be enabled. To enable TLS 1.2 on Exchange 2016, follow the instructions on the Exchange Server TLS configuration best practices article on the Microsoft website.

When running the Hybrid Configuration Wizard for the first time you must add the domains for federation with Office 365. DNS TXT records were added for all four domains but validating failed after validating the first domain. The wizard hangs on validating subsequent domains.

Removing the other three domains from the wizard and thus validating with only one domain solved our problem and the wizard successfully finished. Later on we were able to add the other domains, but only one at a time.

Exchange, Exchange Hybrid, Security

MTA-STS Support in Exchange 2019 and Exchange Online

Leave a comment

My previous blogpost was about DANE and discussed how DANE can be used to make TLS negotiations between mailservers more secure. Another topic in this area is MTA-STS. MTA-STS stands for Message Transfer Server – Strict Transport Security and MTA-STS is a mechanism to enforce the use of TLS and the use of a valid 3rd party server certificate.

MTA-STS, DNS and Policies

When an MTA-STS capable servers wants to send an email, it first retrieves the MX record for the recipient domain. The second step is that the sending server check for an MTA-STS record in DNS. This record looks like:

_mta-sts.exchangelabs.nl. IN TXT "v=STSv1; id=202306242147;"

The id is an identifier and defines the version of the MTA-STS record when changes are made to the MTA-STS record. A good practice is to create an identifier based on the date and time of the last change. In this example, it is June 24, 2023 at 9:47pm.

The next step is that the sending servers looks for a policy. This policy is not stored in DNS, but on a website. The URL for this policy looks like: https://mta-sts.exchangelabs.nl/.well-known/mta-sts.txt. The subdomain mta-sts, the filename mta-sts.txt and the directory .well-known (including the dot) directory are mandatory for the MTA-STS policy. It must also be secured using a valid 3rd party server certificate.

The MTA-STS policy will look something like:

version: STSv1
mode: enforce
mx: smtphost.exchangelabs.nl
mx: *.mail.protection.outlook.com
max_age: 604800

Note. If you have configured DANE for inbound email in Exchange Online, your MX record should be something like Exchangelabs-nl.y-v1.mx.microsoft.

The MTA-STS policy is structured as follows:

  • Version identifies the version of MTA-STS but must always be STSv1 (for now at least).
  • Mode defines how the policy must be applied:
    • Enforce: Sending MTAs MUST NOT deliver the message to hosts that fail MX matching or certificate validation or that do not support STARTTLS.
    • Testing: Sending MTAs that also implement the TLSRPT (TLS Reporting) specification [RFC8460] send a report indicating policy application failures (as long as TLSRPT is also implemented by the recipient domain); in any case, messages maybe delivered as though there were no MTA-STS validation failure.
    • None: In this mode, Sending MTAs should treat the Policy Domain as though it does not have any active policy.
  • MX defines all MX records in use by the recipient domain. This can be one entry, but it can hold multiple MX records, each on a separate line as shown in the policy above.
  • Max_age defines the time (in seconds) that the MTA-STS policy can be cached by a mail server. In this example, the policy is cached for 604800 seconds, which equals to 1 week. When a sending server must send a new email within a week, the policy is still cached. After checking the MX record the server retrieves the TXT record from DNS (as explained in the second step above) and when the identifier has not changed it uses the policy that is cached. If the identifies has changed within the lifetime of the cached policy, a new policy is downloaded.

So, in my example an MTA-STS capable mail server will check the MTA-STS policy and only connects to my mail server using TLS 1.2 (this is enforced with MTA-STS when mode is set to ‘enforce’) and only when a certificate that matches the FQDN is used. When authentication fails for an entry, the sending server continues with the next line in the policy, in my example with the MX record pointing to Exchange Online.

An interesting option in MTA-STS is reporting. DMARC has a reporting function as well, but reports are only sent by receiving domains. Reporting in MTA-STS is performed daily by sending mail servers that supports MTA-STS and TLS-RPT.

To configure the reporting functionality, create a mailbox in Exchange 2019 or Exchange Online and assign it an email address like TLSReports@Exchangelabs.nl. The next step is to configure the following DNS TXT record:

_smtp._tls.exchangelabs.nl. 3600 IN  TXT v=TLSRPTv1;rua=mailto:TLSReports@exchangelabs.nl

There are several online tools available for checking the MTS-STS record. Just like in my other blogs, I often use MXToolbox to check for DNS records as shown in the following screenshot:

As with the DANE checks, the mailhardener site (https://www.mailhardener.com/tools/mta-sts-validator) can also be used to check MTA-STS records as shown in the following screenshot:

When you check the IIS logs of the webserver where the mta-sts.txt file is stored, you can see which servers are using MTA-STS:

2023-07-11 08:29:59 172.16.1.4 GET /.well-known/mta-sts.txt - 443 - 74.125.217.68 Google-SMTP-STS - 200 0 0 119
2023-07-11 13:54:51 172.16.1.4 GET /.well-known/mta-sts.txt - 443 - 104.47.11.254 - - 200 0 0 62
2023-07-11 14:13:37 172.16.1.4 GET /.well-known/mta-sts.txt - 443 - 77.238.178.114 AHC/2.1 - 200 0 0 20

The first one is obvious, the second line is a Microsoft IP address, the third line is a Yahoo IP address. So, Google, Microsoft and Yahoo are using MTA-STS when sending email.

MTA-STS versus DANE

MTA-STS and DANE share a common concept, that is to secure the (initial) communication between mail servers. The ‘problem’ with DANE is that is relies on DNSSEC and the global roll-out of DNSSEC is very slow (to put it mildly).

MTA-STS was developed to overcome the slow roll-out of DNSSEC (since it does not use DNSSEC of course). MTA-STS can be seen as a ‘light-weight’ version of DANE and it will be sufficient for most customers.

And how about Exchange?

Just like with DANE, the ugly part is that Exchange 2019 does not support MTA-STS. You can configure the MTA-STS record in DNS and the policy on a website so that MTA-STS capable servers use your Exchange 2019 server safely. But for sent messages by Exchange 2019, MTA-STS is a no-go, it does not support it and most likely will never do.

Exchange Online on the other hand does support MTA-STS (since the beginning of 2022) for both inbound and outbound messages. The only thing you must do to enable it for inbound messages is create the TXT record in DNS and create and publish the MTA-STS policy.

Edited on July 11, 2023