Exchange 2019 CU13 (2023 H1 Cumulative Update for Exchange server)

Just a quick blogpost about the latest Exchange 2019 release. Yesterday, May 3, 2023 Microsoft has released Exchange 2019 Cumulative Update 13, or 2023 H1 Cumulative Update for Exchange server as it is officially named.

There are two interesting new features in Exchange 2019 CU13:

  • Exchange 2019 CU13 now supports Modern Authentication. Please note that previously Exchange 2019 supported Hybrid Modern Authentication (HMA). Modern Authentication is targeted specifically to customers that do not have any hybrid or any cloud integration as it works with your on-premises ADFS implementatation. At this moment only Outlook clients are supported to work with Modern Authentication, support for other clients is expected later this year.
  • Configuration preservation. When this CU is installed and you have any customized config files, they are now preserved and not overwritten as was the case in earlier Cumulative Updates. When updating to CU13, it reports that config files are preserved:
PS Z:\> .\Setup.EXE /Mode:Upgrade /IAcceptExchangeServerLicenseTerms_DiagnosticDataOn

Microsoft Exchange Server 2019 Cumulative Update 13 Unattended Setup

Copying Files...
File copy complete. Setup will now collect additional information needed for installation.

Languages
Management tools
Mailbox role: Transport service
Mailbox role: Client Access service
Mailbox role: Mailbox service
Mailbox role: Front End Transport service
Mailbox role: Client Access Front End service

Performing Microsoft Exchange Server Prerequisite Check

    Configuring Prerequisites                      COMPLETED
    Prerequisite Analysis                          COMPLETED

Configuring Microsoft Exchange Server

    Language Files                                 COMPLETED
    Restoring Services                             COMPLETED
    Language Configuration                         COMPLETED
    Exchange Management Tools                      COMPLETED
    Mailbox role: Transport service                COMPLETED
    Mailbox role: Client Access service            COMPLETED
    Mailbox role: Mailbox service                  COMPLETED
    Mailbox role: Front End Transport service      COMPLETED
    Mailbox role: Client Access Front End service  COMPLETED
    Finalizing Setup                                       COMPLETED

The Exchange Server setup operation completed successfully.

Exchange Setup preserved the required configurations during upgrade. More details can be found in Exchangesetup.log located in <SystemDrive>:\ExchangeSetupLogs folder. For more information, visit:
https://aka.ms/PreserveExchangeConfig2019.

Unfortunately, support for TLS 1.3 (which is in Windows Server 2022) is not in this Cumulative Update, hopefully support for TLS 1.3 later this year.

More information, tips and downloads:

  • This Cumulative Update contains all binaries from previous Cumulative Updates, including all previously released Security Updates.
  • Download at https://www.microsoft.com/en-us/download/details.aspx?id=105180
  • Knowledgebase article KB5020999 contains more information, including known issues with this release.
  • Before bringing in production, please test thoroughly in your testenvironment.
  • The onlly supported versions of Exchange 2019 are Exchange 2019 Cumulative Update 13 and Cumulative Update 12. Earlier versions are no longer supported.
  • Exchange 2013 is old, and out of extended support. No more Security Updates will be released for Exchange 2013. If you are still running Exchange 2013, make sure you migrate to Exchange 2019 or Exchange Online anytime soon!

Exchange uninstall fails with BuildToBuildUpgrade error

Uninstalling is typically not an issue. One of my Exchange 2019 servers (running on Windows 2022 Server Core) have been running for quite some time, but I had to uninstall it.

Running the setup application failed with a BuildToBuildUpgrade error, almost immediately after starting the setup:

PS C:\> setup.exe /mode:uninstall /IAcceptExchangeServerLIcenseTerms_DiagnosticDataOn

Microsoft Exchange Server 2019 Cumulative Update 12 Unattended Setup

Mailbox role: Mailbox service
Mailbox role: Client Access service
Mailbox role: Transport service
Mailbox role: Front End Transport service
Mailbox role: Client Access Front End service
Management tools
Languages
Setup previously failed while performing the action "BuildToBuildUpgrade". You can't resume setup by performing the action "Uninstall".

The Exchange Server setup operation didn't complete. More details can be found in ExchangeSetup.log located in the <SystemDrive>:\ExchangeSetupLogs folder.

If have seen setups failing before, but normally during an upgrade to a newer setup, but never during an uninstall.
Just like other setup failures, check the registry for Action and Watermark entries in the HKLM\SOFTWARE\Microsoft\ExchangeServer\v15. Only the MailboxRole had these entries, the ClientAccess and HubTransport did not have these entries.

Delete the Action and Watermark keys and resume setup.
Unfortunately, I don’t know why this happened in the first place since my Exchange server has been running without issues for a longer time.

Disk Defragmentation on an Exchange 2019 server

When moving mailboxes from Exchange 2016 to Exchange 2019 (on Windows 2022) I ran into a couple of StalledDueToTarget_Processor issues. These occur regularly and typically nothing to worry about, the move request will automatically continue in minutes.

Next I checked the task manager to see how performance of the Exchange servers were doing, and I noticed that the Disk Defragmenter was running. Processor utilization averaged around 20%, but it consumed also approx. 17GB of memory.

It turns out that the disk optimization is turned on by default on all disk in your server. When you select the properties of a disk, select the Tools tab and click Optimize you can see all disks, the scheduled optimization and the option to turn it off:

Disk optimization makes sense when you have a lot of sequential data or have an application that works with large chunks of data. Exchange server works with relatively small blocks of data and in a complete random order. So, disk optimization does not make sense on an Exchange server, and it is absolutely safe to turn it off on your Exchange server. This is alse mentioned in the Exchange Server storage configuration options article as a best practice. You don’t want to lose any valuable processor, disk and memory resources on a disk optimization process.

Thanks to reader Feras to supplying me the link to the configuration options.

You are currently using an older version of the Exchange Online PowerShell module

When connection to Exchange Online PowerShell using the command Connect-ExchangeOnline nothing special happens. But when executing a PowerShell command like Get-Mailbox you get the following warning:

PS C:\WINDOWS\system32> $mailboxes = Get-mailbox -ResultSize unlimited
Attention!
You are currently using an older version of the Exchange Online PowerShell module which uses RPS. RPS deprecation has been announced and you will need to move to the latest V3 module by June 2023. Read more here: https://aka.ms/RPSDeprecation 
Please install our new REST-based PS V3 module downloadable from https://www.powershellgallery.com/packages/ExchangeOnlineManagement/, which is more secure and reliable.
Please note that you will no longer be able to use -UseRPSSession after June 2023.

Microsoft has introduced a new Exchange Online PowerShell module (v3) and the previous modules will be deprecated in the (near) future.

To update the PowerShell module, the old module needs to be uninstalled first before you can install the latest version. To uninstall the old v2 module, execute the following command:

Uninstall-Module ExchangeOnlineManagement

And to install the new v3 module, execute the following command:

Install-Module -Name ExchangeOnlineManagement -RequiredVersion 3.0.0

To check which PowerShell modules and versions you have installed, execute the Get-InstalledModule command.

Exchange Server in Azure – Part III

In two previous post I blogged about a site-to-site VPN connection to Azure and about installing a Domain Controller (and Azure AD Connect) in Azure. In this blog I will discuss that last server, the Exchange 2019 server in Azure.

The last in most interesting part (in my opinion) is installing an Exchange 2019 server in Azure. This can be your last Exchange server for management purposes, or maybe an Exchange server still hosting (some) mailboxes or performing SMTP Relay. It can be quite expensive to host mailboxes on an Exchange server in Azure, moving Mailboxes to Exchange Online is much cheaper. But then…. If you are already decommissioning your own datacenter… All business requirements are different of course 🙂

Exchange in Azure Setup

Currently Exchange server are located on-premises in a hybrid scenario. There’s already a Domain Controller in Azure, and the Azure AD Connect is running in Azure too. Now I want to install an Exchange 2019 server in Azure and run the Hybrid Configuration Wizard to create a hybrid configuration with the Exchange 2019 server in Azure. Webmail and Autodiscover will point to the Exchange server in Azure as well. In short, the configuration will look something like this:

Inbound mail will be sent to Exchange Online Protection and delivered to mailboxes in Exchange Online. When a mailbox is located on-premises, it will be routed to the Exchange server in Azure and when needed, routed to the Exchange server on-premises.

But first we have to think about sizing the Exchange server in Azure.

Sizing Exchange in Azure

The first question about Exchange server in Azure is sizing the Exchange server. First of all, deploying Exchange on Microsoft Azure virtual machines is supported if all storage volumes used for Exchange databases and database transaction logs (including transport databases) are configured for Azure Premium Storage (please see the Exchange Server Virtualization article on Microsoft Learn.

When you are planning to host active mailboxes on the Exchange server running in Azure then you should use the Exchange 2019 storage requirements calculator to figure out the correct size for the Exchange server. You can find the storage calculator on the Exchange 2019 ISO images in the \Support directory.

If you are planning to install an Exchange server in Azure just for management purposes, or for lab purposes, I would recommend using a ‘Standard d8s v3’ VM. This is a VM with 8 vCPU, 32 GB memory and a maximum of 16 additional disks.
When you create a new VM in the Azure Portal and select ‘see all sizes’ it will show a list of most used virtual machines by Azure users as shown below:

For my own lab environment I selected the DS3_V2 VM. Not the fastest VM, but for my own test- and management purposes it is sufficient, but your mileage may vary of course. Attached to my VM are two SSD disks of each 1TB. Be aware that the disks can be a costly thing, especially in a lab environment! The network interface is connected to the internal VNET that was created earlier, and it is connected to the Microsoft network using a public IP address. I also created a DNS name (exchlabsnl.westeurope.cloudapp.azure.com which can be used as a CNAME record for some other DNS name) but that’s not really needed since I can use the public IP address in a regular DNS A record.

Installing and Configuring Exchange in Azure

When you have the VM up and running in Azure and you have configured the storage as required, you can install the Exchange server. This is not different from a regular Exchange server. Install the prerequisite software, patch the machine, install the latest Exchange 2019 Cumulative Update and install the latest security update for this CU.

Configuring the Exchange server in Azure is similar to configuring the Exchange server on-premises. Install a proper certificate, configure the virtual directories (hopefully needless to say, but use a different FQDN in Azure since it is a different site in Active Directory) and configure the transport database on a different disk.

Configure the Network Security Group (NSG) to allow access on port 443 and port 25 from the Internet. Depending on your security requirements, you can configure the NSG of the Exchange server with the IP ranges of Exchange Online (https://learn.microsoft.com/en-gb/microsoft-365/enterprise/urls-and-ip-address-ranges) so only Microsoft can fully access your Exchange server in Azure. If you have clients that need to connect to the Exchange server, they can use a VPN connection (that’s what I typically recommend to customers these days, and what I do in my lab environment).

If you are planning an Exchange server just for management purposes, I would recommend not using a public IP at all and only connect to the Exchange server using the private Vnet. In this scenario there’s no need to publish the Exchange server to the Internet, and all changes to recipients are replicated to Exchange Online using Azure AD Connect. No Internet connection is less prone to external attacks of course. And another interesting thing when using a management server only, you can turn it off when not in use. This can save you quite some money.

The next step is to run the Hybrid Configuration Wizard which you can download from https://aka.ms/hybridwizard. When going through the wizard, select the Exchange server in Azure for the Receive Connector configuration and for the Send Connector configuration.

When finished, add the public IP address of your Exchange server in Azure to your SPF record for outbound mail, or configure the SMTP connector to Exchange Online to route all outbound mail via Exchange Online, that’s up to your requirements of course.

What happened in my scenario, my public IP address was assigned by Microsoft and you never know what happened previously with this IP address, but I had to delist it from SpamHaus before I was able to send out email from my Exchange server.

Then move your resources from the on-premises Exchange server to the Exchange server in Azure and decommission your Exchange server on-premises when needed.

Summary

In the past three blogpost I explained what it takes to install an Exchange server in Microsoft Azure and it contains of the following steps:

  • Create a site-to-site connection between your on-premises network and a Vnet in Azure.
  • Install a Domain Controller in Azure, connected to the Vnet. Optionally you can install an Azure AD Connect server in Azure as well.
  • Install and configure an Exchange server in Azure, connected to the Vnet (and to the Internet when needed)

An Exchange server in Azure is fully supported, as long as the Mailbox database and transaction logfiles are located on premium storage, that should be no problem.

Sizing your Exchange server in Azure is like an on-premises Exchange server. Use the storage calculator to determine the size of your Exchange server, and look for a VM that matches these requirements.

When installing an Exchange server in Azure just for management purposes it’s much easier. You can use a relatively lightweight Exchange server, there’s no need to publish it to the Internet (much safer) and you can turn it off when not in use.