Tag Archives: Exchange

January 2022 Exchange Security Updates

On january 11, 2022 Microsoft released new Security Updates for Exchange versions:

  • Exchange 2013 CU23
  • Exchange 2016 CU21, Exchange 2016 CU22
  • Exchange 2019 CU10, Exchange 2019 CU11

The following vulnerabilities have been addressed in these Security Updates:

No exploits have been found in the wild, but it is recommended to install these Security Updates as soon as possible.

These updates are targeted toward Exchange server on-premises, including Exchange servers used in a hybrid configuration.

Please note the following:

  • Run the Exchange Server Healthcheck script on your Exchange server to get an overview of all issues in your environment, including installed Security Updates and Cumulative Updates versions.
  • If running an old (and unsupported!) version of Exchange server, please update to the latest CU to get in a supported state and install these Security Updates.
  • When installing manually, start the update from a command prompt with elevated privileges. If you fail to do so, it will look like installation successfully finishes, but various issues will occur. This is not needed when installing using Windows Update or WSUS.
  • Security Updates are also cumulative, so this Security Updates contains all previous Security Updates for this specific Cumulative Update. There’s no need to install previous Security Updates before installing this Security Update.
  • The December 2021 Cumulative Update is postponed, check the link on the Microsoft site. Microsoft does not release Security Updates and Security Updates in the same month, so do not except a new Cumulative Update anytime soon.
  • This Security Update does not contain a fix for the Y2K22 problem that popped up on January 1, see the Email stuck in Exchange on-premises Transport Queues article which also contains the solution.
  • As always, download and deploy in your test environment to see if it all works well in your environment.
Exchange versionDownloadKnowledge base
Exchange 2013 CU23https://www.microsoft.com/en-us/download/details.aspx?id=103857KB5008631
Exchange 2016 CU21https://www.microsoft.com/en-us/download/details.aspx?id=103856KB5008631
Exchange 2016 CU22https://www.microsoft.com/en-us/download/details.aspx?id=103855KB5008631
Exchange 2019 CU10https://www.microsoft.com/en-us/download/details.aspx?id=103853KB5008631
Exchange 2019 CU11https://www.microsoft.com/en-us/download/details.aspx?id=103854KB5008631

The FIP-FS Scan Process failed initialisation. Mail is queued on Exchange servers.

According to a post on Reddit it seems that Exchange 2016 and Exchange 2019 on-premises are queueing email messages starting januari 1, 2022 at midnight (UTC).

Besides mail queueing you will also see EventID 1106 in the application event log stating “The FIP-FS Scan Process failed initialization. Error: 0x80004005. Error Details: Unspecified error”

This might be a bug is how the date is handles inside the scan engine, causing it to fail after midnight UTC on Januari 1, 2022 (or is it December 31, 2021).

As there is no fix from Microsoft yet, the workaround is to disable anti-malware scanning on all your Exchange servers and restart the Transport service:

CD $ExScripts
.\Disable-AntiMalwareScanning.ps1
Restart-Service MSExchangeTransport

Update 1 on January 1 at 9PM GMT+1. Microsoft is aware of this issue and working on a solution. Check the Exchange team blog: https://techcommunity.microsoft.com/t5/exchange-team-blog/email-stuck-in-transport-queues/ba-p/3049447

Update 2 on January 2 at 10AM GMT+1. Microsoft has released a solution for this issue. A script can be downloaded from https://aka.ms/ResetScanEngineVersion. This will stop the services, delete the %ProgramFiles%\Microsoft\Exchange Server\V15\FIP-FS\Data\Engines\amd64\Microsoft and the %ProgramFiles%\Microsoft\Exchange Server\V15\FIP-FS\Data\Engines\metadata directories and download new scan engines. This can take a couple of minutes, but the script can be run in parallel on all your Exchange servers.

When the script has finished, check the eventlog and you should see EventID 6036 that all is well.

Although not documented, I had to reboot my Exchange servers. I’ve several other reports from people that they had to reboot too. Another thing, when you disabled the anti-malware as a workaround, you have to re-enable the anti-malware manually. You can check this using the Get-TransportAgent “malware agent” command.

It is also possible to manually update your Exchange servers, this is also documented in the Microsoft article https://techcommunity.microsoft.com/t5/exchange-team-blog/email-stuck-in-transport-queues/ba-p/3049447

One warning though….. All Exchange 2016 and 2019 servers worldwide are suffering from this issue and they all will queue messages. Queues expire after 48 hours, so when not fixed the Exchange servers will generate NDR messages on Sunday night. Worst case scenario, millions of NDRs will be generated which in turn will result in tons of helpdesk calls. If you read this, most likely you have fixed your Exchange servers and it is the Exchange environment of the intended recipients.

Let’s hope it will be quiet again for some time now 🙂

Exchange Security Updates October 2021

On October 12, 2021 Microsoft released Security Updates for vulnerabilities found in Exchange server 2013 CU23, Exchange server 2016 (CU21/CU22) and Exchange server 2019 (CU10/CU11). Severity is marked as ‘important’.

If you are running one of these versions, it is recommended to apply these security updates. Please note that the security updates are CU specific, and these are not interchangeable. Security updates are also cumulative, so these security updates contain all previous security updates for the same cumulative update. If you are running an older version of Exchange, it is strongly recommended to upgrade to the latest Cumulative Update and apply the security updates. You can use the healthchecker script to inventory your environment.

Please use the Microsoft Security Update Guide for more specific information about the vulnerabilities.

As always, after downloading the security updates, start the installation from an elevated command prompt (‘run as administrator’). This does not apply when installing from Windows Update or WSUS. And of course, please the security updates in a test environment first before installing in production.

You can download the security updates for the following products here:

July 2021 Security Updates for Exchange

On July 13, 2021 Microsoft has released a number of Security Updates for Exchange. Security Updates are released for:

  • Exchange 2013 CU23
  • Exchange 2016 CU20 and CU21
  • Exchange 2019 CU9 and CU10

Some of the issues are marked ‘critical’ (Remote Code Execution) but no evidence have been found for any exploits in the wild, but it is strongly recommended to install these Security Updates as soon as possible.
The following CVE’s are addressed in these Security Updates:

Detailed information regarding the vulnerabilities can be found in the Security Update Guide.

As always, when installing the Security Update manually from a command prompt, use elevated privileges. If you do not, installation will succeed but under the hood things break! This is not an issue when installing using Windows Update.

Note. This Security Update has a dependency on the Schema update that came with Exchange 2016 CU21 and Exchange 2019 CU10. If you are running an older version of these CUs, please update the Schema first to the latest level. If you are still running Exchange 2013, and only Exchange 2013 at the latest level, you can install the Security Update, but you must run setup.exe /PrepareSchema from the V15\bin directory. The SU installation will install the latest schema files in the V15\bin directory which will be used by the setup application to make the schema changes. Failure to do so will result in an unprotected Exchange 2013 environment.

Exchange server patching performance and windows defender

Patching an Exchange server, whether it be Windows Update, a Cumulative Update or a Security Update always takes a long time. When looking at the task manager, it is always the Antimalware Service Executable (Windows Defender Antivirus Service) that is responsible for this. It just consumes a lot of processor cycles:

To overcome this and speed up the overall performance of patching the Exchange server you can temporarily disable Windows Defender.

For Exchange 2016 running on Windows 2016 follow these steps:

Start | Settings | Update and Security | Windows Defender

For Exchange 2019 running on Windows 2019 follow these steps:

Start | Settings | Update and Security | Windows Security | Open Windows Security I Virus & Threat protection I Manage Settings

And switch Real-time protection to off as shown in the following screenshot:

Much easier is using PowerShell, just execute this command:

Set-MpPreference -DisableRealtimeMonitoring $True

When patching the Exchange server you will notice how much faster it will be. When patched and rebooted, enable Windows Defender by executing the following PowerShell command:

Set-MpPreference -DisableRealtimeMonitoring $False

You can check the status of Windows defender using one of the following commands:

Get-MpPreference | select DisableRealtimeMonitoring
Get-MpComputerStatus

Check the output for RealTimeProtectionEnabled, this should be set to True. As a sidenote, there is a lot of other interesting information when executing Get-MpComputerStatus for anti-malware.