Tag Archives: Bug

The FIP-FS Scan Process failed initialisation. Mail is queued on Exchange servers.

According to a post on Reddit it seems that Exchange 2016 and Exchange 2019 on-premises are queueing email messages starting januari 1, 2022 at midnight (UTC).

Besides mail queueing you will also see EventID 1106 in the application event log stating “The FIP-FS Scan Process failed initialization. Error: 0x80004005. Error Details: Unspecified error”

This might be a bug is how the date is handles inside the scan engine, causing it to fail after midnight UTC on Januari 1, 2022 (or is it December 31, 2021).

As there is no fix from Microsoft yet, the workaround is to disable anti-malware scanning on all your Exchange servers and restart the Transport service:

CD $ExScripts
.\Disable-AntiMalwareScanning.ps1
Restart-Service MSExchangeTransport

Update 1 on January 1 at 9PM GMT+1. Microsoft is aware of this issue and working on a solution. Check the Exchange team blog: https://techcommunity.microsoft.com/t5/exchange-team-blog/email-stuck-in-transport-queues/ba-p/3049447

Update 2 on January 2 at 10AM GMT+1. Microsoft has released a solution for this issue. A script can be downloaded from https://aka.ms/ResetScanEngineVersion. This will stop the services, delete the %ProgramFiles%\Microsoft\Exchange Server\V15\FIP-FS\Data\Engines\amd64\Microsoft and the %ProgramFiles%\Microsoft\Exchange Server\V15\FIP-FS\Data\Engines\metadata directories and download new scan engines. This can take a couple of minutes, but the script can be run in parallel on all your Exchange servers.

When the script has finished, check the eventlog and you should see EventID 6036 that all is well.

Although not documented, I had to reboot my Exchange servers. I’ve several other reports from people that they had to reboot too. Another thing, when you disabled the anti-malware as a workaround, you have to re-enable the anti-malware manually. You can check this using the Get-TransportAgent “malware agent” command.

It is also possible to manually update your Exchange servers, this is also documented in the Microsoft article https://techcommunity.microsoft.com/t5/exchange-team-blog/email-stuck-in-transport-queues/ba-p/3049447

One warning though….. All Exchange 2016 and 2019 servers worldwide are suffering from this issue and they all will queue messages. Queues expire after 48 hours, so when not fixed the Exchange servers will generate NDR messages on Sunday night. Worst case scenario, millions of NDRs will be generated which in turn will result in tons of helpdesk calls. If you read this, most likely you have fixed your Exchange servers and it is the Exchange environment of the intended recipients.

Let’s hope it will be quiet again for some time now 🙂

Open attachment in Shared Mailbox using OWA

Last September Microsoft released their quarterly Cumulative Updates for Exchange, Exchange 2016 CU18 and Exchange 2019 CU7. This was quickly followed by a security update, KB4581424 that addresses the CVE-2020-16969 Microsoft Exchange Information Disclosure vulnerability.

Unfortunately, the Exchange 2016 CU18 and Exchange 2019 CU7 contain a nasty bug. If you use OWA, open a shared mailbox and try to access an attachment, OWA redirects to Office 365 instead of the on-premises Exchange 2016/2019 server to download it. This happens in an hybrid environment, but also in a pure on-premises Exchange deployment without any Office 365 connection.

The error message reads:

Hmmm… can’t reach this page
It looks like the webpage at
https://outlook.office365.com/owa/sharedmailbox@contoso.com/services.svc/s/GetAttachmentDownloadToken?redirect=%2fowa%sharedmailbox%40contoso.com%fservices.svc%2fservices.svc%2fsGetFileAttachment….

Microsoft is aware of this issue and it will be fixed in the next Cumulative Updates for Exchange 2016 and Exchange 2019. Looking at the quarterly cadence this should be by the end of this year.

If you have a Microsoft Premier support contract and this is an issue that impacts your business you can open a support ticket and request a fix for this. This service is available for Premier support customers only.

This fix is a replacement for the KB4581424 security update, as such it contains all the fixes in KB4581424, plus the OWA Attachment hotfix. If you are a Premier support customer and do have this fix available, make sure that you uninstall the KB4581424 first before installing this update.
One workaround that I’ve seen in a newsgroup is not to open the Shared Mailbox as “Open another mailbox” but as “Add shared folder”. This should work also, but I have not tested it. I do have a customer with a Premier support contract, I can confirm the problem is fixed in the interim update.