Tag Archives: Exchange 2016

Exchange

Exchange 2016 Setup RecoverServer fails with internal transport certificate warning

Leave a comment

I am currently working with a customer on their Exchange 2016 design, implementation and disaster recovery process. While writing a new Exchange 2016 disaster recovery document I ran into this issue in my lab environment while running “Setup.exe /Mode:RecoverServer /IAcceptExchangeServerLicenseTerms”.

image

For search engine options this is a part of the actual error message.

Mailbox role: Transport service FAILED

The following error was generated when “$error.Clear(); Install-ExchangeCertificate -DomainController $RoleDomainController -Services SMTP” was run: “System.InvalidOperationException: The internal transport certificate for the local server was damaged or missing in Active Directory. The problem has been fixed. However, if you have existing Edge Subscriptions, you must subscribe all Edge Transport servers again by using the New-EdgeSubscription cmdlet in the Shell.

Edit: I ran into the same issue with Exchange 2019 running on Windows 2019 Server Core.

The solution looks simple since it says “the problem has been fixed”. However, running the setup application again results in the next error message.

image

Again, for search engine possibilities:

Performing Microsoft Exchange Server Prerequisite Check

Configuring Prerequisites COMPLETED

Prerequisite Analysis FAILED

A Setup failure previously occurred while installing the HubTransportRole role. Either run Setup again for just this role, or remove the role using Control Panel.

For more information, visit: http://technet.microsoft.com/library(EXCHG.150)/ms.exch.setupreadiness.InstallWatermark.aspx

The Exchange Server setup operation didn’t complete. More details can be found in ExchangeSetup.log located in the :\ExchangeSetupLogs folder.

To remove the watermark, start the registry editor on the Exchange 2016 server and go to HKLM\Software\Microsoft\ExchangeServer\v15\HubTransportRole and delete the Watermark and Action entries.

image

Rerunning the setup application unfortunately results in the 1st error, despite the “the problem has been fixed” and the removal of the watermark entries.

It turns out that I have two Edge Transport servers in my environment, with an Edge Subscription. This Edge subscription is using the self-signed certificate for encryption purposes, and since this self-signed certificate on the new Exchange 2016 server differs from the original (before the crash) self-signed certificate the encryption possibilities fail.

To resolve this, using ADSI Edit to find the msExchEdgeSyncCredential on the Exchange 2016 server you are recovering, and delete all credential entries.

image

When running the Setup application with the /RecoverServer option again (for the third time ) it will succeed and successfully recover the Exchange 2016 server.

Exchange

Exchange 2016 CU10 and Exchange 2013 CU21 released

6 Comments

On June 19, 2016 Microsoft released Exchange 2016 CU10 and Exchange 2013 CU21, exactly 90 days after the previous CUs. Perfectly aligned with their regular quarterly release 🙂

Besides regular hotfixes there are a couple of important things to notice:

  • Exchange 2016 CU10 and Exchange 2013 CU21 need the .NET Framework 4.7.1. This is a hard requirement, so if .NET Framework 4.7.1 is not installed, the setup application will halt and generate an error message. You can use the Get-DotnetVersion.ps1 script that fellow MVP Michel de Rooij wrote to check the .NET version in advance.
  • A new requirement is the VC++ 2013 runtime library. This component provides WebReady Document Viewing in Exchange Server 2010 and 2013 and Data Loss Prevention in Exchange Server 2013 and 2016. In the (near) future the VC++ 2013 runtime library will be forced to install.
  • Standard support for Exchange 2013 ended on April 10th, 2018 and thus Exchange 2013 entered extended support. Exchange 2013 CU21 is the last planned CU. Customers need to install this CU to stay in a supported configuration, and to be able to install future Security Releases.
  • When running a hybrid configuration with Exchange Online, customers are required to install the latest Cumulative Update for Exchange 2013 or Exchange 2016, or install the latest Update Rollup for Exchange 2010 SP3.
  • None of these releases bring Active Directory Schema changes. You have to run Setup.exe /PrepareAD to activate new features like the following:
  • A new feature in Exchange 2016 CU10 and Exchange 2013 CU21 is the option to create shared mailboxes in Office 365 using the *-RemoteMailbox cmdlets. For example, after creating a user account in Active Directory you can use the following command to create a Shared Mailbox in Office 365 directly:
    Enable-RemoteMailbox -Identity <account> -Shared -RemoteRoutingAddress account@contoso.mail.onmicrosoft.com

Microsoft also released Update Rollup 22 for Exchange 2010 SP3. This Update Rollup brings support for Windows 2016 Domain Controllers (and corresponding Domain Functional Level and Forest Functional Level) and it fixes an issue with Web Services impersonation.

As always you should thoroughly test the new Cumulative Updates or Update Rollups in your test environment before installing in your production environment.

Installing a Cumulative Update hasn’t changed much over the years, so you can follow my previous blogpost about installing Exchange 2013 CU9, which is especially important when installing a Cumulative Update in a Database Availability Group.

More information and downloads:

Exchange

Exchange 2016 Database Availability Group and Cloud Witness

Leave a comment

When implementing a Database Availability Group (in Exchange 2010 and higher) you need a File Share Witness (FSW). This FSW is located on a Witness Server which can be any domain joined server in your internal network, as long as it is running a supported Operating System. It can be another Exchange server, as long as the Witness Server is not a member of the DAG you are deploying.

A long time ago (I don’t recall exactly, but it could well be around Exchange 2013 SP1) Microsoft started to support using Azure for hosting the Witness server. In this scenario you would host a Virtual Machine in Azure. This VM is a domain joined VM, for which you most likely also host a Domain Controller in Azure, and for connectivity you would need a site-2-site VPN connection to Azure. Not only from your primary datacenter, but also from your secondary datacenter, i.e. a multi-site VPN Connection, as shown in the following picture:

image

While this is possible and fully supported, it is costly adventure, and personally I haven’t seen any of my customers deploy it yet (although my customers are still interested).

Windows 2016 Cloud Witness

In Windows 2016 the concept of ‘Cloud Witness’ was introduced. The Cloud Witness concept is the same as the Witness server, but instead of using a file share it is using Azure Blob Storage for read/write purposes, which is used as an arbitration point in case of a split-brain situation.

The advantages are obvious:

  • No need for a 3rd datacenter hosting your Witness server.
  • No need for an expensive VM in Azure hosting you Witness server.
  • Using standard Azure Blob Storage (thus cheap).
  • Same Azure Storage Account can be used for multiple clusters.
  • Built-in Cloud Witness resource type (in Windows 2016 of course).

Looking at all this it seems like a good idea to use the Cloud Witness when deploying Windows 2016 failover clusters, or when deploying a Database Availability Group when running Exchange 2016 on Windows 2016.

Unfortunately, this is not a supported scenario at this point. All information you find on the Internet is most likely not officially published by the Microsoft Exchange team. If at one point the Cloud Witness becomes a supported solution for Exchange 2016, you can find it on the Exchange blog. When this happens, I’ll update this page as well.

More information

Using a Microsoft Azure VM as a DAG witness server – https://technet.microsoft.com/en-us/library/dn903504(v=exchg.160).aspx

Exchange

Exchange 2016 Edge Transport Server and IPv6

Leave a comment

I’ve never paid too much attention to IPv6, except for turning it off completely in case of strange issues. And admit it, most of you do the same.

Security is getting more and more important, and as a messaging consultant you want your Exchange environment top notch. In the Dutch community NGN I was pointed to internet.nl where you can check your presence on the Internet. Lots of red crosses when it comes to messaging and IPv6, reason for me to start looking into that.

In this blogpost I will focus on the Exchange 2016 Edge Transport server (I have two for inbound and outbound email) and the Exchange 2016 Mailbox server, which is load balanced behind a Kemp LoadMaster LM3600.

Exchange 2016 Edge Transport server

Although a lot of Exchange admins disable IPv6 on their Exchange servers (through a registry key) in case of strange issues, it is not a recommended solution.

I have two Exchange 2016 Mailbox servers, one Exchange 2013 multi-role server and two Edge Transport servers (one Exchange 2013 and the other Exchange 2016) for inbound and outbound SMTP traffic. There are two MX records which point to these Edge Transport servers. Both have an external IPv4 address.

The first step of course is to add an IPv6 address to the network adapter of the Edge Transport servers, your provider should be able to supply you with a sufficient IP range.

image

This should not result in too much issues. If you want to ping your server on IPv6 make sure that the File and Printer Sharing (Echo request – ICMPv6-In) inbound rule is enabled in Windows Firewall.

The next step is to enable the Edge Transport server for IPv6 usage. The Mailbox server has everything setup by default, but the Edge Transport server is only configured for IPv4.

Continue reading Exchange 2016 Edge Transport Server and IPv6 →

Office365

MigrationTransientException: Target database GUID cannot be used (Mailbox database size limits in Exchange Online)

4 Comments

If you are designing Exchange 2016 (or have been designing Exchange 2013) environment you are aware of the The Exchange 2016 Preferred Architecture (https://blogs.technet.microsoft.com/exchange/2015/10/12/the-exchange-2016-preferred-architecture/) and articles like Ask the Perf Guy: How big is too BIG? (https://blogs.technet.microsoft.com/exchange/2015/06/19/ask-the-perf-guy-how-big-is-too-big/) which explain pretty much how to design an Exchange solid (and large) Exchange environment.

When it comes to Mailbox databases, the recommended size limit for non-replicated databases is 200GB and for replicated databases 2 TB (when running 3 or 4 copies of a Mailbox database).

One can only guess how Microsoft has designed their Exchange servers in Exchange Online, but we can assume that the Preferred Architecture is written with their Exchange Online experiences in mind.

Sometimes error messages that are generated in Exchange Online can reveal more information. While moving mailboxes from Exchange 2010 to Exchange Online in a hybrid configuration the following error message was returned in a migration batch for a number of Mailbox databases:

Error: MigrationTransientException: Target database ‎’07bdf507-ab94-479b-aeb6-1bfef1458c4c‎’ cannot be used: Current database file size: 1502835900416 Current space available inside database: 100237312 Allowed database growth percentage: 90 Maximum database file size limit: 1622722691784 Is database excluded from provisioning: ‎’False‎’. –> Target database ‎’07bdf507-ab94-479b-aeb6-1bfef1458c4c‎’ cannot be used: Current database file size: 1502835900416 Current space available inside database: 100237312 Allowed database growth percentage: 90 Maximum database file size limit: 1622722691784 Is database excluded from provisioning: ‎’False‎’.

Obviously it’s telling us the migration cannot proceed since the target Mailbox (in Exchange Online!) has reached its size limit. The following sizes are reported:

  • Current database file size: 1502835900416 (1,502,835,900,416 bytes, approx. 1.5TB)
  • Current space available inside database: 100237312 (100.237.312 bytes, approx. 100MB)
  • Maximum database file size limit: 1622722691784 (1.622.722.691.784 bytes, approx. 1.6 TB)

So, the maximum size limit for Exchange 2016 in Exchange is not really used in Exchange Online, but it’s getting close, which is interesting to see.

What I don’t understand is why this issue occurs in the first place. To me it looks like a failing part in the provisioning service but I have to admit I’ve never seen this before in the last couple of years so I expect it’s only one Exchange server that’s failing here.