Category Archives: Uncategorized

Moving from Exchange 2010 to Office 365 Part III

Exchange 2010 hybrid and Edge Transport Server

In two previous blog posts I explained how to setup an Exchange 2010 hybrid environment. In these blog posts I used the Exchange 2010 (multi-role) server for the hybrid configuration, so both the Exchange Web Services (used for free/busy, Mailbox Replication Service, OOF, mail tips) and the SMTP connection between Exchange Online and Exchange 2010.

Now that Exchange 2010 end of support is getting closer (less than a year!) I get more questions regarding the move from Exchange 2010 to Exchange Online. And several questions include the use of an Exchange 2010 Edge Transport server in front of the Exchange 2010 multi-role server.

This configuration will look something like this:

exchange 2010 hybrid edge transport

Inbound mail from Internet is getting through Exchange Online Protection and when the mailbox is still on Exchange 2010 it is routed via the Edge Transport Server to the internal Exchange organization. Outbound mail is leaving the organization via the Edge Transport server or via Exchange Online Protection, depending of the location of the mailbox.

The challenge when configuring this in Exchange 2010 is shown in the following screenshot:

missing edge transport server

Compared to running the HCW on Exchange 2013 or Exchange 2016 there’s no option to configure the Edge Transport server for secure mail transport in Exchange 2010!!
The only option right now is to run the Hybrid Configuration Wizard, configure it using the Client Access and Mailbox servers option, but use the data for the Edge Transport where needed.

So, run the Hybrid Configuration Wizard, and when you need to enter the public IP address of the transport servers, enter the Public IP address of the Edge Transport server and not the public IP address of the load balancer VIP pointing to the Exchange 2010 internal servers). In my environment, the webmail.inframan.nl points to 176.62.196.253 and the Edge Transport Server smtphost.inframan.nl points to 176.62.196.245. This is the IP address I am going to use as shown in the following screenshot:

hcw public ip address

The next step is to select the certificate that’s used on the Edge Transport Server. By default, the HCW will only look at the internal Exchange 2010 server, so it won’t find any certificate installed on the Edge Transport server. To overcome this, I have imported the certificate of the Edge Transport Server in the certificate store of the internal Exchange 2010 server used by the HCW. In the HWC, click on the drop down box and select the certificate of the Edge Transport server, in my environment the smtphost.inframan.nl certificate:

hcw loading certificates

The last step is where the FQDN of the organization needs to be entered. I have a lot of discussion here because most admins want to enter something like ‘hybrid.domain.com’, but the FQDN of the transport server needs to be entered here, so in my environment this is the FQDN of the Edge Transport Server, i.e. smtphost.inframan.nl. This FQDN is used (together with the certificate information) to create a Send Connector from Exchange Online to the Edge Transport server.

hcw organization fqdn

Finish the Hybrid Configuration Wizard. It will be configured in Exchange 2010 and in Exchange Online and after a short time you can close the HCW:

hcw configure organization relationship

Now, when looking at the Exchange Management Console you can see the Send Connector from Exchange 2010 to Exchange Online. It is configured with the FQDN smtphost.inframan.nl as expected, but the source server of the Send Connector is still the internal Hub Transport server as shown in the following screen shot:

outbound to office 365 connector

Remove the Hub Transport server entry and add the Edge Transport server instead. If you have an Edge Synchronization in place you will see it immediately when you click the Add button.

In Exchange Online, the Receive Connector that’s created will check for any certificate with a wildcard like name, so the smtphost.inframan.nl certificate will automatically be accepted. The Send Connector is also created correctly. The FQDN of the Edge Transport server is used as the server to route message to, and the CN of the certificate that was selected in the HCW is also configured as shown in the following screenshot:

office 365 send connector certificate

We’re almost there. The only thing that needs to be done is to configure the Receive Connector on the Edge Transport Server for TLS from Exchange Online. You should have already configured the Edge Transport Server with the correct 3rd party certificate and when setting up an inbound connection it should use the 3rd party certificate. You can test this using the https://checktls.com tool online.

On the Edge Transport server, execute the following command in the Exchange Management Shell:

Get-ReceiveConnector "smtphost\Default internal receive connector SMTPHOST" | Set-ReceiveConnector -Fqdn smtphost.inframan.nl -TlsDomainCapabilities mail.protection.outlook.com:AcceptOorgProtocol

This will make sure the cross-premises email will be treated as internal email (SCL=-1). If you omit this step, there’s always the risk the email will be treated as external (I’ve seen SCL=5 in my environment) and will end up in the user’s Junk Email Folder.

Summary

When configuring an Exchange 2010 hybrid configuration it is not possible to configure an Edge Transport Server in the Hybrid Configuration Wizard. It is possible to configure this in the HCW for Exchange 2013 and Exchange 2016, but for Exchange 2010 this needs some manual changes.

In this blogpost I showed you the steps needed to configure an Edge Transport Server for secure messaging between Exchange Online and Exchange 2010. When configured this way cross-premises email will be seen as internal email and thus treated accordingly.

 

Exchange 2019 released and available on Microsoft Volume License Center

On Monday October 22, 2018 Microsoft has released Exchange 2019 publicly. Well… Publicly… Exchange 2019 is only available via Volume Licensing, so if you don’t have a VL agreement with Microsoft I’m afraid getting a legal version of Exchange 2019 can become a bit annoying.

How exciting is Exchange 2019? Personally, I think it’s a nice improvement compared to Exchange 2016, there are some new admin features for performance and security, and Exchange 2019 also includes some nice features for end users as well. But be aware, at this moment these features are only available in OWA and sometimes not (yet) in Outlook. But let’s have a look at Exchange 2019…

Engineering

In Exchange 2016 a Cumulative Update was released every 90 days, and this Cumulative Update was directly derived from Exchange Online. Within the Exchange product team there was only one ‘branch’ and all versions were coming from this branch.

In Exchange 2019 a separate branch is used, so within Microsoft there are now two branches. One for Exchange Online and one for Exchange 2019. This means that changes and improvements that are introduced in Exchange Online do not necessarily make it into Exchange 2019. At the same time, it is also possible that Microsoft introduces changes in Exchange 2019 that won’t make it into Exchange Online.

building-2019

Look at calendaring for example, a lot of new features are introduced recently in Exchange Online, a new look and feel etc. but these will never make it to Exchange 2019 unfortunately.

Windows Server 2019

Exchange 2019 runs on Windows Server 2019, and it only runs on Windows 2019. There are a lot of security and .NET improvement in Windows Server 2019 which Exchange 2019 uses, and these are not available in Windows 2016. So, Windows 2019 is the way forward when it comes to Exchange 2019….

Also, Windows 2019 Server Core is the recommended version of Windows 2019. Because of the lower footprint of Windows, Server Core reduces the attack surface and thus improves security of the server.

When it comes to Active Directory, the Domain Functional Level (DFL) and Forest Functional Level (FFL) should be at Windows 2012 R2 level. By enforcing this, Microsoft ensures customers are using a recent version of Windows for Domain Controllers, thus improving security of Active Directory.

From a personal perspective I’m curious about the impact of this. A lot of my customers are still running at a lower level of DFL and FFL (and thus a lower version of Windows), and also Windows 2019 might be a challenge to bring into production at this time…

Exchange 2019 Requirements

When it comes to server memory, the amount of recommended server memory is 128 GB (64 GB for Exchange 2019 Edge Transport Server). Exchange 2019 will run fine with less memory of course, but to make use of the performance improvements (in Windows 2019 and .NET Framework) 100 GB or more is needed, hence the 128 GB recommendation. The maximum amount of memory that’s supported is 256 GB.

One might argue that the minimum amount of server memory in Exchange 2016 was ‘only’ 8 GB. While this is true, Exchange 2016 won’t hardly start with 8 GB of memory, and a properly sized Exchange 2016 server also has 48 GB or 64 GB or RAM, so the step to 128 GB is not that big.

The maximum number of processor cores is now 48 (was 24 in Exchange 2016), and the amount of server memory is also related to the amount of processor cores, so less processor cores will certainly result in less memory.

The requirements calculator will be updated soon, and once released we will have more information regarding proper sizing.

If you are still interested, server virtualization is supported, but you can ask yourself if the advantages of server virtualization with these requirements outnumber the added layer of complexity (and potential hit on performance).

Metacache Database (MCDB)

The current hard disk technology is improving, and disk sizes continue to grow and will do the upcoming years. Disk I/O doesn’t grow, and a 16TB disk has approx. the same IOPS as a 4TB disk. The existing Exchange disk usage will suffer from performance issues with these larger disks, so in Exchange 2019 Microsoft has added ‘Metacache Database’ technology, where meta information from Mailbox databases is stored on SSD disks. So, an Exchange server will have JBOD disks where Mailbox databases are stored, but at the same time have SSD disks where this meta information is stored. The ratio SSD to spinning disk is 1:3, so for every spinning disk one SSD disk is used. This meta information can be mailbox information within a database, the folder structure within mailboxes, meta information about individual items or when items are small enough, even these individual items.

For a MCDB implementation, the Exchange server needs to be configured with a reseed-enabled DAG and a symmetric SSD count and size on each server (preferred architecture!), so all servers need to be completely identical.

mcdb-config

Meta information is automatically replicated from JBOD to SSD disks in a background task, so no need to worry about that. What happens if one SSD disk fails? It is a cached system, so all information will always be available in the Mailbox database, so when one SSD disk fails the server will automatically use the Mailbox database, but of course will be hit by a performance penalty.

MCDB will result is faster logon times, faster response times and less latency, something that will be extremely useful when Outlook is running online mode (in a Citrix environment).

Dynamic Database Cache

In previous versions of Exchange, database cache was automatically and equally used by all Mailbox databases, whether they were active copies or passive copies.

Exchange 2019 uses the concept of Dynamic Database Cache, where active Mailbox databases will be assigned (much) more memory than passive copies of Mailbox databases. This will result in much better performance. When a Mailbox database failover occurs, server memory is automatically re-aligned to the new situation.
Compared to Exchange 2016, the dynamic database cache will result in more memory per active mailbox database, which (again) will result in better performance.

dynamic-database

Exchange 2019 Search Engine

Exchange 2019 introduces a new search engine called ‘Big Funnel’, based on Bing technologies. Search indexes are not longer stored on disk per Mailbox database, but search indexes are now stored inside the Mailbox database on a per Mailbox basis. As such, Search Indexes are also replicated to passive copies of the Mailbox database and these are always in sync. When a corrupt page containing search index information is found, page patching occurs to copy a healthy page from a passive copy of the Mailbox database and the active copy of a Mailbox database.

Now that the content indexes are always good, issues with Mailbox database failovers no longer occur, and Mailbox database failover times should decrease dramatically.
Combined with the new MCDB and dynamic database cache search results should be increase dramatically, which is again especially interesting for Outlook running in online mode.

Client improvements

Exchange 2019 also comes with some client improvements but be aware that there’s a difference in client. OWA client behaves differently than an Outlook client on Windows, which is again differs from an Outlook client on Mac…

Recurring meetings with default end date

When creating a recurring appointment there’s a default end date on the appointments. This will prevent your users creating a recurring appointment without an end date, which is annoying when the user leaves the organization ending in a orphaned recurring meeting.

default-end-date

Block calendar during OOF

Very nice improvement in Exchange 2019 is the possibility to block your calendar when setting your Out-of-Office during this period. Besides blocking, you can also select to automatically decline new meeting requests or decline new requests and cancel existing meetings in your OOF time. Very useful!

block-calendar

Remove calendar events

One annoying issue with Exchange is that an Exchange admin cannot remove calendar items created by users. This is an issue when a recurring or future appointment is created by a user that has left the organization, and the meeting is an orphaned meeting.
Exchange 2019 introduces the Remove-CalendarEvents cmdlet, which makes it possible to remove future calendar items created by users.

The following example will remove all meetings created from this day forward for user Chris:

Remove-CalendarEvents -Identity chris@contoso.com -CancelOrganizedMeetings

The next example will remove meetings created by Kim Akers starting on November 1, 2018 for a period of 120 days. This is useful when Kim is on leave for a 4 months period:

Remove-CalendarEvents -Identity “Kim Akers” -CancelOrganizedMeetings -QueryStartDate 11-1-2018 -QueryWindowInDays 120

Email Address Internationalization

Email Address Internationalization (EAI) was already supported in Exchange Online, but now it has arrived in Exchange 2019 as well.

Wat is EAI? As Microsoft announced on the Exchange Team blog “Out of 7.6 billion people in the world, only 360 million are native English speakers” so lots of people don’t even use the character set that we are using currently in Exchange. There are a lot of character sets that are not supported in Exchange.

For example, I’m working with a European organization that acquired a company in Colombia. Importing these users into the existing Active Directory was quite a challenge, because they are using a Latin alphabet with diacritic. Before importing we had to convert all names and email addresses into our regular character set.

Exchange 2019 now supports EAI, and you can send email to users with an EAI address. Unfortunately, you cannot add an EAI proxy or Accepted Domain yet.

Examples of EAI email addresses are:

  • Latin alphabet (with diacritics): Pelé@example.com
  • Greek alphabet: δοκιμή@παράδειγμα.δοκιμή
  • Traditional Chinese characters: 我買@屋企.香港
  • Japanese characters: 甲斐@黒川.日本
  • Cyrillic characters: чебурашка@ящик-с-апельсинами.рф
  • Hindi email address: संपर्क@डाटामेल.भारत

For more information regarding Email Address Internationalization (in Office 365) please check Jeff Guillet’s blog on http://www.expta.com/2017/12/email-address-internalization-eai.html.

Unified Messaging

In Exchange 2019 the Unified Messaging server role is completely removed, so Exchange 2019 does not offer voice mail or auto attendant processing.

You can use the Microsoft cloud voice mail option or a 3rd party PBX solution that can record voicemail messages and have these sent by SMTP to your Exchange 2019 mailbox, but that’s not really a viable replacement I would say.

When moving UM enabled mailboxes from Exchange 2013 or Exchange 2016 to Exchange 2019, these mailboxes will automatically be UM disabled, but existing voicemail messages will remain in the user’s mailbox of course.

The options you have as a UM customer are:

  • Move all users and mailboxes to Office 365.
  • Migrate to Skype for Business Server 2019.
  • Remain on Exchange Server 2016 (supported until 2023).
  • Deploy 3rd party voice mail solution options

Summary

Exchange 2019 was released by Microsoft on October 22nd, 2018 and is the latest version of Exchange Server, targeted toward (large) enterprise customers that cannot move to the cloud (yet). As it is targeted towards enterprise customers, Exchange 2019 is only available via Volume Licensing only. This is not only true for the initial version of Exchange 2019, future CU’s will be made available via VL only as well.

There are a lot of new features in Exchange 2019 like the dynamic database cache and metacache database. Combined with the new hardware requirements and SSD disk this will certainly improve performance, and Outlook clients running in Online Mode (I say Citrix) will benefit from this.

Some new client features as well, like the possibility to block forwarding of meeting requests, block invites during Out-Of-Office and support for Email Address Internationalization.

Time to start playing with Exchange 2019, find out what the true benefits are, and hopefully blog about it in the near future 😊

More information

During Ignite 2018 in Orlando there were a few presentation on Exchange 2019 that are certainly worth having a look at:

 

Exchange 2016 Cumulative Update 11

Most likely you’ve seen this information before, because of my vacation in Dallas and New Orleans I’m a bit behind with blogging 😊

But on October 16, 2018 Microsoft has released Cumulative Update 11 (CU11) for Exchange 2016, this is a little later than expected to align the release of Exchange 2016 Cumulative Updates with the upcoming release of Exchange 2019. . There’s only a release for Exchange 2016, there won’t be any new CU’s for Exchange 2013 since Exchange 2013 is already in extended support. There will be security updates for Exchange 2013 though.

Exchange server and .NET Framework is not a happy marriage and it continues to be a struggle, or at least it looks that way. Exchange 2016 CU11 now supports .NET Framework 4.7.2. This version of .NET Framework is not mandatory, installation of .NET Framework 4.7.2 can be before installing of CU11 or after CU11. The .NET Framework 4.7.2 will be required for a future CU of Exchange 2016.

Another dependency is Visual C++, you might have seen this in previous CU’s and also in Exchange 2010 Update Rollup 23 as well. To avoid any issue, install Visual C++ 2012 (https://www.microsoft.com/download/details.aspx?id=30679) before installing Exchange 2016 CU11.

Exchange 2016 CU11 does not have any schema changes. If you’re upgrading from an older version of Exchange 2016, Active Directory changes (in the configuration container) might be needed. These will automatically be applied by the setup application, but you can also choose to update the configuration partition manually by running setup.exe /PrepareSchema /IAcceptExchangeServerLicenseTerms

As always, you should test a Cumulative Update thoroughly before bringing it to production, it won’t be the first time something goes wrong in production with a CU. But I have to say, I haven’t seen any major blocking issues so far…

More information and downloads of Exchange 2016 CU11:

Connecting to remote server outlook.office365.com failed

In a previous blog I explained how to enable MFA for Admin accounts. This is a great security solution, but unfortunately it breaks Remote PowerShell for Exchange Online.
When you try to connect to Exchange Online using the following commands:

$Cred= Get-Credential
$Session= New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://outlook.office365.com/PowerShell-LiveID -Credential $Cred -Authentication Basic -AllowRedirection

It fails with the following error message:

New-PSSession : [outlook.office365.com] Connecting to remote server outlook.office365.com failed with the following error message : Access is denied. For more information, see the about_Remote_Troubleshooting Help topic.
At line:1 char:11
+ $Session= New-PSSession -ConfigurationName Microsoft.Exchange -Connec …
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OpenError: (System.Manageme….RemoteRunspace:RemoteRunspace) [New-PSSession], PSRemotingTransportException
+ FullyQualifiedErrorId : AccessDenied,PSSessionOpenFailed

As shown in the following screenshot:

connecting-failed

To overcome this issue, Microsoft has a special Exchange Online PowerShell module that supports Multi Factor Authentication. You can download this from the Exchange Admin Center in Exchange Online by selecting hybrid in the navigation pane as shown in the following screenshot:

MFA-Portal

Click Configure followed by Open to download and start the setup application. Click Install to continue. The Exchange Online PowerShell module will be automatically installed in seconds and when finished it will automatically open a PowerShell window as shown in the following screenshot:

EXO-PSSession

You can now use the Get-EXOPSSession -UserPrincipalName admin@tenant.onmicrosoft.com command to logon to Remote PowerShell. A separate windows will be opened requesting your tenant credentials, followed by the MFA option you’ve configured.

If all is entered correctly the Remote PowerShell for Exchange Online is opened with MFA enabled.

 

Choose a password that’s harder for people to guess

When you’ve implemented Self Service Password Reset and a cloud user (i.e. an account that only lives in the Microsoft cloud, not an on-premises Active Directory account) wants to change his password, there’s a chance the user will see the following error message:
“Choose a password that’s harder for people to guess”

pass1word-guess
The odd thing is, when the user changes his password in the SSPR it even says the user is using a strong password as shown in the following screenshot:

pass1word

Note. I tried this with several combinations, like Pass1word, P@ssW0rd and Spring2018.

A similar error message can be “Unfortunately, your password contains a word, phrase or pattern that makes it easily guessable. Please try again with a different password.” as shown in the following screenshot:

guessable

The ‘problem’ here is that the user is hitting the ‘banned password list’ in Azure Active Directory. This banned password list is a list of over 1,000 passwords that can easily be guessed, and as such vulnerable for password spray attacks. These passwords are simple words like spring, summer, autumn, winter, football, company name, qwerty, 123456, welcome, zaq1zaq1 etc etc etc. There’s a list of most common passwords on WikiPedia. Of course there are several variations of passwords, password, Pass1word, Pass!word, Passw0rd, you name it, but Microsoft is using normalization techniques to filter out all replaced characters and thus block these passwords.

Banned passwords are part of the Azure AD Password Protection feature, a feature that’s still in preview at the time of writing (October 2018). When you logon to the Azure Portal (https://portal.azure.com) and navigate to Azure Active Directory | Authentication Methods (in the security section) you’ll see the Azure AD Password Protection feature:

password_protection

The banned password list is enforced by default, there’s no way to disable it. If you have an Azure AD Premium license, you can also use a custom banned password list and maintain you own list of words or phrases that you don’t want to be used as a password.

Summary

If your users run into the Choose a password that’s harder for people to guess error message when changing their password in Azure AD or Office 365, they are hitting the banned password list as part of the Azure AD Password protect feature. A feature that’s enforced by default, and implemented by Microsoft as a means to improve security.
This feature is available for cloud users only by default, but if you have implemented self service password reset (SSPR) with password writeback it also works. The nice thing is, it can also be extended to on-premises Active Directory for password changes on-premises. Nice topic for an upcoming blog.