Category Archives: Security

MTA-STS Support in Exchange 2019 and Exchange Online

My previous blogpost was about DANE and discussed how DANE can be used to make TLS negotiations between mailservers more secure. Another topic in this area is MTA-STS. MTA-STS stands for Message Transfer Server – Strict Transport Security and MTA-STS is a mechanism to enforce the use of TLS and the use of a valid 3rd party server certificate.

MTA-STS, DNS and Policies

When an MTA-STS capable servers wants to send an email, it first retrieves the MX record for the recipient domain. The second step is that the sending server check for an MTA-STS record in DNS. This record looks like:

_mta-sts.exchangelabs.nl. IN TXT "v=STSv1; id=202306242147;"

The id is an identifier and defines the version of the MTA-STS record when changes are made to the MTA-STS record. A good practice is to create an identifier based on the date and time of the last change. In this example, it is June 24, 2023 at 9:47pm.

The next step is that the sending servers looks for a policy. This policy is not stored in DNS, but on a website. The URL for this policy looks like: https://mta-sts.exchangelabs.nl/.well-known/mta-sts.txt. The subdomain mta-sts, the filename mta-sts.txt and the directory .well-known (including the dot) directory are mandatory for the MTA-STS policy. It must also be secured using a valid 3rd party server certificate.

The MTA-STS policy will look something like:

version: STSv1
mode: enforce
mx: smtphost.exchangelabs.nl
mx: *.mail.protection.outlook.com
max_age: 604800

The MTA-STS policy is structured as follows:

  • Version identifies the version of MTA-STS but must always be STSv1 (for now at least).
  • Mode defines how the policy must be applied:
    • Enforce: Sending MTAs MUST NOT deliver the message to hosts that fail MX matching or certificate validation or that do not support STARTTLS.
    • Testing: Sending MTAs that also implement the TLSRPT (TLS Reporting) specification [RFC8460] send a report indicating policy application failures (as long as TLSRPT is also implemented by the recipient domain); in any case, messages maybe delivered as though there were no MTA-STS validation failure.
    • None: In this mode, Sending MTAs should treat the Policy Domain as though it does not have any active policy.
  • MX defines all MX records in use by the recipient domain. This can be one entry, but it can hold multiple MX records, each on a separate line as shown in the policy above.
  • Max_age defines the time (in seconds) that the MTA-STS policy can be cached by a mail server. In this example, the policy is cached for 604800 seconds, which equals to 1 week. When a sending server must send a new email within a week, the policy is still cached. After checking the MX record the server retrieves the TXT record from DNS (as explained in the second step above) and when the identifier has not changed it uses the policy that is cached. If the identifies has changed within the lifetime of the cached policy, a new policy is downloaded.

So, in my example an MTA-STS capable mail server will check the MTA-STS policy and only connects to my mail server using TLS 1.2 (this is enforced with MTA-STS when mode is set to ‘enforce’) and only when a certificate that matches the FQDN is used. When authentication fails for an entry, the sending server continues with the next line in the policy, in my example with the MX record pointing to Exchange Online.

An interesting option in MTA-STS is reporting. DMARC has a reporting function as well, but reports are only sent by receiving domains. Reporting in MTA-STS is performed daily by sending mail servers that supports MTA-STS and TLS-RPT.

To configure the reporting functionality, create a mailbox in Exchange 2019 or Exchange Online and assign it an email address like TLSReports@Exchangelabs.nl. The next step is to configure the following DNS TXT record:

_smtp._tls.exchangelabs.nl. 3600 IN  TXT v=TLSRPTv1;rua=mailto:TLSReports@exchangelabs.nl

There are several online tools available for checking the MTS-STS record. Just like in my other blogs, I often use MXToolbox to check for DNS records as shown in the following screenshot:

As with the DANE checks, the mailhardener site (https://www.mailhardener.com/tools/mta-sts-validator) can also be used to check MTA-STS records as shown in the following screenshot:

When you check the IIS logs of the webserver where the mta-sts.txt file is stored, you can see which servers are using MTA-STS:

2023-07-11 08:29:59 172.16.1.4 GET /.well-known/mta-sts.txt - 443 - 74.125.217.68 Google-SMTP-STS - 200 0 0 119
2023-07-11 13:54:51 172.16.1.4 GET /.well-known/mta-sts.txt - 443 - 104.47.11.254 - - 200 0 0 62
2023-07-11 14:13:37 172.16.1.4 GET /.well-known/mta-sts.txt - 443 - 77.238.178.114 AHC/2.1 - 200 0 0 20

The first one is obvious, the second line is a Microsoft IP address, the third line is a Yahoo IP address. So, Google, Microsoft and Yahoo are using MTA-STS when sending email.

MTA-STS versus DANE

MTA-STS and DANE share a common concept, that is to secure the (initial) communication between mail servers. The ‘problem’ with DANE is that is relies on DNSSEC and the global roll-out of DNSSEC is very slow (to put it mildly).

MTA-STS was developed to overcome the slow roll-out of DNSSEC (since it does not use DNSSEC of course). MTA-STS can be seen as a ‘light-weight’ version of DANE and it will be sufficient for most customers.

And how about Exchange?

Just like with DANE, the ugly part is that Exchange 2019 does not support MTA-STS. You can configure the MTA-STS record in DNS and the policy on a website so that MTA-STS capable servers use your Exchange 2019 server safely. But for sent messages by Exchange 2019, MTA-STS is a no-go, it does not support it and most likely will never do.

Exchange Online on the other hand does support MTA-STS (since the beginning of 2022) for both inbound and outbound messages. The only thing you must do to enable it for inbound messages is create the TXT record in DNS and create and publish the MTA-STS policy.

Edited on July 11, 2023

DNSSEC and DANE support in Exchange server and Exchange Online

In the past I have written about SPF, DKIM and DMARC for email authentication in Exchange server to improve e-mail security. Additional topics to improve security are secure DNS (DNSSEC) and DNS-based Authentication of Named Entities (DANE). Where SPF, DKIM and DMARC are focusing more on the email messages and the sending hosts they are coming from, is DANE more focusing on setting up the TLS connection between mail servers. DANE has a dependency on DNSSEC but since my focus is on Exchange and Exchange Online, I don’t discuss DNSSEC and leave that to my provider (DANE only works when DNSSEC is enabled).

When two mailservers setup a connection, they negotiate a common TLS protocol and this handshake as it is called is unencrypted. As such, this handshake is vulnerable for man-in-the-middle or downgrade attacks.

DANE is using a special DNS record (a TLSA record, Transport Layer Security Authentication) where an organization can publish information regarding the certificate on the mail server and thus the TLS options available.

The TLSA record for my own domain is shown in the following example:

_25._tcp.smtphost.exchangelabs.nl IN TLSA 3 1 1 50167c478a2f536a88ee9ec232b14b0d223c2d5bdc837451eee104c153376cbe

When a mailserver wants to send email to my Exchange environment it retrieves the MX record first which is smtphost.exchangelabs.nl (this is an Exchange 2019 Edge transport server, with a Digicert certificate that matches the servername). When DNSSEC is not used at the recipient domain, opportunistic TLS is used. When DNSSEC is used at the recipient domain, a DANE capable mail server will check for a TLSA record.

When a TLSA record is found and retrieved, the sending server sets up a TLS connection with the receiving server, which in turn returns the fingerprint of the certificate. The sending server compares the fingerprint with the information found in DNS and when it matches, the connection is established and communication between the two servers continue, and the email is sent.

So, how do you create the TLSA record? There are publicly available webservers that can create TLSA records. Shumon Huque is a software engineer and technologist based in Washington DC who has a site that can generate TLSA records. As a bonus his site can also check TLSA records.

To generate a new TLSA record, navigate to his website on https://www.huque.com/bin/gen_tlsa and fill out the necessary information as shown in the following screenshot:

The easiest way to create a PEM format X.509 certificate is using the MMC certificate snap-in. Select your certificate and export it. Select the ‘No, do not export the private key’ option, select the ‘Base-64 encoded X.509 (CER)’ option and export the certificate. You can use Notepad to open the certificate export, and copy-and-past this into the PEM format X.509 textbox.

Select the port number (25), the protocol (TCP, not SMTP) and enter the domain name. This is the FQDN of the receiving server, so in my example it is smtphost.exchangelabs.nl and click Generate.

It will show the generated TLSA record, including the certificate information as shown in the following screenshot:

For my domain the following TLSA record is generated:

_25._tcp.smtphost.exchangelabs.nl. IN TLSA 3 1 1 a63d74fc7ec1acad702017d13479a1b60b36f234ccb96b90f97c9619ba2c91ab

How is this TLSA record structured?

TCP port 25 and the FQDN of the server make up the first part of this TLSA record.

The first number after the TLSA text is the certificate verification or the certificate usage on the server and this number can have 4 values:

  • 0 – CA Constraint. The certificate provided when establishing the TLS connection must be issued by the listed root-CA or one of its intermediate Certificate Authorities.
  • 1 – Service Certificate constraint. – The certificate used must match the TLSA record, and it must also pass the certification path validation to a trusted root-CA.
  • 2 – Trust Anchor Assertion. The TLSA record must match the certificate of the root CA, or one of the intermediate CAs of the certificate in use by the mailserver.
  • 3 – Domain Issued Certificate. The TLSA record matches the used certificate itself. The certificate does not need to be signed by other parties and as such a self-signed certificate can be used (this is interesting on an Exchange server).

In my TLSA record a value of 3 is used for the certificate usage, so the TLSA record must match the certificate on the Exchange 2019 server.

The second number that is used is the selector. The selector can have two values:

  • 0 – The entire certificate is used for matching.
  • 1 – Only the public key of the certificate is used for matching.

In my TLSA record the value is 1 for the selector, so only the public key is used.

The third number that is used is the matching type. The matching type can have three values:

  • 0 – the entire information select is present in the certificate associated data (the last text string in the TLSA record)
  • 1 – The SHA-256 hash of the public key of the certificate must match the certificate associated data.
  • 2 – The SHA-512 hash of the public key of the certificate must match the certificate associated data.

In my TLSA record a value of 1 is used for matching, to a SHA-256 has of the public key of the Exchange server certificate must match the data in the TLSA record.

The next step is to add the generated TLSA record in DNS and check the TLSA record once added.

You can use the same site (https://www.huque.com/bin/danecheck-smtp) to check the TLSA record. The site queries the MX record for your domain and checks the accompanying TLSA record. The result is shown in the following screenshot:

But there are more sites that can check your TLSA records. Mailhardener (https://www.mailhardener.com/tools/dane-validator) for example can do the same as shown in the following screenshot:

In this blog I am discussing DANE for mail server, but it can also be used for other services. For a website a TLSA record can be:

_443._tcp.www.exchangelabs.nl. IN TLSA 3 1 1 a63d74fc7ec1acad702017d13479a1b60b36f234ccb96b90f97c9619ba2c91ab

Or for IMAP it can be:

_995._tcp.mail.exchangelabs.nl. IN TLSA 3 1 1 a63d74fc7ec1acad702017d13479a1b60b36f234ccb96b90f97c9619ba2c91ab

The ugly part is that Exchange 2019 does not support DANE out of the box. For incoming mail, you can configure the TLSA record based on the certificate that is installed on the Exchange (Edge Transport) server. A DANE capable mail server automatically checks for a TLSA record and when available, use this to setup the TLS connection.

Exchange Online on the other hand only supports DANE for outbound connections at the time of writing. As an Exchange Online customer, there’s nothing you must configure, it’s available for everyone. DANE for inbound mail will become available in the future, but when is unclear. According to Microsoft it is ‘near future’ but unfortunately this is already the case for quite some time.

One final (and very important) closing note: When you have configured a TLSA record for your (inbound) Exchange server you MUST update your TLSA record when renewing the Exchange certificate. You must also do this when the Exchange server is used in a hybrid configuration, since Exchange Online will check for a TLSA record. Mail flow from Exchange Online to Exchange 2019 will halt and the following error message is shown in Exchange Online message trace:

Reason: [{LED=450 4.7.323 tlsa-invalid: The domain failed DANE validation [Message=450 4.7.323 tlsa-invalid: The domain failed DANE validation] [LastAttemptedServerName=smtphost.exchangelabs.nl] [LastAttemptedIP=185.116.41.45:25] [SmtpSecurity=11;-1] [HE1EUR01FT069.eop-EUR01.prod.protection.outlook.com 2023-06-06T10:23:06.0. OutboundProxyTargetIP: 185.116.41.45. OutboundProxyTargetHostName: smtphost.exchangelabs.nl

This is solved when a new TLSA record is generated and published in DNS.

The federation server proxy configuration could not be updated with the latest configuration on the federation service

After the latest Windows Updates (June 2022) I could not logon to Office 365 using a federated domain (Microsoft Teams in particular, the mailboxes are in Exchange 2019 for this domain), regular domains did not experience any issues.

Trying the regular ADFS URLs did not help, the site was not available:

When checking the WAP server I noticed the WAP service was not running and was not willing to start. At the same time, EventID 224 was logged in the eventlog with the “The federation server proxy configuration could not be updated with the latest configuration on the federation service” error messages as shown in the following screenshot:

Especially the additional data reveals a lot:

Retrieval of proxy configuration data from the Federation Server using trust certificate with thumbprint ‘76426A7DB45871F25A7BD5D883F2C5196B82E0DA’ failed with status code ‘Unauthorized’. The remote server returned an error: (401) Unauthorized.

At the same time, Event ID 276 is logged on the internal ADFS Server:

Obviously, the trust between the proxy server and the ADFS server is broken (it has been some time when I look at the timestamps, this happens in a test environment 😊) so the trust relationship needs to be re-established.

This can be done using the wizard in the Remote Access Management Console:

If you get a warning message like “Web Application Proxy could not connect to the AD FS configuration storage and could not load the configuration” you must change the ProxyConfigurationStatus in the registry (HKLM\Software\Microsoft\ADFS) from “2” to “1” as shown in the following screenshot.

Follow the wizard, select the appropriate certificate, check the changes and click the Configure button as shown in the following two screenshots:

When you check the eventlog, you’ll see Event ID 252 with the configuration changes:

And you can see that the ADFS Proxy server can authenticate successfully:

The server is now fully functional again.

on-Premises Azure Active Directory Password Protection

Last year I wrote a blogpost on password in Azure Active Directory (Choose a password that’s harder for people to guess – https://jaapwesselius.com/2018/10/15/choose-a-password-thats-harder-for-people-to-guess/) in which I mentioned the banned password lists and the Azure AD Password Protect feature. Back then this was only for Azure AD, but right now it is also available for on-premises Domain Controller as well (for some time already). It is possible for on-premises Domain Controllers to use the password protect functionality in Azure AD and thus block the possibility to use weak passwords in your on-premises environment. Let’s see how it works.

The password protection feature on-premises uses a Password Protection Agent that’s running on the on-premises Domain Controllers. When a user initiates a password change, the new password is validated by the Azure AD Password Protection agent, which request a password policy from the Azure AD Password Protection proxy service. This Password Protection service requests a password policy from Azure AD. The new password is never sent to Azure AD. This is shown in the following picture (borrowed from the Microsoft website):

azure-ad-password-protection

After receiving the password policy, the agent returns pass or fail for the new password. In case of fail the user must try it again.

Installation of the password protect consists of two steps:

  • The Azure AD Password Protection Proxy service using the AzureADPasswordProtectionProxySetup.exe software installer. This is installed on a domain joined computer that has access to the Internet and proxies the password policy request to Azure Active Directory.
  • The DC Agent service for password protection by using the AzureADPasswordProtectionDCAgentSetup.msi package. This runs on the Domain Controllers and send the password policy requests to the server running the proxy service.

Both can be downloaded from the Microsoft download center on https://www.microsoft.com/en-us/download/details.aspx?id=57071

Password Protection Proxy Installation

The first step is to install the password protection service. This server should be able to access Azure AD and since the Domain Controller does not have an internet connection this should be installed on a separate server. In my lab environment I have installed the password protection service on the Azure AD Connect server.

Installation of the password protection proxy is straightforward; you can use the GUI or the command line setup with the /quit switch for unattended install (and Server Core). After installation use PowerShell to register the proxy in Azure AD by using the following commands:

[PS] C:\> Import-Module AzureADPasswordProtection
[PS] C:\> Register-AzureADPasswordProtectionProxy -AccountUpn 'administrator@tenant.onmicrosoft.com'

This command can work when you have MFA enabled for admin accounts, if you don’t require MFA on your admin accounts (which is a bad practice IMHO) you can use the following command:

[PS] C:\> $globalAdminCredentials = Get-Credential
[PS] C:\> Register-AzureADPasswordProtectionProxy -AzureCredential $globalAdminCredentials

The last step is to register the forest in Azure Active Directory. This is very similar to the registration process of the proxy service. You can use the following PowerShell commands to register the forest:

Register-AzureADPasswordProtectionForest

[PS] C:\> Register-AzureADPasswordProtectionForest -AccountUpn ‘yourglobaladmin@yourtenant.onmicrosoft.com’

Again, when MFA is not enabled you can use the following command to register your forest in Azure AD:

[PS] C:\> $globalAdminCredentials = Get-Credential 'yourglobaladmin@yourtenant.onmicrosoft.com'
[PS] C:\> Register-AzureADPasswordProtectionForest -AzureCredential $globalAdminCredentials

Note. A multi-forest scenario is supported for the Password Protection service, you can install multiple forest using these commands. Multiple domains against one tenant is supported, one domain against multiple tenants is a not-supported scenario.

Some remarks:

  • The server where the password proxy agent server is installed should have .NET Framework 4.7 or higher installed.
  • For high availability it is recommended to install the password protection agents on multiple servers
  • The password protection proxy supports an in-place upgrade, so a newer version can be installed without uninstalling the previous version.

So how does this work, and how does the password protection service find the proxy server (or servers)?

When the Password Protection Proxy is installed it is registered in Active Directory with a well-know GUID. The Password Protection Agent checks Active Directory for this well-know GUID and finds the server where the Password Protection Agent is installed.

You can use the following PowerShell commands to find the Password Protection Proxy:

$SCP = "serviceConnectionPoint"
$Keywords = "{ebefb703-6113-413d-9167-9f8dd4d24468}*"
Get-ADObject -SearchScope Subtree -Filter {objectClass -eq $SCP -and keywords -like $Keywords }

It returns the server, and you can use ADSIEdit to inspect the computer:

Azure AD Password Protection Proxy SCP

This is much like how domain-joined Outlook clients find the Autodiscover SCP in Active Directory.

Installing the DC agent service

When the proxy service is installed and registered the Domain Controller agent service can be installed. It is just an MSI package that can be installed (using the GUI, accept license agreement and click install) or you can install it on the command line using the following command (use elevated privileges):

C:\> msiexec.exe /i AzureADPasswordProtectionDCAgentSetup.msi /quiet /qn

Note. Installation of the DC agent requires a restart, but you can use the /norestart switch to reboot at a more convenient time.

After rebooting the Domain Controller the password protection service is ready for use.

Some remarks:

  • Azure AD Password protection service requires an Azure AD Premium P1 or P2 license.
  • Domain Controllers should be Windows 2012 or higher.
  • Domain Controllers should have .NET Framework 4.5 or higher installed.
  • You never know which Domain Controller is going to process a password change. Therefore, the Password Protection service need to be installed on all Domain Controllers. For a straightforward environment this should not be a problem, but for large enterprises with lots of DC’s it can be an issue (I deliberately do not that about security officers at this point :-))
  • Both the proxy service and the DC agent support an in-place upgrade, so a newer version can be installed without uninstalling the old version.

Testing the Azure AD Password Protection service

So, after installing the Password Protection Proxy and the DC agent it’s time to test which is relatively simple. Logon to a domain-joined workstation, use CTRL-ALT-DELETE to change the password. When using a simple password like “Summer2019” or something it fails with the following error message.

Unable to update the password

From this moment on it is no longer possible to use weak passwords, locally enforced by Azure Active Directory and again a step closer to a safer environment.

An error occurred trying to connect the WSUS server

You might not expect a WSUS blog post on a site maintained by an Exchange consultant, but there are still customers using Exchange servers on-premises, and these need to be patched as well (and so are the clients of course).

After installing and a new WSUS server running on Windows 2016 I quickly ran into an annoying issue after configuring the WSUS server and downloading the updates. The console would no longer connect and generated a ‘Connection Error’ popup saying “An error occurred trying to connect the WSUS server. This error can happen for a number of reasons. Check connectivity with the server. Please contact your network administrator if the problem persists.”

Error Connection Error

When you click the copy error to clipboard button the following is copied:

The WSUS administration console was unable to connect to the WSUS Server via the remote API.
Verify that the Update Services service, IIS and SQL are running on the server. If the problem persists, try restarting IIS, SQL, and the Update Services Service.
The WSUS administration console has encountered an unexpected error. This may be a transient error; try restarting the administration console. If this error persists, Try removing the persisted preferences for the console by deleting the wsus file under %appdata%\Microsoft\MMC\.

If IISRESET was executed, it runs again for some time, but then the issue happens again. When looking at the IIS console when this error occurs it turns out that the WsusPool was stopped as can be seen in the following screenshot:

WSUS App Pool

Starting the WsusPool solves the problem temporarily, but after some time it stops again. And again… and again…

It turns out to be a private memory issue in the WsusPool which seems to be depleted quickly. It is possible to assign more memory, but since I have no clue how much memory to assign I changed the setting to ‘0’ (1,843,200 KB is default) so the WsusPool can use anything it needs.

WSUS App Pool advanced settings

After changing the private memory limit for the WsusPool the error no longer occurs.