Tag Archives: Active Directory

Active Directory recycle bin

I was under the impression I blogged about this years ago (while working on an Exchange 2010 –> 2016 migration) but I couldn’t find my own blog, so here it goes (again).

The Active Directory Recycle Bin saved my life a couple of times, not only my life but also my customer’s life. With the Active Directory Recycle Bin you can restore deleted object from Active Directory, and when using Azure AD Connect also automatically restore object in Azure Active Directory (assuming the deleted object was synchronized before the actual deletion of course).

 Too bad that’s it disabled by default, but enabling the Active Directory Recycle Bin to just a matter of start the Active Directory Administrative Center, select and right-click your root domain and select ‘Enable Recycle Bin…” as shown in the following screenshot:

Of course, it is also possible to enable this using PowerShell, just execute the following commands:

PS C:\> Import-Module ActiveDirectory
PS C:\> Enable-ADOptionalFeature "Recycle Bin Feature" -Scope ForestOrConfigurationSet -Target "ProExchangeAdmin.com"
WARNING: Enabling 'Recycle Bin Feature' on 'CN=Partitions,CN=Configuration,DC=ProExchangeAdmin,DC=com' is an irreversible action! You will not be able to disable 'Recycle Bin Feature' on 'CN=Partitions,CN=Configuration,DC=ProExchangeAdmin,DC=com' if you proceed.
Are you sure you want to perform this action?
Performing the operation "Enable" on target "Recycle Bin Feature".
[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [S] Suspend  [?] Help (default is "Y"): y
PS C:\>

Wait until all Domain Controllers have replicated this change and you are all set.

But how does this work under the hood? Deleted object don’t stay in the recycle bin forever. By default, a deleted object continues to exist in Active Directory for 180 days, which is set on the msDS-deletedObjectLifetime attribute of the Directory Service object. On the object that is deleted, the isDeleted and isRecycled property come into play.

When a user object is deleted, the isDeleted property on the user is set to True. At this point it is logically deleted, but physically still available in Active Directory (in the Deleted Items container) and it can be restored. When the deleted object lifetime has passed, the isRecycled property is set to True and the Active Directory garbage collector knows it can remove this object from Active Directory (physically removed from the database).

This is graphically shown in the following figure.

It is possible to increase the deleted object lifetime by stamping a higher value on the msDS-DeletedObjectLifetime property. For example, to increase the default lifetime of 180 days to 2 years (=730 days) you can use the following PowerShell command:

PS C:\> Set-ADObject -Identity "CN=Directory Service,CN=Windows NT,CN=Services,CN=Configuration,DC=ProExchangeAdmin,DC=COM" -Partition "CN=Configuration,DC=ProExchangeAdmin,DC=COM" -Replace:@{"msDS-DeletedObjectLifetime" = 730}

Restoring a deleted object from the recycle bin is easy. Open Active Directory Administrative Center, navigate to your domain and select deleted objects. Select and right-click the object and select Restore as shown in the following figure:

It is also possible to use PowerShell to restore a deleted object. For example, to find the user in the previous example we can use the following Powershell command:

PS C:\> Get-ADObject -Filter 'samaccountname -eq "Johns"' -IncludeDeletedObjects

Deleted           : True
DistinguishedName : CN=Labs | John Smith\0ADEL:7ee41efd-0034-46d7-9313-62360fff43fb,CN=Deleted Objects,DC=labs,DC=local
Name              : Labs | John Smith
ObjectClass       : user
ObjectGUID        : 7ee41efd-0034-46d7-9313-62360fff43fb

PS C:\>

Pipe this output to the Restore-ADObject command and the deleted user will be restored.
Very simple and very useful, I always recommend enabling this.

Transfer FSMO Role Win32 error returned is 0x20af

For a client I had to test some domain controller scenarios and issues. There are two domain controllers (DC01 and DC02) and a new one (NewDC) is added. FSMO roles are moved to NewDC and DC01 must be decommissioned. No big deal you would say….

Domain FSMO roles (RID Master, PDC and Infrastructure master) moved in an instant, but moving the forest FSMO roles (Schema master and Domain Naming master) failed with the following error, both in the GUI as well as NTDSUTIL:

Win32 error returned is 0x20af(The requested FSMO operation failed. The current FSMO holder could not be contacted.)
Depending on the error code this may indicate a connection, ldap, or role transfer error.

Using Netdom query FSMO I could see the roles.

This happened when logged on the AD01 as well as NewDC, connectivity was not an issue and I did not notice any strange issues with Exchange and Outlook or OWA clients.

Using Google I found several other similar issues, and the overall recommendation was to seize the FSMO roles. Seizing FSMO roles is an option when the old Domain Controller is no longer available, but in my scenario I had to keep the old Domain Controller for testing.

But instead of using Google I had to look in the eventlog in the first place. The first entry I found was a confirmation that the roles could not be transferred, but Additional Data Error value: 4 was not very helpful:

An attempt to transfer the operations master role represented by the following object failed.

Current operations master role:
CN=NTDS Settings,CN=DC01,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=msexchangebook,DC=com
Proposed operations master role:
CN=NTDS Settings,CN=NIEUWEDC,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=msexchangebook,DC=com

Additional Data
Error value:

Subsequent entries were more helpful, Event 2019, ActiveDirectory_DomainService indicated replication issues:

Or in tekst for Search Optimization:

This server is the owner of the following FSMO role, but does not consider it valid. For the partition which contains the FSMO, this server has not replicated successfully with any of its partners since this server has been restarted. Replication errors are preventing validation of this role.

Operations which require contacting a FSMO operation master will fail until this condition is corrected.

FSMO Role: CN=Partitions,CN=Configuration,DC=msexchangebook,DC=com

User Action:

  1. Initial synchronization is the first early replications done by a system as it is starting. A failure to initially synchronize may explain why a FSMO role cannot be validated. This process is explained in KB article 305476.
  2. This server has one or more replication partners, and replication is failing for all of these partners. Use the command repadmin /showrepl to display the replication errors. Correct the error in question. For example there maybe problems with IP connectivity, DNS name resolution, or security authentication that are preventing successful replication.
  3. In the rare event that all replication partners are expected to be offline (for example, because of maintenance or disaster recovery), you can force the role to be validated. This can be done by using NTDSUTIL.EXE to seize the role to the same server. This may be done using the steps provided in KB articles 255504 and 324801 on http://support.microsoft.com.

The following operations may be impacted:
Schema: You will no longer be able to modify the schema for this forest.
Domain Naming: You will no longer be able to add or remove domains from this forest.
PDC: You will no longer be able to perform primary domain controller operations, such as Group Policy updates and password resets for non-Active Directory Domain Services accounts.
RID: You will not be able to allocation new security identifiers for new user accounts, computer accounts or security groups.
Infrastructure: Cross-domain name references, such as universal group memberships, will not be updated properly if their target object is moved or renamed.

In the end it turned out that DNS was not configured correctly, hence the replication issues. Fixed the DNS settings on the Domain Controllers and after a short while the Domain Controllers resumed replication and I was able to move the Forest FSMO roles to the new Domain Controller.

So in short, always check eventlog, replication and DNS when troubleshooting Domain Controller issues 😉

Exchange 2010 – This mailbox database contains one or more mailboxes, mailbox plans, archive mailboxes, or arbitration mailboxes

After migrating all mailboxes to Exchange 2016 and Exchange Online it is time to decommission the old Exchange 2010 servers. One of the servers could not be removed and the dreaded “This mailbox database contains one or more mailboxes, mailbox plans, archive mailboxes, or arbitration mailboxes” was shown:

This typically means that there is some sort of mailbox (arbitration or archive) is still available in the database, causing this issue. Unfortunately using the -Verbose switch did not reveal any other useful information.

Also, when trying to use Remote PowerShell directly (Add-PSSnapin Microsoft.Exchange.Management.PowerShell.E2010, which bypasses RBAC and sometimes help) dit not help either this time.

Note. I was able to find dozens of similar threads on the Internet about this warning, but not a single one applied to my scenario. Ok, I found one where they removed the mailbox database using ADSI Edit, but on the long term that will cause other issues I’m afraid. You can do this, but there are still references to this mailbox database in Active Directory, so stuff will be lingering if you do this. Also, a lot of the threads are about Exchange 2013 and higher, but we were decommissioning Exchange 2010.

Last resort is to look directly into the mailbox database properties in Active Directory using LDP.exe. In LDP, navigate to the mailbox database and open the details (or dump it to a text file). Using the dump a user account was found that still referenced this mailbox database for an archive mailbox:

The user account was fixed and we were able to remove the mailbox database.

Configure OAuth authentication in Exchange 2016

As long as I can remember the Hybrid Configuration Wizard finishes successfully, and itgenerates the error about the OAuth portion of the hybrid configuration.

Configure Intra-Organization Connector

HCW8064 – The HCW has completed, but was not able to perform the OAuth portion of your Hybrid configuration. If you need features that rely on OAuth, you can try running the HCW again or manually configure OAuth using these manual steps.

The Learn more option redirects to the Microsoft page Configure OAuth authentication between Exchange and Exchange Online organizations. I used that article for the PowerShell commands in this blogpost.

OAuth is used cross-premises to logon to other services, on behalf of the user. So, if you are logged on to some Microsoft service, this service can use OAuth to access services in Exchange on-premises and vice versa.

Example of these cross-premises services are:

  • Message Records Management (MRM).
  • Exchange in-place eDiscovery.
  • Exchange in-place Archiving.
  • Teams calendaring.

The HCW can configure Azure Active Directory for OAuth authentication, it can create the IntraOrganizationConnectors, but it cannot export and import the (self-signed) certificate on the Exchange server, nor can it (or does it) create the authorization server objects in Active Directory. So, time to test, guided by the Microsoft article and write down my experiences.

Note. This only works for Exchange 2013 and higher, I have been working on this in a mixed Exchange 2016 and Exchange 2019 environment.

Configuring OAuth between Office 365 and Exchange Online involve a number of steps.

Create Authorization server objects in Exchange on-premises

To create the authorization server objects in your on-premises environment enter the following commands in the Exchange Management Shell.

New-AuthServer -Name "WindowsAzureACS" -AuthMetadataUrl "https://accounts.accesscontrol.windows.net/contoso.com/metadata/json/1"
New-AuthServer -Name "evoSTS" -Type AzureAD -AuthMetadataUrl https://login.windows.net/contoso.com/federationmetadata/2007-06/federationmetadata.xml

Your verified domain contoso.com (in the command) should be something like exchangelabs.nl, and not <your tenant name> as outlined in the Microsoft article.


Enable the partner application for use with Exchange Online

The partner application was created in the previous step (the first command) and this should be enabled. Do this using the following command in Exchange Management Shell (on-premises):

Get-PartnerApplication | ?{$_.ApplicationIdentifier -eq "00000002-0000-0ff1-ce00-000000000000" -and $_.Realm -eq ""} | Set-PartnerApplication -Enabled $true

Export the Exchange authorization certificate

Authentication cross-premises is using certificates, so the on-premises certificate needs to be exported to Azure Active Directory. In case you were wondering where the CN=Microsoft Exchange Server Auth Certificate certificate was coming from when running the Get-ExchangeCertificate command in Exchange Management Shell, here you go.

Use the following PowerShell commands and store them in a PowerShell script called ExportAuthCert.ps1 or something and run it. This should export the OAuth certificate to a file called OAuthCert.cer.

$ThumbPrint = (Get-AuthConfig).CurrentCertificateThumbprint
If((Test-Path $ENV:SYSTEMDRIVE\OAuthConfig) -eq $false)
$oAuthCert = (dir Cert:\LocalMachine\My) | ?{$_.ThumbPrint -Match $ThumbPrint}
$CertType = [System.Security.Cryptography.X509Certificates.X509ContentType]::Cert
$CertBytes = $oAuthCert.Export($CertType)
$CertFile = "$env:SYSTEMDRIVE\OAuthConfig\OAuthCert.cer"
[System.IO.File]::WriteAllBytes($CertFile, $CertBytes)


Note. The Export-ExchangeCertificate command doesn’t work in this scenario since the self-signed certificate isn’t exportable.

Import the Exchange authorization certificate into Azure AD

The next step is to import the OAuthCert.cer certificate into Azure AD. Connect to the Microsoft Online service (Connect-MSOLService, if you don’t have this installed you can use the Install-Module MSOnline command) and run the following commands when connected:

$Cred = Get-Credential
Connect-MSOLService -Credential $Cred
$CertFile = "$ENV:SYSTEMDRIVE\OAuthConfig\OAuthCert.cer"
$objFSO = New-Object -ComObject Scripting.FileSystemObject
$CertFile = $objFSO.GetAbsolutePathName($CertFile)
$CER = New-Object System.Security.Cryptography.X509Certificates.X509Certificate
$binCert = $cer.GetRawCertData()
$CredValue = [System.Convert]::ToBase64String($binCert)
$ServiceName = "00000002-0000-0ff1-ce00-000000000000"
$P = Get-MsolServicePrincipal -ServicePrincipalName $ServiceName
New-MsolServicePrincipalCredential -AppPrincipalId $P.AppPrincipalId -Type asymmetric -Usage Verify -Value $credValue

This will import the self-signed certificate from the Exchange server into Azure AD so it can be used for mutual authentication.

I did not run the commands mentioned above on my Exchange server but on my Azure AD Connect server since the MSOL module was loaded on that server. For importing the certificate file I had to use the following command accessing the certificate file (instead of the $ENV:System variable):

$CertFile = "\\AMS-EXCH01\C$\OAuthConfig\OAuthCert.cer"

Register endpoints in Azure Active Directory

The last step is to register the endpoints of your on-premises Exchange environment into Azure Active Directory. You can use the following commands to register the endpoints:

$ServiceName = "00000002-0000-0ff1-ce00-000000000000";
$x = Get-MsolServicePrincipal -AppPrincipalId $ServiceName;
Set-MSOLServicePrincipal -AppPrincipalId $ServiceName -ServicePrincipalNames $x.ServicePrincipalNames;

Instead of webmail.exchangeserver.com and Autodiscover.exchangeserver.com you must use your own local FQDNs of the Exchange server (shown below in the verification screenshot).

You can use the the following command to check if this was configured correctly.

Get-MsolServicePrincipal -AppPrincipalId 00000002-0000-0ff1-ce00-000000000000 | select -ExpandProperty ServicePrincipalNames

As shown in the following screenshot:

Register Endpoints

Note. Over the years I have run the HCW several times. Domains have been added, but not (automatically) deleted in Azure Active Directory.

IntraOrganizationConnectors and AvailabilityAddressSpace

The Hybrid Configuration Wizard created the IntraOrganizationConnectors (both in Exchange 2016 as well as Exchange Online) and configured the AvailabilityAddressSpace. There’s no need to create these, but you have to check them using the Get-IntraOrganizationConnector and the Get-AvailabilityAddressSpace commands)

Verify the OAuth configuration

To verify the OAuth configuration you can use the Test-OAuthConnectivity command. You must do this on the on-premises Exchange server and in Exchange Online.

On the on-premises Exchange server use the Exchange Online Uri and a mailbox on-premises:

Test-OAuthConnectivity -Service EWS -TargetUri https://outlook.office365.com/ews/exchange.asmx -Mailbox onprem-user@exchangeserver.com -Verbose | Format-List

In Exchange Online, use the Exchange on-premises Uri with a mailbox in Exchange Online:

Test-OAuthConnectivity -Service EWS -TargetUri https://webmail.exchangeserver.com/metadata/json/1 -Mailbox Online-User@exchangeserver.com -Verbose | Format-List

The output is an extended list, but in the end you should see ResultType: Success and IsValid:True on your console:


You have now configured the OAuth between Exchange Online and Exchange On-Premises.


on-Premises Azure Active Directory Password Protection

Last year I wrote a blogpost on password in Azure Active Directory (Choose a password that’s harder for people to guess – https://jaapwesselius.com/2018/10/15/choose-a-password-thats-harder-for-people-to-guess/) in which I mentioned the banned password lists and the Azure AD Password Protect feature. Back then this was only for Azure AD, but right now it is also available for on-premises Domain Controller as well (for some time already). It is possible for on-premises Domain Controllers to use the password protect functionality in Azure AD and thus block the possibility to use weak passwords in your on-premises environment. Let’s see how it works.

The password protection feature on-premises uses a Password Protection Agent that’s running on the on-premises Domain Controllers. When a user initiates a password change, the new password is validated by the Azure AD Password Protection agent, which request a password policy from the Azure AD Password Protection proxy service. This Password Protection service requests a password policy from Azure AD. The new password is never sent to Azure AD. This is shown in the following picture (borrowed from the Microsoft website):


After receiving the password policy, the agent returns pass or fail for the new password. In case of fail the user must try it again.

Installation of the password protect consists of two steps:

  • The Azure AD Password Protection Proxy service using the AzureADPasswordProtectionProxySetup.exe software installer. This is installed on a domain joined computer that has access to the Internet and proxies the password policy request to Azure Active Directory.
  • The DC Agent service for password protection by using the AzureADPasswordProtectionDCAgentSetup.msi package. This runs on the Domain Controllers and send the password policy requests to the server running the proxy service.

Both can be downloaded from the Microsoft download center on https://www.microsoft.com/en-us/download/details.aspx?id=57071

Password Protection Proxy Installation

The first step is to install the password protection service. This server should be able to access Azure AD and since the Domain Controller does not have an internet connection this should be installed on a separate server. In my lab environment I have installed the password protection service on the Azure AD Connect server.

Installation of the password protection proxy is straightforward; you can use the GUI or the command line setup with the /quit switch for unattended install (and Server Core). After installation use PowerShell to register the proxy in Azure AD by using the following commands:

[PS] C:\> Import-Module AzureADPasswordProtection
[PS] C:\> Register-AzureADPasswordProtectionProxy -AccountUpn 'administrator@tenant.onmicrosoft.com'

This command can work when you have MFA enabled for admin accounts, if you don’t require MFA on your admin accounts (which is a bad practice IMHO) you can use the following command:

[PS] C:\> $globalAdminCredentials = Get-Credential
[PS] C:\> Register-AzureADPasswordProtectionProxy -AzureCredential $globalAdminCredentials

The last step is to register the forest in Azure Active Directory. This is very similar to the registration process of the proxy service. You can use the following PowerShell commands to register the forest:


[PS] C:\> Register-AzureADPasswordProtectionForest -AccountUpn ‘yourglobaladmin@yourtenant.onmicrosoft.com’

Again, when MFA is not enabled you can use the following command to register your forest in Azure AD:

[PS] C:\> $globalAdminCredentials = Get-Credential 'yourglobaladmin@yourtenant.onmicrosoft.com'
[PS] C:\> Register-AzureADPasswordProtectionForest -AzureCredential $globalAdminCredentials

Note. A multi-forest scenario is supported for the Password Protection service, you can install multiple forest using these commands. Multiple domains against one tenant is supported, one domain against multiple tenants is a not-supported scenario.

Some remarks:

  • The server where the password proxy agent server is installed should have .NET Framework 4.7 or higher installed.
  • For high availability it is recommended to install the password protection agents on multiple servers
  • The password protection proxy supports an in-place upgrade, so a newer version can be installed without uninstalling the previous version.

So how does this work, and how does the password protection service find the proxy server (or servers)?

When the Password Protection Proxy is installed it is registered in Active Directory with a well-know GUID. The Password Protection Agent checks Active Directory for this well-know GUID and finds the server where the Password Protection Agent is installed.

You can use the following PowerShell commands to find the Password Protection Proxy:

$SCP = "serviceConnectionPoint"
$Keywords = "{ebefb703-6113-413d-9167-9f8dd4d24468}*"
Get-ADObject -SearchScope Subtree -Filter {objectClass -eq $SCP -and keywords -like $Keywords }

It returns the server, and you can use ADSIEdit to inspect the computer:

Azure AD Password Protection Proxy SCP

This is much like how domain-joined Outlook clients find the Autodiscover SCP in Active Directory.

Installing the DC agent service

When the proxy service is installed and registered the Domain Controller agent service can be installed. It is just an MSI package that can be installed (using the GUI, accept license agreement and click install) or you can install it on the command line using the following command (use elevated privileges):

C:\> msiexec.exe /i AzureADPasswordProtectionDCAgentSetup.msi /quiet /qn

Note. Installation of the DC agent requires a restart, but you can use the /norestart switch to reboot at a more convenient time.

After rebooting the Domain Controller the password protection service is ready for use.

Some remarks:

  • Azure AD Password protection service requires an Azure AD Premium P1 or P2 license.
  • Domain Controllers should be Windows 2012 or higher.
  • Domain Controllers should have .NET Framework 4.5 or higher installed.
  • You never know which Domain Controller is going to process a password change. Therefore, the Password Protection service need to be installed on all Domain Controllers. For a straightforward environment this should not be a problem, but for large enterprises with lots of DC’s it can be an issue (I deliberately do not that about security officers at this point :-))
  • Both the proxy service and the DC agent support an in-place upgrade, so a newer version can be installed without uninstalling the old version.

Testing the Azure AD Password Protection service

So, after installing the Password Protection Proxy and the DC agent it’s time to test which is relatively simple. Logon to a domain-joined workstation, use CTRL-ALT-DELETE to change the password. When using a simple password like “Summer2019” or something it fails with the following error message.

Unable to update the password

From this moment on it is no longer possible to use weak passwords, locally enforced by Azure Active Directory and again a step closer to a safer environment.