During MEC 2014 I delivered a presentation on “Backup, Restore and Disaster Recovery” in Exchange 2013. In two demos I showed replay of transaction log files and an offline backup (see previous post) and what happens when creating a VSS snapshot using the low level tool called Diskshadow. This is todays topic.

VSS Framework

The VSS Framework was first introduced in Windows Server 2003 and consists of:

• VSS – Volume Shadow Copy Service;
• Requestor – typically a backup application;
• Writers – the glue between the VSS service and the application. There are writers for Exchange, Active Directory, NTFS, IIS, Hyper-V and lots of other applications;
• Provider – the part that controls the disk. This can be a Windows provider (provided by Microsoft) or a hardware specific provider. This is provided by a hardware vendor like HP, EMC or Dell (Equallogic) for their hardware storage solution.

You can use the VSSadmin tool to show the available writers on a given server. To do this, open an command prompt and enter the following command:

Vssadmin list writers

As shown in the following figure:

If you have issues with VSS backups you can always use the VSSadmin tool. In this figure the Last Error is showing “No Error” so you should be fine here. If VSS is causing headaches this is the first place to check.

If a VSS backup is created the following steps are performed:

1. The backup application (requestor) sends a command to the VSS to make a shadow copy of the mailbox database.
2. The VSS sends a command to the Exchange Writer to prepare for a snapshot backup.
3. The VSS then sends a command to the appropriate provider to create a shadow copy of the mailbox database. The storage provider can be a hardware storage provider (provided by the hardware vendor) or the default Windows storage provider.
4. The Exchange Writer temporarily stops, or quiesces, the mailbox database and puts it into read-only mode; all data in server memory is then flushed to the mailbox database. Also, a log file rollover is performed to make sure that all data will be in the backup set. This holds for a couple of seconds for the snapshot to be created (in the next step). All write IOs are queued at this point.
5. The shadow copy is created.
6. The VSS releases the Exchange Server to resume ordinary operations, and all queued write IOs are completed.
7. The VSS queries the Exchange Writer to confirm that the write IOs were successfully held during the snapshot creation. If the write operations were not successfully held, there could be a potentially inconsistent shadow copy. If this is the case, the shadow copy is deleted and the requestor is notified of the failed snapshot.
8. If the snapshot was successful, the requestor verifies the integrity of the backup set (the clone). If the clone integrity is good, the requestor informs Exchange Server that the snapshot was successful. The snapshot can now be transferred to a backup device.
9. When all data is succesfully moved to the backup device, the requestor informs VSS that the backup was successful and that the log files can be purged.

Diskshadow

You can use the Diskshadow tool to create VSS snapshots from the command line. Please not that this is just a snapshot and not a backup solution. For a backup you need to perform additional steps as I’ll outline in the last part of this blog.

To create a snapshot of just Exchange and nothing else you have to exclude all writers that are available on the server and only include the Exchange writer. You can achieve this by opening the Diskshadow utility and enter the following commands:

SET verbose on

SET context persistent

# Exclude other writers on Exchange Server

# Can be retrieved using VSSADMIN List Writers

Writer Exclude {d61d61c8-d73a-4eee-8cdd-f6f9786b7124}

Writer Exclude {75dfb225-e2e4-4d39-9ac9-ffaff65ddf06}

Writer Exclude {0bada1de-01a9-4625-8278-69e735f39dd2}

Writer Exclude {e8132975-6f93-4464-a53e-1050253ae220}

Writer Exclude {be000cbe-11fe-4426-9c58-531aa6355fc4}

Writer Exclude {1072ae1c-e5a7-4ea1-9e4a-6f7964656570}

Writer Exclude {afbab4a2-367d-4d15-a586-71dbb18f8485}

Writer Exclude {4dc3bdd4-ab48-4d07-adb0-3bee2926fd7f}

Writer Exclude {542da469-d3e1-473c-9f4f-7847f01fc64f}

Writer Exclude {4969d978-be47-48b0-b100-f328f07ac1e0}

Writer Exclude {a6ad56c2-b509-4e6c-bb19-49d8f43532f0}

Writer Exclude {2a40fd15-dfca-4aa8-a654-1f8c654603f6}

Writer Exclude {7e47b561-971a-46e6-96b9-696eeaa53b2a}

Writer Exclude {41e12264-35d8-479b-8e5c-9b23d1dad37e}

Writer Exclude {59b1f0cf-90ef-465f-9609-6ca8b2938366}

# Exchange writer is required

Writer Verify {76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}

This is shown in the following figure. Please note that the Writer Verify command is not shown in this figure but it is executed though.

The next step is to select the disk we want to backup (disk F:\ contains the Mailbox database and the transaction log files), assign an alias to this disk and create the snapshot. You can do this to execute the following commands in the Diskshadow utility:

# Take the actual snapshot

begin backup

add volume F: alias VSS_Backup_F

create

This is shown in the following figure:

When the create command is executed steps 1 through 7 as outlined in the beginning of this article are executed. You can see this in the eventlog as multiple events are logged at this point:

At this point only the VSS snapshot is created. It is not visible yet and the transaction log files are not purged yet. To look at the actual snapshot the VSS snapshot needs to be exposed to the operating system. This can be achieved by entering the following command in Diskshadow:

expose %VSS_Backup_F% S:

End backup

The End Backup command will actual stop the VSS snapshot process and purge the transaction log files. When opening Windows Explorer drive S:\ will show up, containing the actual snapshot of the Mailbox database disk. The contents of this snapshot will be read-only of course.

When you check the eventlog you’ll that the transaction log files are purged. However, it can take some time before the transaction log files are actually deleted after the snapshot process was stopped:

When you open a command prompt and navigate to the VSS snapshot (Drive S:\) you can inspect the header of the Mailbox datababase using the following command:

ESEUTIL /MH MDB01.EDB

This is shown in the following figure:

When you look at the state of this Mailbox database you’ll see that it is in a Dirty Shutdown mode. To bring it in a consistent state only the transaction log file E00000001D4.log is needed. Typicall more log files are needed but the VSS process will flush all the data to the Mailbox database so only one transaction log file is needed. All other data is safely stored in the Mailbox database. To check the integrity of the Mailbox database you can use the following ESEUTIL command:

ESEUTIL /K MDB01.EDB

This is shown in the following figure:

To use this as an actual backup you have to copy the contents of drive S:\ (which is a combined view of the diff area and the non-changed blocks on disk) to a safe location and keep it there safely. Once done you have to unexpose drive S:\ from the operating system and remove the VSS snapshot using the following commands in Diskshadow:

unexpose S:

Delete Shadows All

Note. It can happen that a regular backup fails, the backup application stops but a VSS snapshot is left open on the Exchange server. It looks like you’re missing quite an amount of data from the actual disk. You can use the Diskshadow tool to get an overview of all available VSS snapshots (using the LIST SHADOWS ALL command) and if needed delete them (using the DELETE SHADOWS ALL) command. It’s best to do this after consulting with your backup vendor or Microsoft support.

Another important note is that when you try this in a production environment your regular backup application most likely will notice the creation of this backup and your regular backup schedule will be messed up!

The commands used in the previous section can be stored in a regular text file and saved to the local disk of the Exchange server (you can download it here). To use this textfile with diskshadow from the command line enter the following command:

diskshadow /s Diskshadow.txt /l c:\temp\diskshadow-logs.txt

This will perform all the steps as outlined above, with the exception of the unexpose of the snapshot and the deletion of the snapshot of course. These need to be executed later on when you’re finished. These commands however are in the textfile (but they are commented).

Hyper-V backup

You can also use VSS to create snapshots from Virtual Machines running on Hyper-V. The process is fairly identical. But when the VSS snapshot is created a signal is sent to the VSS service running inside the Virtual Machine (using the Integration Components) that a VSS snapshot is created. VSS inside the VM will perform the same steps as outlined in the beginning of this blog post and once frozen the actual snapshot of the VM is created. When the snapshot is created a signal is sent to VSS inside the VM to resume normal operation and purge the tranaction log files:

It is possible to prevent purging of the log files inside the Virtual Machine by setting the following registry key inside the Virtual Machine:

Location: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Virtualization\VssRequestor
Name: BackupType
Type: REG_DWORD
Value: 0 or 1

If this registry entry is created and its value is set to 1, application backup will not be affected by the virtual machine backup operation on the server that is running Hyper-V.

Summary

In this blogpost, following my presentation at MEC 2014, I tried to explain what’s happening during a VSS snapshot creation. You can do this (in a lab environment) to see for yourself what’s happening under the hood. There are (few) people who actually use this technique to create backups from their environment, but this is not something I would recommend to do.