Backup Image Format & Resilient File System

This page contains details about the interrelated topics

Backup Image Format,
Resilient File System (ReFS),
Sparse Files,

Backup Image Format

Lindenberg Software Backup uses different variants of the virtual hard disk (vhd) format. To be specific, Lindenberg Software Backup uses dynamic virtual hard disks for initial backups, and differential virtual hard disks for incremental backups, and also supports both .vhd and .vhdx via the Virtual Disk API. A big advantage of using a standard format is that you can use standard tools to verify backup operations or restore data. The use of .vhdx is usually preferable as Microsoft designed it to be more performant and more robust than .vhd, and it also supports 4KB sector sizes of both the virtual disk and the physical disk it is stored on. You may however use .vhd in case you want to access the backups on a system using Windows 7 or Windows Server 2008 which happen not to support the .vhdx format.

As Lindenberg Software Backup creates one new virtual hard disk file per backup, Backup will occasionally merge these to preserve a daily backup for the last 3 days, then weekly backups, and so on. Backups with errors are merged more aggressively as they may be inconsistent.

Resilient File System (ReFS)

In Windows Server 2016 and Windows 10 1703 Microsoft optimized virtual machine operations with ReFS by introducing new features:

Feature	Advantage	ReFS and Windows Versions
Block cloning on ReFS	Block clone accellerates the merge operations done regularly and also allows to clone a virtual disk (using Convert-VHD) in seconds rather than hours.	ReFS 3.1, Windows 10 1703, Window Server 2016
Sparse virtual disks	virtual disks consume significantly less disk space, allocating space in cluster units rather than the standard 0.5MB for .vhd or 2MB for .vhdx.	ReFS 3.2, Windows 10 1703, Windows Server Core 1709

Therefore the recommendation is to use Windows 10 or Windows Server Core 1709 with ReFS 3.2 or higher for highest performance and lowest disk space used.

Unfortunately, Microsoft removed the support to create volumes with ReFS in Windows 10 1709 (Fall Creators Update) in all editions except the "Enterprise" and "for Workstation" editions. However creating volumes is the only functionality removed, all Windows 10 editions are still capable to read and write ReFS volumes, and can also resize ReFS partitions. Thus you can follow one of the following two options to create a volume with ReFS under Windows:

use a version of Windows that supports to create ReFS volumes. This included Windows 10 Workstation or Enterprise, or any Windows Server version starting with 2012, including Hyper-V. All of these are available as evaluation versions. All of them can be installed as a physical or virtual system. When using a virtual system you can attach the initially blank disk you want to use with ReFS as a pass-through disk.
use Lindenberg Software Backup to restore a backup of a ReFS volume as follows:
1. download and unzip the virtual disk image with a ReFS partition to any location, e.g. your download folder,
2. setup a temporary Lindenberg Software Backup server with the image path pointing to the folder above (the folder you got unzipping should be a sub folder),
3. use Lindenberg Software Backup to restore the image to an empty hard disk,
4. open Disk Management (type Windows, then "diskmgmt.msc", go),
5. find the hard disk you restored to, find the partition, and resize the partition as appropriate, usually using all space.
6. Configure your Lindenberg Software Backup server to use the ReFS volume created.
Note: the disk and volume id is obviously not unique. This is no issue unless you connect two disks with the same ids to the same system. If you do, you will notice that the system takes the second offline until you resolve the conflict using Disk Management or Diskpart "online".

There is one caveat you have to be aware of: when upgrading to a new version of Windows, the new version may upgarde the ReFS version on your drive (e.g. Windows 10 1709 updates to ReFS 3.3). If you later decide to downgrade your Windows, you may not be able to read the disk on the downgraded system (e.g. Windows 10 1703 is not capable to read ReFS 3.3 but only up to ReFS 3.1).

Sparse Files

Sparse files are files that consume less space on disk because empty areas are not allocated. Windows supports sparse files with NTFS for a long time, however with exceptions. An important exception was that virtual disks cannot be sparse on NTFS or ReFS before 3.2. Lindenberg Software Backup uses sparse files for hash files, but starting with ReFS 3.2 also for virtual disks, which reduces the disk space needed especially for differecing disks used by incremental backups significantly. The caveat however is that standard file copy tools like windows explorer, copy, or robocopy do not know how to special case sparse files which causes them to be inflated to their nominal size. In order to avoid this you need to use special tools like http://www.flexhex.com/docs/articles/alternate-streams.phtml#cstool or https://tzworks.net/prototype_page.php?proto_id=9 when copying backup files from one disk to another one.