What is an in­cre­men­tal backup?

In­cre­men­tal backups allow for con­tin­u­ous storage of changing data sets. This minimizes the time, bandwidth, and storage re­quire­ments compared to repeated full backups. In­cre­men­tal backups begin with a full backup, followed by a chain of smaller in­cre­men­tal backups, but results in higher com­plex­i­ty of backup and restore processes. Special software is used to cope with this.

An in­cre­men­tal backup is a backup method that is often used as part of a wider backup strategy. Unlike a full backup, it does not create a complete copy of the dataset to be backed up. Instead, only changes made to files as of the last backup are saved.

Unlike a dif­fer­en­tial backup, the rec­on­cil­i­a­tion of changes does not nec­es­sar­i­ly refer to the last full backup. Rather, the changes since the last full or in­cre­men­tal backup are saved.

In general, there are two broad cat­e­gories of ap­proach­es to in­cre­men­tal data pro­tec­tion:

1. The gran­u­lar­i­ty of changes
2. The use of the strategy to be employed

Usually, hybrid ap­proach­es are used. For example, the popular tool Rsync creates “block-level synthetic full backups” in normal use. Let’s look at the different types of in­cre­men­tal backup in detail.

Digitally stored data consists of almost in­fi­nite­ly long chains of zeros and ones. These bits are grouped into logical units, of which files are the best known. Below file level, however, there are “blocks” and “bytes” as logical groupings of in­di­vid­ual bits.

Comparing changes between the last backup and the current state of the dataset can refer to different levels. The closer the matching of changes gets to the in­di­vid­ual bits, the more efficient the in­cre­men­tal backup becomes. While files vary in size, blocks and bytes have a defined, fixed size. For example, a byte comprises eight con­sec­u­tive bits; blocks are usually between 512 and 4,096 bytes long.

A file-level in­cre­men­tal backup only dis­tin­guish­es whether a file has been changed. The extent of the change is not taken into account. If even a single bit has been changed, the entire file is backed up again in the course of the in­cre­men­tal backup. To il­lus­trate this, imagine a large book man­u­script that is stored as a single file on a data storage device. If even a single letter is changed, the entire man­u­script is backed up again as part of a file-level in­cre­men­tal backup.

A file-level in­cre­men­tal backup is the easiest to implement because file systems log the mod­i­fi­ca­tion date of files. To reconcile the changes, it is suf­fi­cient to compare the time­stamps of the last mod­i­fi­ca­tion of a file on the source and target systems. If the timestamp on the source system is newer, the file has been modified and must be backed up again. Thus, file-level in­cre­men­tal backups are in­ef­fi­cient for small changes to large files.

The term “block” orig­i­nates from data storage. Hard disks and other mass storage devices organize sections of con­sec­u­tive bytes as a con­tigu­ous logical area. When creating the block-level in­cre­men­tal backup, only the changed blocks are backed up. The advantage is the smaller size of the data to be trans­ferred and stored. Thus, only the changed sections of in­di­vid­ual files are backed up.

The main dis­ad­van­tage is that a mechanism is needed to log which blocks have changed. Some storage tech­nolo­gies are able to manage this in­for­ma­tion and thus can implement block-level in­cre­men­tal backups. Otherwise, the full backup at block level would need to be compared with the current state to detect changes.

A byte-level in­cre­men­tal backup is an even more detailed version than a block-level in­cre­men­tal backup. Only the changed bytes of a block are backed up. In terms of the time required and the storage space needed, the byte-level in­cre­men­tal backup may be sig­nif­i­cant­ly more efficient. However, the process is more demanding. This is because a journal of the byte-level changes is required to identify the changed bytes. The ad­di­tion­al effort for byte-level in­cre­men­tal backups is useful for selective changes to very large files.

In­cre­men­tal backups are part of a chain of in­di­vid­ual backups that contain changes to the previous state. At the beginning of the chain there is always a full backup. There are various methods of con­struct­ing the chain and creating new full backups. This has ad­van­tages in terms of com­plex­i­ty and duration of backup and restore processes. Here are some strate­gies for creating in­cre­men­tal backups.

A tra­di­tion­al in­cre­men­tal backup strategy requires periodic full backups. Creating a full backup is time-consuming and slow. With synthetic full backups, a new full backup is created. The original full backup and the sub­se­quent in­cre­men­tal backups are used. This saves the copying of the entire dataset from the source system that would otherwise be required when creating a full backup. The process is also known as “forward in­cre­men­tal backup”.

The in­cre­men­tal forever backup is optimized for backing up hard disks and similar media. Initially, a full backup is created; af­ter­wards only the changed blocks are stored in a “forever” con­tin­u­ous chain. Unlike a synthetic full or a reverse in­cre­men­tal backup, no further full backups are created after the initial one. This saves storage space and minimizes the data trans­ferred during the copy op­er­a­tions.

The reverse in­cre­men­tal backup works similar to the synthetic full backup. After each in­cre­men­tal backup, changes are merged with the last full backup. This means a full backup exists by the end of the backup chain. It is true that creating a reverse in­cre­men­tal backup takes longer than a forward in­cre­men­tal backup. However, if necessary, the source system can be restored without delay, since the last current status is already available as a complete copy.

An enhanced in­cre­men­tal backup can be of any type. A special feature is that it also detects changes to files caused by moving or renaming them. Provided large files are involved, this boosts ef­fi­cien­cy.

The biggest advantage of in­cre­men­tal backups is that in­di­vid­ual backups are small. Since only the dif­fer­ences as of the last backup are saved, the backup process requires rel­a­tive­ly little time, bandwidth, and storage space. The pre­req­ui­site is that the period between the in­di­vid­ual backups is kept short. Otherwise, large amounts of changed data ac­cu­mu­late and have to be trans­ferred.

The inherent advantage of in­cre­men­tal backups is well il­lus­trat­ed by the term “backup window”. This is the time period within which a backup can be created without dis­rupt­ing op­er­a­tions. If the amount of data is so large that the copying process would take longer than the backup window allows, it becomes im­pos­si­ble to create a full backup during operation. In that case it is advisable to create small in­cre­men­tal backups at short intervals.

The immediate dis­ad­van­tage of in­cre­men­tal backups is that spe­cial­ized software is usually necessary. Planning the backup strategy also requires more effort than creating full backups. Restoring the original state from the backups is similar. Because data is spread across several backups, the process is more complex. There is also a higher risk of data loss. If one in­cre­men­tal backup in a chain is damaged, the integrity of all sub­se­quent backups is affected.

In­cre­men­tal backups are primarily aimed at min­i­miz­ing the storage re­quire­ments and time needed to create a backup. They are therefore always used when the repeated creation of complete backups would be lo­gis­ti­cal­ly non­sen­si­cal. Let’s look at a few scenarios in detail.

To create a backup on Mac, it is best to use the macOS built-in backup tool “Time Machine”. The tool backs up changes to the internal data storage in­cre­men­tal­ly to an external hard drive. As usual with in­cre­men­tal backups, an initial full backup is created during the first run. Sub­se­quent changes to the file system are logged and are available for further backups.

Time Machine allows you to restore in­di­vid­ual files to earlier points in time. Fur­ther­more, the complete system can be re­con­struct­ed from the backup. This is practical in case of faulty hardware or if you want to migrate your own system to new hardware. Its simple handling of the backup software is par­tic­u­lar­ly im­pres­sive. The user only has to connect an external hard disk and launch the backup process; the rest is automated.

In­cre­men­tal backups are also available on Windows. On the one hand, a backup can be created in Windows 10 with the help of Windows Backup. On the other hand, you can use the Robocopy backup tool in the command line to in­cre­men­tal­ly backup the contents of a directory. Let’s look at an example:

The /MIR option here stands for “mirror”. The command mirrors the source directory to the des­ti­na­tion path. If a directory with the same name already exists, an in­cre­men­tal backup is performed. Robocopy then transfers only the changes as of the last backup operation.

Robocopy exists only under Windows. To create a server backup with Rsync under Linux, an in­cre­men­tal backup is used. First, a full backup is created. When the command is next called up, Rsync transfers the blockwise changes only as of the last backup. The trans­ferred data is merged with the existing data set. Thus, the result of the backup operation is a synthetic full backup. Here is an example of a cor­re­spond­ing Rsync call: