Differential Backup

Terminology Definition

Full backups comprise entire data backup sets, regardless of already existing backups or data change circumstances.

  • Differential backups backup Kafka messages that have changed since the most recently completed or most recently restored full backup.

For example, a full backup happened at tid:100, a schema change happened at tid:150 and a differential backup happened at tid:200.

Additionally, for example, if you have full backup A and full backup B, full backup A happened before full backup B. If you restore full backup A, then the next differential backup is based on full backup A, instead of full backup B, although full backup B was completed more recently.

How Differential Backup works

To illustrate how differential backup works, take into account the user scenario below.

Example 1. User Scenario

  1. A user creates a full backup every Sunday (Or Day N).

  2. During the week, between the two full backup days (Sunday and Next Sunday) TigerGraph provides differential backups for every change since that first full backup.

  3. A differential backup is taken until the backup data is as large as the backup on Sunday

  4. Another full backup is taken on Next Sunday and the circle repeats steps 2-4.

Enable Differential Backup

If a user wants to make differential backups based on a full backup they have made, they should run: gadmin backup create <tag> with the --incremental flag.

Ex.
gadmin backup create <tag> --incremental

For more details on gadmin backup create see Back up a Database Cluster Procedure.

To restore a differential backup, users need to find the correct <tag> from the backup list. To see the list of backups, run:

Ex.
gadmin backup list
backup list
Figure 1. gadmin backup list example result.

This will list all the backup tags. Notice the type column in the result. This is how users can identify which backups are FULL or not. To restore the specific backup, run:

Ex.
gadmin backup restore <specifc backup tag>

Disable Differential Backup

If a user wants to make a full backup WITHOUT differential backup enabled, they can do so by excluding the --incremental flag in gadmin backup create.

Ex.
gadmin backup create <tag>

Limitations

Differential backup does have some know limitations:

  • Kafka reset or Kafka purging of large amounts of Kafka message data is not yet supported by differential backups.

  • Differential Backup must based on last full snapshot backup and cannot be based on a previous days differential backup.

    • For example: If a user did a full backup on Sunday and then one differential backup on Monday. Then, if the user wants to run a differential backup on Tuesday, they will have to base it on Sunday’s full backup and save all changes since then. So part of your Tuesday differential backup is actually the same as Monday’s.