Backup and Restore Configurations

This page describes the configuration options available for backup and restore on TigerGraph and how to set them.

Prerequisites

You have access to the TigerGraph Linux user account on your cluster. All commands must be run from the TigerGraph Linux user.

Configuration parameters

The following is a list of configurations available for backup and restore.

Configuration parameter Description Default

Configuration parameter	Description	Default
System.Backup.Local.Enable	Whether to store the database backup data to a local path. If this parameter is set to true, `System.Backup.S3.Enable` cannot be set to true.	`false`
System.Backup.Local.Path	Local path to store the backup files. Required if backup is to be stored locally.	`null`
System.Backup.S3.AWSAccessKeyID	The access key for the cloud storage bucket. For AWS, the S3 access key ID; for GCS, the "Access Key". For Azure, the blob "Account Name".	`null`
System.Backup.S3.AWSSecretAccessKey	The {secret access key, secret, secret} for the {AWS S3, Azure Blob, GCS} bucket, respectively. NOTE: If setting this in interactive mode, store the key in a file and provide the path to the file, e.g., `@/tmp/test_secret`.	`null`
System.Backup.S3.RoleARN	The AWS role for accessing s3 buckets. S3 Role ARN takes priority over access keys. For more information, see AWS role ARN documentation. NOTE: This is only for AWS S3, and TigerGraph assumes the credentials for using `sts:AssumeRole` have been set up. You can verify the credentials are ready by running aws sts assume-role. One way to set up credentials is to configure access key id, secret access key and region with AWS CLI `aws configure`.	`nan`
System.Backup.S3.BucketName	The {bucket, container, bucket} name for {AWS, Azure, GCS}, respectively.	null
System.Backup.S3.Enable	Data is backed up to various S3-compatible storage services, including AWS S3, S3-compatible services such as Ceph S3, and specifically supported services like Google Cloud Storage (GCS) and Azure Blob Storage. The specific storage service used for the backup is determined by the `System.Backup.S3.Endpoint` configuration setting. If this parameter is set to true, `System.Backup.Local.Enable` cannot be set to true.	`false`
System.Backup.S3.Endpoint	A fully qualified URI, such as https://s3.amazonaws.com/, is used to specify the storage service’s endpoint for the client. It is used in the following situations: To backup to a private AWS S3, typically set it to "https://s3.amazonaws.com/" or any available URI. To backup to an S3-compatible service, set it to its corresponding service URI. To backup to Google Cloud Storage (GCS), set it to "https://storage.googleapis.com" To backup to Azure Blob, set it to "https://<your storage account name>.blob.core.windows.net"." Except for the above specific situations, leave it empty.	`null`
System.Backup.TimeoutSec	Timeout limit for the backup operation in seconds	`18000`
System.Backup.CompressProcessNumber	Number of concurrent processes for compression during backup. `0` means the number of processes used to compress is equal to the number of CPU cores on each node. We recommending keeping the default value `10`.	`10`
System.Backup.DecompressProcessNumber	The number of concurrent processes for decompression during the restore.	`8`
System.Backup.CompressionLevel	The backup compression level strikes a balance between size and speed. The better compression, the longer it takes. ("BestSpeed", "DefaultCompression", "BestCompression")	"DefaultCompression"

System.Backup.Local.Enable

Whether to store the database backup data to a local path. If this parameter is set to true, System.Backup.S3.Enable cannot be set to true.

false

System.Backup.Local.Path

Local path to store the backup files. Required if backup is to be stored locally.

null

System.Backup.S3.AWSAccessKeyID

The access key for the cloud storage bucket. For AWS, the S3 access key ID; for GCS, the "Access Key". For Azure, the blob "Account Name".

null

System.Backup.S3.AWSSecretAccessKey

The {secret access key, secret, secret} for the {AWS S3, Azure Blob, GCS} bucket, respectively.

NOTE: If setting this in interactive mode, store the key in a file and provide the path to the file, e.g., @/tmp/test_secret.

null

System.Backup.S3.RoleARN

The AWS role for accessing s3 buckets. S3 Role ARN takes priority over access keys. For more information, see AWS role ARN documentation.

NOTE: This is only for AWS S3, and TigerGraph assumes the credentials for using sts:AssumeRole have been set up. You can verify the credentials are ready by running aws sts assume-role. One way to set up credentials is to configure access key id, secret access key and region with AWS CLI aws configure.

nan

System.Backup.S3.BucketName

The {bucket, container, bucket} name for {AWS, Azure, GCS}, respectively.

null

System.Backup.S3.Enable

Data is backed up to various S3-compatible storage services, including AWS S3, S3-compatible services such as Ceph S3, and specifically supported services like Google Cloud Storage (GCS) and Azure Blob Storage. The specific storage service used for the backup is determined by the System.Backup.S3.Endpoint configuration setting.

If this parameter is set to true, System.Backup.Local.Enable cannot be set to true.

false

System.Backup.S3.Endpoint

A fully qualified URI, such as https://s3.amazonaws.com/, is used to specify the storage service’s endpoint for the client. It is used in the following situations:

To backup to a private AWS S3, typically set it to "https://s3.amazonaws.com/" or any available URI.
To backup to an S3-compatible service, set it to its corresponding service URI.
To backup to Google Cloud Storage (GCS), set it to "https://storage.googleapis.com"
To backup to Azure Blob, set it to "https://<your storage account name>.blob.core.windows.net"."

Except for the above specific situations, leave it empty.

null

System.Backup.TimeoutSec

Timeout limit for the backup operation in seconds

18000

System.Backup.CompressProcessNumber

Number of concurrent processes for compression during backup. 0 means the number of processes used to compress is equal to the number of CPU cores on each node.

We recommending keeping the default value 10.

10

System.Backup.DecompressProcessNumber

The number of concurrent processes for decompression during the restore.

8

System.Backup.CompressionLevel

The backup compression level strikes a balance between size and speed. The better compression, the longer it takes. ("BestSpeed", "DefaultCompression", "BestCompression")

"DefaultCompression"

Configure backup and restore

Running gadmin config entry backup allows you to enter the value for each parameter individually.

Alternatively, you can use gadmin config set <parameter> to change the value of any parameter.

After configuring the parameters, run gadmin config apply to apply the new parameter values.

Configure System.Backup.S3.Endpoint

Typically, there’s no need to configure the System.Backup.S3.Endpoint parameter on a TigerGraph Server. This is because the system auto-detects the regional endpoint for AWS S3 backups.

Users should configure this parameter only for special cases, such as:

To backup to a private AWS S3, typically set it to "https://s3.amazonaws.com/" or any available URI.
To backup to an S3-compatible service, set it to its corresponding service URI.
To backup to Google Cloud Storage (GCS), set it to "https://storage.googleapis.com"
To backup to Azure Blob, set it to "https://<your storage account name>.blob.core.windows.net"."

Except for the above specific situations, leave it empty.

Backup to AWS S3

To configure backup files to an AWS S3 Bucket for an on-premises TigerGraph Server cluster, complete the following steps:

Create an S3 bucket in AWS
Create an AWS IAM user

Create an IAM policy that ensures the IAM user has sufficient access to the bucket itself, and contents within the bucket

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "s3:PutObject",
                "s3:ListBucket",
                "s3:GetObject",
                "s3:GetBucketLocation"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::<bucket-name>",
                "arn:aws:s3:::<bucket-name>/*"
            ]
        }
    ]
}

Create an AccessKeyID and SecretAccessKey for the IAM user

TigerGraph clusters use long-lived credentials to authenticate to AWS as the IAM user, allowing TigerGraph access to put backup files into the S3 bucket. These credentials are also used to read and copy files during a Restore process.

Configure each of the following parameters on the linux command line:

Enable storing backup data in S3

gadmin config set "System.Backup.S3.Enable" "true"

Specify bucket name

gadmin config set "System.Backup.S3.BucketName" "<bucket-name>"

Set S3 backup AccessKeyID

gadmin config set "System.Backup.S3.AWSAccessKeyID" "<access-key-id>"

Set S3 backup SecretAccessKey

gadmin config set "System.Backup.S3.AWSSecretAccessKey" "<secret-access-key>"

Alternatively, instead of using AccessKeyID and SecretAccessKey, you may use AWS Role ARN for the authentication.

gadmin config set "System.Backup.S3.RoleARN" "arn:aws:iam::account:role/role-name-with-path"

Apply the new parameter values

gadmin config apply -y

Backup to Azure Blob

Similar to AWS S3, once the Azure Blob Container is created and configured properly, then configure it to be your backup storage via the following steps. You may find Microsoft’s Introduction to Azure Blob Storage to be helpful. users only need to complete the configurations by the following steps:

Enable storing backup data in S3

gadmin config set "System.Backup.S3.Enable" "true"

Specify the backup S3 Endpoint

gadmin config set "System.Backup.S3.Endpoint" "https://<account-name>.blob.core.windows.net"

Specify bucket name

gadmin config set "System.Backup.S3.BucketName" "<container-name>"

Set S3 backup AccessKeyID

gadmin config set "System.Backup.S3.AWSAccessKeyID" "<account-name>"

Set S3 backup SecretAccessKey

gadmin config set "System.Backup.S3.AWSSecretAccessKey" "<account-key>"

Apply the new parameter values
```
gadmin config apply -y
```