You can repartition a cluster to meet the changing needs of your workload. Repartitioning a cluster changes the replication factor and the partitioning factor of your cluster without adding or removing nodes from the cluster.
You can also change the replication factor when you expand or shrink a cluster. See Expand a Cluster and Shrink a Cluster.
No loading jobs, queries, or REST requests are running on the cluster.
Obtain a few key measures for the state of your data before the repartition, such as vertex counts/edge counts or certain query results. This will be useful in verifying data integrity after the repartition completes.
To repartition a cluster, calculate the new replication factor for your cluster. You can calculate the replication factor by dividing the total number of nodes in the cluster by the number of partitions you'd like your cluster to have.
To repartition a cluster, use the gadmin cluster expand
command like below. Use the --ha
option to indicate the new replication factor of the cluster. For example, the command below will change the replication factor to 2:
The partitioning factor of your cluster will change automatically based on your specified replication factor. Its updated value will be the total number of nodes divided by the replication factor.
If the total number of nodes cannot be fully divided by the replication factor, the remainder nodes will be left idle. For example, if you currently have a 5-node cluster with a replication factor of 1 and a partitioning factor of 5. Changing the replication factor to 2 without adding new nodes will change the distribution of your cluster to be 2*2, with one node being left idle. To avoid nodes being left idle, ensure that you pick a replication factor that can fully divide the total number of nodes you have in the cluster.
Extra disk space is required during cluster repartition. If more space is not available on the same disk, you can supply a staging location on a different disk to hold temporary data:
If you choose to supply a staging location, make sure that the TigerGraph Linux user has write permission to the path you provide. The overall amount of space required for cluster repartition on each node is (1 + Math.ceil(oldPartition/newPartition) ) * dataRootSize
. oldPartition
and newPartition
each stand for the partitioning factor of the cluster before and after repartition; dataRootSize
stands for the size of the data root folder on the node.
For example, if you are repartitioning from a 6-node cluster with a replication factor of 2 and a partitioning factor of 3, to a 6-node cluster with a replication factor of 3 and a partitioning factor of 2, and the size of the data root folder on a node is 50GB. Then you would need more than (1 + Math.ceil(3/2)) * 50) = 150 GB
of free space on the staging path.
When the repartition completes, you should see a message confirming the completion of the cluster change. The message will also include the location of the temporary files created during the repartition.
Verify data integrity by comparing vertex/edge counts or query results. After confirming a successful repartition, delete the temporary files to free up disk space.
As your workload changes, it makes sense to resize your cluster to meet your production needs. The section describes the steps to perform the following cluster resizing operations:
Shrinking a cluster can make sense for a few reasons:
Your workload has changed and you can operate a cluster with fewer resources.
Your data volume is lower than projected.
Shrinking a cluster removes nodes from the cluster. The data stored on those nodes will be redistributed to the remaining nodes.
No loading jobs, queries, or REST requests are running on the new node or the cluster.
Obtain a few key measures for the state of your data before shrinking, such as vertex counts/edge counts or certain query results. This will be useful in verifying data integrity after shrinking completes.
Before running any commands to shrink a cluster, make sure you have a clear idea of how the new cluster should be distributed. You should have the following information:
The new replication factor of the cluster
The new partitioning factor of the cluster
The names and IP addresses of the nodes to be removed from the cluster
To shrink the cluster, run the gadmin cluster shrink
command like below. If the shrinking involves changing the replication factor, use the --ha <replicattion_factor>
option to indicate the new replication factor:
node_ip_list
is the list of nodes you are removing from the cluster mapped to their IP addresses with a colon(:
), and separated by a comma. Below is an example:
Extra disk space is required during cluster shrinking. If more space is not available on the same disk, you can supply a staging location on a different disk to hold temporary data:
If you choose to supply a staging location, make sure that the TigerGraph Linux user has write permission to the path you provide. The overall amount of space required for cluster shrinking on each node is (1 + Math.ceil(oldPartition/newPartition) ) * dataRootSize
. oldPartition
and newPartition
each stand for the partitioning factor of the cluster before and after shrinking; dataRootSize
stands for the size of the data root folder on the node.
For example, if you are shrinking from a 10-node cluster with a replication factor of 2 and a partitioning factor of 5, to a 6-node cluster with a replication factor of 2 and a partitioning factor of 3, and the size of the data root folder on a node is 50GB. Then you would need more than (1 + Math.ceil(5/3)) * 50) = 150 GB
of free space on the staging path.
When shrinking completes, you should see a message confirming the completion of the cluster change. The message will also include the location of the temporary files created during the operation.
Verify data integrity by comparing vertex/edge counts or query results to what they were before the shrinking. After confirming a successful cluster contraction, delete the temporary files to free up disk space.
Uninstall TigerGraph from the removed nodes by removing all root directories (app
, data
and tmp
) except the log directory of TigerGraph.
For security reasons, guninstall
is disabled on the removed nodes. Therefore, you need to delete TigerGraph by manually deleting the root directories.
As your workload changes, you can expand your cluster to improve its query performance, system availability, and fault tolerance. Expanding a cluster adds more nodes to the cluster. During an expansion, you can also change the replication factor of your cluster.
TigerGraph must already be installed on the new nodes in exactly the same version as the cluster.
No loading jobs, queries, or REST requests are running on the new node or the cluster.
Obtain a few key measures for the state of your data before the expansion, such as vertex counts/edge counts or certain query results. This will be useful in verifying data integrity after the expansion completes.
If the original cluster is a single node installation, make sure the IP used is not a local loopback address such as 127.0.0.1.
Before running any commands to expand a cluster, make sure you have a clear idea of how the new cluster should be distributed. You should have the following information:
The new replication factor of the cluster
The new partitioning factor of the cluster
The IP addresses of the new nodes to be added to the cluster
To expand the cluster, run the gadmin cluster expand
command like below. If the expansion involves changing the replication factor, use the --ha
option to indicate the new replication factor:
node_ip_list
is the machine aliases of the nodes you are adding to the cluster mapped to their IP addresses with a colon(:
), and separated by a comma. Below is an example:
We suggest naming the new nodes following the convention of m<serial>
, such as m1
, m2
, and m3
for a 3-node cluster. If you are adding a fourth node, then the fourth node would be named m4
. If you decide to name them differently, make sure that all names are unique within the cluster.
Extra disk space is required during cluster expansion. If more space is not available on the same disk, you can supply a staging location on a different disk to hold temporary data:
If you choose to supply a staging location, make sure that the TigerGraph Linux user has write permission to the path you provide. The overall amount of space required for expansion on each node is (1 + Math.ceil(oldPartition/newPartition) ) * dataRootSize
. oldPartition
and newPartition
each stand for the partitioning factor of the cluster before and after expansion; dataRootSize
stands for the size of the data root folder on the node.
For example, if you are expanding from a 6 node cluster with a replication factor of 2 and a partitioning factor of 3, to a 10-node cluster with a replication factor of 2 and a partitioning factor of 5, and the size of the data root folder on a node is 50GB. Then you would need more than (1 + Math.ceil(3/5)) * 50) = 100 GB
of free space on the staging path.
When the expansion completes, you should see a message confirming the completion of the cluster change. The message will also include the location of the temporary files created during the expansion.
Verify data integrity by comparing vertex/edge counts or query results. After confirming a successful expansion, delete the temporary files to free up disk space.