Compute-on-demand Elastic Cluster

TigerGraph allows you to spin up an Elastic Read-only (ER) cluster from your primary cluster to handle compute-intensive online analytical processing (OLAP) queries on demand. Operations on the ER cluster do not interfere with the primary cluster in any way.

This feature is currently in the preview stage only available for enterprise TigerGraph Cloud customers. If you are a paid TigerGraph Cloud enterprise customer and want to set up an ER cluster for your environment, please open a support ticket.

The elasticity afforded by on-demand ER clusters can provide the following benefits:

  • Better availability. Diverting memory-intensive and time-consuming OLAP queries to an isolated ER cluster allows your primary cluster to have better availability for other workloads.

  • Better cost management. Provisioning dedicated machines to process OLAP can prove quite expensive. When the queries are not running, the excess hardware sits idle, which is a waste of resources. The elasticity of ER clusters make sure that you are only paying for computing power you need when you need it.

Architecture

The ER cluster shares storage with the primary cluster. When the ER cluster is set up, it reads from the shared storage to retrieve all data from the primary cluster. After setup, it no longer receives updates from the primary cluster.

Diagram of an ER cluster. A primary production cluster has a two-way arrow connecting it to a shared storage area. The shared storage area has one-way arrows leading from itself to two on-demand read-only clusters for OLAP.
Figure 1. Architectural diagram for ER cluster

The ER clusters are able to process memory-intensive OLAP queries in an isolated environment with no impact to the primary cluster. After the queries finish, the ER cluster should be shut down. The next time there is a demand for a memory-intensive OLAP query, you can set up a new ER cluster to meet the demand.

Use cases

If your workload involves online analytical processing (OLAP) queries that access large amounts of data and are very time-consuming, you can benefit from using ER clusters for your OLAP queries.

ER cluster for scaling up

For example, suppose you have an application powered by a 6-node TigerGraph online transactional processing (OLTP) cluster with a total memory of 192 GB, and the regular workload of the cluster uses 120 GB of memory. However, for your quarterly business review, you need to run an OLAP query that will require the processing machines to have 300 GB of memory.

In this case, you cannot have your primary cluster process this workload, since it does not have enough memory, and you also do not want your regular workload to be affected. Upgrading your primary cluster would be expensive as it would add to your regular operating expenses. At the same time, the excess computing power would not be utilized efficiently since your regular workload does not require so much memory.

Instead, you can provision an ER cluster that meets the memory requirement and run the OLAP query on the ER cluster. After the query has finished processing, you shut down the cluster. This allows you to process your OLAP queries as well as save costs.

ER cluster for compute isolation

Even if your existing hardware is able to handle the OLAP workload, ER can still benefit your workload by preserving your cluster’s regular functions without disruption.

For example, suppose you have an application powered by a 6-node TigerGraph OLTP cluster with a total memory of 192 GB, and the regular workload of the cluster uses 120 GB of memory. Every week, you also want to run an OLAP query to analyze user activities for the week that will use up 150 GB of memory.

In this case, even though your existing hardware is able to handle the OLAP workload, running the query will still disrupt your regular workload.

Instead, you can provision an ER cluster that meets the memory requirement and run the OLAP query on the ER cluster. After the query has finished processing, you shut down the cluster. This allows you to process your OLAP queries as well as avoid disruptions to your regular workload.