Manage Data Sources

Loading jobs which using the Kafka Loader (from cloud storage, data warehouses, or external Kafka) make use of data sources, objects that you define and which become part of a graph’s catalog. Data sources are defined with a configuration file, which specify the configurations to apply to the data source.

This page describes the permissions required for different operations concerning data sources, as well as how to delete a data source.

Data source operation permissions

A data source can be either global or local:

  • A global data source can only be created by a user with WRITE_DATASOURCE privilege on the global scope, who can grant it to any graph.

  • A local data source belongs to a graph and cannot be accessed by other graphs.

The following are examples of permitted DATA_SOURCE operations.

  • Users with the WRITE_DATASOURCE privilege on the global scope may create a global level data source without assigning it to a particular graph:

GSQL > CREATE DATA_SOURCE k1 = "/path/to/config"

  • Users with the WRITE_DATASOURCE privilege on the global scope may grant/revoke a data source to/from one or more graphs:

GSQL > GRANT DATA_SOURCE k1 TO GRAPH graph1, graph2
GSQL > REVOKE DATA_SOURCE k1 FROM GRAPH graph1, graph2

  • Users with the WRITE_DATASOURCE privilege for a particular graph user may create a local data source for that graph:

GSQL > CREATE DATA_SOURCE k1 = "/path/to/config" FOR GRAPH test_graph

In the above statement, the local data_source k1 is only accessible to graph test_graph. A superuser cannot grant it to another graph.

Delete a data source

A data source variable can be dropped by a user who has sufficient privileges. A global data source can only be dropped by a users with global WRITE_DATASOURCE privilege. Users with WRITE_DATASOURCE privilege for one graph can drop data sources on that graph. The syntax for the DROP DATA_SOURCE command is as follows:

GSQL > DROP DATA_SOURCE <source1>[<source2>...] | * | ALL

Below are several examples of Kafka data source CREATE and DROP commands.

GSQL > CREATE DATA_SOURCE k1 = "/home/tigergraph/kafka.conf"
GSQL > CREATE DATA_SOURCE k2 = "/home/tigergraph/kafka2.conf"

GSQL > DROP DATA_SOURCE k1, k2
GSQL > DROP DATA_SOURCE *
GSQL > DROP DATA_SOURCE ALL

SHOW DATA_SOURCE

The SHOW DATA_SOURCE command will display a summary of all existing data sources for which the user has privilege:

GSQL > SHOW DATA_SOURCE *

// The sample output
Data Source:
  - KAFKA k1 ("127.0.0.1:9092")
The global data source will be shown in global scope. The graph scope will only show the data source it has access to.