CREATE DATA_SOURCEstatement defines a data_source variable with a subtype of KAFKA:
SETcommand to specify the path to a configuration file for that data source.
"kafka_config"key. For its value, provide a list of key-value pairs. For example:
WRITE_DATASOURCEprivilege on the global scope may create a global level data source without assigning it to a particular graph:
WRITE_DATASOURCEprivilege on the global scope may grant/revoke a data source to/from one or more graphs:
WRITE_DATASOURCEprivilege for a particular graph user may create a local data source for that graph:
WRITE_DATASOURCEprivilege. Users with
WRITE_DATASOURCEprivilege for one graph can drop data sources on that graph. The syntax for the
DROP DATA_SOURCEcommand is as follows:
SHOW DATA_SOURCEcommand will display a summary of all existing data_sources for which the user has privilege:
DEFINE FILENAMEstatement should be used to assign a loader
FILENAMEvariable to a Kafka data source name and the path to its config file.
RUN LOADING JOBstatement with the
USINGclause. The filename value set by a
RUNstatement overrides the value set in the
CREATE LOADING JOB.
DEFINE FILENAMEfor use with the Kakfa Loader. In the syntax,
$DATA_SOURCE_NAMEis the Kafka data source name, and the path points to a configuration file with topic and partition information of the Kafka server. The Kafka configuration file must be in JSON format.
"topic"key is required. Optionally, a
"partition_list"array can be included to specify which topic partitions to read and what start offsets to use. If the
"partition_list"key is missing or empty, all partitions in this topic will be used for loading. The default offset for loading is
"-1", which means you will load data from the most recent message in the topic, i.e., the end of the topic. If you want to load from the beginning of a topic, the "
start_offset"value should be "-2".
"default_start_offset"in the Kafka topic configuration file. For example,
RUN LOADING JOBsyntax:
SHOW LOADING STATUS
ABORT LOADING JOB
RESUME LOADING JOB
SHOW LOADING STATUScommand is as follows:
RUN LOADING JOBis executed. For each loading job, the above command reports the following information :