By default, only one loading job may run at a time. Additional job requests are held in a wait queue. You can increase the number of concurrent active jobs by changing the following configurations:
For local file loading:
$ gadmin config set FileLoader.ReplicaNumber <new_limit> $ gadmin config apply -y && gadmin restart gse gpe restpp -y
For Kafka Loader loading:
$ gadmin config set KafkaLoader.ReplicaNumber <new_limit> $ gadmin config apply -y && gadmin restart gse gpe restpp -y
You can also configure how many resources are permitted to work on loading jobs.
Tasks per CPU (for KafkaConnect)
This parameter sets the average CPU workload per server, in a distributed cluster. The allowed range is [0.5, 10], but we recommend going no higher than 2. The default value is 1.5. In a 4-server cluster, that means the maximum number of Kafka connector tasks is
0.5 * 4 = 2.
If you get an error message saying that the Connector has hit the upper limit on the number of tasks, you can slightly increase the limit to allow more connector tasks.
$ gadmin config set KafkaConnect.AllowedTaskPerCPU <new_limit> $ gadmin config apply -y && gadmin restart kafkaconn -y