Adding a New LLM Generation Service

Create a new file in the app/llm_services directory. The file should be named service_name.py where service_name is the name of your service.

Define your service. The service should be a valid Python class that inherits from the LLM_Model class defined in the app/llm_services/base_llm.py file.

Add your service to the app/llm_services/init.py file. This file should contain an import statement for your service. For example: from .service_name import ServiceName

Import and instantiate your service in the app/main.py file. For example:

from app.llm_services import ServiceName

# Within the instantiation of the Agent class elif block
elif llm_config["completion_service"]["llm_service"].lower() == "my_service":
    logger.debug(f"/{graphname}/query request_id={req_id_cv.get()} llm_service=my_service agent created")
    agent = TigerGraphAgent(AzureOpenAI(llm_config["completion_service"]), conn, embedding_service, embedding_store)

Test your service. Run the service and test your service to ensure that it works as expected.