Skip to content

Module llm

get_azure_llm(engine='gpt-4o')

Get the Azure OpenAI language model.

Parameters:

Name Type Description Default
engine str

The engine to use for the Azure OpenAI model. Default is "gpt-4o".

 'gpt-4o'

Returns:

Name Type Description
AzureOpenAI AzureOpenAI

An instance of the AzureOpenAI language model.
Source code in nbs/llm.py 
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
def get_azure_llm(engine="gpt-4o") -> AzureOpenAI:
    """
    Get the Azure OpenAI language model.

    Args:
        engine (str): The engine to use for the Azure OpenAI model. Default is "gpt-4o".

    Returns:
        AzureOpenAI: An instance of the AzureOpenAI language model.
    """
    AZURE_API_KEY, AZURE_ENDPOINT, AZURE_API_VERSION = get_azure_openai_keys()
    llm = AzureOpenAI(
        engine=engine,
        api_key=AZURE_API_KEY,
        azure_endpoint=AZURE_ENDPOINT,
        api_version=AZURE_API_VERSION,
        # increase timeout to 300 seconds (5 minutes) to avoid timeout on long transcriptions
        timeout=300.0,
    )
    Settings.llm = llm
    return llm

get_gemini_llm(model='gemini-1.5-flash')

Get the Gemini language model.

This function configures the Gemini API using the API key obtained from the environment variables and returns an instance of the GenerativeModel.

Parameters:

Name Type Description Default
model str

The model to use for the Gemini language model. Default is "gemini-1.5-flash".

 'gemini-1.5-flash'

Returns:

Name Type Description
GenerativeModel GenerativeModel

An instance of the GenerativeModel.
Source code in nbs/llm.py 
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
def get_gemini_llm(model="gemini-1.5-flash") -> genai.GenerativeModel:
    """
    Get the Gemini language model.

    This function configures the Gemini API using the API key obtained from the environment variables
    and returns an instance of the GenerativeModel.

    Args:
        model (str): The model to use for the Gemini language model. Default is "gemini-1.5-flash".

    Returns:
        GenerativeModel: An instance of the GenerativeModel.
    """
    genai.configure(api_key=get_gemini_api_key())
    llm = genai.GenerativeModel(model)
    return llm

get_vertex_llm(model='gemini-1.5-flash-001')

Get the Vertex language model.

This function configures the Vertex API using the project ID and API key obtained from the environment variables and returns an instance of the Vertex model.

Parameters:

Name Type Description Default
model str

The model to use for the Vertex language model. Default is "gemini-1.5-flash-001".

 'gemini-1.5-flash-001'

Returns:

Name Type Description
Vertex Vertex

An instance of the Vertex language model.
Source code in nbs/llm.py 
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
def get_vertex_llm(model="gemini-1.5-flash-001") -> Vertex:
    """
    Get the Vertex language model.

    This function configures the Vertex API using the project ID and API key obtained from the environment variables
    and returns an instance of the Vertex model.

    Args:
        model (str): The model to use for the Vertex language model. Default is "gemini-1.5-flash-001".

    Returns:
        Vertex: An instance of the Vertex language model.
    """
    # Set up necessary variables
    credentials = {
        "project_id": get_google_projectID(),
        "api_key": get_gemini_api_key(),
    }

    # filename = get_google_auth_file()
    # credentials_service: service_account.Credentials = (
    #     service_account.Credentials.from_service_account_file(filename)
    # )

    # Create an instance of the Vertex class
    llm = Vertex(
        model=model,
        project=credentials["project_id"],
        credentials=credentials,
        context_window=100000,
    )
    # llm = Vertex(
    #     model=model,
    #     project=credentials_service.project_id,
    #     credentials=credentials_service,
    #     context_window=100000,    )
    Settings.llm = llm
    return llm