Accessing AI Verde¶

• AI Verde is a massive LLM distribution platform details of which can be found here.

Step 0: Get access to AI Verde¶

• If you are new to AI Verde drop an email to mithunpaul@arizona.edu detailing
• who you are (e.g. student, faculty, researcher)
• what kind of facilities of AI Verde would you like to use (e.g. Chatbot access, API access to 75+ LLM models, RAG etc)

Once an admin approves your access, you will minimally have access to a chatbot. If you asked for more programming level access, you will be provided with a unique api-token. Here are details on how to access these keys. Once you have the keys here are the further steps on how to access the LLM models from a python code interface.

Step 1: Export the API key as an environment variable¶

export LLM_URL="https://llm-api.cyverse.ai"
export LLM_API_KEY="paste your key here"

Step 2: Set up a conda environment¶

For example Open a linux command line window from the tools in de.cyverse and type

bash conda create --name verde python

```bash conda activate verde

## Step 3: Install chatur chains

`pip install chatur-chains`

## Step 4: Write the Python code


```python
import os
from pathlib import Path

Import the API keys form the environment which you exported earlier

llm_url = os.environ.get('LLM_URL')
api_key = os.environ.get('LLM_API_KEY')

Next use this command to import the build_llm_proxy function from chatur chains.

from chains.llm_proxy import build_llm_proxy

Now use the code below to initiate a connection to your favorite llm.

llm = build_llm_proxy(
    model="anvilgpt/llama3.1:latest",
    url=llm_url,
    engine="OpenAI",
    temperature=0.9,
    api_key=api_key,
)

Note: The list of LLMs (for example anvilgpt/llama3.1:latest in the above code) you can access is dependant per user permissions. To find out what models you have access to you can use the below curl command, where LLM_API_KEY is the same as given to you by the AI Verde admin.

curl -s -L "https://llm-api.cyverse.ai/v1/models" -H "Authorization: Bearer $LLM_API_KEY" -H 'Content-Type: application/json'

Next you can start asking question to this LLM as shown below

message = llm.invoke("what is the largest wildfire in arizona")

print(message.content)

Step 5: Run the code¶

Next save this editor file, as verde.py and run the python script from command line like this:

python verde.py

Adapted from CyVerse Learning Materials (last source update 2025-03-14), CC BY 4.0. Spotted a problem? Open an issue.