How to Build an AI Assistant with OpenAI + Python | by Shaw Talebi | Feb, 2024

Sooner than diving into the occasion code, I want to briefly differentiate an AI chatbot from an assistant. Whereas these phrases are generally used interchangeably, proper right here, I make the most of them to indicate varied issues.

A chatbot is an AI you’ll have a dialog with, whereas an AI assistant is a chatbot which will use devices. A software program could also be points like web looking out, a calculator, a Python interpreter, or the remaining that expands the capabilities of a chatbot [1].

For example, must you use the free mannequin of ChatGPT, that’s a chatbot on account of it solely comes with a major chat efficiency. Nonetheless, must you use the premium mannequin of ChatGPT, that’s an assistant on account of it comes with capabilities akin to web looking out, info retrieval, and film period.

Whereas setting up AI assistants (i.e., AI brokers) simply isn’t a model new idea, OpenAI’s new Assistants API provides a easy choice to create these sorts of AIs. Proper right here, I’ll use the API to make a YouTube comment responder outfitted with info retrieval (i.e. RAG) from one among my Medium articles. The subsequent occasion code is obtainable at this publish’s GitHub repository.

Vanilla Assistant

We start by importing Python libraries and organising communication with the OpenAI API.

from openai import OpenAI
from sk import my_sk # import secret key from .py file
client = OpenAI(api_key=my_sk)

Bear in mind that for this step, you need an OpenAI API key. If you don’t have an API key or don’t understand how to get one, I stroll through how to do that in a previous article. Proper right here, I’ve my secret key outlined in a separate Python file generally known as sk.py, which was imported throughout the above code block.

Now we’re in a position to create a major assistant (technically a chatbot since no devices however). This can be carried out in a single line of code, nonetheless I make the most of numerous further for readability.

intstructions_string = "ShawGPT, functioning as a digital info science 
advertising advisor on YouTube, communicates in clear, accessible language, escalating 
to technical depth upon request. 
It reacts to strategies aptly and concludes with its signature '–ShawGPT'. 
ShawGPT will tailor the scale of its responses to match the viewer's comment, 
providing concise acknowledgments to transient expressions of gratitude or 
strategies, thus defending the interaction pure and interesting."
assistant = client.beta.assistants.create(
determine="ShawGPT",
description="Information scientist GPT for YouTube suggestions",
instructions=intstructions_string,
model="gpt-4-0125-preview"
)

As confirmed above, we’re in a position to set the assistant determine, description, instructions, and model. The inputs most associated to the assistant’s effectivity are the instructions and model. Rising good instructions (i.e. prompt engineering) is an iterative course of nonetheless value spending some time on. Furthermore, I make the most of the most recent obtainable mannequin of GPT-4. Nonetheless, older (and cheaper) fashions are moreover available [2].

With the “assistant” prepare, we’re in a position to ship it a message to generate a response. That’s carried out throughout the code block beneath.

# create thread (i.e. object that handles dialog between particular person and assistant)
thread = client.beta.threads.create()
# add an individual message to the thread
message = client.beta.threads.messages.create(
thread_id=thread.id,
place="particular person",
content material materials="Good content material materials, thanks!"
)
# ship message to assistant to generate a response
run = client.beta.threads.runs.create(
thread_id=thread.id,
assistant_id=assistant.id,
)

A variety of points are happening throughout the above code block. First, we create a thread object. This handles message passing between particular person and assistant, thus avoiding the need for us to jot down boilerplate code to do that. Subsequent, we add an individual message to the thread. These are the YouTube suggestions for our use case. Then, lastly, we ship the thread to the assistant to generate a response by means of the run object.

After numerous seconds, we get the subsequent response from the assistant:

You might be welcome! I'm glad you found it helpful. If you've obtained any further questions 
or issues you might be interested by, be at liberty to ask. –ShawGPT

Whereas this may increasingly seem like a nice response, it’s not one factor I’d ever say. Let’s see how we’re in a position to improve the assistant by means of so-called few-shot prompting.

Few-shot Prompting

Few-shot prompting is the place we embrace input-output examples throughout the assistant’s instructions from which it could presumably research. Proper right here, I append 3 (precise) suggestions and responses to the sooner instruction string.

intstructions_string_few_shot = """ShawGPT, functioning as a digital info 
science advertising advisor on YouTube, communicates in clear, accessible language, 
escalating to technical depth upon request. 
It reacts to strategies aptly and concludes with its signature '–ShawGPT'. 
ShawGPT will tailor the scale of its responses to match the viewer's comment, 
providing concise acknowledgments to transient expressions of gratitude or 
strategies, thus defending the interaction pure and interesting.
Listed under are examples of ShawGPT responding to viewer suggestions.
Viewer comment: This was a very thorough introduction to LLMs and answered many questions I had. Thanks.
ShawGPT: Good to take heed to, glad it was helpful :) -ShawGPT
Viewer comment: Epic, very useful for my BCI class
ShawGPT: Thanks, glad to take heed to! -ShawGPT
Viewer comment: Truly basically probably the most simple rationalization I've ever watched. Great fantastic work Shaw. Thanks. It's so unusual to hunt out good communicators similar to you!
ShawGPT: Thanks, glad it was clear -ShawGPT"""

Uncover that I added the sign-off “— ShawGPT” to each occasion (though these weren’t part of my exact response). That’s essential to get the desired habits from the assistant.

With the updated instructions, we’re in a position to create a model new assistant within the equivalent method as sooner than.

assistant = client.beta.assistants.create(
determine="ShawGPT",
description="Information scientist GPT for YouTube suggestions",
instructions=intstructions_string_few_shot,
model="gpt-4-0125-preview"
)

Passing the equivalent check out comment (i.e. “Good content material materials, thanks!”) to this assistant as sooner than, we now get the subsequent response:

You might be welcome, utterly joyful to take heed to you found it useful! -ShawGPT

Although that’s (nonetheless) just a little bit longer than my typical response, it’s considerably higher than what we seen with the vanilla assistant.

Let’s try one different (further technical) question. We are going to use the equivalent syntax as sooner than to cross the particular person message to the assistant, nonetheless now let’s ask, “What’s fat-tailedness?”

Proper right here’s the assistant response.

Fat-tailedness is an concept from statistics that refers again to the properties of a 
particular form of probability distribution. In a fat-tailed distribution, the 
tails (the far ends of the distribution, representing extreme values faraway from 
the indicate) are heavier or "fatter" than they is perhaps in a standard (Gaussian) 
distribution. This means there's a higher chance of most outcomes 
occurring compared with what you may anticipate if the information adopted a standard 
distribution.
In smart phrases, if a dataset is described as having a fat-tailed 
distribution, it signifies that unusual, extreme events (these throughout the tails) 
happen further ceaselessly than will be predicted by fashions based mostly totally on common 
distributions. This concept is awfully obligatory in fields like finance, 
economics, and hazard administration, the place underestimating the chance of most
events can have vital penalties.
Fat-tailed distributions distinction with thin-tailed distributions, similar to the 
common distribution, the place extreme values are a lot much less seemingly, and most of the 
prospects are concentrated throughout the indicate. Understanding whether or not or not a dataset
follows a fat-tailed distribution can significantly affect how we model and 
predict phenomena, notably with reference to assessing risks and anticipating
unusual nonetheless impactful events. -ShawGPT

Whereas (as soon as extra) this response is reasonably priced, it reads further like one factor from ChatGPT than what I’d say. Significantly, it makes use of a definite definition of fat tails (i.e. a distribution with heavier tails than a standard distribution) than the one I make the most of in my series on the topic (i.e. a distribution pushed by extreme events).

To boost this aspect of the assistant’s responses, we are able to present it the pliability to entry info from a Medium article I’ve written on fat-tailedness.

Knowledge Retrieval

Whereas organising a RAG system consists of a handful of nontrivial steps (i.e. chunking paperwork, organising vector database, setting up semantic search carry out, fusing search outcomes into context window), the complete course of is diminished to 2 steps throughout the Assistants API.

First, we add our paperwork for retrieval. Second, we added the retrieval performance to our assistant.

Throughout the code block beneath, I add my article on quantifying fat tails to my OpenAI account. To try this, we now have to specify the file path/determine and the purpose of the file. Proper right here, we set the purpose as “assistants”. The other alternative is “fine-tune”, which we’ll see shortly.

file = client.recordsdata.create(
file=open("articles/4 Strategies to Quantify Fat Tails with Python _ by Shaw Talebi _ In course of Information Science.pdf", "rb"),
perform="assistants"
)

Subsequent, we’re in a position to create a model new assistant very similar to sooner than, nonetheless now enabling retrieval and specifying the recordsdata obtainable to the assistant. Whereas I solely use one doc for retrieval proper right here, we’re in a position to specify as a lot as 20 recordsdata by along with the associated file IDs throughout the file_ids itemizing.

assistant = client.beta.assistants.create(
determine="ShawGPT",
description="Information scientist GPT for YouTube suggestions",
instructions=intstructions_string_few_shot,
devices=[{"type": "retrieval"}],
file_ids=[file.id],
model="gpt-4-0125-preview"
)

Passing the equivalent technical question to the assistant, we get the subsequent response.

Fat-tailedness describes the diploma to which unusual events significantly 
contribute to the combination statistics of a distribution. This concept 
highlights that in certain distributions, a small number of observations can 
disproportionately have an effect on the final picture. A fundamental occasion is the 
Pareto distribution, sometimes referenced throughout the context of wealth distribution, 
the place a small proportion of individuals can preserve the overwhelming majority of wealth. In 
statistical phrases, distributions range from thin-tailed (e.g., Gaussian, the place 
extreme events are extraordinarily unbelievable) to very fat-tailed (e.g., Pareto, the place 
extreme events are a lot much less unusual than in Gaussian distributions).
Fat-tailedness simply is not a few binary categorization of distributions as each 
"Power Regulation" or not nonetheless acknowledges a continuum or spectrum of "tail heaviness." 
This understanding provides a further nuanced view, allowing for further right and 
actual categorization of data distributions. Determining and quantifying the 
fat-tailed nature of a dataset is important in fields like finance, insurance coverage protection, 
hazard administration, and social sciences, the place understanding the chance and 
affect of most events is important.
In smart phrases, fat-tailedness could also be quantified using utterly completely different heuristics,
which may comprise mathematical computations such as a result of the Power Regulation Tail Index, 
Kurtosis, Log-normal's σ, and Taleb's κ metric. Each of these measures provides 
a definite lens through which to judge the fat-tailed nature of a 
distribution, thereby providing useful insights into the habits of most 
events all through the dataset -ShawGPT

This response is way nearer to the best way during which I think about (and make clear) fat-tailedness. The assistant did a seamless job of incorporating key concepts from the article into its response. For example, defining fat-tailedness by the use of unusual events, fat-tailedness dwelling on a spectrum, and 4 heuristics for measuring them.

Up up to now, we’ve gotten pretty far using prompt engineering and info retrieval to create our assistant. Nonetheless, the responses nonetheless don’t utterly be taught like one factor I’d write. To further improve this aspect of the assistant, we’re in a position to flip to fine-tuning.

Whereas prompt engineering could also be a easy choice to program an assistant, it’s not always obvious the best way to best instruct the model to point out the desired habits. In these circumstances, it could be advantageous to fine-tune the model.

Fine-tuning is as soon as we observe a pre-existing model with additional examples for a particular course of. Throughout the OpenAI Incredible-tuning API this consists of providing occasion user-assistant message pairs [3].

For the YouTube comment responder use case, this suggests gathering pairs of viewer suggestions (i.e., particular person message) and their associated responses (i.e., assistant message).

Although this further data-gathering course of makes fine-tuning further work upfront, it may end up in vital enhancements in model effectivity [3]. Proper right here, I stroll through the fine-tuning course of for this particular use case.

Information Preparation

To generate the user-assistant message pairs, I manually went through earlier YouTube suggestions and copy-pasted them proper right into a spreadsheet. I then exported this spreadsheet as a .csv file (obtainable on the GitHub repo).

Whereas this .csv file has the entire important info wished for fine-tuning, it will possibly’t be used straight. We must always first rework it into a particular format to cross it into the OpenAI API.

Additional notably, we now have to generate a .jsonl file, a textual content material file the place each line corresponds to a training occasion throughout the JSON format. If you happen to’re a Python particular person unfamiliar with JSON, you could think about it like a dictionary (i.e. an info development consisting of key-value pairs) [4].

To get our .csv into the required .jsonl format, I first create Python lists for each form of comment. That’s carried out by learning the raw .csv file line by line and storing each message throughout the acceptable itemizing.

import csv
import json
import random
comment_list = []
response_list = []
with open('info/YT-comments.csv', mode="r") as file:
file = csv.reader(file)
# be taught file line by line
for line in file:
# skip first line
if line[0]=='Comment':
proceed
# append suggestions and responses to respective lists
comment_list.append(line[0])
response_list.append(line[1] + " -ShawGPT")

Subsequent, to create the .jsonl file, we must always create a list of dictionaries the place each side corresponds to a training occasion. The key for each of these dictionaries is “messages”, and the price is (yet one more) itemizing of dictionaries similar to the system, particular person, and assistant messages, respectively. A visual overview of this info development is given beneath.

The Python code for taking our comment_list and response_list objects and creating the itemizing of examples is given beneath. That’s carried out by going through comment_list and response_list, side by side, and creating three dictionaries at each step.

These correspond to the system, particular person, and assistant messages, respectively, the place the system message is equivalent instructions we used to make our assistant by means of few-shot prompting, and the particular person/assistant messages come from their respective lists. These dictionaries are then saved in a list that serves as the value for that particular person teaching occasion.

example_list = []
for i in range(len(comment_list)):
# create dictionaries for each place/message    
system_dict = {"place": "system", "content material materials": intstructions_string_few_shot}
user_dict = {"place": "particular person", "content material materials": comment_list[i]}
assistant_dict = {"place": "assistant", "content material materials": response_list[i]}
# retailer dictionaries into itemizing
messages_list = [system_dict, user_dict, assistant_dict]
# create dictionary for ith occasion and add it to example_list
example_list.append({"messages": messages_list})

On the end of this course of, we’ve obtained a list with 59 elements similar to 59 user-assistant occasion pairs. One different step that helps think about model effectivity is to separate these 59 examples into two datasets, one for teaching the model and the other for evaluating its effectivity.

That’s carried out throughout the code block beneath, the place I randomly sample 9 out of 59 examples from example_list and retailer them in a model new itemizing generally known as validation_data_list. These examples are then away from example_list, which may perform our teaching dataset.

# create observe/validation break up
validation_index_list = random.sample(range(0, len(example_list)-1), 9)
validation_data_list = [example_list[index] for index in validation_index_list]
as an illustration in validation_data_list:
example_list.take away(occasion)

Lastly, with our teaching and validation datasets prepared, we’re in a position to write them to .jsonl recordsdata. This can be carried out throughout the following method.

# write examples to file
with open('info/training-data.jsonl', 'w') as training_file:
as an illustration in example_list:
json.dump(occasion, training_file)
training_file.write('n')
with open('info/validation-data.jsonl', 'w') as validation_file:
as an illustration in validation_data_list:
json.dump(occasion, validation_file)
validation_file.write('n')

Incredible-tuning job

With the information preparation carried out, we’re in a position to run the fine-tuning job in 2 steps. First, we add the teaching and validation recordsdata to our OpenAI account. Second, we run the teaching course of [3].

We add recordsdata like we did when organising doc retrieval for an assistant, nonetheless now setting the file perform as “fine-tune”. That’s carried out for every the teaching and validation datasets beneath.

# add fine-tuning recordsdata
training_file = client.recordsdata.create(
file = open("info/training-data.jsonl", "rb"),
perform = "fine-tune"
)
validation_file = client.recordsdata.create(
file = open("info/validation-data.jsonl", "rb"),
perform = "fine-tune"
)

Now, we’re in a position to run the fine-tuning job. For this, we now have to specify the teaching recordsdata and model we need to fine-tune. Primarily probably the most superior model obtainable for fine-tuning is gpt-3.5-turbo, which I make the most of proper right here. I moreover set a suffix that gives a user-defined string to the fine-tuned model’s ID. The code for that’s confirmed beneath.

client.fine_tuning.jobs.create(
training_file = training_file.id,
validation_file = validation_file.id,
suffix = "ShawGPT",
model = "gpt-3.5-turbo"
)

The teaching course of will take spherical quarter-hour. When full, we’re in a position to make use of the fine-tuned model by means of the completions API, as confirmed beneath.

test_comment = "Good content material materials, thanks!"
response = client.chat.completions.create(
model="ft:gpt-3.5-turbo-0613:personal:shawgpt:8mUeVreo",
messages=[
{"role": "system", "content": intstructions_string_few_shot},
{"role": "user", "content": test_comment}
]
)

Uncover how we generated the response throughout the code block above differs from how we did it beforehand. It’s as a result of fine-tuned fashions are not presently supported throughout the Assistants API.

A very powerful draw again is that if we want to improve the fine-tuned model with devices (i.e. make it an assistant), we must always assemble out these functionalities by means of libraries like LangChain or LlamaIndex (as an illustration).

Whereas this makes making a fine-tuned assistant further work, the responses we see out of the sector are nearer to at least one factor that I’d say. Listed under are numerous responses to the check out comment.

Thanks, I acknowledge it! -ShawGPT
Thanks, glad you favored it! -ShawGPT
Glad it was helpful! -ShawGPT

Let’s see the best way it handles the equivalent technical question as sooner than i.e. “What’s fat-tailedness?”

Good question! The fat tail phenomenon represents the size of outlier (extreme) 
events relative to a standard (Gaussian) distribution. In several phrases, there's a 
bigger probability of most events occurring compared with a standard 
distribution. -ShawGPT

Although the model defines fat tails in a number of phrases than I’d, the scale and magnificence of the response are considerably higher than what we seen with the Assistants API pre-RAG. Which means if we had been in order so as to add RAG to this fine-tuned model, it may generate significantly larger responses than what we seen sooner than.

Developing a personalized AI assistant is less complicated than ever sooner than. Proper right here, we seen a simple choice to create an AI assistant by means of OpenAI’s Assistant’s API and the best way to fine-tune a model by means of their Incredible-tuning API.

Whereas OpenAI presently has basically probably the most superior fashions for creating the form of AI assistant talked about proper right here, these fashions are locked behind their API, which limits what/how we’re in a position to assemble with them.

A pure question, on account of this reality, is how could we develop associated packages using open-source choices. This is perhaps coated throughout the subsequent articles of this assortment, the place I’ll deal with the best way to fine-tune a model using QLoRA and improve a chatbot by means of RAG.

Additional on LLMs 👇