Retrieval-augmented Generation with CrewAI | by Xin Cheng | Mar, 2024

In previous article, we talked about CrewAI, LLM-powered agent programming framework, using bing search to go searching public internet sites. This textual content covers one of many important frequent use circumstances: Retrieval-augmented Generation (might have devoted articles for this topic). This enables us to answer questions using private info.

https://docs.crewai.com/tools/WebsiteSearchTool/

All RAG devices in the mean time can solely use openAI to generate embeddings, we’re engaged on together with assist for various suppliers.

We have to use Azure OpenAI embedding. Until it’s natively supported, we’ve bought to stipulate our custom-made instrument.

Steps

1. Define custom-made instrument
2. Define agent
3. Define course of

from langchain_community.vectorstores import FAISS
from langchain_openai import AzureChatOpenAI, AzureOpenAIEmbeddings
import os
from dotenv import load_dotenv
load_dotenv()
import openai
api_key = os.getenv("AZURE_OPENAI_API_KEY")
openai.api_key = api_key
os.environ['OPENAI_API_KEY'] = os.getenv("AZURE_OPENAI_API_KEY")
api_version = os.getenv("AZURE_OPENAI_API_VERSION")
llm = AzureChatOpenAI(
deployment_name=os.getenv("AZURE_OPENAI_DEPLOYMENT_NAME"),
openai_api_version=api_version
)
embedding = AzureOpenAIEmbeddings(
azure_deployment=os.getenv("AZURE_OPENAI_EMBEDDINGS_MODEL_NAME"),
openai_api_version=api_version,
)
from langchain_community.vectorstores import FAISS
vectorstore = FAISS.from_texts(
["harrison worked at kensho"], embedding=embedding
)
retriever = vectorstore.as_retriever()

Define custom-made instrument

The precept stuff is title, description and _run methodology

from crewai_tools import BaseTool
class WorkInfoSearchTool(BaseTool):
title: str = "Work Info Search Instrument"
description: str = "Search work related data."
def _run(self, query: str) -> str:
# Implementation goes proper right here
return retriever.get_relevant_documents(query)

Define agent

from crewai import Agent, Course of, Crew, Course of
devices=[WorkInfoSearchTool()]
websearch_agent = Agent(
operate='Work data search',
goal="Uncover reply for work data. Reply "No data" in case you occur to can"t uncover data',
backstory="""Uncover reply for work data associated to the individual. """,
llm=llm,
verbose=True,
allow_delegation=True,
devices=devices
)

Define course of

task1 = Course of(
description="""the place did harrison work?""",
expected_output="""work data""",
agent=websearch_agent
)

Run agent

# use openai embedding, learn how to make use of Azure OpenAI embedding in CrewAI technique?
crew = Crew(
brokers=[websearch_agent],
duties=[task1],
verbose=2,
course of=Course of.sequential
)
# Kick off the crew to start on it's duties
consequence = crew.kickoff()
print("######################")
print(consequence)

End result

 [DEBUG]: == Working Agent: Work data search
[INFO]: == Starting Course of: the place did harrison work?
> Moving into new CrewAgentExecutor chain...
I wish to search out particulars concerning the place Harrison labored.
Movement: Work Info Search Instrument
Movement Enter: {'query': 'Harrison work historic previous'} 
[Document(page_content="harrison worked at kensho")]
Thought: 
I found a doc that mentions Harrison labored at Kensho.
Closing Reply: 
Harrison labored at Kensho.
> Accomplished chain.
[DEBUG]: == [Work information search] Course of output: Harrison labored at Kensho.
######################
Harrison labored at Kensho.

Damaging sample

task1 = Course of(
description="""the place did greg work?""",
expected_output="""work data""",
agent=websearch_agent
)

End result

 [DEBUG]: == Working Agent: Work data search
[INFO]: == Starting Course of: the place did greg work?
> Moving into new CrewAgentExecutor chain...
I ought to make use of the Work Info Search Instrument to hunt out particulars concerning the place Greg labored.
Movement: 
Work Info Search Instrument
Movement Enter:
{
"query": "the place did Greg work"
}
[Document(page_content="harrison worked at kensho")]
Thought: 
The search consequence reveals that Harrison labored at Kensho, not Greg. I wish to go searching as soon as extra.
Movement: 
Work Info Search Instrument
Movement Enter:
{
"query": "the place did Greg work"
}
I tried reusing the equivalent enter, I have to stop using this movement enter. I'm going to aim one factor else in its place.
Thought: 
I ought to aim a particular query to hunt out particulars concerning the place Greg labored.
Movement: 
Work Info Search Instrument
Movement Enter:
{
"query": "Greg's work historic previous"
}
[Document(page_content="harrison worked at kensho")]
Thought: 
I ought to aim a particular query to hunt out particulars concerning the place Greg labored.
Movement: 
Work Info Search Instrument
Movement Enter:
{
"query": "Greg work historic previous"
}
[Document(page_content="harrison worked at kensho")]
Thought: 
The search consequence continues to be displaying particulars about Harrison working at Kensho. This is not helpful for finding the place Greg labored. I've to aim one different query.
Movement: 
Work Info Search Instrument
Movement Enter:
{
"query": "Greg's employment historic previous"
}
[Document(page_content="harrison worked at kensho")]
Thought: 
The search consequence continues to be not providing any particulars concerning the place Greg labored. I've to aim a particular technique.
Movement: 
Work Info Search Instrument
Movement Enter:
{
"query": "Greg earlier jobs"
}
[Document(page_content="harrison worked at kensho")]
Thought: 
I've exhausted the entire queries I can take into account to hunt out particulars concerning the place Greg labored. I couldn't uncover any associated data.
Closing Reply: No data
> Accomplished chain.
[DEBUG]: == [Work information search] Course of output: No data
######################
No data

One different hostile sample

task1 = Course of(
description="""the place is harrison's ardour?""",
expected_output="""ardour data""",
agent=websearch_agent
)

End result

[DEBUG]: == Working Agent: Work data search
[INFO]: == Starting Course of: the place is harrison's ardour?
> Moving into new CrewAgentExecutor chain...
I wish to search out particulars about Harrison's ardour. To do that, I am going to use the Work Info Search Instrument.
Movement: Work Info Search Instrument
Movement Enter: {
"query": "Harrison's ardour"
}
[Document(page_content="harrison worked at kensho")]
Thought: The search outcomes solely level out that Harrison labored at Kensho, nonetheless there is no such thing as a such factor as a particulars about his ardour. I've to take care of searching.
Movement: Work Info Search Instrument
Movement Enter: {
"query": "Harrison's ardour Kensho"
}
[Document(page_content="harrison worked at kensho")]
Thought:
The search outcomes nonetheless solely level out that Harrison labored at Kensho, nonetheless there is no such thing as a such factor as a particulars about his ardour. I've to aim a particular technique.
Movement: Work Info Search Instrument
Movement Enter: {
"query": "Harrison's ardour non-public"
}
[Document(page_content="harrison worked at kensho")]
Thought: I wish to search out particulars about Harrison's ardour. To do that, I am going to use the Work Info Search Instrument.
Movement: Work Info Search Instrument
Movement Enter: {
"query": "Harrison's ardour"
}
[Document(page_content="harrison worked at kensho")]
Thought: The search outcomes solely level out that Harrison labored at Kensho, nonetheless there is no such thing as a such factor as a particulars about his ardour. I've to take care of searching.
Movement: Work Info Search Instrument
Movement Enter: {
"query": "Harrison's ardour Kensho"
}
[Document(page_content="harrison worked at kensho")]
Thought:
The search outcomes nonetheless solely level out that Harrison labored at Kensho, nonetheless there is no such thing as a such factor as a particulars about his ardour. I've to aim a particular technique.
Movement: Work Info Search Instrument
Movement Enter: {
"query": "Harrison's ardour non-public"
}
I tried reusing the equivalent enter, I have to stop using this movement enter. I'm going to aim one factor else in its place.
Thought: I wish to search out particulars about Harrison's ardour. To do that, I am going to use the Work Info Search Instrument.
Movement: Work Info Search Instrument
Movement Enter: {
"query": "Harrison's ardour"
}
[Document(page_content="harrison worked at kensho")]
Thought: The search outcomes solely level out that Harrison labored at Kensho, nonetheless there is no such thing as a such factor as a particulars about his ardour. I've to take care of searching.
Movement: Work Info Search Instrument
Movement Enter: {
"query": "Harrison's ardour Kensho"
}
[Document(page_content="harrison worked at kensho")]
Thought:
The search outcomes nonetheless solely level out that Harrison labored at Kensho, nonetheless there is no such thing as a such factor as a particulars about his ardour. I've to aim a particular technique.
Movement: Work Info Search Instrument
Movement Enter: {
"query": "Harrison's ardour non-public life"
}
[Document(page_content="harrison worked at kensho")]
Thought: I wish to search out particulars about Harrison's ardour. To do that, I am going to use the Work Info Search Instrument.
Movement: Work Info Search Instrument
Movement Enter: {
"query": "Harrison's ardour"
}
[Document(page_content="harrison worked at kensho")]
Thought: The search outcomes solely level out that Harrison labored at Kensho, nonetheless there is no such thing as a such factor as a particulars about his ardour. I've to take care of searching.
Movement: Work Info Search Instrument
Movement Enter: {
"query": "Harrison's ardour Kensho"
}
[Document(page_content="harrison worked at kensho")]
Thought:
The search outcomes nonetheless solely level out that Harrison labored at Kensho, nonetheless there is no such thing as a such factor as a particulars about his ardour. I've to aim a particular technique.
Movement: Work Info Search Instrument
Movement Enter: {
"query": "Harrison's ardour non-public life"
}
I tried reusing the equivalent enter, I have to stop using this movement enter. I'm going to aim one factor else in its place.
Thought: I wish to search out particulars about Harrison's ardour. To do that, I am going to use the Work Info Search Instrument.
Movement: Work Info Search Instrument
Movement Enter: {
"query": "Harrison's ardour"
}
[Document(page_content="harrison worked at kensho")]
Closing Reply: There is not a data obtainable about Harrison's ardour.
> Accomplished chain.
[DEBUG]: == [Work information search] Course of output: There is not a data obtainable about Harrison's ardour.
######################
There is not a data obtainable about Harrison's ardour.

Conclusion

You’ll be capable of program RAG merely with langchain, llamaindex moreover. However, CrewAI implementation supplies completely different layer to verify the usual (e.g. with native RAG, the retrieve might return Harrison’s work when requested about “Greg’s work”. You might need to set matching threshold, which is a bit information). Using CrewAI, you could merely setup reply, validator agent.