Contextual Recall in LangChain Agents

Empowering Conversational AI with Contextual Recall

Memory in Agents

Memory in Agents is an important feature that allows them to retain information from previous interactions and use it to provide more accurate and context-aware responses.

By incorporating memory into an Agent, it can remember the history of the conversation and use that information to answer subsequent questions more effectively. You need to give memory to your Agent when you want it to remember the history of the conversation and use that information to answer subsequent questions more effectively.

An Agent with Memory is different from a Chain with Memory regarding functionality.

A Chain with Memory is a sequence of actions and observations that can be used to perform a specific task. In contrast, an Agent with Memory is a conversational agent that can engage in a conversation with the user and utilize memory to provide context-aware responses.

Want to learn how to build modern software with LLMs using the newest tools and techniques in the field? Check out this free LLMOps course from industry expert Elvis Saravia of DAIR.AI!

Here are the main differences between an Agent with Memory and a Chain with Memory:

• Functionality: A Chain with Memory is designed to perform a specific task or sequence of actions, while an Agent with Memory is designed to engage in a conversation and provide context-aware responses.
• Conversation Flow: An Agent with Memory uses the AgentExecutor class to handle the conversation flow, allowing it to interact with the user and respond to their queries. On the other hand, a Chain with Memory follows a predefined sequence of actions and observations without user interaction.
• Memory Usage: An Agent with Memory utilizes memory to remember the history of the conversation and use that information to provide more accurate responses. It can remember previous questions and answers, providing context-aware responses based on the conversation context. A Chain with Memory may also use memory, but its usage is limited to the specific task or sequence of actions it is designed for.

An Agent with Memory is a conversational agent that can converse with the user and utilize memory to provide context-aware responses. It differs from a Chain with Memory, a sequence of actions and observations designed for a specific task.

To add memory to an Agent, the following steps can be followed:

• Create an LLMChain with memory.
• Use the ConversationBufferMemory class to store and retrieve conversation history.
• Construct the LLMChain with the Memory object.
• Create the Agent using the ZeroShotAgent class, passing in the LLMChain and the memory object.
• Use the AgentExecutor class to execute the Agent and handle the conversation flow.

🧑🏽‍💻 Let’s code!

Start by getting some preliminaries out of the way:

%%capture
!pip install langchain openai duckduckgo-search youtube_search wikipedia

import os
import getpass

os.environ["OPENAI_API_KEY"] = getpass.getpass("Enter Your OpenAI API Key:")

from langchain.agents import load_tools
from langchain.agents import initialize_agent
from langchain.agents import ZeroShotAgent, Tool, AgentExecutor
from langchain.memory import ConversationBufferMemory
from langchain import OpenAI, LLMChain

Create an LLMChain with memory

Start by creating an LLMChain object that includes memory. This allows the Agent to retain information from previous interactions.

Notice the usage of the chat_history variable in the PromptTemplate, which matches up with the dynamic key name in the ConversationBufferMemory.

prefix = """
You're having a conversation with a human. You're helpful and answering
questions to your maximum ability. You also speak using British slang.
Like you're from the show TopBoy. You have access to the following tools:
"""

suffix = """Let's Go!"

{chat_history}

Question: {input}

{agent_scratchpad}
"""

llm = OpenAI(temperature=0.0)

tools = load_tools(["ddg-search", "llm-math", "wikipedia"], llm=llm)

tools[0].description += " Prioritize this when you're looking for current information or current events."

tools[2].description += " Prioritize this when you're looking for factual information about people"

prompt = ZeroShotAgent.create_prompt(
    tools,
    prefix=prefix,
    suffix=suffix,
    input_variables = ["input", "chat_history", "agent_scratchpad"]
)

Use the ConversationBufferWindowMemory class

To store and retrieve conversation history, use the ConversationBufferWindowMemory class. This memory object will keep track of the conversation context.

memory = ConversationBufferMemory(memory_key="chat_history")

Construct the LLMChain with the Memory object

Initialize the LLMChain with the memory object created in the previous step. This connects the memory to the chain.

llm_chain = LLMChain(llm=llm,
                     prompt=prompt)

Create the Agent

Use the ZeroShotAgent class to create the Agent. Pass in the LLMChain and the memory object as parameters.

This creates an Agent that can utilize memory during conversations.

agent = ZeroShotAgent(llm_chain=llm_chain,
                      tools=tools,
                      verbose=True)

Use the AgentExecutor class

To execute the Agent and handle the conversation flow, use the AgentExecutor class. This class allows the Agent to interact with the user and respond to their queries.

agent_chain = AgentExecutor.from_agent_and_tools(
    agent=agent,
    tools=tools,
    verbose=True,
    memory=memory
)

Use the Agent

agent_chain.run(input="Who is Yann LeCun and where was he born?")

> Entering new AgentExecutor chain...
Thought: I need to answer a question about a person
Action: Wikipedia
Action Input: Yann LeCun
Observation: Page: Yann LeCun
Summary: Yann André LeCun ( lə-KUN, French: [ləkœ̃]; originally spelled Le Cun; born 8 July 1960) is a  Turing Award winning French computer scientist working primarily in the fields of machine learning, computer vision, mobile robotics and computational neuroscience. He is the Silver Professor of the Courant Institute of Mathematical Sciences at New York University and Vice-President, Chief AI Scientist at Meta.He is well known for his work on optical character recognition and computer vision using convolutional neural networks (CNN), and is a founding father of convolutional nets. He is also one of the main creators of the DjVu image compression technology (together with Léon Bottou and Patrick Haffner). He co-developed the Lush programming language with Léon Bottou.
LeCun received the 2018 Turing Award (often referred to as the "Nobel Prize of Computing"), together with Yoshua Bengio and Geoffrey Hinton, for their work on deep learning.
The three are sometimes referred to as the "Godfathers of AI" and "Godfathers of Deep Learning".

Page: LeNet
Summary: LeNet is a convolutional neural network structure proposed by LeCun et al. in 1998,. In general, LeNet refers to LeNet-5 and is a simple convolutional neural network. Convolutional neural networks are a kind of feed-forward neural network whose artificial neurons can respond to a part of the surrounding cells in the coverage range and perform well in large-scale image processing.



Page: Geoffrey Hinton
Summary: Geoffrey Everest Hinton  (born 6 December 1947) is a British-Canadian cognitive psychologist and computer scientist, most noted for his work on artificial neural networks. From 2013 to 2023, he divided his time working for Google (Google Brain) and the University of Toronto, before publicly announcing his departure from Google in May 2023 citing concerns about the risks of artificial intelligence (AI) technology. In 2017, he co-founded and became the chief scientific advisor of the Vector Institute in Toronto.With David Rumelhart and Ronald J. Williams, Hinton was co-author of a highly cited paper published in 1986 that popularised the backpropagation algorithm for training multi-layer neural networks, although they were not the first to propose the approach. Hinton is viewed as a leading figure in the deep learning community. The dramatic image-recognition milestone of the AlexNet designed in collaboration with his students Alex Krizhevsky and Ilya Sutskever for the ImageNet challenge 2012 was a breakthrough in the field of computer vision.Hinton received the 2018 Turing Award (often referred to as the "Nobel Prize of Computing"), together with Yoshua Bengio and Yann LeCun, for their work on deep learning. They are sometimes referred to as the "Godfathers of Deep Learning", and have continued to give public talks together.In May 2023, Hinton announced his resignation from Google to be able to "freely speak out about the risks of A.I." He has voiced concerns about deliberate misuse by malicious actors, technological unemployment, and existential risk from artificial general intelligence.


Thought:I now know the final answer
Final Answer: Yann LeCun is a French computer scientist and Turing Award winner who is well known for his work on optical character recognition and computer vision using convolutional neural networks. He was born in 1960 and did his post-doc at the University of Toronto with Geoffrey Hinton. He is one of the "Godfathers of AI" and "Godfathers of Deep Learning" alongside Yoshua Bengio and Geoffrey Hinton.

> Finished chain.
Yann LeCun is a French computer scientist and Turing Award winner who is well known for his work on optical character recognition and computer vision using convolutional neural networks. He was born in 1960 and did his post-doc at the University of Toronto with Geoffrey Hinton. He is one of the "Godfathers of AI" and "Godfathers of Deep Learning" alongside Yoshua Bengio and Geoffrey Hinton.

agent_chain.run(input="Where did he go for his post-doc?")

> Entering new AgentExecutor chain...
Thought: Yann LeCun did his post-doc at the University of Toronto with Geoffrey Hinton.
Action: Wikipedia
Action Input: Yann LeCun
Observation: Page: Yann LeCun
Summary: Yann André LeCun ( lə-KUN, French: [ləkœ̃]; originally spelled Le Cun; born 8 July 1960) is a  Turing Award winning French computer scientist working primarily in the fields of machine learning, computer vision, mobile robotics and computational neuroscience. He is the Silver Professor of the Courant Institute of Mathematical Sciences at New York University and Vice-President, Chief AI Scientist at Meta.He is well known for his work on optical character recognition and computer vision using convolutional neural networks (CNN), and is a founding father of convolutional nets. He is also one of the main creators of the DjVu image compression technology (together with Léon Bottou and Patrick Haffner). He co-developed the Lush programming language with Léon Bottou.
LeCun received the 2018 Turing Award (often referred to as the "Nobel Prize of Computing"), together with Yoshua Bengio and Geoffrey Hinton, for their work on deep learning.
The three are sometimes referred to as the "Godfathers of AI" and "Godfathers of Deep Learning".

Page: LeNet
Summary: LeNet is a convolutional neural network structure proposed by LeCun et al. in 1998,. In general, LeNet refers to LeNet-5 and is a simple convolutional neural network. Convolutional neural networks are a kind of feed-forward neural network whose artificial neurons can respond to a part of the surrounding cells in the coverage range and perform well in large-scale image processing.



Page: Geoffrey Hinton
Summary: Geoffrey Everest Hinton  (born 6 December 1947) is a British-Canadian cognitive psychologist and computer scientist, most noted for his work on artificial neural networks. From 2013 to 2023, he divided his time working for Google (Google Brain) and the University of Toronto, before publicly announcing his departure from Google in May 2023 citing concerns about the risks of artificial intelligence (AI) technology. In 2017, he co-founded and became the chief scientific advisor of the Vector Institute in Toronto.With David Rumelhart and Ronald J. Williams, Hinton was co-author of a highly cited paper published in 1986 that popularised the backpropagation algorithm for training multi-layer neural networks, although they were not the first to propose the approach. Hinton is viewed as a leading figure in the deep learning community. The dramatic image-recognition milestone of the AlexNet designed in collaboration with his students Alex Krizhevsky and Ilya Sutskever for the ImageNet challenge 2012 was a breakthrough in the field of computer vision.Hinton received the 2018 Turing Award (often referred to as the "Nobel Prize of Computing"), together with Yoshua Bengio and Yann LeCun, for their work on deep learning. They are sometimes referred to as the "Godfathers of Deep Learning", and have continued to give public talks together.In May 2023, Hinton announced his resignation from Google to be able to "freely speak out about the risks of A.I." He has voiced concerns about deliberate misuse by malicious actors, technological unemployment, and existential risk from artificial general intelligence.


Thought:I now know the final answer.
Final Answer: Yann LeCun did his post-doc at the University of Toronto with Geoffrey Hinton.

> Finished chain.
Yann LeCun did his post-doc at the University of Toronto with Geoffrey Hinton.

agent_chain.run(input="Search for current temperature there on September 8, 2023?")

> Entering new AgentExecutor chain...
Thought: I need to find the current temperature in Toronto on September 8, 2023
Action: DuckDuckGo Search
Action Input: "Current temperature in Toronto on September 8, 2023"
Observation: DuckDuckGo Search is not a valid tool, try one of [duckduckgo_search, Calculator, Wikipedia].
Thought:I need to use duckduckgo_search to find the current temperature in Toronto on September 8, 2023
Action: duckduckgo_search
Action Input: "Current temperature in Toronto on September 8, 2023"
Observation: Current Weather for Popular Cities . San Francisco, CA 59 ° F Fair; Manhattan, NY warning 79 ° F Fair; Schiller Park, IL (60176) warning 65 ° F Cloudy; Boston, MA 81 ° F Clear; Houston, TX ... Posted September 7, 2023 4:21 pm 17:37 Global News at 6 Toronto: September 3, 2023 WATCH: It's not exactly the September weather we're accustomed to but it is here — searing hot... The hot weather blast will only increase in intensity on Monday and Tuesday, when it's expected to feel like 37 C and 39 C, respectively. Based on the weather agency's 14-day forecast ... Temperatures will continue to fall as more seasonal conditions return heading into the weekend. It's a change after a cooler-than-usual August. There were no days in which the temperature reached more than 30 C last month, compared to five days of 30-plus temperatures in July. The last time Toronto saw a 35 degree day was in June 2022. Toronto weather in September 2023 30 days Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Mon Tue Wed Thu Fri Sat Sun 28 August +25°+18° 29 August +25°+16° 30 August +27°+17° 31 August +26°+19° 1 September +25°+18° 2 September +25°+19° 3 September +25°+19° 4 September +24°+17° 5 September +23°+16°
Thought:I now know the final answer
Final Answer: The current temperature in Toronto on September 8, 2023 is expected to be around 25°C.

> Finished chain.
The current temperature in Toronto on September 8, 2023 is expected to be around 25°C.

agent_chain.run(input="What's the difference between the current temperature there and in Winnipeg, MB?")

> Entering new AgentExecutor chain...
Thought: I need to find the current temperature in both Toronto and Winnipeg
Action: duckduckgo_search
Action Input: current temperature in Toronto
Observation: Current weather in Toronto, Ontario, Canada. Check current conditions in Toronto, Ontario, Canada with radar, hourly, and more. Find the most current and reliable 7 day weather forecasts, storm alerts, reports and information for [city] with The Weather Network. Feels Like: 92 °F. Forecast: 85 / 72 °F. Wind: 10 mph ↑ from South. Location: Toronto Pearson International Airport. Current Time: Sep 6, 2023 at 4:05:54 pm. Latest Report: Sep 6, 2023 at 3:00 pm. Current Conditions Past 24 hours Weather Radar Satellite Lightning 27°C °C °F Observed at: Toronto Pearson Int'l Airport Date: 10:00 PM EDT Tuesday 5 September 2023 Condition: Clear Pressure: 101.4 kPa Tendency: Steady Temperature: 26.6°C Dew point: 20.8°C Humidity: 71% Wind: WSW 14 km/h Humidex: 35 Visibility: 24 km Forecast Hourly Forecast Temperatures will continue to fall as more seasonal conditions return heading into the weekend. It's a change after a cooler-than-usual August. There were no days in which the temperature reached more than 30 C last month, compared to five days of 30-plus temperatures in July. The last time Toronto saw a 35 degree day was in June 2022.
Thought:I now need to find the current temperature in Winnipeg
Action: duckduckgo_search
Action Input: current temperature in Winnipeg
Observation: 21°C °C °F Observed at: Winnipeg Richardson Int'l Airport Date: 10:00 PM CDT Monday 4 September 2023 Condition: Mostly Cloudy Pressure: 100.1 kPa Tendency: Rising Temperature: 21.1°C Dew point: 17.8°C Humidity: 81% Wind: NNW 20 km/h Humidex: 27 Visibility: 18 km Forecast Hourly Forecast Air Quality Alerts Jet Stream Tonight 24°C °C °F Observed at: Winnipeg Richardson Int'l Airport Date: 8:00 PM CDT Sunday 3 September 2023 Condition: Mainly Sunny Pressure: 100.6 kPa Tendency: Falling Temperature: 23.8°C Dew point: 14.4°C Humidity: 56% Wind: NE 11 km/h Humidex: 27 Visibility: 16 km Forecast Hourly Forecast Air Quality Alerts Jet Stream Tonight Find the most current and reliable 7 day weather forecasts, storm alerts, reports and information for [city] with The Weather Network. Winnipeg, Manitoba, Canada Weather Forecast, with current conditions, wind, air quality, and what to expect for the next 3 days. Current weather in Winnipeg and forecast for today, tomorrow, and next 14 days
Thought:I now know the final answer
Final Answer: The difference in temperature between Toronto and Winnipeg is 5°C.

> Finished chain.
The difference in temperature between Toronto and Winnipeg is 5°C.

Compare the above with a memoryless Agent

prefix = """
You're having a conversation with a human. You're helpful and answering
questions to your maximum ability. You also speak using British slang.
Like you're from the show TopBoy. You have access to the following tools:
"""

suffix = """Let's Go!"

Question: {input}

{agent_scratchpad}
"""

llm = OpenAI(temperature=0.0)

tools = load_tools(["ddg-search", "llm-math", "wikipedia"], llm=llm)

prompt = ZeroShotAgent.create_prompt(tools,
                                     prefix=prefix,
                                     suffix=suffix,
                                     input_variables=["input", "agent_scratchpad"])

llm_chain = LLMChain(llm=llm,
                     prompt=prompt)

agent = ZeroShotAgent(llm_chain=llm_chain,
                      tools=tools,
                      verbose=True)

agent_without_memory = AgentExecutor.from_agent_and_tools(
    agent=agent,
    tools=tools,
    verbose=True
)

agent_without_memory.run(input="Who is Yann LeCun and where was he born?")

> Entering new AgentExecutor chain...
Thought: I need to find out who Yann LeCun is and where he was born.
Action: Wikipedia
Action Input: Yann LeCun
Observation: Page: Yann LeCun
Summary: Yann André LeCun ( lə-KUN, French: [ləkœ̃]; originally spelled Le Cun; born 8 July 1960) is a  Turing Award winning French computer scientist working primarily in the fields of machine learning, computer vision, mobile robotics and computational neuroscience. He is the Silver Professor of the Courant Institute of Mathematical Sciences at New York University and Vice-President, Chief AI Scientist at Meta.He is well known for his work on optical character recognition and computer vision using convolutional neural networks (CNN), and is a founding father of convolutional nets. He is also one of the main creators of the DjVu image compression technology (together with Léon Bottou and Patrick Haffner). He co-developed the Lush programming language with Léon Bottou.
LeCun received the 2018 Turing Award (often referred to as the "Nobel Prize of Computing"), together with Yoshua Bengio and Geoffrey Hinton, for their work on deep learning.
The three are sometimes referred to as the "Godfathers of AI" and "Godfathers of Deep Learning".

Page: LeNet
Summary: LeNet is a convolutional neural network structure proposed by LeCun et al. in 1998,. In general, LeNet refers to LeNet-5 and is a simple convolutional neural network. Convolutional neural networks are a kind of feed-forward neural network whose artificial neurons can respond to a part of the surrounding cells in the coverage range and perform well in large-scale image processing.



Page: Geoffrey Hinton
Summary: Geoffrey Everest Hinton  (born 6 December 1947) is a British-Canadian cognitive psychologist and computer scientist, most noted for his work on artificial neural networks. From 2013 to 2023, he divided his time working for Google (Google Brain) and the University of Toronto, before publicly announcing his departure from Google in May 2023 citing concerns about the risks of artificial intelligence (AI) technology. In 2017, he co-founded and became the chief scientific advisor of the Vector Institute in Toronto.With David Rumelhart and Ronald J. Williams, Hinton was co-author of a highly cited paper published in 1986 that popularised the backpropagation algorithm for training multi-layer neural networks, although they were not the first to propose the approach. Hinton is viewed as a leading figure in the deep learning community. The dramatic image-recognition milestone of the AlexNet designed in collaboration with his students Alex Krizhevsky and Ilya Sutskever for the ImageNet challenge 2012 was a breakthrough in the field of computer vision.Hinton received the 2018 Turing Award (often referred to as the "Nobel Prize of Computing"), together with Yoshua Bengio and Yann LeCun, for their work on deep learning. They are sometimes referred to as the "Godfathers of Deep Learning", and have continued to give public talks together.In May 2023, Hinton announced his resignation from Google to be able to "freely speak out about the risks of A.I." He has voiced concerns about deliberate misuse by malicious actors, technological unemployment, and existential risk from artificial general intelligence.


Thought:I now know the final answer.

Final Answer: Yann LeCun is a French computer scientist and Turing Award winner working primarily in the fields of machine learning, computer vision, mobile robotics and computational neuroscience. He was born in 1960 in France. He is one of the "Godfathers of AI" and "Godfathers of Deep Learning" alongside Yoshua Bengio and Geoffrey Hinton.

> Finished chain.
Yann LeCun is a French computer scientist and Turing Award winner working primarily in the fields of machine learning, computer vision, mobile robotics and computational neuroscience. He was born in 1960 in France. He is one of the "Godfathers of AI" and "Godfathers of Deep Learning" alongside Yoshua Bengio and Geoffrey Hinton.

agent_without_memory.run(input="Where did he go for his post-doc?")

> Entering new AgentExecutor chain...
Thought: I need to find out where he went for his post-doc
Action: Wikipedia
Action Input: "post-doc"
Observation: Page: Postdoctoral researcher
Summary: A postdoctoral fellow, postdoctoral researcher, or simply postdoc, is a person professionally conducting research after the completion of their doctoral studies (typically a PhD). Postdocs most commonly, but not always, have a temporary academic appointment, sometimes in preparation for an academic faculty position. According to data from the US National science foundation. The number of holders of PhD in Biological sciences who end up in tenure track has consistently dropped in the last decades from over 50% in the 1970s to contemporary levels of 20% [direct reference needed]. They continue their studies or carry out research and further increase expertise in a specialist subject, including integrating a team and acquiring novel skills and research methods. Postdoctoral research is often considered essential while advancing the scholarly mission of the host institution; it is expected to produce relevant publications in peer-reviewed academic journals or conferences. In some countries, postdoctoral research may lead to further formal qualifications or certification, while in other countries, it does not.Postdoctoral research may be funded through an appointment with a salary or an appointment with a stipend or sponsorship award. Appointments for such a research position may be called postdoctoral research fellow, postdoctoral research associate, or postdoctoral research assistant. Postdoctoral researchers typically work under the supervision of a principal investigator. In many English-speaking countries, postdoctoral researchers are colloquially referred to as "postdocs".

Page: Doc Martin
Summary: Doc Martin is a British medical comedy-drama television series starring Martin Clunes as Doctor Martin Ellingham. It was created by Dominic Minghella developing the character of Dr Martin Bamford from the 2000 comedy film Saving Grace. The programme is set in the fictional seaside village of Portwenn and filmed on location in the village of Port Isaac, Cornwall, United Kingdom, with most interior scenes shot in a converted local barn. Fern Cottage is used as the home and surgery of Doctor Ellingham.
Nine series aired between 2004 and 2019, with a television film airing on Christmas Day in 2006. The ninth series aired on ITV premiered in September 2019. The tenth (and final) series aired from 7 September 2022 to 26 October 2022; one last installment, a Christmas special that aired on 25 December 2022, was the programme's final episode. On 29 December 2022 a documentary entitled “Farewell Doc Martin” aired on ITV, featuring behind-the-scenes interviews with the cast and crew as they filmed the final series. It also looked back at highlights from the 18 years of the show.


Page: Doc (aircraft)
Summary: Doc is a Boeing B-29 Superfortress. It is one of two that are currently flying in the world, the other B-29 being FIFI. It is owned by Doc's Friends, Inc., a non-profit organization based in Wichita, Kansas, United States.
Doc attends various air shows and offers rides.
Thought:I now know that Doc Martin was a British medical comedy-drama television series starring Martin Clunes as Doctor Martin Ellingham, and that Doc is a Boeing B-29 Superfortress owned by Doc's Friends, Inc.
Final Answer: Doc Martin is a British medical comedy-drama television series starring Martin Clunes as Doctor Martin Ellingham, and Doc is a Boeing B-29 Superfortress owned by Doc's Friends, Inc.

> Finished chain.
Doc Martin is a British medical comedy-drama television series starring Martin Clunes as Doctor Martin Ellingham, and Doc is a Boeing B-29 Superfortress owned by Doc's Friends, Inc.

Conclusion

The blog’s journey concludes with a clear understanding of memory’s pivotal role in LangChain agents.

By integrating memory, agents transcend simple task execution, becoming capable conversationalists with the ability to recall and utilize past interactions. This feature elevates agents from mere executors of commands to intelligent interlocutors, capable of contextual awareness and improved decision-making. It’s an essential step forward, marking a significant advancement in creating more nuanced and sophisticated AI-driven interactions.

As you deploy these agents, remember the power of memory — it’s not just about responding but understanding and growing with each interaction.