Agent executor enters and endless loop where the Tool output is ignored and then used again and again

[Onturenio]

I'm following the Custom LLM Agent tutorial. I have created an agent with the following prompt template:

You are an AI assistant. You are a clever and friendly assistant using emojis to better express your ideas. Answer the following questions from the users as best you can in the language used by the user.

To fulfill these objectives, you have access to the following tools:

{tools}

Always reply following the following format:

Question: the input question you must answer
Language: the detected language used in the input question
Thought: you should always think about what to do and if you have the date you need for invoking the tools.
Action: the action to take. It should be one of [{tool_names}]. If you do not need a tool, proceed directly to Final Answer.
Action Input: the input to the action.
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can repeat N times, or maybe you can just proceed to Final Answer if you know how to answer the question)
Thought: I now know the final answer
Final Answer (in Language): the final answer to the original input question.  Always use the Language you detected. The final answer is friendly and includes emojis to better express your ideas.

If the user queries you to do a task beyond your objectives, you must politely
decline and proceed to `Final Answer` where you suggest to ask another
friendship-related question.

Today is 20230622

Begin!
Question: {query}

Then I define a custom prompt based on the CustomPromptTemplate from the tutorial. I also take the CustomOutputParser from there.

I have define an ultra simple tool that outputs two simple sentences:

def friendinfo(args):
    return "Joan is the best friend of Pedro.\nPedro is the son of Luis."

friends_info_tool = Tool(name = "FriendsInfo",
                     func=friendinfo,
                     description=("useful when you need to know more about friends."))

With this, I go ahead and create and use the agent:

output_parser = CustomOutputParser()
llm = ChatOpenAI(temperature=0)
# LLM chain consisting of the LLM and a prompt
llm_chain = LLMChain(llm=llm, prompt=prompt)
tool_names = [tool.name for tool in tools]
agent = LLMSingleActionAgent(
    llm_chain=llm_chain,
    output_parser=output_parser,
    stop=["\nObservation:"],
    allowed_tools=tool_names)

agent_executor = AgentExecutor.from_agent_and_tools(agent=agent, tools=tools, verbose=True)

agent_executor.run("Who is the best friend of Pedro?")

And the problem is that this starts an endless loop where the tool is correctly used, it outputs what it has to, but the Agent does ignore this and keeps using the tool forever:

> Entering new  chain...
Language: English
Thought: I need to use the FriendsInfo tool to find out who is Pedro's best friend.
Action: FriendsInfo
Action Input: Pedro

Observation:
Joan is the best friend of Pedro.
Pedro is the son of Luis.

Language: English
Thought: I need to use the FriendsInfo tool to find out who is Pedro's best friend.
Action: FriendsInfo
Action Input: Pedro

Observation:
Joan is the best friend of Pedro.
Pedro is the son of Luis.

Language: English
Thought: I need to use the FriendsInfo tool to find out who is Pedro's best friend.
Action: FriendsInfo
Action Input: Pedro

Observation:
Joan is the best friend of Pedro.
Pedro is the son of Luis.

<- I have to quit here

Am I missfollowing the guidelines? Should I use a different type of Agent? I'm very close to believe that this is some sort of bug in LangChain, or at least in the documentation.

[antonygunawan94]

Hey @Onturenio, try adding enter ("\n") after "Observation:" word inside your prompt template.
It works for me.

Before:

Observation: the result of the action

After:

Observation: 
the result of the action

[wiesenfeK]

It does not seem to help, maybe it improves things a little..

[protyze]

I'm having the same issue of an endless loop with a basic Tools Agent:

    agent = (
            RunnablePassthrough.assign(
                agent_scratchpad=lambda x: format_to_openai_tool_messages(
                    x["intermediate_steps"]
                )
            )
            | prompt
            | llm_with_tools
            # | OpenAIToolsAgentOutputParser()
            | CustomOpenAIToolsAgentOutputParser()
    )

    agent_executor = AgentExecutor(tools=tools, agent=agent, verbose=True)

When I use the OpenAIToolsAgentOutputParser it works as intended.
(from langchain.agents.output_parsers.openai_tools import OpenAIToolsAgentOutputParser)

However, when I use my CustomOpenAIToolsAgentOutputParser it enters an endless loop. BUT: I copied the exact same code from OpenAIToolsAgentOutputParser over and verified that it produces the exact same output.

Any ideas what the problem could be?

[Joshua-Shepherd]

Same, however in my case.. it only happens intermittently.
I can get it to happen reliably though. By introducing a CallbackHandler to the AgentExecutor.

[alonme]

I Solved this issue by changing the prompt template

Changed from this

prompt = ChatPromptTemplate.from_messages(
    [
        ("system", get_base_prompt()),
        MessagesPlaceholder(variable_name="history", optional=True),
        (
            "human",
            """

        {input}


        {agent_scratchpad}""",
        ),
    ]
)

To this

prompt = ChatPromptTemplate.from_messages(
    [
        ("system", get_base_prompt()),
        MessagesPlaceholder(variable_name="history", optional=True),
        ("human","{input}"),
        MessagesPlaceholder(variable_name="agent_scratchpad", optional=True),
    ]
)