原始实现: https://github.com/ysymyth/ReAct.git

instruction = """
Solve a question answering task with interleaving Thought, Action, Observation steps. Thought can reason about the current situation, and Action can be three types: 
(1) Search[entity], which searches the exact entity on Wikipedia and returns the first paragraph if it exists. If not, it will return some similar entities to search.
(2) Lookup[keyword], which returns the next sentence containing keyword in the current passage.
(3) Finish[answer], which returns the answer and finishes the task.
Here are some examples.
"""

example = """
Question: Were Pavel Urysohn and Leonid Levin known for the same type of work?
Thought 1: I need to search Pavel Urysohn and Leonid Levin, find their types of work, then find if they are the same.
Action 1: Search[Pavel Urysohn]
Observation 1: Pavel Samuilovich Urysohn (February 3, 1898 – August 17, 1924) was a Soviet mathematician who is best known for his contributions in dimension theory.
Thought 2: Pavel Urysohn is a mathematician. I need to search Leonid Levin next and find its type of work.
Action 2: Search[Leonid Levin]
Observation 2: Leonid Anatolievich Levin is a Soviet-American mathematician and computer scientist. 
Thought 3: Leonid Levin is a mathematician and computer scientist. So Pavel Urysohn and Leonid Levin have the same type of work. 
Action 3: Finish[yes]
"""

prompt = instruction + example + "Question: ..."

简单来说是个循环

messages = []  
messages.append("Question: ...")  # user
round_num = 1
while True:
    s = f"Thought {round_num} ... \n Action {round_num}: ...\n"  # assistant
    messages.append(s)
    if "Finish" in s:
        break
    messages.append(f"Observation {round_num} ...\n")            # user/tool output
    round_num += 1

而 Search 工具是搜索维基百科, Lookup 工具是在页面中查找关键词, 具体实现上大致是这个流程 (最主要的 trick 是生成模型可以设置结束字符串):

[原始 Prompt]
Question: Were Pavel Urysohn and Leonid Levin known for the same type of work?

[LLM GENERATE: 设置 stop=["Observation 1"]]

Thought 1: I need to search Pavel Urysohn and Leonid Levin, find their types of work, then find if they are the same.
Action 1: Search[Pavel Urysohn]
Observation 1:

[工具调用: 解析 Action 中的工具]

Pavel Samuilovich Urysohn (February 3, 1898 – August 17, 1924) was a Soviet mathematician who is best known for his contributions in dimension theory.

[手动拼接上: "Thought 2:"]

Thought 2:

[LLM GENERATE: 设置 stop=["Observation 2"]]

...