以前说过基于结构化数据配置进行函数调用的，以下是直接使用llama-cpp-agent 自带的StructuredOutputAgent

环境准备

需要安装llama-cpp-agent 以及启动一个llama-server

  • 安装
pip install llama-cpp-agent
  • 服务启动
    Linux 环境自己编译的llama-server
llama-server  -m rubra-mistral-7b-instruct-v0.3.Q4_K_M.gguf  --host 0.0.0.0

llama-cpp-agent 代码调用

  • demo3.py
from llama_cpp_agent import StructuredOutputAgent
from llama_cpp_agent.providers import LlamaCppServerProvider
from  pydantic  import BaseModel
 
provider = LlamaCppServerProvider("http://localhost:8080")
# 注意我们可以通过pydantic 进一步对于类型进行说明，使用上比较类似instructor 
class User(BaseModel):
    name: str
    age: int
 
agent = StructuredOutputAgent(
    llama_llm=provider,
    debug_output=True
    )
txt = "大龙，今年30岁了"
msg = agent.create_object(User,txt)
print(msg)
效果
<|im_start|>system
Read and follow the instructions below:
 
<system_instructions>
You are an advanced AI agent. You are tasked to assist the user by creating structured output in JSON format.
 
Output Model: User
  Fields:
    name (str)
    age (int)
</system_instructions>
 
 
Your output should be structured as JSON and represent one of the following output models: 
 
<output_models>
Output Model: User
  Fields:
    name (str)
    age (int)
</output_models>
 
Your JSON output should have the following fields:
 
 
The 'model' field should contain the name of the specific model that you are outputting, based on the system instructions. 
 
The 'fields' field should contain the actual fields and content of the model you are outputting, filled out according to the system instructions.<|im_end|>
<|im_start|>user
Create an JSON response based on the following input.
 
Input:
 
大龙，今年30岁了<|im_end|>
<|im_start|>assistant
{
  "model": 
    "User",
  "fields": {
    "name": "大龙",
    "age": 30
  }
}
name='大龙' age=30
from llama_cpp_agent.providers import LlamaCppServerProvider

说明

llama-cpp-agent 基于grammars 以及json schema 的结构化数据还是比较稳定的，项目中值得试用下

参考资料

https://llama-cpp-agent.readthedocs.io/en/latest/get-started/#create-agent