hai I'm building langchain api and try
hai I'm building langchain api and try use langfuse to trace my chain but why is it only working with invoke method but not the batch method here how I define the callback to integrate with langfuse??
langfuse_config = RunnableConfig(callbacks=[langfuse_handler]
async def batch_api(api_chain, path: str, request: Request) -> Response:
"""
Handle a request for the specified API.
"""
api_handler = APIHandler(api_chain, path=path, include_callback_events=True)
return await api_handler.batch(request, server_config=langfuse_config)7 Replies
I think it's because, on is different from . Is there any support or documentation for that?
i have not found any
assuming you are using langserve, does this screenshot from a wip cookbook help?
apparently what's most robust here is to pass the config to the llm/chain instead of to the .batch of the handler
in case I misunderstood APIHandler let me know
Yes, Im using langserve. Is there a way to do it without passing the callback config into the llm, cause as my understanding the APIHandler doen't work the same way as add_routes, and maybe can you explain why its only work with invoke handler but not on batch method. And I'm also make an issue on github in this link: https://github.com/orgs/langfuse/discussions/1982
GitHub
trace doesn't work with .batch handler on langserve · langfuse · Di...
I'm using langserve and make two handler using .invoke and .batch, the trace is working fine for invoke but seem doesn't show record with the .batch handler from langserve import APIHandler...
we have just added a notebook, does this help?
https://langfuse.com/guides/cookbook/integration_langserve
Cookbook: Langserve Integration - Langfuse
Examples on how to natively integrate with LangChain's Langserve
I have read the notebook and all the examples that you provide are using invoke method. I don't really have a problem when tracing invoke method. The problem only occurs when I'm using batch method to create multiple outputs from multiple inputs,. That's when langfuse doesn't trace any of my input and output prompt. I used Langsmith before and this wasn't a problem in Langsmith