DB connections aren’t reused when self hosting Langfuse
on further inspection, this sharp increase of db connections from langfuse instance has occured in the past too. Only now have we realised the issue due to error logs
11 Replies
We’re investigating
Thanks for the additional details, that helps
Do you run a single instance of Langfuse?
yes single instance
running the docker file using AWS ECS. just one task
Ok perfect
Please update when you can if you are able to find the fix/recreate it. Thank you for the help
Yes. Will do
Sumanth, can you extend the history of the graph by any chance? Would be interesting to see whether we introduced this somewhere accidentially
@sumanth balaji
any luck ? @Max
on further analysis, this issue seems to arise from LiteLLM proxy layer. I have informed the maintainer, shall coordinate with them to resolve this. Did further testing and seems to not be a problem caused by langfuse
We tried a lot of things and weren’t yet able to reproduce
Will keep trying to make sure Langfuse is safe in this regard
LiteLLM has pushed a release for this bug
following are the details
Okay the error you were seeing is fixed here: https://github.com/BerriAI/litellm/pull/2363
- It only occurs when there is a fallback
- It was caused because the
metadata field had a previous_models field which had a circular ref
- Pr fixes this error for Langfuse logging + using LiteLLM DB
How we'll prevent this in the future
- Adding testing to ensure we can run json.dumps() on the callbacks metadata field
Thank you to the people at langfuse for looking into the issueGitHub
(Fix) High Traffic Fix - handle litellm circular ref error by ishaa...
Proxy - High Traffic Fix
When a fallback occurs Prisma + Langfuse hit circular ref errors
This leads to Prisma retrying each request 5 times
User hits Prisma DB limits
Perfect, thanks for sharing