sumanth balaji
LLangfuse
•Created by sumanth balaji on 3/5/2024 in #get-support
DB connections aren’t reused when self hosting Langfuse
LiteLLM has pushed a release for this bug
following are the details
Okay the error you were seeing is fixed here: https://github.com/BerriAI/litellm/pull/2363
- It only occurs when there is a fallback
- It was caused because the
metadata field had a previous_models field which had a circular ref
- Pr fixes this error for Langfuse logging + using LiteLLM DB
How we'll prevent this in the future
- Adding testing to ensure we can run json.dumps() on the callbacks metadata field
Thank you to the people at langfuse for looking into the issue17 replies
LLangfuse
•Created by sumanth balaji on 3/5/2024 in #get-support
DB connections aren’t reused when self hosting Langfuse
on further analysis, this issue seems to arise from LiteLLM proxy layer. I have informed the maintainer, shall coordinate with them to resolve this. Did further testing and seems to not be a problem caused by langfuse
17 replies
LLangfuse
•Created by sumanth balaji on 3/5/2024 in #get-support
DB connections aren’t reused when self hosting Langfuse
any luck ? @Max
17 replies
LLangfuse
•Created by sumanth balaji on 3/5/2024 in #get-support
DB connections aren’t reused when self hosting Langfuse
17 replies
LLangfuse
•Created by sumanth balaji on 3/5/2024 in #get-support
DB connections aren’t reused when self hosting Langfuse
Please update when you can if you are able to find the fix/recreate it. Thank you for the help
17 replies
LLangfuse
•Created by sumanth balaji on 3/5/2024 in #get-support
DB connections aren’t reused when self hosting Langfuse
running the docker file using AWS ECS. just one task
17 replies
LLangfuse
•Created by sumanth balaji on 3/5/2024 in #get-support
DB connections aren’t reused when self hosting Langfuse
yes single instance
17 replies
LLangfuse
•Created by sumanth balaji on 2/12/2024 in #get-support
the following is the logs when debug=
Through further debugging i understand that the error is caused because of the TaskManager in Langfuse not creating threads and sending requests
Below is the logs that should have been seen if successful (which is missing in my ECS service)
DEBUG:langfuse:item size 239
DEBUG:langfuse:joining 1 consumer threads
DEBUG:langfuse:uploading batch of 2 items
DEBUG:langfuse:making request: {"batch": [{"id": "7f9b82e1-260f-41df-9638-1419c09df678", "type": "score-create", "body": {"id": "489af865-52dd-4466-ad73-19f1a193d201", "traceId": "brrrrrruuur", "name": "faithfulness", "value": 1.0}, "timestamp": "2024-02-12T09:49:48.768478Z"}, {"id": "98a501d5-d51b-4b79-b8b8-1864b0978f62", "type": "score-create", "body": {"id": "1f974df8-f288-41e3-944c-d228f0fb0e9b", "traceId": "brrrrrruuur", "name": "context_relevancy", "value": 0.0}, "timestamp": "2024-02-12T09:49:48.768811Z"}], "metadata": {"batch_size": 2, "sdk_integration": "default", "sdk_name": "python", "sdk_version": "2.9.0", "public_key": "pk-lf-071f366b-0f9a-4066-9d11-feb72b49e45f"}} to http://langfuse-258775943.eu-west-1.elb.amazonaws.com/api/public/ingestion
DEBUG:langfuse:received response: {"errors":[],"successes":[{"id":"7f9b82e1-260f-41df-9638-1419c09df678","status":201},{"id":"98a501d5-d51b-4b79-b8b8-1864b0978f62","status":201}]}
DEBUG:langfuse:successfully uploaded batch of 2 items
DEBUG:langfuse:consumer thread 0 joined
What could be reason for the TaskManager to not be able to join threads and make the api request?
My ECS task definition alots the following cpu and memory reservation for this task
"cpu": 1024,
"memoryReservation": 1000,
1 replies