stupidretard6969
LLangfuse
•Created by Ray on 6/20/2024 in #feature-suggestion
Aggregating user feedback
Hey @Marc
I would like to add to this but in a different way.
The user feedback features from LLM outputs are really good but our org would like to use them in a different manner. Specifically for internal reviews of output.
We want specific people to simply go through a bunch of different outputs and basically just score and give feed back foreach.
Like select these specific traces/generations are sent to a person for review. Each round of review has its own results and feedback from internal reviews can be referenced later.
Before we would paste each output into a google sheets and simply have our reviewers give feedback as comments.
But now we would like to link versioned prompts and outputs to specific feedback and vice versa.
As a custom solution we are planning to just have a webpage internally that loads specific outputs and prompts by calling langfuse API.
Then our reviewers can just give feedback for each output as a custom score/comment.
If this doesnt make much sense then please let me know as I can flesh it out further.
Would love to contribute in anyway to get this idea off the ground as that would literally make langfuse an all in one for my employer.
6 replies