Summary evaluators. f1 score

How to calculate and display f1 score on langfuse?

Solution:

Hi @maximlupey, summary evaluators bound to dataset runs are on our roadmap, tracking this here: https://github.com/orgs/langfuse/discussions/2511 Feel free to add your thoughts or ideas...

GitHub

Scoring dataset runs, e.g. precision, recall, f-value · langfuse · ...

Describe the feature or potential improvement In LangSmith, for example, there is a feature to get precision, recall, f-value in experiment feature on a dataset. https://docs.smith.langchain.com/ho...

Jump to solution

2 Replies

maximlupey•2mo ago

Is there support for summary evaluators?

Solution

Marc•2mo ago

Hi @maximlupey, summary evaluators bound to dataset runs are on our roadmap, tracking this here: https://github.com/orgs/langfuse/discussions/2511 Feel free to add your thoughts or ideas

GitHub

Scoring dataset runs, e.g. precision, recall, f-value · langfuse · ...