Matt•5mo ago

FYI, I am getting wonderful tracing for

FYI, I am getting wonderful tracing for the whole program by using this code... but I need to break out each input into its own trace:

@observe()
def parent_function():
    langfuse_callback_handler = langfuse_context.get_current_llama_index_handler()
    langfuse_context.update_current_trace(
        name='rag-eval',
        user_id='rag-evaluator'
    )
    Settings.callback_manager = CallbackManager([langfuse_callback_handler])

@observe()
def parent_function():
    langfuse_callback_handler = langfuse_context.get_current_llama_index_handler()
    langfuse_context.update_current_trace(
        name='rag-eval',
        user_id='rag-evaluator'
    )
    Settings.callback_manager = CallbackManager([langfuse_callback_handler])

7 Replies

Matt•5mo ago

Oh also, I have found that creating a new instance of the callbackhandler makes it difficult/impossible to get the trace id in order to add new spans to the trace. Using get_current_llama_index_handler works better... do you know a workaround for that?

Marc•5mo ago

dont do this langfuse = Langfuse() root_trace = langfuse.trace(name='rag-eval', user_id='rag-evaluator') langfuse_callback_handler.set_root(root_trace) this is mixing up the lowlevel sdk and the decorator thereby you'll get lots of duplicates i guess

from langfuse.decorators import langfuse_context, observe

def parent_function(input):
    for i in input:
        llama_index_function(i.question)


@observe()
def llama_index_function(question):
    #new trace should begin now

    ###the next five lines are generating traces all over the place
    langfuse_callback_handler = LlamaIndexCallbackHandler()
    Settings.callback_manager = CallbackManager([langfuse_callback_handler])
    langfuse = Langfuse()
    root_trace = langfuse.trace(name='rag-eval', user_id='rag-evaluator')
    langfuse_callback_handler.set_root(root_trace)
    validate(question) # span is created inside this function
    response = send_question_to_llama_index(question) #normal llama index generations created here
    validate(response) # span is created inside this function


    return response

def validate(question):
    trace = langfuse.trace(id=langfuse_context.get_current_trace_id())
    span = trace.span(name="validation span", input=question)
    # validate happens

    span.end()

from langfuse.decorators import langfuse_context, observe

def parent_function(input):
    for i in input:
        llama_index_function(i.question)


@observe()
def llama_index_function(question):
    #new trace should begin now

    ###the next five lines are generating traces all over the place
    langfuse_callback_handler = LlamaIndexCallbackHandler()
    Settings.callback_manager = CallbackManager([langfuse_callback_handler])
    langfuse = Langfuse()
    root_trace = langfuse.trace(name='rag-eval', user_id='rag-evaluator')
    langfuse_callback_handler.set_root(root_trace)
    validate(question) # span is created inside this function
    response = send_question_to_llama_index(question) #normal llama index generations created here
    validate(response) # span is created inside this function


    return response

def validate(question):
    trace = langfuse.trace(id=langfuse_context.get_current_trace_id())
    span = trace.span(name="validation span", input=question)
    # validate happens

    span.end()

this looks good, if you remove the low-level related lines you should be good

Matt•5mo ago

removing those lines improves things. I'm still not able to add a span to the trace when I instantiate a new LlamaIndexCallbackHandler. I was able to do this previously when using the get_current_llama_index_handler function and then running Langfuse().get_current_trace_id() but that returns None now and so a new trace is recorded when I manually create spans I suspect that's flushing related?

Marc•5mo ago

mhh, would love to have a look at this together chat 5 min about this tomorrow? going to sleep now (currently in CET)

Marc•5mo ago

https://cal.com/marc-kl/5

Cal.com

5 Min Meeting | Marc Klingen | Cal.com

5 Min Meeting

Matt•5mo ago

Thanks for your time, Marc. I scheduled a 5 min call tomorrow. I hope that time works considering we're 8 hours apart.

Matt•5mo ago

I've continued to work on this and am now having a slightly different issue. I am now using the @observe() decorator on each function and using langfuse_context.update_current_observation to manage the spans. I run the same function three times in one execution, trying to generate three traces. The first trace includes most of what I need now (although the spans and generations are out of order). But the next two are broken out into rag-eval which contains my custom observations and LlamaIndex_chat which seems to be auto-generated by the llama index handler and includes the LLM generations. Does this ring any bells for you? (screenshots included) Finally, I notice that the retrieve span is only created when I put the @observe() decorator at my top level function.