Hey all! I'm using Langfuse with LiteLLM
Hey all! I'm using Langfuse with LiteLLM (using success_callback set to
langfuse) and I am getting the following error: langfuse.request.APIError: {'message': 'Invalid public key'} (401): None
Does anyone know why this is happening?25 Replies
These are the docs I'm following, by the way: https://litellm.vercel.app/docs/proxy/logging#logging-proxy-inputoutput---langfuse
🔎 Logging - Custom Callbacks, Langfuse, s3 Bucket, Sentry, OpenTele...
Log Proxy Input, Output, Exceptions using Custom Callbacks, Langfuse, OpenTelemetry, LangFuse, DynamoDB, s3 Bucket
Figured it out. I selected a us based location for langfuse, and had to set the "LANGFUSE_HOST" environment variable in addition to my public and secret keys for it to work. I figured this out by searching the LiteLLM repo for "langfuse" and saw an option for setting that environment variable (see this search https://github.com/search?q=repo%3ABerriAI%2Flitellm+cloud.langfuse&type=code), which wasn't in the documentation
GitHub
Build software better, together
GitHub is where people build software. More than 100 million people use GitHub to discover, fork, and contribute to over 420 million projects.
thanks for sharing, adding it to the docs rn
sorry! I had this issue as well and I forgot to document it.
no worries, if you spot other issues or things that'd be helpful to add to the litellm docs, let us know
The documentation is currently quite purist
what's the tl;dr on how the pub/secret key auth works?
get created in an api route, then secret key hashed and saved to db, displayed once in UI
how are both used on the HTTP request going to Langfuse? 🙂
is the pubkey sent in the request anywhere?
yes, basic auth,
base64(pk:sk)
+ pk as header for debugging purposes as it is not sensitiveah sweet. So you could make the
pk a JWT, and set the aud to the Langfuse host it's intended for.
if you see somebody sends a JWT that isn't meant for your server, you can reject it with a message saying it's the wrong server.Do you want to improve the error messages if the wrong host is specified?
I'd suggest to just add "confirm that you se the correct host" to the 401 error message lol
But we could technically check if the pk exists on the instance
But this could be a security problem in the end, the easy route is the safe route I guess
yeah the JWT is just a lazy way to avoid needing to make an expensive ish call back to the database
since you can validate the JWT first before calling out to the db.
if the JWT isn't valid for the host, no need to waste time with a query to the db.
fair, currently thinking about using redis to cache api keys, has not been an issue though as the db query is relatively cheap since the index works well
the main pro with JWTs is it's super easy to offload that first check to the edge (like on a Worker or Lambda@Edge); scaling redis out like that sounds like more work.
it just depends on how much you expect to get DDoS'd over time.
my other pro-JWT opinion is that you can generate them now but opt to just use them like a random string for now.
so it won't take up any real dev time now, and is just for future proofing
agree, considered it in the past but went the faster route, can always switch
ddos has not been an issue luckily
(cloudflare is amazing)