Liam•11mo ago

Datasets

Hi, I'm experiencing some issues trying to create a Dataset, My code is very simple and taken from the docs:

import os

import openai
from dotenv import load_dotenv
from langfuse import Langfuse
from langfuse.model import CreateDatasetItemRequest, CreateDatasetRequest

os.chdir("./Evaluation")
load_dotenv("../.env")
# get keys for your project from https://cloud.langfuse.com
os.environ["LANGFUSE_PUBLIC_KEY"] = os.environ.get("DEV_LANGFUSE_PUBLIC_KEY")
os.environ["LANGFUSE_SECRET_KEY"] = os.environ.get("DEV_LANGFUSE_SECRET_KEY")

# your openai key
os.environ["OPENAI_API_KEY"] = os.environ.get("OPENAI_API_KEY")

# if you do not use Langfuse Cloud
os.environ["LANGFUSE_HOST"] = "http://host.docker.internal:3000"
langfuse = Langfuse()

print(openai.__version__)

langfuse.create_dataset(CreateDatasetRequest(name="example_dataset"))

import os

import openai
from dotenv import load_dotenv
from langfuse import Langfuse
from langfuse.model import CreateDatasetItemRequest, CreateDatasetRequest

os.chdir("./Evaluation")
load_dotenv("../.env")
# get keys for your project from https://cloud.langfuse.com
os.environ["LANGFUSE_PUBLIC_KEY"] = os.environ.get("DEV_LANGFUSE_PUBLIC_KEY")
os.environ["LANGFUSE_SECRET_KEY"] = os.environ.get("DEV_LANGFUSE_SECRET_KEY")

# your openai key
os.environ["OPENAI_API_KEY"] = os.environ.get("OPENAI_API_KEY")

# if you do not use Langfuse Cloud
os.environ["LANGFUSE_HOST"] = "http://host.docker.internal:3000"
langfuse = Langfuse()

print(openai.__version__)

langfuse.create_dataset(CreateDatasetRequest(name="example_dataset"))

The last line throws:

pydantic.v1.error_wrappers.ValidationError: 1 validation error for ParsingModel[str]
__root__
  str type expected (type=type_error.str)

pydantic.v1.error_wrappers.ValidationError: 1 validation error for ParsingModel[str]
__root__
  str type expected (type=type_error.str)

7 Replies

Marc•11mo ago

Hi Liam, with local docker setups we often see network config issues causing the Langfuse server to not be available. Can you check whether you get the same error when using Langfuse Cloud (free account)

Liam•11mo ago

It does indeed work in the cloud.

Marc•11mo ago

Good to know! Feel free to share details on your setup, happy to have a look

Liam•11mo ago

I'm not sure which details will be usefull to you, im running my code in a devcontainer with the following dockerfile:

FROM "mcr.microsoft.com/devcontainers/python:3.11-bullseye"
EXPOSE 80
RUN apt-get update
RUN apt-get install -y wget lttng-tools
RUN wget https://packages.microsoft.com/config/debian/12/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
RUN sudo dpkg -i packages-microsoft-prod.deb
RUN rm packages-microsoft-prod.deb
RUN apt-get update
RUN sudo apt-get update && sudo apt-get install -y dotnet-sdk-7.0
RUN sudo apt-get update && sudo apt-get install -y aspnetcore-runtime-7.0

FROM "mcr.microsoft.com/devcontainers/python:3.11-bullseye"
EXPOSE 80
RUN apt-get update
RUN apt-get install -y wget lttng-tools
RUN wget https://packages.microsoft.com/config/debian/12/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
RUN sudo dpkg -i packages-microsoft-prod.deb
RUN rm packages-microsoft-prod.deb
RUN apt-get update
RUN sudo apt-get update && sudo apt-get install -y dotnet-sdk-7.0
RUN sudo apt-get update && sudo apt-get install -y aspnetcore-runtime-7.0

Marc•11mo ago

Do you run Langfuse within the devcontainer (eg via docker compose) or separately? In the latter case, you could try to ping/curl the Langfuse app to see whether it’s accessible

Liam•11mo ago

I had to put this on the backburner for a bit because i had some other work to take care of first, but i've fixed it today. For anyone else seeing this im unsure which change fixed it but i: set up a docker network, made new api keys, and added a langfuse.flush(). Again, not sure which one did the trick but hope it helps 😅

Marc•11mo ago

Thanks for sharing