langfuse/langfuse-js

🪢 Langfuse JS/TS SDKs - Instrument your LLM app and get detailed tracing/observability. Works with any LLM or framework

Sign up · Self Host · Demo (live data)

Docs · Report Bug · Feature Request · Changelog · Roadmap · Discord

Langfuse uses Github Discussions for Support and Feature Requests.

Overview

Unmute video for voice-over

langfuse-overview-3min.mp4

Develop

• Observability: Instrument your app and start ingesting traces to Langfuse (Quickstart, Integrations Tracing)
• Langfuse UI: Inspect and debug complex logs (Demo, Tracing)
• Prompt Management: Manage, version and deploy prompts from within Langfuse (Prompt Management)
• Prompt Engineering: Test and iterate on your prompts with the LLM Playground

Monitor

• Analytics: Track metrics (cost, latency, quality) and gain insights from dashboards & data exports (Analytics)
• Evals: Collect and calculate scores for your LLM completions (Scores & Evaluations)
  • Run model-based evaluations (Model-based evaluations) within Langfuse
  • Collect user feedback (User Feedback)
  • Manually score observations in Langfuse (Manual Scores)

Test

• Experiments: Track and test app behaviour before deploying a new version
  • Datasets let you test expected in and output pairs and benchmark performance before deploying (Datasets)
  • Track versions and releases in your application (Experimentation, Prompt Management)

Get started

Langfuse Cloud

Managed deployment by the Langfuse team, generous free-tier (hobby plan), no credit card required.

» Langfuse Cloud

Localhost (docker)

# Clone repository
git clone https://github.com/langfuse/langfuse.git
cd langfuse

# Run server and database
docker compose up -d

→ Learn more about deploying locally

Self-host (docker)

Langfuse is simple to self-host and keep updated. It currently requires only a single docker container. → Self Hosting Instructions

Templated deployments: Railway, GCP Cloud Run, AWS Fargate, Kubernetes and others

Get Started

API Keys

You need a Langfuse public and secret key to get started. Sign up here and find them in your project settings.

Ingesting Data · Instrumenting Your Application

Note: We recommend using our fully async, typed SDKs that allow you to instrument any LLM application with any underlying model. They are available in Python (Decorators) & JS/TS. The SDKs will always be the most fully featured and stable way to ingest data into Langfuse.

You may want to use another integration to get started quickly or implement a use case that we do not yet support. However, we recommend to migrate to the Langfuse SDKs over time to ensure performance and stability.

See the → Quickstart to integrate Langfuse.

Integrations

Integration	Supports	Description
SDK	Python, JS/TS	Manual instrumentation using the SDKs for full flexibility.
OpenAI	Python, JS/TS	Automated instrumentation using drop-in replacement of OpenAI SDK.
Langchain	Python, JS/TS	Automated instrumentation by passing callback handler to Langchain application.
LlamaIndex	Python	Automated instrumentation via LlamaIndex callback system.
Haystack	Python	Automated instrumentation via Haystack content tracing system.
LiteLLM	Python, JS/TS (proxy only)	Use any LLM as a drop in replacement for GPT. Use Azure, OpenAI, Cohere, Anthropic, Ollama, VLLM, Sagemaker, HuggingFace, Replicate (100+ LLMs).
API		Directly call the public API. OpenAPI spec available.

Packages that integrate with Langfuse:

Name	Description
Instructor	Library to get structured LLM outputs (JSON, Pydantic)
Mirascope	Python toolkit for building LLM applications.
AI SDK by Vercel	Typescript SDK that makes streaming LLM outputs super easy.
Flowise	JS/TS no-code builder for customized LLM flows.
Langflow	Python-based UI for LangChain, designed with react-flow to provide an effortless way to experiment and prototype flows.
Superagent	Open Source AI Assistant Framework & API for prototyping and deployment of agents.

Questions and feedback

Ideas and roadmap

• GitHub Discussions
• Feature Requests
• Roadmap

Support and feedback

In order of preference the best way to communicate with us:

• GitHub Discussions: Contribute ideas support requests and report bugs (preferred as we create a permanent, indexed artifact for other community members)
• Discord: community support
• Privately: contact at langfuse dot com

Contributing to Langfuse

• Vote on Ideas
• Raise and comment on Issues
• Open a PR - see CONTRIBUTING.md for details on how to setup a development environment.

License

This repository is MIT licensed, except for the ee folders. See LICENSE and docs for more details.

Misc

GET API to export your data

GET routes to use data in downstream applications (e.g. embedded analytics).

Security & Privacy

We take data security and privacy seriously. Please refer to our Security and Privacy page for more information.

Telemetry

By default, Langfuse automatically reports basic usage statistics of self-hosted instances to a centralized server (PostHog).

This helps us to:

1. Understand how Langfuse is used and improve the most relevant features.
2. Track overall usage for internal and external (e.g. fundraising) reporting.

None of the data is shared with third parties and does not include any sensitive information. We want to be super transparent about this and you can find the exact data we collect here.

You can opt-out by setting TELEMETRY_ENABLED=false.

langfuse-js's Issues

JS SDKs: take env variables for instantiation

Currently, the Langfuse SDK is not able to take environment variables for configuration. Instead, it relies on options on the constructor.

We want to have one function, which takes params: { publicKey?: string; secretKey?: string } & LangfuseCoreOptions and returns the same type.

• This function should have the following behavior:
  • check if values for publicKey, secretKey, or variables within LangfuseCoreOptions were provided.
  • If not, check the environment variables and take values from there.
  • If still no variable was defined, log an error if the input param was required but not provided by either the constructor or the env variable. No exception is thrown.
• use existing getEnv utility to access environment variables
• the following variables are available:
  • LANGFUSE_RELEASE
  • LANGFUSE_PUBLIC_KEY
  • LANGFUSE_SECRET_KEY
  • LANGFUSE_DEBUG
  • LANGFUSE_HOST
• Please adjust all environment variables in the CI and tests to reflect the change.
• The function should be called as first action in the two Langfuse objects (

  langfuse-js/langfuse/src/langfuse.ts

  Line 20 in 04a75b4

  export class Langfuse extends LangfuseCore {

  , https://github.com/langfuse/langfuse-js/blob/04a75b406010368326bf3e12336f2a9bb9d4c210/langfuse-node/src/langfuse-node.ts#L28) and LangfuseWeb (

  langfuse-js/langfuse/src/langfuse.ts

  Lines 76 to 77 in 04a75b4

  	export class LangfuseWeb extends LangfuseWebStateless {
  	private _storage: LangfuseStorage;

  )
  • As of there, the values returned from that function will be used internally

Overview over all constructors:

• Langfuse: constructor(params: { publicKey: string; secretKey: string } & LangfuseOptions)

  • langfuse-js/langfuse/src/langfuse.ts

    Line 20 in 04a75b4

    export class Langfuse extends LangfuseCore {

  • extends LangfuseCore and is the object exposed to SDK users for the backend
• Langfuse (node): constructor(params: { publicKey: string; secretKey: string } & LangfuseOptions)

  • langfuse-js/langfuse-node/src/langfuse-node.ts

    Line 28 in 04a75b4

    constructor(params: { publicKey: string; secretKey: string } & LangfuseOptions) {

  • Backend SDK for users on old node versions, also extends LangfuseCore
• LangfuseCore: constructor(params: { publicKey: string; secretKey: string } & LangfuseCoreOptions)
  • extends LangfuseCoreStateless
  • Contains additional logic to the LangfuseCoreStateless class to improve developer experience
• LangfuseWeb: constructor(params: { publicKey: string } & LangfuseOptions)

  • langfuse-js/langfuse/src/langfuse.ts

    Lines 76 to 77 in 04a75b4

    	export class LangfuseWeb extends LangfuseWebStateless {
    	private _storage: LangfuseStorage;

  • extends LangfuseWebStateless
  • This is the SDK which our users can use in react applications
• LangfuseWebStateless: constructor(params: { publicKey: string } & LangfuseCoreOptions)
  • extends LangfuseCoreStateless
• LangfuseCoreStateless: constructor(params: { publicKey: string; secretKey?: string } & LangfuseCoreOptions)

  • langfuse-js/langfuse-core/src/index.ts

    Line 100 in 04a75b4

    constructor(params: { publicKey: string; secretKey?: string } & LangfuseCoreOptions) {

  • LangfuseCoreStateless is extended by LangfuseCore
  • LangfuseCoreStateless contains all API calls to our server

Circular references in the storage cache when using observeOpenAI

Bug description

So, I've been using traces successfully within an application, all traces and spans get stored correctly and it works great.
However, I recently added the langfuse error handler and started logging errors from it, and noticed that every time traces occur I see the following errors being logged:

2024-05-23 22:26:31 Error while flushing Langfuse TypeError: Converting circular structure to JSON
2024-05-23 22:26:31     --> starting at object with constructor 'Object'
2024-05-23 22:26:31     |     property 'queue' -> object with constructor 'Array'
2024-05-23 22:26:31     |     index 2 -> object with constructor 'Object'
2024-05-23 22:26:31     |     ...
2024-05-23 22:26:31     |     property 'client' -> object with constructor 'Langfuse'
2024-05-23 22:26:31     --- property '_storageCache' closes the circle
2024-05-23 22:26:31     at JSON.stringify (<anonymous>)
2024-05-23 22:26:31     at Langfuse.setPersistedProperty (file:///usr/src/app/node_modules/langfuse/lib/index.mjs:177:50)
2024-05-23 22:26:31     at Langfuse.flush (file:///usr/src/app/node_modules/langfuse-core/lib/index.mjs:598:10)
2024-05-23 22:26:31     at file:///usr/src/app/node_modules/langfuse-core/lib/index.mjs:574:14
2024-05-23 22:26:31     at new Promise (<anonymous>)
2024-05-23 22:26:31     at Langfuse.flushAsync (file:///usr/src/app/node_modules/langfuse-core/lib/index.mjs:572:12)

I don't see what I might be doing wrong that is causing a circular reference to end up in the storage cache, since I'm just either manually creating a trace or a span just with a name and some input and output (which get shown correctly in langfuse), plus the observeOpenAI wrapper for OpenAI completions.

How to reproduce

Traces that are manually created don't trigger the error, neither do spans.
The error appears as soon as it's a Generation automatically created by the observeOpenAI wrapped openAI client.

Related sub-libraries

• All of them
• [X ] langfuse
• langfuse-node

Additional context

Completions in my case are mostly done as streams. Not sure if that info helps. Even completions with tools, since I'm running an Agent with tools with streamed responses.
I'm using langfuse version 3.11.1 from npm.

Thank you for your bug report – we love squashing them!

Langchain integration

Describe the solution you'd like

Add Langchain Integration to the JS/TS SDKs which currently only exists in the Python SDKs.

Why is this important? Reducing the integration effort is key to help more people use Langfuse for their project. The Langchain intgeration (Python only) is currently the fastest way to integrate. It would be awesome if this was also usable for JS/TS Langchain projects.

I'm happy to support on this, dm on Discord

Additional context

• Langchain Callbacks: https://js.langchain.com/docs/modules/callbacks/
• Langfuse Python implementation (probably a lot can be copied/ported)
  • Docs: https://langfuse.com/docs/integrations/langchain
  • Callback handler: https://github.com/langfuse/langfuse-python/blob/main/langfuse/callback.py
  • Tests: https://github.com/langfuse/langfuse-python/blob/main/tests/test_langchain.py

OpenAI response_format in chat completion body breaks observeOpenAI()

Adding response_format to a child OpenAI observation within a trace will cause the observability to fail. Uncomment the specified line to test.

Example adapted from the Get Started docs page

  const langfuse = new Langfuse();
  const openai = new OpenAI();

  const trace = langfuse.trace({ name: "country-poem-generator" });
  const span = trace.span({ name: "poem-span" });

  const poem = (
    await observeOpenAI(openai, {
      parent: span,
      generationName: "generate-poem",
    }).chat.completions.create({
      model: "gpt-3.5-turbo",
      messages: [
        {
          role: "system",
          content: "You are a poet. Create a haiku about this city.",
        },
        { role: "user", content: "France" },
      ],
      // Uncommenting the line below will break the Langfuse observability
      // OpenAI API will respond as desired
      // response_format: { type: "text" },
    })
  ).choices[0].message.content;

  console.log(poem)
  span.end();

[LFJ-2] Test

_{From SyncLinear.com | LFJ-2}

LlamaIndex.TS integration

Is your feature request related to a problem?

Add Llamaindex Integration to the JS/TS SDKs which currently only exists in the Python SDKs.

Additional context

• https://ts.llamaindex.ai/observability/

All Langfuse Issues are in langfuse/langfuse

Hi all, since many of the features overlap across the core application and the client sdks, we centralize issues in the main application repo: langfuse/langfuse (filter by Javascript SDK)

Request to Add Get list of traces Functionality to langfuse-js SDK

Is your feature request related to a problem?

I've noticed that while there is a REST API available for getting the list of traces (GET /api/public/traces), this functionality is not yet available in the SDK.

To retrieve a list of traces, we currently need to directly interact with the REST API, which involves handling HTTP requests and responses manually. This is not as efficient and seamless as using the SDK functions.

Describe the solution you'd like

It would be highly beneficial to include a function in the langfuse-js SDK that encapsulates this API call. This addition would streamline development by reducing the need for boilerplate code and by providing a more consistent and simplified interface.

Describe alternatives you've considered

Related sub-libraries

• All of them
• langfuse
• langfuse-node

Additional context

Thank you for your feature request – we love each and every one!