|
Hi [First name] [First name],
|
|
|
|
Exciting news: We’re launching Langfuse AI Gateway today (public beta).
|
|
|
|
|
|
|
Many of you have been asking for a gateway, now it’s here and we are very much looking forward to your feedback on it. Highlights of what we launch today: Fully open source (MIT), built in Rust for performance and reliability, natively integrated with the Langfuse platform, and an ambitious roadmap for the coming weeks based on your input.
|
|
|
|
Why build a gateway?
|
|
AI usage has sprawled in organizations, from coding agent use to custom agents. Keeping track of AI usage in an organization across providers is hard, and in many cases there’s a jungle of issued API keys and no transparency on cost.
|
|
|
|
Langfuse AI Gateway Specs
|
|
The Langfuse AI Gateway is a single entry point for all AI traffic between your apps or coding agents and model providers, and can (opt-in) turn every model call into a Langfuse trace.
|
|
|
|
For builders
|
|
|
|
|
-
See what your calls cost: each trace carries model, tokens, and cost, including coding-agent sessions.
|
|
|
|
|
|
|
|
For team leads and admins
|
|
|
|
|
-
Privacy controls: record usage and cost only (default), full prompts and responses, or nothing at all. You keep your provider contracts and rates, e.g. zero data retention.
|
|
|
|
|
|
Performance and availability
|
-
Built in Rust: <3ms median latency with tracing on and 1,000+ concurrent streams on a single 2 vCPU node. Self-hosted, it’s a single 22MB Docker image with no extra dependencies.
|
-
Supports OpenAI and Anthropic (more coming very soon, let us know what you need).
|
-
Available on Langfuse Cloud or self-hosted. Cloud: no token markup. Recorded calls are billed as regular units; unrecorded calls are free.
|
|
|
|
|
|
|
Org admins can turn it on today. Not an admin? Forward this email to whoever manages your Langfuse org.
|
|
|
This is only the beginning, we have a strong roadmap for the AI Gateway, incl. spend limits (very soon), fallbacks, guardrails, and much more. Feedback? Reply to this email or join the GitHub Discussion.
|
|
|
See a demo and ask questions live during our Langfuse Community Town Hall this Thursday, Nov 12 → Sign up
|
|
|
Also: Next to Langfuse AI Gateway, ClickHouse announces a series of products today across Managed Postgres, Observability (if you like Langfuse, you will like this), Security and AI Engineering. Read more here.
|
|
|
|
See you on Thursday during the Town Hall,
|
|
Marc
|
|
|
PS: If you want to support the launch, feel free to drop a like or comment on our socials (X/LinkedIn).
|
|
|
|
—
|
|
Co-founder & CEO, langfuse.com
|
|
🐧 @marcklingen
|
|
⭐️ github.com/langfuse/langfuse
|
|
|
|
|