Best LLM Observability Tools in 2026: LangSmith, Helicone, Humanloop, PromptLayer and Alternatives

Two of the four tools in this comparison have changed status: Humanloop is shut down and Helicone is in maintenance mode. Here is how LangSmith and PromptLayer stack up, and which tools to pick instead.

Key takeaways

  • Humanloop is gone. Its platform and API were sunset on September 8, 2025, after its founders joined Anthropic, so it is not a 2026 option.
  • Helicone was acquired by Mintlify on March 3, 2026 and is in maintenance mode: security fixes, bug fixes and new-model support continue, but active feature development has ended.
  • LangSmith is the deepest choice if you build on LangChain or LangGraph. Watch the seat and trace metering.
  • PromptLayer fits teams that want prompt management, evals and logging in one place, with a clear $0 / $49 / $500 price ladder.
  • If you are starting fresh, shortlist Langfuse, Braintrust, Arize Phoenix and W&B Weave next to LangSmith and PromptLayer.

A note on how this guide was put together

The title asks whether we tested these four tools. We did not run a hands-on benchmark, and I would be suspicious of any 2026 article claiming to have tested Humanloop, because you cannot sign up for it anymore. What follows is a research-based comparison built from vendor documentation, pricing pages, and third-party reviews from Vellum, Logz.io, Tsuga and others. Where a claim comes from a vendor's own blog, I say so.

That framing matters more than usual here. Half the tools in the title have changed status, and a buyer comparing them as if all four were healthy would make a bad decision.

What LLM observability actually covers

Classic logging tells you when something broke. LLM apps are different because a run can finish with a 200 status code and still be wrong. The model invents a fact, calls the wrong tool, or drifts from your policy. Nothing throws an error.

So a useful observability tool needs to do a few things:

  • Capture traces of every step in a chain or agent run, including tool calls and retrieval.
  • Track cost, latency and token usage per request, user and prompt version.
  • Run evaluations on production traffic, either with LLM-as-a-judge or human review.
  • Turn bad runs into test cases so the same failure does not come back.
  • Manage prompts with versioning, so you know which change caused which regression.

The LangChain guide makes a fair point here (though it is vendor-authored): most teams can answer "what went wrong" and then stall on who reviews it, how it becomes test coverage, and how it reaches production. Tracing is the easy part.

LangChain's guide to LLM observability tools for production AI agents, with a comparison of nine platforms

Status check: the four tools in the title

ToolStatus in late 2026TypePricing (verify before buying)Best for
LangSmithActiveTracing, evals, monitoringDeveloper $0/seat, Plus $39/seat/mo, Enterprise customLangChain and LangGraph teams
HeliconeMaintenance modeProxy gateway and loggingFree, Pro $79/mo, Team $799/mo (per LangChain guide)Existing users needing cost tracking and caching
HumanloopDiscontinuedPrompt management and evalsNot availableNobody, migrate off
PromptLayerActivePrompt management, evals, loggingFree, Pro $49/mo, Team $500/mo, Enterprise customPrompt-centric teams, including non-engineers

LangSmith

Favicon of LangSmith

LangSmith

Debug, test, and monitor LangChain applications
View more
Screenshot of LangSmith website

LangSmith comes from the LangChain team, and the depth follows the framework. Runs render as full execution trees, and in LangGraph you can see the state at each node instead of inferring it. A traceable wrapper covers code outside LangChain, and the platform is OpenTelemetry-compliant according to Logz.io.

Vellum's review gave it a score of 84 and called out these strengths:

  • Run-level debugging for graph-based agents.
  • Online LLM-as-a-judge evaluators that score production runs at a configurable sampling rate.
  • Annotation queues that route selected runs to human reviewers and feed evaluation datasets.
  • An LLM Gateway with fallback chains, still in private beta.

One claim to treat carefully: a Data Science Collective article says LangSmith runs on a purpose-built store called SmithDB with sub-second trajectory queries. That number is LangChain's own, so I would test it on your data volume rather than take it as given.

LangSmith pricing and pitfalls

The LangChain guide, dated August 16, 2026, lists Developer at $0 per seat per month, Plus at $39 per seat per month, and Enterprise as custom. Confirm on the LangSmith site before budgeting.

The pricing is where I would slow down. Vellum flags several issues:

  • Seats and traces meter separately, and there is no read-only seat. A reviewer who only reads results costs the same as an engineer.
  • Trace counts include every run inside a chain, so a retrieval, rerank and generation pipeline bills three times per query.
  • Self-hosting requires Enterprise.
  • Trace depth is thinner outside LangChain and LangGraph.

That last point is the real decision. Choosing LangSmith is partly a bet on a framework. If your stack is LangGraph, the bet is easy. If you run a custom orchestrator or a mix of SDKs, you will get less from it.

If you are building on the framework itself, you may also want to look at LangChain directly.

Favicon of LangChain

LangChain

Framework for building LLM-powered applications
View more
Screenshot of LangChain website

Helicone

Favicon of Helicone

Helicone

AI Gateway & LLMOps platform for routing, debugging, and mon
View more
Screenshot of Helicone website

Helicone took a deliberately lightweight approach: change your base URL, route requests through its proxy, and get logging, cost analytics, caching and routing. Product Hunt reviewers praised the quick setup and clean interface. It integrates with OpenAI, Anthropic, Gemini, OpenRouter, the Vercel AI SDK, Together AI, AWS Bedrock, Groq, LiteLLM and Azure OpenAI.

The catch is the status. Mintlify announced it was acquiring Helicone on March 3, 2026. Per Mintlify's announcement, security updates, bug fixes and new-model support continue, but active feature development has ended. Logz.io's list labels it "Hosted, maintenance mode." A third-party vendor wiki (druce.ai, re-checked July 27, 2026) reports no sunset date and says the team is helping customers migrate away. Treat that last part as secondary, since it is not from Helicone or Mintlify.

The LangChain guide still lists Helicone at Free, Pro $79/month and Team $799/month. That guide is vendor-authored and I could not verify the current prices, so check helicone.ai/pricing.

My take: if you already use Helicone and it works, there is no emergency. But I would not start a new project on a tool whose roadmap has stopped. A proxy also sits in your request path, which is a real dependency to take on for a product with no new features coming.

Humanloop

Favicon of Humanloop

Humanloop

Prompt versioning and monitoring platform
View more
Screenshot of Humanloop website

Humanloop used to be a popular choice for prompt versioning, evaluation and monitoring. Then it was an acqui-hire. The three co-founders (Raza Habib, Peter Hayes and Jordan Burgess) and most of the engineering and research team joined Anthropic, as reported by Silicon Republic. Humanloop's own changelog announced that the platform and API would be sunset on September 8, 2025.

So there is nothing to review. If you still have prompts, datasets or evaluation configs there, they should already have been exported. Weights & Biases has positioned Weave as its migration alternative, and Langfuse, Braintrust and PromptLayer are all reasonable landing spots depending on how much you used the prompt-management side.

PromptLayer

Favicon of PromptLayer

PromptLayer

Prompt tracking and version control for AI applications
View more
Screenshot of PromptLayer website

PromptLayer started as a request logger and grew into a prompt-management platform covering evaluations, observability, dataset management and prompt chaining. Its angle is that prompts are a shared asset, so product managers and domain experts can edit and version them without a deploy.

The pricing from the official page (I did not capture a date, so recheck it):

PlanPriceKey limits
Free$0/month5 users, 2.5k requests/month, 10 prompts, 10 playground runs/day
Pro$49/monthUnlimited prompts, playgrounds and workspaces, 150MB per dataset, $0.003 overage per transaction
Team$500/month25 users, 100k+ requests/month, 1GB per dataset, webhooks, $0.002 overage
EnterpriseCustomRBAC, SSO, deployment approvals, HIPAA with BAA, self-hosting on GCP, AWS or Azure, EU cloud

Free, Pro and Team are US cloud-hosted SaaS only. If you have EU data residency needs, you are looking at Enterprise. PromptLayer lists SOC 2 Type 2, GDPR, HIPAA and CCPA certifications.

What I could not verify: how deep its tracing goes for multi-step agents, and whether it supports OpenTelemetry. If either matters to you, ask before you commit. For pure agent debugging, LangSmith or Arize Phoenix will likely give you more trace detail. For prompt workflow with non-engineers involved, PromptLayer is a stronger fit.

Replacements worth shortlisting

Because two of the four named tools are out of the running for new projects, here is what other 2026 lists keep naming. Logz.io, Tsuga and the LangChain guide all point at roughly the same set.

Langfuse

Favicon of Langfuse

Langfuse

Open-source LLM observability and prompt engineering platfor
View more
Screenshot of Langfuse website

Open-source tracing, prompt management and evals. It can be self-hosted, which is the main reason teams pick it over LangSmith, where self-hosting requires Enterprise. It is framework-agnostic, so it suits mixed stacks.

Braintrust

Favicon of Braintrust

Braintrust

End-to-end prompt management and evaluation platform
View more
Screenshot of Braintrust website

Evaluation-first. If your pain is regression testing and turning production failures into datasets, it is built around that workflow.

Arize Phoenix and AX

Favicon of Arize AI

Arize AI

End-to-end LLM observability and agent evaluation platform
View more
Screenshot of Arize AI website

OpenTelemetry-based tracing with strong evaluation tooling. Phoenix is open source, and AX is the commercial platform for larger teams.

Weights & Biases Weave

Favicon of Weights & Biases Weave

Weights & Biases Weave

Track and evaluate LLM applications
View more
Screenshot of Weights & Biases Weave website

A natural fit if your team already uses W&B for experiment tracking. It is also the vendor's stated path for former Humanloop users.

Other options

Vellum ([tool:vellum]) covers prompt engineering and evaluation in one workspace. Promptfoo ([tool:promptfoo]) is an open-source testing framework that pairs well with a tracing tool rather than replacing one. LangWatch ([tool:langwatch]) focuses on simulated-user testing for agents.

Favicon of Vellum

Vellum

Prompt engineering and model evaluation workspace
View more
Screenshot of Vellum website
Favicon of Promptfoo

Promptfoo

Open-source LLM testing and evaluation framework
View more
Screenshot of Promptfoo website
Favicon of LangWatch

LangWatch

Test AI agents with simulated users, prevent regressions in
View more
Screenshot of LangWatch website

Side-by-side comparison

ToolOpen source / self-hostStrongest atMain drawback
LangSmithSelf-host only on EnterpriseAgent trace depth, online evals, annotation queuesPer-seat plus per-trace metering, framework bet
PromptLayerSelf-host on EnterprisePrompt versioning, non-engineer workflowsAgent tracing depth unverified
HeliconeSelf-hosted gateway optionCost tracking, caching, routingMaintenance mode, no new features
LangfuseYesOpen-source tracing and promptsYou run more of it yourself if self-hosting
BraintrustNoEvals and regression testingLess focused on raw gateway features
Arize PhoenixYes (Phoenix)OpenTelemetry tracing and evalsTwo products (Phoenix and AX) to understand
W&B WeavePart of W&BTeams already on W&BLess compelling outside that ecosystem

How to choose

Start with your framework, then your hosting constraints, then who touches prompts.

  • You build on LangGraph: LangSmith. Budget carefully and model your trace counts first.
  • You need self-hosting without an enterprise contract: Langfuse or Arize Phoenix.
  • Product managers edit prompts: PromptLayer, or Langfuse's prompt management.
  • Your problem is regressions, not debugging: Braintrust, possibly alongside Promptfoo.
  • You are on Helicone today: no rush, but plan a migration. Pick a tool that supports OpenTelemetry so you are not locked in again.

A practical test: instrument one real agent flow in two candidates and compare what you can see when a run fails. Count how many traces a single user query generates, then multiply by your monthly volume. That one calculation has more influence on your bill than the headline plan price, especially with LangSmith's per-run counting.

Observability is not the same as AI search visibility

One last distinction, since the terms get mixed up. LLM observability watches your own application's model calls. It says nothing about how ChatGPT, Gemini or Perplexity describe your brand to customers. If that second question is the one on your mind, tools like Promptwatch track how AI engines mention and cite a brand, which is a separate job from tracing your own agents. The GEO software directory at bestgeosoftware.com lists others in that category.

Favicon of Promptwatch

Promptwatch

Track and optimize your brand visibility in AI search engines
View more
Screenshot of Promptwatch website

Bottom line

The honest 2026 shortlist is LangSmith and PromptLayer from the original four, plus Langfuse, Braintrust and Arize Phoenix. Helicone is fine to keep if you already run it, and Humanloop is simply gone. Verify every price on the vendor's page before you commit, because several figures here come from third-party summaries or vendor-authored comparisons.

Data Science Collective article on top LLM observability platforms in 2026

Share:

© 2026 Surferstack · Find the best Marketing tools for your GTM motion · RSS

Surferstack is an affiliate review site. When you click links to vendors or buy through links on our site, we may earn an affiliate commission at no extra cost to you.

Surferstack is a review website based on user reviews on Reddit and G2, and on publicly available information. We keep everything as up to date as possible, but pricing and features can change. Always confirm the details with the vendor before purchasing.