Key takeaways
- Humanloop is gone. Its platform and API were sunset on September 8, 2025, after its founders joined Anthropic, so it is not a 2026 option.
- Helicone was acquired by Mintlify on March 3, 2026 and is in maintenance mode: security fixes, bug fixes and new-model support continue, but active feature development has ended.
- LangSmith is the deepest choice if you build on LangChain or LangGraph. Watch the seat and trace metering.
- PromptLayer fits teams that want prompt management, evals and logging in one place, with a clear $0 / $49 / $500 price ladder.
- If you are starting fresh, shortlist Langfuse, Braintrust, Arize Phoenix and W&B Weave next to LangSmith and PromptLayer.
A note on how this guide was put together
The title asks whether we tested these four tools. We did not run a hands-on benchmark, and I would be suspicious of any 2026 article claiming to have tested Humanloop, because you cannot sign up for it anymore. What follows is a research-based comparison built from vendor documentation, pricing pages, and third-party reviews from Vellum, Logz.io, Tsuga and others. Where a claim comes from a vendor's own blog, I say so.
That framing matters more than usual here. Half the tools in the title have changed status, and a buyer comparing them as if all four were healthy would make a bad decision.
What LLM observability actually covers
Classic logging tells you when something broke. LLM apps are different because a run can finish with a 200 status code and still be wrong. The model invents a fact, calls the wrong tool, or drifts from your policy. Nothing throws an error.
So a useful observability tool needs to do a few things:
- Capture traces of every step in a chain or agent run, including tool calls and retrieval.
- Track cost, latency and token usage per request, user and prompt version.
- Run evaluations on production traffic, either with LLM-as-a-judge or human review.
- Turn bad runs into test cases so the same failure does not come back.
- Manage prompts with versioning, so you know which change caused which regression.
The LangChain guide makes a fair point here (though it is vendor-authored): most teams can answer "what went wrong" and then stall on who reviews it, how it becomes test coverage, and how it reaches production. Tracing is the easy part.

Status check: the four tools in the title
| Tool | Status in late 2026 | Type | Pricing (verify before buying) | Best for |
|---|---|---|---|---|
| LangSmith | Active | Tracing, evals, monitoring | Developer $0/seat, Plus $39/seat/mo, Enterprise custom | LangChain and LangGraph teams |
| Helicone | Maintenance mode | Proxy gateway and logging | Free, Pro $79/mo, Team $799/mo (per LangChain guide) | Existing users needing cost tracking and caching |
| Humanloop | Discontinued | Prompt management and evals | Not available | Nobody, migrate off |
| PromptLayer | Active | Prompt management, evals, logging | Free, Pro $49/mo, Team $500/mo, Enterprise custom | Prompt-centric teams, including non-engineers |
LangSmith
LangSmith comes from the LangChain team, and the depth follows the framework. Runs render as full execution trees, and in LangGraph you can see the state at each node instead of inferring it. A traceable wrapper covers code outside LangChain, and the platform is OpenTelemetry-compliant according to Logz.io.
Vellum's review gave it a score of 84 and called out these strengths:
- Run-level debugging for graph-based agents.
- Online LLM-as-a-judge evaluators that score production runs at a configurable sampling rate.
- Annotation queues that route selected runs to human reviewers and feed evaluation datasets.
- An LLM Gateway with fallback chains, still in private beta.
One claim to treat carefully: a Data Science Collective article says LangSmith runs on a purpose-built store called SmithDB with sub-second trajectory queries. That number is LangChain's own, so I would test it on your data volume rather than take it as given.
LangSmith pricing and pitfalls
The LangChain guide, dated August 16, 2026, lists Developer at $0 per seat per month, Plus at $39 per seat per month, and Enterprise as custom. Confirm on the LangSmith site before budgeting.
The pricing is where I would slow down. Vellum flags several issues:
- Seats and traces meter separately, and there is no read-only seat. A reviewer who only reads results costs the same as an engineer.
- Trace counts include every run inside a chain, so a retrieval, rerank and generation pipeline bills three times per query.
- Self-hosting requires Enterprise.
- Trace depth is thinner outside LangChain and LangGraph.
That last point is the real decision. Choosing LangSmith is partly a bet on a framework. If your stack is LangGraph, the bet is easy. If you run a custom orchestrator or a mix of SDKs, you will get less from it.
If you are building on the framework itself, you may also want to look at LangChain directly.
Helicone
Helicone took a deliberately lightweight approach: change your base URL, route requests through its proxy, and get logging, cost analytics, caching and routing. Product Hunt reviewers praised the quick setup and clean interface. It integrates with OpenAI, Anthropic, Gemini, OpenRouter, the Vercel AI SDK, Together AI, AWS Bedrock, Groq, LiteLLM and Azure OpenAI.
The catch is the status. Mintlify announced it was acquiring Helicone on March 3, 2026. Per Mintlify's announcement, security updates, bug fixes and new-model support continue, but active feature development has ended. Logz.io's list labels it "Hosted, maintenance mode." A third-party vendor wiki (druce.ai, re-checked July 27, 2026) reports no sunset date and says the team is helping customers migrate away. Treat that last part as secondary, since it is not from Helicone or Mintlify.
The LangChain guide still lists Helicone at Free, Pro $79/month and Team $799/month. That guide is vendor-authored and I could not verify the current prices, so check helicone.ai/pricing.
My take: if you already use Helicone and it works, there is no emergency. But I would not start a new project on a tool whose roadmap has stopped. A proxy also sits in your request path, which is a real dependency to take on for a product with no new features coming.
Humanloop
Humanloop used to be a popular choice for prompt versioning, evaluation and monitoring. Then it was an acqui-hire. The three co-founders (Raza Habib, Peter Hayes and Jordan Burgess) and most of the engineering and research team joined Anthropic, as reported by Silicon Republic. Humanloop's own changelog announced that the platform and API would be sunset on September 8, 2025.
So there is nothing to review. If you still have prompts, datasets or evaluation configs there, they should already have been exported. Weights & Biases has positioned Weave as its migration alternative, and Langfuse, Braintrust and PromptLayer are all reasonable landing spots depending on how much you used the prompt-management side.
PromptLayer

PromptLayer started as a request logger and grew into a prompt-management platform covering evaluations, observability, dataset management and prompt chaining. Its angle is that prompts are a shared asset, so product managers and domain experts can edit and version them without a deploy.
The pricing from the official page (I did not capture a date, so recheck it):
| Plan | Price | Key limits |
|---|---|---|
| Free | $0/month | 5 users, 2.5k requests/month, 10 prompts, 10 playground runs/day |
| Pro | $49/month | Unlimited prompts, playgrounds and workspaces, 150MB per dataset, $0.003 overage per transaction |
| Team | $500/month | 25 users, 100k+ requests/month, 1GB per dataset, webhooks, $0.002 overage |
| Enterprise | Custom | RBAC, SSO, deployment approvals, HIPAA with BAA, self-hosting on GCP, AWS or Azure, EU cloud |
Free, Pro and Team are US cloud-hosted SaaS only. If you have EU data residency needs, you are looking at Enterprise. PromptLayer lists SOC 2 Type 2, GDPR, HIPAA and CCPA certifications.
What I could not verify: how deep its tracing goes for multi-step agents, and whether it supports OpenTelemetry. If either matters to you, ask before you commit. For pure agent debugging, LangSmith or Arize Phoenix will likely give you more trace detail. For prompt workflow with non-engineers involved, PromptLayer is a stronger fit.
Replacements worth shortlisting
Because two of the four named tools are out of the running for new projects, here is what other 2026 lists keep naming. Logz.io, Tsuga and the LangChain guide all point at roughly the same set.
Langfuse
Open-source tracing, prompt management and evals. It can be self-hosted, which is the main reason teams pick it over LangSmith, where self-hosting requires Enterprise. It is framework-agnostic, so it suits mixed stacks.
Braintrust

Evaluation-first. If your pain is regression testing and turning production failures into datasets, it is built around that workflow.
Arize Phoenix and AX
OpenTelemetry-based tracing with strong evaluation tooling. Phoenix is open source, and AX is the commercial platform for larger teams.
Weights & Biases Weave

A natural fit if your team already uses W&B for experiment tracking. It is also the vendor's stated path for former Humanloop users.
Other options
Vellum ([tool:vellum]) covers prompt engineering and evaluation in one workspace. Promptfoo ([tool:promptfoo]) is an open-source testing framework that pairs well with a tracing tool rather than replacing one. LangWatch ([tool:langwatch]) focuses on simulated-user testing for agents.
Side-by-side comparison
| Tool | Open source / self-host | Strongest at | Main drawback |
|---|---|---|---|
| LangSmith | Self-host only on Enterprise | Agent trace depth, online evals, annotation queues | Per-seat plus per-trace metering, framework bet |
| PromptLayer | Self-host on Enterprise | Prompt versioning, non-engineer workflows | Agent tracing depth unverified |
| Helicone | Self-hosted gateway option | Cost tracking, caching, routing | Maintenance mode, no new features |
| Langfuse | Yes | Open-source tracing and prompts | You run more of it yourself if self-hosting |
| Braintrust | No | Evals and regression testing | Less focused on raw gateway features |
| Arize Phoenix | Yes (Phoenix) | OpenTelemetry tracing and evals | Two products (Phoenix and AX) to understand |
| W&B Weave | Part of W&B | Teams already on W&B | Less compelling outside that ecosystem |
How to choose
Start with your framework, then your hosting constraints, then who touches prompts.
- You build on LangGraph: LangSmith. Budget carefully and model your trace counts first.
- You need self-hosting without an enterprise contract: Langfuse or Arize Phoenix.
- Product managers edit prompts: PromptLayer, or Langfuse's prompt management.
- Your problem is regressions, not debugging: Braintrust, possibly alongside Promptfoo.
- You are on Helicone today: no rush, but plan a migration. Pick a tool that supports OpenTelemetry so you are not locked in again.
A practical test: instrument one real agent flow in two candidates and compare what you can see when a run fails. Count how many traces a single user query generates, then multiply by your monthly volume. That one calculation has more influence on your bill than the headline plan price, especially with LangSmith's per-run counting.
Observability is not the same as AI search visibility
One last distinction, since the terms get mixed up. LLM observability watches your own application's model calls. It says nothing about how ChatGPT, Gemini or Perplexity describe your brand to customers. If that second question is the one on your mind, tools like Promptwatch track how AI engines mention and cite a brand, which is a separate job from tracing your own agents. The GEO software directory at bestgeosoftware.com lists others in that category.

Bottom line
The honest 2026 shortlist is LangSmith and PromptLayer from the original four, plus Langfuse, Braintrust and Arize Phoenix. Helicone is fine to keep if you already run it, and Humanloop is simply gone. Verify every price on the vendor's page before you commit, because several figures here come from third-party summaries or vendor-authored comparisons.








