LLM Observability
Token overruns, slow API calls, broken call chains — you need more than alerts. You need to see the real cost and behavior behind every request. TrueWatch LLM observability unifies token usage, cost trends, call traces, and root cause analysis so every decision is backed by data.
product features
LLM token monitoring and request volume tracking
The real-time dashboard shows cumulative Total Token consumption with a Prompt Token vs. Completion Token breakdown — split by model and application for average Input/Output token counts. Evaluate the cost and complexity of every LLM call during development, before unexpected bills appear in production. Set usage thresholds to trigger instant alerts when a single request goes over limit.
LLM cost optimization and spend trend analysis
LLM API call counts, average cost per call, and cumulative spend — all in one chart. Trend lines give product and finance teams a clear view of where costs are heading, so budget planning is data-driven rather than guesswork. Get instant notifications when cost or request volume in any dimension hits a threshold, before a traffic spike turns into a budget overrun.
LLM tracing and prompt monitoring across call history
Use Trace IDs to look up the complete metadata for every inference request — start and end time, model version, temperature, prompt length, and output summary. Filter for failed or timed-out requests in one click, then replay the issue using input/output logs. Export history as CSV or JSON for cross-team sharing and downstream analysis.
Flame graph root cause analysis for LLM latency and performance
The trace detail view automatically renders each sub-Span's latency distribution as a flame graph — so you can see at a glance which call stage is consuming the most time. P75, P90, and P99 response time curves help you distinguish sporadic slow requests from systemic LLM performance bottlenecks. Once you've pinpointed the issue, jump directly to the relevant code or config — diagnose and fix without leaving the platform.
