Overview
Observability is a retrospective tool for organization administrators. It gives you a structured view of past threads, playbook runs, agents, dashboards, and connectors across your workspace. After threads complete, TextQL automatically analyzes them for quality signals: gaps in context, execution errors, signs of user frustration, potential inaccuracies. These appear as warnings attached to individual threads, which you can drill into to understand exactly what happened and where things broke down. The goal is to give administrators a systematic way to:- Monitor analysis quality across your organization over time
- Identify patterns — recurring warning types, problematic connectors, or topic areas where Ana consistently struggles
- Apply targeted fixes — improving ontology coverage, enriching the Ontology, or adjusting connector configuration based on what you find
- Drive user education — spotting where users are asking questions Ana cannot yet answer well, and helping them prompt more effectively in the meantime
Observability is available to organization administrators only. If you do not see it in your sidebar, confirm that you have admin privileges.
Time Range
Use the time range selector (top right) to control the window of data shown across all tabs. Options are Last 7 days, Last 14 days, Last 30 days, and Last 90 days.Overview Tab
The Overview tab is the main monitoring surface. It shows thread and playbook activity, warning trends, and a full searchable table of runs.Stats Bar
Summary cards at the top show aggregate metrics for the selected time range, each with a delta vs. the previous equivalent period:- Total Runs — all thread runs across the workspace
- Playbook Runs — runs triggered by scheduled playbooks
- Slack — runs initiated via the Slack integration
- Feed Agents — visible when Feed Agents are enabled or when feed agent activity exists in the selected range
- Warnings — total warnings detected, with a warn rate percentage
Run Volume Chart
A bar chart breaking down run volume over time, color-coded by source: Threads, Playbooks, Slack, and Feed Agents.Warning Breakdown
Next to the run volume chart, the Warning Breakdown panel ranks the most frequent warning types by count. Click any warning type to filter the thread table below to show only threads with that warning. Warnings are color-coded by category:- Causes (blue) — configuration issues you can fix
- Symptoms (orange) — execution problems
- Outcomes (red) — user-facing impact
Warning Types
Causes — Configuration issues you can fix
These warnings indicate that the agent lacked the information it needed. Fixing them usually means improving your ontology or semantic layer.Symptoms — Execution problems
These warnings indicate something went wrong while the agent was running.Outcomes — User-facing impact
These warnings indicate the user had a negative experience.Filters
The filter bar appears above the table:- Search — Search threads by summary content
- Source — Filter by thread source: All Sources, Threads, Playbooks. A Feed option appears when Feed Agents are enabled or when feed agent activity exists.
- Users — Multiselect dropdown to filter by one or more team members
- Warning — Filter by warning status: All threads, Any warning, or a specific warning type
Thread Table
The main table lists every thread and playbook run within the selected time range. Each row shows:- Timestamp — When the thread was created
- Source — Thread, Playbook, or Slack
- Summary — A short description of what the user asked
- User — The email of the member who created the thread
- Warnings — Any quality warnings detected on the thread
Thread Detail Panel
Backfilling Warnings
New threads are scanned for warnings automatically. Threads created before Observability was enabled will not have warnings attached. To analyze older threads, use the Scan for Warnings feature.
- Time range — Choose how far back to scan: Last 7 days, Last 14 days, Last 30 days, or Last 90 days
- Re-analyze all threads in range — When enabled, re-evaluates every thread in the range including already-analyzed ones. Off by default.
Users Tab
The Users tab shows a breakdown of activity and ACU consumption by individual team member.Summary Cards
- Active Users — Number of users who ran at least one thread in the selected period
- Avg ACU / User — Average ACUs consumed per active user
- Top Spender — The user who consumed the most ACUs, with their total for the period
Users Table
Agents Tab
The Agents tab shows activity and performance for Feed Agents across your organization.Summary Cards
- Active Agents — Agents with at least one run in the period, out of the total configured
- ACUs Consumed — Total ACUs consumed by feed agents this period
- Warn Rate — Percentage of agent threads that have at least one warning
Agents Table
Playbooks Tab
The Playbooks tab shows scheduled Playbook usage, run counts, and cost breakdown.Summary Cards
- Playbooks — Active playbooks (with runs in the period) vs. total configured
- ACUs Consumed — Total ACUs consumed by playbook runs this period
- Total Runs — Number of chats triggered by playbooks
Playbooks Table
Dashboards Tab
The Dashboards tab shows usage and cost for all Dashboards in your organization.Summary Cards
- Dashboards — Total dashboards in the organization
- ACUs Consumed — Total ACUs consumed by dashboard refreshes this period
- Total Views — Page views across all dashboards in the period
Dashboards Table
Connectors Tab
The Connectors tab gives a health and usage view of every data connector in your organization.Summary Cards
- Total Connectors — Number of connectors configured in the org
- Total Queries — Number of queries run against connectors this period
- Avg Error Rate — Average error rate across all connectors
Connectors Table
High error rates on a connector typically indicate schema mismatches, permission issues, or stale ontology definitions. Drill into the Overview tab and filter by that connector’s users to find specific failing threads.
Further Reading
- Setting Up Context for Best Results — act on what you find by improving your context and metric definitions
- What is Ontology? — close ontology gaps surfaced by Observability warnings
- Writing Better Prompts — share with users whose threads show recurring issues