What an ontology is here
An ontology is the mapping between your technical assets (tables, catalogs, documents) and your business (metrics, workflows, terminology). In TextQL it’s just files — Markdown for human context and.tql for a typed, SQL-rendering semantic layer — kept in a git-backed repository with review, versioning, and access control. Ana reads it, renders inspectable warehouse SQL from it, and helps you extend it.
The principles
- Just files. Business definitions, SQL templates, notes, and operating rules live in one versioned file tree. Portable, diff-able, and yours.
- Locality of behavior. You should understand a metric by reading its file and the few it imports — no black-box compiler.
.tqlrenders inspectable native warehouse SQL. - Governed by default. Semantic changes are proposed → diffed → reviewed → approved → audited, like software.
- Ana does the heavy lifting. She inspects a
.tqlfile, renders SQL from parameters, and executes it — preferring a governed surface over ad-hoc SQL. - Discovery first. Build from what’s actually there — start by reading the schema and the documents, not from assumptions.
Why this approach wins — field lessons
Most data leaders you’ll build with have spent months on a clean semantic layer in Looker, Omni, Collibra, or Palantir — and they’re rightly proud of it. The rigid-tool trap: those tools force big design decisions upfront, before anyone has seen a single real question from a real analyst. So the model captures what seemed important at modeling time and drifts from what actually turns out to matter. Our approach inverts that.- Their prior work isn’t wasted — it’s a corpus. dbt models, LookML, Omni configs, Power BI datasets, wikis, spreadsheets — none needs to be reformatted or migrated. Drop it in; Ana reads it and starts from a warm place.
- You’re not writing the ontology — Ana is. She makes most of the structural decisions and physically writes the content. You are the captain and reviewer, keeping her directionally correct. Ana is self-referential: she knows what context an agent needs.
- You’re not building blind — you need a North Star. Establish what the ontology is for before anything goes in the repo. BI parity, analyst self-serve, and operational/embedded queries each want a different shape (governance, strictness, flexibility). An ontology without a North Star is expensive guesswork.
The cardinal rule —Do not hand a skeptic a form to fill out. Templates, question libraries, and “North Star archetypes” exist to signal expertise (you’ve solved this in their domain) and to force the right conversation — not as a schema to deploy. Naming domain-specific failure modes (“here’s where naive implementations break”) earns your opinions. Then bias hard toward action: the answer to “where do we start?” is “let me show you.” Establish the North Star → feed existing context → start asking real questions → review & ratify what Ana proposes. The model improves every cycle.
The research behind it
This isn’t just philosophy — it’s measured. TextQL’s paper “Malleability Is All You Need: A Self-Maintaining Semantic Layer for Data Agents” (Baumstark, Tomitsuka, Ma — VLDB 2026 / NOVAS) names the “amnesia tax”: a stateless agent that re-explores the warehouse on every question spends 92–94% of its tokens rediscovering the same tables, joins, and filters. Cost scales with task count, not novelty; identical questions resolve to divergent SQL (non-reproducible); long contexts rot (latency and failure rates climb with depth); and naive memory is non-portable across engines.- Invert the lifecycle: the agent reads prior knowledge → explores only the frontier → answers → commits new definitions back to a version-controlled, executable semantic layer. Discovery is amortized across tasks and agents, not re-incurred per request.
- Measured results: ≥30% fewer tokens (up to ~50% on complex multi-join queries), ~41% fewer SQL probe-turns, per-query cost falls toward a “novelty floor” as the layer accretes knowledge (most of a warehouse learned in ~35 queries), and switching backends (DuckDB→Postgres) required no change — definitions are dialect-independent.
- Trust by construction: every write-back is a git commit with an author + diff; definitions are executable, so they’re validated mechanically (recompile + run) rather than re-read as prose; high-stakes paths route to human review (CODEOWNERS-style).
The repository shape
Where each input lands
| Input | Lands as |
|---|---|
| Business documentation (SOPs, policies, glossaries) | notes/*.md + term definitions |
| Metrics documentation | governed .tql query surfaces + notes |
| Database + data dictionaries | schema backings + relations + table docs |
| Process-flow diagrams | entities + relationships + workflow notes |
| Call transcripts / unstructured text | terminology, intents, entities → glossary + entities |
| Golden datasets | golden-query test fixtures (accuracy) |
| Validation sets | eval cases that verify answers |
| Governed / access-controlled documents | fail-closed access rules + governed notes |
How you know it’s right
- Every query renders inspectable SQL — you can read exactly what ran.
- Golden-query tests pin a known-correct value and alert on drift.
- Flexibility ↔ accuracy is a per-metric dial: exact where it must be, flexible where exploration matters.
The expert shortcut — seed from what you already know
Discovery (Modules 2–5) is for when the knowledge doesn’t exist yet. If your team already has the join logic, ERDs, dbt models, or canonical SQL — don’t make Ana rediscover it. Feed it in as a corpus and let her formalize it:Prompt
You’ll see: Ana turn your existing SQL/ERD knowledge into governed ontology entries in one pass — discovery reserved for what your assets don’t cover.
Two rules for a long, live session —1 · Checkpoint every couple of modules. Long threads have a ceiling. After every module or two, ask Ana: “Save a handoff document summarizing what we’ve built, what we decided, and what’s next — so we can continue in a new thread.” If a thread ever maxes out, you lose nothing — open a new thread, paste the handoff, keep going.
2 · Pin the scope in every prompt. Name the entity you’re analyzing and the source-of-truth tables in each prompt (“…for [entity X], using the [base audit] tables, not the summary table”). Without that, Ana may drift to a convenient summary table or query every source at once — and a plausible-looking answer from the wrong table is worse than no answer.
2 · Pin the scope in every prompt. Name the entity you’re analyzing and the source-of-truth tables in each prompt (“…for [entity X], using the [base audit] tables, not the summary table”). Without that, Ana may drift to a convenient summary table or query every source at once — and a plausible-looking answer from the wrong table is worse than no answer.
✅ Checkpoint
- You can explain “just files” and why git-backed matters
- You know the six folders of the repo shape and what goes in each
- You know where a metrics doc vs. a call transcript lands in the model
- You can pitch a skeptic: prior work = corpus, Ana writes it, North Star first, never a form
- You can cite the research: the “amnesia tax” and the ≥30% token reduction from a malleable layer