Skip to main content

What an ontology is here

An ontology is the mapping between your technical assets (tables, catalogs, documents) and your business (metrics, workflows, terminology). In TextQL it’s just files — Markdown for human context and .tql for a typed, SQL-rendering semantic layer — kept in a git-backed repository with review, versioning, and access control. Ana reads it, renders inspectable warehouse SQL from it, and helps you extend it.

The principles

  • Just files. Business definitions, SQL templates, notes, and operating rules live in one versioned file tree. Portable, diff-able, and yours.
  • Locality of behavior. You should understand a metric by reading its file and the few it imports — no black-box compiler. .tql renders inspectable native warehouse SQL.
  • Governed by default. Semantic changes are proposed → diffed → reviewed → approved → audited, like software.
  • Ana does the heavy lifting. She inspects a .tql file, renders SQL from parameters, and executes it — preferring a governed surface over ad-hoc SQL.
  • Discovery first. Build from what’s actually there — start by reading the schema and the documents, not from assumptions.

Why this approach wins — field lessons

Most data leaders you’ll build with have spent months on a clean semantic layer in Looker, Omni, Collibra, or Palantir — and they’re rightly proud of it. The rigid-tool trap: those tools force big design decisions upfront, before anyone has seen a single real question from a real analyst. So the model captures what seemed important at modeling time and drifts from what actually turns out to matter. Our approach inverts that.
  • Their prior work isn’t wasted — it’s a corpus. dbt models, LookML, Omni configs, Power BI datasets, wikis, spreadsheets — none needs to be reformatted or migrated. Drop it in; Ana reads it and starts from a warm place.
  • You’re not writing the ontology — Ana is. She makes most of the structural decisions and physically writes the content. You are the captain and reviewer, keeping her directionally correct. Ana is self-referential: she knows what context an agent needs.
  • You’re not building blind — you need a North Star. Establish what the ontology is for before anything goes in the repo. BI parity, analyst self-serve, and operational/embedded queries each want a different shape (governance, strictness, flexibility). An ontology without a North Star is expensive guesswork.
The cardinal ruleDo not hand a skeptic a form to fill out. Templates, question libraries, and “North Star archetypes” exist to signal expertise (you’ve solved this in their domain) and to force the right conversation — not as a schema to deploy. Naming domain-specific failure modes (“here’s where naive implementations break”) earns your opinions. Then bias hard toward action: the answer to “where do we start?” is “let me show you.” Establish the North Star → feed existing context → start asking real questions → review & ratify what Ana proposes. The model improves every cycle.

The research behind it

This isn’t just philosophy — it’s measured. TextQL’s paper “Malleability Is All You Need: A Self-Maintaining Semantic Layer for Data Agents” (Baumstark, Tomitsuka, Ma — VLDB 2026 / NOVAS) names the “amnesia tax”: a stateless agent that re-explores the warehouse on every question spends 92–94% of its tokens rediscovering the same tables, joins, and filters. Cost scales with task count, not novelty; identical questions resolve to divergent SQL (non-reproducible); long contexts rot (latency and failure rates climb with depth); and naive memory is non-portable across engines.
  • Invert the lifecycle: the agent reads prior knowledge → explores only the frontier → answers → commits new definitions back to a version-controlled, executable semantic layer. Discovery is amortized across tasks and agents, not re-incurred per request.
  • Measured results:30% fewer tokens (up to ~50% on complex multi-join queries), ~41% fewer SQL probe-turns, per-query cost falls toward a “novelty floor” as the layer accretes knowledge (most of a warehouse learned in ~35 queries), and switching backends (DuckDB→Postgres) required no change — definitions are dialect-independent.
  • Trust by construction: every write-back is a git commit with an author + diff; definitions are executable, so they’re validated mechanically (recompile + run) rather than re-read as prose; high-stakes paths route to human review (CODEOWNERS-style).
That’s the “why” behind everything below: you’re not asking the customer to build a perfect model upfront — you’re standing up a living one that gets cheaper, more accurate, and more reproducible the more it’s used.

The repository shape

Where each input lands

InputLands as
Business documentation (SOPs, policies, glossaries)notes/*.md + term definitions
Metrics documentationgoverned .tql query surfaces + notes
Database + data dictionariesschema backings + relations + table docs
Process-flow diagramsentities + relationships + workflow notes
Call transcripts / unstructured textterminology, intents, entities → glossary + entities
Golden datasetsgolden-query test fixtures (accuracy)
Validation setseval cases that verify answers
Governed / access-controlled documentsfail-closed access rules + governed notes

How you know it’s right

  • Every query renders inspectable SQL — you can read exactly what ran.
  • Golden-query tests pin a known-correct value and alert on drift.
  • Flexibility ↔ accuracy is a per-metric dial: exact where it must be, flexible where exploration matters.

The expert shortcut — seed from what you already know

Discovery (Modules 2–5) is for when the knowledge doesn’t exist yet. If your team already has the join logic, ERDs, dbt models, or canonical SQL — don’t make Ana rediscover it. Feed it in as a corpus and let her formalize it:
Prompt
You’ll see: Ana turn your existing SQL/ERD knowledge into governed ontology entries in one pass — discovery reserved for what your assets don’t cover.
Two rules for a long, live session1 · Checkpoint every couple of modules. Long threads have a ceiling. After every module or two, ask Ana: “Save a handoff document summarizing what we’ve built, what we decided, and what’s next — so we can continue in a new thread.” If a thread ever maxes out, you lose nothing — open a new thread, paste the handoff, keep going.

2 · Pin the scope in every prompt. Name the entity you’re analyzing and the source-of-truth tables in each prompt (“…for [entity X], using the [base audit] tables, not the summary table”). Without that, Ana may drift to a convenient summary table or query every source at once — and a plausible-looking answer from the wrong table is worse than no answer.

✅ Checkpoint

  • You can explain “just files” and why git-backed matters
  • You know the six folders of the repo shape and what goes in each
  • You know where a metrics doc vs. a call transcript lands in the model
  • You can pitch a skeptic: prior work = corpus, Ana writes it, North Star first, never a form
  • You can cite the research: the “amnesia tax” and the ≥30% token reduction from a malleable layer