Skip to main content
A practical reference for customer admins, power users, internal developers, and Ana AI.

Motivation

The core primitive for knowledge management in TextQL is the git-backed ontology repo. Business definitions, SQL templates, notes, examples, dashboard snippets, spreadsheet references, and operating rules should live in the same versioned file tree and cross-reference one another naturally. .tql is the SQL-native semantic layer file format for that repository. It replaces application-stored ontology objects, relations, and metrics with reviewable files. Those files define governed metrics, dimensions, filters, joins, query-backed objects, reusable functions, and runtime-aware access logic while still compiling to warehouse-native SQL.

Why files and folders?

  • Frontier AI is strongest when working with files, folders, search, terminal tools, diffs, and review workflows.
  • Files let Ana choose the right exploration strategy: lexical search, semantic search, structural search, import following, local previews, and temporary scratch edits.
  • Any context file can sit near the model it supports: prose explanations, generated SQL examples, source statistics, dashboard components, style references, spreadsheet exports, and presentation templates.
  • Git gives teams diffs, approvals, history, rollback, auditability, and a familiar workflow for improving Ana’s behavior over time.
  • TextQL RBAC, SCIM, and SSO can control who can view, propose, approve, and use ontology repo content.

Locality of behavior

A major design goal is locality of behavior: the behavior of a semantic unit should be as obvious as possible by reading that unit and the nearby files it imports. .tql intentionally keeps the declaration of a metric, dimension, filter, or join close to the SQL it affects. This is the main contrast with highly dispersed semantic systems. A query should not require a reader to mentally stitch together distant UI state, hidden application objects, injected prompts, and warehouse SQL. A .tql file should show:
  • what parameters callers can provide,
  • what metric labels mean,
  • what dimension labels select,
  • which joins are trusted,
  • which filters are valid,
  • which runtime roles or client attributes are required, and
  • what SQL is ultimately assembled.
.tql still supports reuse through imports and helper files. The point is not to inline every implementation detail into one huge file. The point is to make the invocation and declaration of behavior visible where the behavior matters, and to keep shared logic in small, nearby modules with obvious names.

No query compiler

.tql is not centered around a black-box query compiler. It is centered around authored SQL fragments, sanitized interpolation, lambdas, and helper patterns for common BI workflows. That makes the generated SQL inspectable, keeps behavior local, and avoids a large hidden translation layer between the model and the warehouse. This matters for AI. Ana can read the same source that produces the SQL. She can see the metric formula, the join, the filter registry, and the runtime guard in the file rather than reverse-engineering behavior from compiler output or opaque semantic-model state. If a .tql entry does not support exactly what Ana needs, it is still valuable. Ana can read the entry as a reference for trusted metric definitions, joins, table aliases, filter keys, and access rules, then manually write a one-off SQL query using that context. This is a major advantage over semantic modeling languages that only expose a compiler: .tql is both executable and directly useful as AI-readable source material.

Why not only text-to-SQL?

Freeform text-to-SQL is useful for discovery, but it is not enough for governed analytics. Teams need stable definitions for revenue, active customer, ARR, churn, valid status, trusted joins, tenant scoping, and role-aware access. .tql gives Ana a governed surface she can rely on, while preserving an escape hatch for direct SQL when no model covers the question.

Why not only a rigid semantic UI?

Real warehouses contain imperfect schemas, legacy tables, bridge joins, role-playing dimensions, bespoke access rules, and business-specific calculations. .tql keeps a SQL-shaped escape hatch so teams can model reality without forcing every definition through a narrow UI schema or requiring TextQL to build complex multi-dialect semantic compilation for every warehouse edge case.

Mental Model

A .tql file is a typed program that renders SQL.
  • Authored SQL fragments are trusted. They come from ontology files that are reviewed and governed.
  • Parameter values are untrusted data. They can become bind values or escaped inline literals, but they cannot become SQL structure by themselves.
  • Sets choose authored structure. A caller can request "revenue", but only the authored matchSet arm decides what SQL "revenue" means.
  • Filters are compiled from a registry. A caller can request an allowed key and operator, but only authored filterables decide which SQL expression is filtered.
  • Runtime context is explicit. Role names and client attributes are available under _tql so access-aware behavior lives in versioned code.
Ana can take three different actions on .tql entries:
  • inspect: parse the file and return parameter names, types, defaults, and docs from -- comments in the params block.
  • render: render SQL for a supplied JSON parameter object.
  • execute: render SQL and run it against a connector.
If an entry does not support what Ana wants directly, she can use it as a reference for manually writing a query. The file still tells her which business definitions, joins, filters, and runtime constraints are trusted.

Short Language Spec

File forms

.tql files have two common shapes.

Plain SQL template

Use this when the file is basically SQL with direct parameters. In a plain body, interpolation is limited to parameter names.

Expression body

Use this when you need branching, reusable fragments, imports, helper functions, semantic views, or conditional joins. The expression body must evaluate to a SqlFragment, usually with sql"..." or sql'' ... ''.

Imports

Project templates support imports between .tql files. Relative imports resolve from the importing file. Absolute imports resolve from the project root.
Whole-record imports bind the exported record to a local name. Destructured imports bind named fields from the exported record.

Params block

Parameters are declared in a params { ... } block. Use one declaration per line and no commas.
? marks a nullable parameter. Omitted nullable params resolve to null. Non-nullable params without defaults are required. Defaults are supported for scalar literals and empty lists or sets.

Pure expressions in Set signatures

Set type signatures can use pure expressions, not only literal unions. This is a key technique for multi-file ontology repo layouts: object modules can export metric and dimension key lists, and query entrypoints can compose those lists into their public parameter types.

Expressions

  • Literals: numbers, strings, booleans, null, lists, and records.
  • Bindings: let ... in ..., evaluated in order. Duplicate binding names are rejected.
  • Records: { expr = sql"b.name", join = sql"JOIN buyers b ON ..." }.
  • Record shorthand: { expr, join }.
  • Field access: dims.buyer.expr.
  • Conditionals: if condition then value else value. Only the selected branch is evaluated.
  • Lambdas: \col val -> sql"${col} = ${val}". Function application is whitespace-separated.
  • Set matching: matchSet dimensions { "buyer" -> ... }.

Operators

Precedence, from highest to lowest: field access and function application, not, ++, comparisons, equality, &&, ||. Use parentheses when mixing boolean operators.

SQL fragments and interpolation

Use sql"..." for short fragments and sql'' ... '' for multiline SQL.
Inside sql, ${expr} evaluates an expression and lowers it into SQL. Scalar values become bind values or escaped inline values. null becomes NULL. Lists of scalar values become SQL tuples. SqlFragment values splice in directly. Never quote interpolations yourself.
Nullable params are not omitted automatically. If created_after is null, created_at >= ${created_after} renders a NULL comparison. Use branching for optional predicates.
List interpolation already adds parentheses. Write WHERE id IN ${ids}, not WHERE id IN (${ids}).

Builtins

FilterInput

At the JSON boundary, filters are objects with key, op or operator, and usually value or values.
Supported canonical operators include equals, not_equals, gt, gte, lt, lte, like, not_like, starts_with, not_starts_with, ends_with, not_ends_with, between, in, not_in, is_null, and is_not_null. Aliases such as eq, equal, ilike, greater_than_or_equal, and less_than_or_equal are normalized. Each filterable still controls which operators are allowed for its key.

Runtime context

Chat-scoped runtime values are available under _tql without declaring params:
  • _tql.roleset_by_rolename: set of effective chat role names.
  • _tql.role_names: list of effective chat role names.
  • _tql.client_attributes_json: client attribute object from the chat/API key context.
  • _tql.client_attributes: alias of _tql.client_attributes_json.

Features

Typed query interfaces

A .tql file exposes a small typed API to callers. Required params, nullable params, defaults, set labels, expression-backed set signatures, and filter shapes can be inspected before execution.

Semantic-view query surfaces

The standard reusable pattern is a view with metrics, dimensions, and filters. Callers get a compact surface. Authors keep SQL structure, join logic, grouping behavior, and filter rules in code.

Safe parameter interpolation

Caller values do not become SQL structure directly. They render as bind values or inline-escaped values. Authored fragments are the only way to introduce SQL structure.

Transparent SQL assembly

.tql uses sanitized string interpolation and lambdas instead of hiding logic behind a query compiler. Authors can represent imperfect warehouse reality directly, including hand-tuned SQL, custom formulas, conditional joins, role-playing dimensions, warehouse-specific functions, and messy legacy conventions that are hard to encode in conventional semantic layers.

Governed arbitrary filtering

filterWhere allows flexible caller filters without raw SQL from the caller. The author controls filter keys, SQL expressions, and allowed operators.

Modular reuse

Imports allow table backing definitions, relation modules, dimensions, measures, filters, key lists, and query templates to live in separate files. This keeps definitions small without hiding behavior in application state. Because Set signatures can reference pure expressions such as obj_order.metric_keys or obj_order.dimension_keys ++ obj_customer.dimension_keys, multi-file modules can define the allowed query surface once and reuse it in entrypoints.

Access-aware modeling

.tql can branch on role names and client attributes through _tql. This lets teams model tenant scoping, regional scoping, role-based visibility, and personalization in versioned files.

Versioned governance

Because .tql lives in the ontology repo, semantic changes can be proposed, diffed, reviewed, approved, reverted, and audited like software changes.

Ana-readable context

Ana can read a .tql file and learn the business terms, valid dimensions, trusted joins, metric definitions, filter grammar, runtime access rules, and final SQL shape. This makes .tql both executable code and high-signal context.

Current limitations

  • Date and Timestamp params are strings at validation time. Document format expectations in comments.
  • Plain SQL bodies only support param-name interpolation. Use expression-style .tql for computed fragments.
  • Structured source assembly with compiler-managed join aliases is a design direction, not the current authoring surface. Today, joins are authored as SQL fragments.

Patterns

1. Plain template query

Use when the user-facing API is a few direct values and the SQL shape is fixed.

Example input

Rendered SQL

2. Semantic view

Use when callers should select governed metrics, dimensions, and filters.

Example input

Rendered SQL

3. File reuse with imports

Use imports when several views share backing table names, helper predicates, measures, or dimension definitions. Keep files small and named after their role.

Example input

Rendered SQL

4. Fact and dimension star schema pattern

The fact and dimension pattern separates reusable object modules from query entrypoints. Object modules export backing tables, source helpers, metric keys, dimension keys, filter keys, semantic key checks, joins, metrics, dimensions, and filterables. Query files import the relevant object modules and compose them into a fact-centered view. This pattern is important because Set param signatures can be pure expressions. A query can expose Set<obj_order.metric_keys> or Set<obj_order.dimension_keys ++ obj_customer.dimension_keys>, so the allowed query surface is assembled from imported files rather than copied into every entrypoint. This is also the pattern used by the legacy ontology auto-migration, but it is a useful hand-authored pattern whenever a domain has clear fact tables and dimension tables. Object modules usually follow this shape:
Query entrypoints compose object modules into one callable surface:
The important details are:
  • Object modules define semantic keys once.
  • Query params compose imported key lists with pure expressions inside Set type signatures.
  • needs makes conditional joins local to the object that owns the semantic keys.
  • join_if keeps query entrypoints concise.
  • dedupeByKey allows several objects to contribute dimensions and filterables without duplicate keys.
  • filter turns all imported dimension definitions into only the requested dimensions.

Example input

Rendered SQL

5. Fail-closed row-level scoping

Use error when an access condition is not met. Do not silently render unscoped SQL.

Example runtime context

Rendered SQL

Intentional error


Best Practices

Modeling

  • Prefer a plain SQL template when a few direct params are enough.
  • Prefer a semantic view when users need reusable metrics, dimensions, and filters.
  • Use the same labels that business users use, even if the warehouse column names differ.
  • Alias every computed metric with AS stable_metric_name.
  • Keep metric definitions at the grain where they are valid. Do not join facts at different grains unless the query explicitly protects against duplication.
  • Make bridge joins and many-to-many joins obvious in comments and names.
  • For date dimensions, expose useful grains such as day, week, month, or year as separate dimension labels.

Locality and reuse

  • Keep behavior near the query surface it affects.
  • Extract shared logic only when it removes real duplication or prevents inconsistent definitions.
  • Put shared backing table definitions in schema.tql or _defs/*.tql so table renames are local.
  • Keep imported modules small: relations, dimensions, measures, filters, and query entrypoints are easier to navigate than a single large utility file.
  • Prefer obvious import names: orders, customers, dims, filters, revenue.

Parameters and docs

  • Document every user-facing param with -- comments immediately above the declaration.
  • For date and timestamp params, document accepted formats and timezone assumptions.
  • For filters, document allowed keys and operators because inspect exposes those comments to users and Ana.
  • Use nullable params only when null has clear meaning.
  • Use Set<"..."> when callers select from a known set of SQL structures.

Safety and correctness

  • Never quote interpolations inside SQL string literals.
  • Never accept raw SQL as a string param to splice into a query.
  • Use filterWhere instead of hand-rolled string concatenation for arbitrary filters.
  • Use if and wrap for optional clauses.
  • Use error to reject unsupported parameter combinations or missing runtime access context.
  • Render generated SQL before execution when changing a .tql file.

Ana usage

  • Ana should prefer a relevant .tql file over raw SQL when the file covers the user’s question.
  • Ana should inspect params before rendering or executing unfamiliar .tql.
  • Ana should use direct SQL for exploration when no .tql file fits, then propose a .tql improvement if the analysis pattern becomes reusable.
  • Ana should keep proposed .tql patches minimal, reviewable, and colocated with related files.

Small Example

This example shows a tiny ontology repo area with table reuse, semantic dimensions, reusable filter helpers, and a query entrypoint.

Directory layout

schema.tql

relations/orders.tql

dimensions/customer.tql

filters/compare.tql

queries/revenue_by_customer.tql

Example render params

Example rendered SQL

Intentional business logic error

The error builtin is useful when a request is syntactically valid but violates a business or warehouse rule. In this example, year is required because the table is partitioned by year.

Why this example works

  • The public query surface is small: metrics, dimensions, and filters.
  • Warehouse table names live in schema.tql.
  • The customer join is declared near customer dimensions.
  • The query only emits the customer join when a dimension or filter needs it.
  • Filter keys and operators are governed by filterables.
  • Ana can inspect the query params, read related files, render SQL, and explain the business behavior.

Guidance For Ana

When answering analytics questions, use .tql as the first-class semantic surface.
  • Search for relevant .tql files by business term, metric name, table name, and filter key.
  • Prefer an existing .tql file when its params can express the user’s request.
  • Inspect before use so parameter docs, defaults, and filter contracts are visible.
  • Render before execute when using a .tql file for the first time or after editing it.
  • Use raw SQL for discovery when no .tql file fits, but do not treat raw SQL discoveries as governed definitions.
  • When a raw SQL pattern is repeated or important, propose a small .tql file or a small edit to an existing one.
  • Do not invent metric definitions when a .tql file defines the metric. Use the authored definition or explain why the .tql file is insufficient.
  • Respect runtime scoping. If a .tql file uses _tql, preserve its fail-closed behavior.

Checklist

  • Params use params { ... }, one declaration per line, no commas.
  • Param docs use -- comments immediately above declarations.
  • String defaults use double quotes.
  • Equality tests use == and !=, not bare =.
  • Optional clauses use if, isEmpty, and wrap.
  • Interpolations are not wrapped in SQL quotes.
  • List interpolations are not wrapped in extra parentheses.
  • Metrics have stable AS ... aliases.
  • Filter params document allowed keys and operators.
  • Filter behavior uses filterWhere instead of caller-provided SQL.
  • Conditional joins include dimensions and filter keys that require each join.
  • Runtime access checks fail closed with error.
  • Shared table backing definitions live in a local shared file.
  • The file has been inspected and rendered before approval.