Motivation
The core primitive for knowledge management in TextQL is the git-backed ontology repo. Business definitions, SQL templates, notes, examples, dashboard snippets, spreadsheet references, and operating rules should live in the same versioned file tree and cross-reference one another naturally..tql is the SQL-native semantic layer file format for that repository. It replaces application-stored ontology objects, relations, and metrics with reviewable files. Those files define governed metrics, dimensions, filters, joins, query-backed objects, reusable functions, and runtime-aware access logic while still compiling to warehouse-native SQL.
Why files and folders?
- Frontier AI is strongest when working with files, folders, search, terminal tools, diffs, and review workflows.
- Files let Ana choose the right exploration strategy: lexical search, semantic search, structural search, import following, local previews, and temporary scratch edits.
- Any context file can sit near the model it supports: prose explanations, generated SQL examples, source statistics, dashboard components, style references, spreadsheet exports, and presentation templates.
- Git gives teams diffs, approvals, history, rollback, auditability, and a familiar workflow for improving Ana’s behavior over time.
- TextQL RBAC, SCIM, and SSO can control who can view, propose, approve, and use ontology repo content.
Locality of behavior
A major design goal is locality of behavior: the behavior of a semantic unit should be as obvious as possible by reading that unit and the nearby files it imports..tql intentionally keeps the declaration of a metric, dimension, filter, or join close to the SQL it affects.
This is the main contrast with highly dispersed semantic systems. A query should not require a reader to mentally stitch together distant UI state, hidden application objects, injected prompts, and warehouse SQL. A .tql file should show:
- what parameters callers can provide,
- what metric labels mean,
- what dimension labels select,
- which joins are trusted,
- which filters are valid,
- which runtime roles or client attributes are required, and
- what SQL is ultimately assembled.
.tql still supports reuse through imports and helper files. The point is not to inline every implementation detail into one huge file. The point is to make the invocation and declaration of behavior visible where the behavior matters, and to keep shared logic in small, nearby modules with obvious names.
No query compiler
.tql is not centered around a black-box query compiler. It is centered around authored SQL fragments, sanitized interpolation, lambdas, and helper patterns for common BI workflows. That makes the generated SQL inspectable, keeps behavior local, and avoids a large hidden translation layer between the model and the warehouse.
This matters for AI. Ana can read the same source that produces the SQL. She can see the metric formula, the join, the filter registry, and the runtime guard in the file rather than reverse-engineering behavior from compiler output or opaque semantic-model state.
If a .tql entry does not support exactly what Ana needs, it is still valuable. Ana can read the entry as a reference for trusted metric definitions, joins, table aliases, filter keys, and access rules, then manually write a one-off SQL query using that context. This is a major advantage over semantic modeling languages that only expose a compiler: .tql is both executable and directly useful as AI-readable source material.
Why not only text-to-SQL?
Freeform text-to-SQL is useful for discovery, but it is not enough for governed analytics. Teams need stable definitions for revenue, active customer, ARR, churn, valid status, trusted joins, tenant scoping, and role-aware access..tql gives Ana a governed surface she can rely on, while preserving an escape hatch for direct SQL when no model covers the question.
Why not only a rigid semantic UI?
Real warehouses contain imperfect schemas, legacy tables, bridge joins, role-playing dimensions, bespoke access rules, and business-specific calculations..tql keeps a SQL-shaped escape hatch so teams can model reality without forcing every definition through a narrow UI schema or requiring TextQL to build complex multi-dialect semantic compilation for every warehouse edge case.
Mental Model
A.tql file is a typed program that renders SQL.
- Authored SQL fragments are trusted. They come from ontology files that are reviewed and governed.
- Parameter values are untrusted data. They can become bind values or escaped inline literals, but they cannot become SQL structure by themselves.
- Sets choose authored structure. A caller can request
"revenue", but only the authoredmatchSetarm decides what SQL"revenue"means. - Filters are compiled from a registry. A caller can request an allowed key and operator, but only authored
filterablesdecide which SQL expression is filtered. - Runtime context is explicit. Role names and client attributes are available under
_tqlso access-aware behavior lives in versioned code.
.tql entries:
inspect: parse the file and return parameter names, types, defaults, and docs from--comments in theparamsblock.render: render SQL for a supplied JSON parameter object.execute: render SQL and run it against a connector.
Short Language Spec
File forms
.tql files have two common shapes.
Plain SQL template
Use this when the file is basically SQL with direct parameters. In a plain body, interpolation is limited to parameter names.Expression body
Use this when you need branching, reusable fragments, imports, helper functions, semantic views, or conditional joins. The expression body must evaluate to aSqlFragment, usually with sql"..." or sql'' ... ''.
Imports
Project templates support imports between.tql files. Relative imports resolve from the importing file. Absolute imports resolve from the project root.
Params block
Parameters are declared in aparams { ... } block. Use one declaration per line and no commas.
| .tql type | JSON shape | Notes |
|---|---|---|
Int | integer | Non-integer numbers are rejected. |
Float | number | Integers are accepted. |
String | string | Used for literal values, not SQL identifiers. |
Bool | boolean | Use with if or boolean operators. |
Date | string | Document the expected format. .tql does not deeply parse dates. |
Timestamp | string | Document timezone expectations. |
Set<"..."> | array of strings | Values are deduped and validated against the allowed labels. |
List<T> | array | Elements are validated recursively. |
FilterInput | object | Usually used as List<FilterInput>. |
? marks a nullable parameter. Omitted nullable params resolve to null. Non-nullable params without defaults are required. Defaults are supported for scalar literals and empty lists or sets.
Pure expressions in Set signatures
Set type signatures can use pure expressions, not only literal unions. This is a key technique for multi-file ontology repo layouts: object modules can export metric and dimension key lists, and query entrypoints can compose those lists into their public parameter types.
Expressions
- Literals: numbers, strings, booleans,
null, lists, and records. - Bindings:
let ... in ..., evaluated in order. Duplicate binding names are rejected. - Records:
{ expr = sql"b.name", join = sql"JOIN buyers b ON ..." }. - Record shorthand:
{ expr, join }. - Field access:
dims.buyer.expr. - Conditionals:
if condition then value else value. Only the selected branch is evaluated. - Lambdas:
\col val -> sql"${col} = ${val}". Function application is whitespace-separated. - Set matching:
matchSet dimensions { "buyer" -> ... }.
Operators
| Category | Operators |
|---|---|
| Boolean | ||, &&, not |
| Equality | ==, != |
| Comparison | >, <, >=, <= |
| Concatenation | ++ for two SqlFragment values or two lists. |
not, ++, comparisons, equality, &&, ||. Use parentheses when mixing boolean operators.
SQL fragments and interpolation
Usesql"..." for short fragments and sql'' ... '' for multiline SQL.
sql, ${expr} evaluates an expression and lowers it into SQL. Scalar values become bind values or escaped inline values. null becomes NULL. Lists of scalar values become SQL tuples. SqlFragment values splice in directly.
Never quote interpolations yourself.
created_after is null, created_at >= ${created_after} renders a NULL comparison. Use branching for optional predicates.
WHERE id IN ${ids}, not WHERE id IN (${ids}).
Builtins
| Builtin | Purpose |
|---|---|
concatSep | Concatenates a list of SqlFragment values with an authored string literal separator. Empty fragments are skipped. |
wrap | Adds an authored prefix and suffix only when the fragment is non-empty. |
isEmpty | Returns true for empty fragments, strings, sets, lists, and null. |
map | Applies a function to each item in a list. |
any | Returns true if any item in a boolean list is true. |
contains | Checks membership in a set or list. |
matchSet | Maps a set param to authored expressions in authored arm order. |
filter | Keeps list items where a predicate returns true. |
dedupeByKey | Deduplicates record lists by their key field, preserving the first definition. |
filterWhere | Compiles FilterInput values against an authored filter registry. |
error | Stops rendering with an explicit message. |
FilterInput
At the JSON boundary, filters are objects withkey, op or operator, and usually value or values.
equals, not_equals, gt, gte, lt, lte, like, not_like, starts_with, not_starts_with, ends_with, not_ends_with, between, in, not_in, is_null, and is_not_null. Aliases such as eq, equal, ilike, greater_than_or_equal, and less_than_or_equal are normalized. Each filterable still controls which operators are allowed for its key.
Runtime context
Chat-scoped runtime values are available under_tql without declaring params:
_tql.roleset_by_rolename: set of effective chat role names._tql.role_names: list of effective chat role names._tql.client_attributes_json: client attribute object from the chat/API key context._tql.client_attributes: alias of_tql.client_attributes_json.
Features
Typed query interfaces
A.tql file exposes a small typed API to callers. Required params, nullable params, defaults, set labels, expression-backed set signatures, and filter shapes can be inspected before execution.
Semantic-view query surfaces
The standard reusable pattern is a view withmetrics, dimensions, and filters. Callers get a compact surface. Authors keep SQL structure, join logic, grouping behavior, and filter rules in code.
Safe parameter interpolation
Caller values do not become SQL structure directly. They render as bind values or inline-escaped values. Authored fragments are the only way to introduce SQL structure.Transparent SQL assembly
.tql uses sanitized string interpolation and lambdas instead of hiding logic behind a query compiler. Authors can represent imperfect warehouse reality directly, including hand-tuned SQL, custom formulas, conditional joins, role-playing dimensions, warehouse-specific functions, and messy legacy conventions that are hard to encode in conventional semantic layers.
Governed arbitrary filtering
filterWhere allows flexible caller filters without raw SQL from the caller. The author controls filter keys, SQL expressions, and allowed operators.
Modular reuse
Imports allow table backing definitions, relation modules, dimensions, measures, filters, key lists, and query templates to live in separate files. This keeps definitions small without hiding behavior in application state. BecauseSet signatures can reference pure expressions such as obj_order.metric_keys or obj_order.dimension_keys ++ obj_customer.dimension_keys, multi-file modules can define the allowed query surface once and reuse it in entrypoints.
Access-aware modeling
.tql can branch on role names and client attributes through _tql. This lets teams model tenant scoping, regional scoping, role-based visibility, and personalization in versioned files.
Versioned governance
Because.tql lives in the ontology repo, semantic changes can be proposed, diffed, reviewed, approved, reverted, and audited like software changes.
Ana-readable context
Ana can read a.tql file and learn the business terms, valid dimensions, trusted joins, metric definitions, filter grammar, runtime access rules, and final SQL shape. This makes .tql both executable code and high-signal context.
Current limitations
DateandTimestampparams are strings at validation time. Document format expectations in comments.- Plain SQL bodies only support param-name interpolation. Use expression-style
.tqlfor computed fragments. - Structured source assembly with compiler-managed join aliases is a design direction, not the current authoring surface. Today, joins are authored as SQL fragments.
Patterns
1. Plain template query
Use when the user-facing API is a few direct values and the SQL shape is fixed.Example input
Rendered SQL
2. Semantic view
Use when callers should select governed metrics, dimensions, and filters.Example input
Rendered SQL
3. File reuse with imports
Use imports when several views share backing table names, helper predicates, measures, or dimension definitions. Keep files small and named after their role.Example input
Rendered SQL
4. Fact and dimension star schema pattern
The fact and dimension pattern separates reusable object modules from query entrypoints. Object modules export backing tables, source helpers, metric keys, dimension keys, filter keys, semantic key checks, joins, metrics, dimensions, and filterables. Query files import the relevant object modules and compose them into a fact-centered view. This pattern is important becauseSet param signatures can be pure expressions. A query can expose Set<obj_order.metric_keys> or Set<obj_order.dimension_keys ++ obj_customer.dimension_keys>, so the allowed query surface is assembled from imported files rather than copied into every entrypoint.
This is also the pattern used by the legacy ontology auto-migration, but it is a useful hand-authored pattern whenever a domain has clear fact tables and dimension tables.
Object modules usually follow this shape:
- Object modules define semantic keys once.
- Query params compose imported key lists with pure expressions inside
Settype signatures. needsmakes conditional joins local to the object that owns the semantic keys.join_ifkeeps query entrypoints concise.dedupeByKeyallows several objects to contribute dimensions and filterables without duplicate keys.filterturns all imported dimension definitions into only the requested dimensions.
Example input
Rendered SQL
5. Fail-closed row-level scoping
Useerror when an access condition is not met. Do not silently render unscoped SQL.
Example runtime context
Rendered SQL
Intentional error
Best Practices
Modeling
- Prefer a plain SQL template when a few direct params are enough.
- Prefer a semantic view when users need reusable metrics, dimensions, and filters.
- Use the same labels that business users use, even if the warehouse column names differ.
- Alias every computed metric with
AS stable_metric_name. - Keep metric definitions at the grain where they are valid. Do not join facts at different grains unless the query explicitly protects against duplication.
- Make bridge joins and many-to-many joins obvious in comments and names.
- For date dimensions, expose useful grains such as day, week, month, or year as separate dimension labels.
Locality and reuse
- Keep behavior near the query surface it affects.
- Extract shared logic only when it removes real duplication or prevents inconsistent definitions.
- Put shared backing table definitions in
schema.tqlor_defs/*.tqlso table renames are local. - Keep imported modules small: relations, dimensions, measures, filters, and query entrypoints are easier to navigate than a single large utility file.
- Prefer obvious import names:
orders,customers,dims,filters,revenue.
Parameters and docs
- Document every user-facing param with
--comments immediately above the declaration. - For date and timestamp params, document accepted formats and timezone assumptions.
- For
filters, document allowed keys and operators becauseinspectexposes those comments to users and Ana. - Use nullable params only when
nullhas clear meaning. - Use
Set<"...">when callers select from a known set of SQL structures.
Safety and correctness
- Never quote interpolations inside SQL string literals.
- Never accept raw SQL as a string param to splice into a query.
- Use
filterWhereinstead of hand-rolled string concatenation for arbitrary filters. - Use
ifandwrapfor optional clauses. - Use
errorto reject unsupported parameter combinations or missing runtime access context. - Render generated SQL before execution when changing a
.tqlfile.
Ana usage
- Ana should prefer a relevant
.tqlfile over raw SQL when the file covers the user’s question. - Ana should inspect params before rendering or executing unfamiliar
.tql. - Ana should use direct SQL for exploration when no
.tqlfile fits, then propose a.tqlimprovement if the analysis pattern becomes reusable. - Ana should keep proposed
.tqlpatches minimal, reviewable, and colocated with related files.
Small Example
This example shows a tiny ontology repo area with table reuse, semantic dimensions, reusable filter helpers, and a query entrypoint.Directory layout
schema.tql
relations/orders.tql
dimensions/customer.tql
filters/compare.tql
queries/revenue_by_customer.tql
Example render params
Example rendered SQL
Intentional business logic error
Theerror builtin is useful when a request is syntactically valid but violates a business or warehouse rule. In this example, year is required because the table is partitioned by year.
Why this example works
- The public query surface is small: metrics, dimensions, and filters.
- Warehouse table names live in
schema.tql. - The customer join is declared near customer dimensions.
- The query only emits the customer join when a dimension or filter needs it.
- Filter keys and operators are governed by
filterables. - Ana can inspect the query params, read related files, render SQL, and explain the business behavior.
Guidance For Ana
When answering analytics questions, use.tql as the first-class semantic surface.
- Search for relevant
.tqlfiles by business term, metric name, table name, and filter key. - Prefer an existing
.tqlfile when its params can express the user’s request. - Inspect before use so parameter docs, defaults, and filter contracts are visible.
- Render before execute when using a
.tqlfile for the first time or after editing it. - Use raw SQL for discovery when no
.tqlfile fits, but do not treat raw SQL discoveries as governed definitions. - When a raw SQL pattern is repeated or important, propose a small
.tqlfile or a small edit to an existing one. - Do not invent metric definitions when a
.tqlfile defines the metric. Use the authored definition or explain why the.tqlfile is insufficient. - Respect runtime scoping. If a
.tqlfile uses_tql, preserve its fail-closed behavior.
Checklist
- Params use
params { ... }, one declaration per line, no commas. - Param docs use
--comments immediately above declarations. - String defaults use double quotes.
- Equality tests use
==and!=, not bare=. - Optional clauses use
if,isEmpty, andwrap. - Interpolations are not wrapped in SQL quotes.
- List interpolations are not wrapped in extra parentheses.
- Metrics have stable
AS ...aliases. - Filter params document allowed keys and operators.
- Filter behavior uses
filterWhereinstead of caller-provided SQL. - Conditional joins include dimensions and filter keys that require each join.
- Runtime access checks fail closed with
error. - Shared table backing definitions live in a local shared file.
- The file has been inspected and rendered before approval.