Skip to main content

​
Inference Overview

​
Overview

Large language models are at the core of our product, so making the way we use language models transparent and customizable for your organization is paramount.

​
In-Depth

There are three main types of language model deployment that we support
  • Foundational Model Providers
  • In-your-cloud Foundation Models
  • Self-hosted Open Source Models
The best performing versions of TextQL use some variation of a closed-source foundational model to run core LLM functions. For our multi-tenant deployment this defaults to OpenAI (GPT-4o, o1-mini) as we’ve found this to have the best performance. We’ve also found comparable performance from Anthropic’s Claude 3.5 Sonnet and thus this is also and officially supported deployment type. Lastly we have the ability to run TextQL with publicly available open source models, though performance trails OpenAI and Anthropic and it is not recommended.

​
Foundational Model Provider

​
Overview

The fastest and easiest way to get industry leading language model performance, all without sacrificing privacy and security

​
In-Depth

Working directly with foundational model providers means getting access to the state of the art language models right as they’re released. We work closely with the teams at both OpenAI and Anthropic to use the latest and greatest LLMs that they put out, all while maintaining the data security and privacy that you expect from an enterprise product. We have DPAs in place with both providers and review their SOC 2 Type 2 audits. For OpenAI we use
  • gpt-4o
  • o1-mini
For Anthropic we use
  • claude-3.5-sonnet
  • claude-3.5-haiku

​
Why?

  • Best in class models
  • High Availability
  • SOC2, GDPR, HIPAA certified/compliant

​
Who it’s for

  • Organization looking for the best performance and pricing

​
In-Your-Cloud Model Providers

​
Overview

Use best-in-class models all while making sure data never leaves the VPC.
** We can connect to In-Your-Cloud Model Providers within your VPC from Multi and Single tenant managed deployments (via PrivateLink or internet). Contact us for more information.

​
In-Depth

Foundational model providers now partner with leading clouds to allow you to deploy the same best in class LLMs available on their own APIs directly within your VPC. Each of the major public clouds has partnerships with either Anthropic or OpenAI and we support both. For AWS:
  • AWS Bedrock within you VPC
    • Anthropic Claude 3.5 Sonnet and Haiku
For GCP:
  • GCP Vertex AI
    • Anthropic Claude 3.5 Sonnet and Haiku
For Azure:
  • Azure OpenAI
    • OpenAI GPT-4o and o1-mini

​
Why

  • Best in class models
  • High availability
  • Cloud inference priced into your cloud bill
  • Data stays in your VPC

​
Who it’s for

  • Organization with large cloud credit balances
  • Organizations seeking to keep data from leaving their VPC

​
Self-hosted Models

​
Overview

For maximum security and control, run models where you know every weight and bias.

​
In-Depth

For organizations that require visibility into model weights and runner environments, we offer the ability to run TextQL with open source models. This performs meaningfully worse than the models offered by OpenAI and Anthropic so we reserve this for organizations in highly regulated industries where this is required.

​
Why

  • Full control over models
  • Ability to fine tune or additionally pre-train

​
Who it’s for

  • Organization with restrictions on LLM use