ktx.yaml reference

ktx.yaml is the single source of truth for a ktx project. The file lives at the project root and tells ktx which databases to read, which context sources to ingest, which LLM and embedding providers to use, how to store state, and how the scan and agent layers behave. Every block below is optional and falls back to a documented default, so a minimal ktx.yaml is just one connection.

This page is the canonical reference for the file. For the guided flow that writes it, see ktx setup.

Where blocks fit

ktx.yaml has eight top-level keys. They group into three layers: what to read, how to think, and where to put the results.

Minimal config

A working ktx.yaml needs one entry in connections. Everything else accepts defaults. The example below is enough for ktx ingest warehouse to run a fast schema scan against a local Postgres.

connections:
  warehouse:
    driver: postgres
    url: env:DATABASE_URL

Secret references

Several fields accept either a literal value or a reference. References keep secrets out of ktx.yaml so the file can stay in git.

References work in: warehouse url, Metabase api_key / api_key_ref, Looker client_secret / client_secret_ref, Notion / dbt / LookML / MetricFlow auth_token / auth_token_ref, and any api_key under the llm and ingest.embeddings blocks.

connections

The connections block is a map from a connection ID you choose to the configuration for that connector. The connection ID is what every other part of ktx uses to address a connector - ktx ingest warehouse, ktx sql --connection warehouse, the semantic-layer path semantic-layer/warehouse/, and so on.

Each entry is discriminated by the driver field. Warehouse drivers and context-source drivers share the map.

Warehouse drivers

Warehouse connections are open objects: the listed fields are validated, and any other field is preserved and passed through to the connector. Use enabled_tables to scope deep ingest to a specific list of schema.table names - useful for smoke tests.

connections:
  warehouse:
    driver: postgres
    url: env:DATABASE_URL
    enabled_tables:
      - public.orders
      - public.customers

Connector-specific scope fields let setup and scan use the same warehouse boundary:

connections:
  mysql-warehouse:
    driver: mysql
    url: env:MYSQL_URL
    schemas: [analytics, mart]
  clickhouse-warehouse:
    driver: clickhouse
    url: env:CLICKHOUSE_URL
    database: analytics
    databases: [analytics, mart]
  bigquery-warehouse:
    driver: bigquery
    credentials_json: file:./service-account.json
    location: US
    dataset_ids: [analytics, mart]

For Postgres, MySQL, SQL Server, and Snowflake connections, set maxConnections when scan or ingest work needs to stay below the target's connection cap. Postgres, MySQL, and SQL Server default to 10; Snowflake defaults to 4. This caps all concurrent SQL work for that connector instance, including schema introspection, table sampling, relationship profiling, relationship validation, and read-only SQL execution. BigQuery and ClickHouse do not expose maxConnections because their connectors don't use client-side connection pools.

For Postgres, BigQuery, and Snowflake, historicSql and context.queryHistory toggle query-history ingest. The shape is connector-specific; the setup wizard writes these fields when you pass --enable-query-history.

connections:
  warehouse:
    driver: postgres
    url: env:DATABASE_URL
    context:
      queryHistory:
        enabled: true
        minExecutions: 5

Metabase

connections:
  metabase:
    driver: metabase
    api_url: https://metabase.example.com
    api_key_ref: env:METABASE_API_KEY
    mappings:
      databaseMappings:
        "1": warehouse        # Metabase DB id "1" -> ktx connection "warehouse"
      syncMode: ALL           # ALL | ONLY | EXCEPT

Looker

connections:
  looker:
    driver: looker
    base_url: https://looker.example.com
    client_id: ktx-integration
    client_secret_ref: env:LOOKER_CLIENT_SECRET
    mappings:
      connectionMappings:
        prod_warehouse: warehouse

LookML

connections:
  lookml:
    driver: lookml
    repoUrl: git@github.com:org/lookml.git
    branch: main
    path: lookml/
    auth_token_ref: env:GITHUB_TOKEN
    mappings:
      expectedLookerConnectionName: prod_warehouse

dbt

connections:
  dbt_main:
    driver: dbt
    source_dir: ../dbt-project
    target: prod

MetricFlow

connections:
  metricflow:
    driver: metricflow
    metricflow:
      repoUrl: git@github.com:org/sl-config.git
      branch: main
      path: semantic_models/
      auth_token_ref: env:GITHUB_TOKEN

The MetricFlow connector wraps its fields in a nested metricflow block. repoUrl is required; the rest mirrors the LookML / dbt git fields.

Notion

connections:
  notion:
    driver: notion
    auth_token_ref: env:NOTION_TOKEN
    crawl_mode: selected_roots
    root_database_ids:
      - 9f30c2c4d4f24a8d9a8d8e2c1b2a3d4e
    max_pages_per_run: 500
    max_knowledge_creates_per_run: 5
    max_knowledge_updates_per_run: 25

setup

Captured by the setup wizard. The only field ktx still reads is database_connection_ids, which tells the ingest layer which entries in connections are primary warehouses. When omitted, every warehouse-typed connection is treated as primary.

setup:
  database_connection_ids:
    - warehouse

storage

storage controls where ktx keeps its own state and search index, and how state changes are committed. Defaults work for a single-user local project.

storage:
  state: sqlite          # sqlite | postgres
  search: sqlite-fts5    # sqlite-fts5 | postgres-hybrid
  git:
    auto_commit: true
    author: "ktx <ktx@example.com>"

llm

The llm block selects the LLM provider, lets you override the model used for specific roles, and tunes prompt caching.

llm:
  provider:
    backend: anthropic
    anthropic:
      api_key: env:ANTHROPIC_API_KEY
  models:
    default: claude-sonnet-4-6
    triage: claude-haiku-4-5
  promptCaching:
    enabled: true
    systemTtl: 1h
    toolsTtl: 1h
    historyTtl: 5m
    vertexFallbackTo5m: true

Provider

Model roles

models overrides the per-role model. Keys are fixed; values are provider-specific model identifiers.

Prompt caching

ingest

ingest controls how ktx builds context from your stack. It lists the adapters to run, the embedding provider used when adapters embed documents, and the concurrency and failure policy for work units.

ingest:
  adapters:
    - live-database
    - dbt
    - metabase
  embeddings:
    backend: openai
    model: text-embedding-3-small
    dimensions: 1536
    openai:
      api_key: env:OPENAI_API_KEY
  workUnits:
    stepBudget: 40
    maxConcurrency: 2
    failureMode: continue

Adapters

adapters is a list of adapter IDs that should run. Each ID matches a connector that ktx ships locally:

Embeddings

The embeddings block can also appear inside scan.enrichment; that override wins when present.

Work units

A work unit is one unit of agent-driven ingest work (for example one table or one Metabase question). These knobs bound how long it runs and how the run handles failures.

scan

scan configures how schema-level inputs become structured context: column-level enrichment and inferred relationships between tables.

scan:
  enrichment:
    mode: llm           # none | deterministic | llm
  relationships:
    enabled: true
    llmProposals: true
    validationRequiredForManifest: true
    acceptThreshold: 0.85
    reviewThreshold: 0.55
    maxLlmTablesPerBatch: 40
    maxCandidatesPerColumn: 25
    profileSampleRows: 10000
    profileConcurrency: 4
    validationConcurrency: 4
    validationBudget: all

Enrichment

Relationships

The relationship discovery step proposes joins between tables, scores them, and optionally validates each one against the database before writing it to the manifest.

agent

agent carries feature flags for ktx-side agent behavior. Today the only block is run_research, which gates the research agent invoked by ktx mcp and CLI research tools.

agent:
  run_research:
    enabled: true
    max_iterations: 20
    default_toolset:
      - sl_query
      - wiki_search
      - sl_read_source

memory

memory controls the agent memory subsystem.

memory:
  auto_commit: true

A full example

Combining the blocks above:

connections:
  warehouse:
    driver: postgres
    url: env:DATABASE_URL
  metabase:
    driver: metabase
    api_url: https://metabase.example.com
    api_key_ref: env:METABASE_API_KEY
    mappings:
      databaseMappings:
        "1": warehouse
      syncMode: ALL
setup:
  database_connection_ids:
    - warehouse
storage:
  state: sqlite
  search: sqlite-fts5
  git:
    auto_commit: true
    author: "ktx <ktx@example.com>"
llm:
  provider:
    backend: claude-code
  models:
    default: sonnet
ingest:
  adapters:
    - live-database
    - metabase
  embeddings:
    backend: openai
    model: text-embedding-3-small
    dimensions: 1536
    openai:
      api_key: env:OPENAI_API_KEY
  workUnits:
    maxConcurrency: 2
scan:
  enrichment:
    mode: llm
  relationships:
    acceptThreshold: 0.85
    reviewThreshold: 0.55
agent:
  run_research:
    enabled: true
memory:
  auto_commit: true

Validating your config

ktx validates ktx.yaml strictly: unknown keys at the top level or inside strict blocks cause setup and CLI commands to fail with a precise path (scan.relationships.acceptThreshhold: Unrecognized key). Warehouse connections accept extra driver-specific fields, so passthrough values like historicSql and context.queryHistory are allowed.

To re-validate without running anything else:

ktx status

ktx status parses ktx.yaml, surfaces validation issues, and reports which inputs are ready.

Related references

• ktx setup - the guided flow that writes most of these fields for you.
• ktx status - readiness check for the current ktx.yaml.
• LLM configuration - provider-specific setup notes.
• Primary sources and Context sources - connector-specific details and credentials.