🧭 elevata Documentation Index¶

Welcome to the elevata Documentation Hub - your guide to metadata-defined, discoverable, controllable, and executable data architecture.

This documentation describes the current behavior and architecture of elevata. Release history and feature introductions are documented in the CHANGELOG.

This index gives you an overview of all major topics and how they fit together.

🗺️ Table of Contents¶

🚀 Getting Started¶

Getting Started
Install elevata, run the first migration, and open the UI.
Secure Metadata Connectivity
Configure profiles, environment variables, secrets, peppers and secure access to source/target systems.

🧩 Metadata Model & Generation¶

Generation Logic
How metadata is transformed into Logical Plans and final SQL. Includes dataset types (RAW, STAGE, CORE, …), dependencies and generation rules.
Metadata Naming Guidance
Deterministic, project-specific naming assistance for TargetColumn modeling based on existing mappings.
Incremental Load Architecture
Incremental patterns, MERGE semantics, deletion handling, and how elevata models change propagation.
Load SQL Architecture
How elevata transforms lineage and metadata into executable SQL through the logical plan, renderer, and dialect adapters - covering full loads, merge operations, and delete detection.
Historization Architecture
Complete SCD Type 2 historization model: versioning, change detection, deletion, surrogate keys, lineage-based attribute mapping, and SQL generation.
Schema Evolution
Schema evolution is metadata-driven, deterministic, and lineage-safe. Structural changes are never inferred implicitly from SQL but are always derived from explicit metadata changes.

🎨 SQL Rendering & Dialects¶

SQL Rendering Conventions
General rules for how SQL is formatted and rendered (identifiers, literals, ordering, readability).
Dialect System
Overview of the dialect abstraction, the dialect registry, and how BigQuery, Databricks, DuckDB, Fabric Warehouse, MSSQL, Postgres and Snowflake are implemented.
Target Backends
Which engines are supported and how they fit into a Lakehouse or Warehouse architecture.

▶️ Execution & Observability¶

Load Execution & Orchestration Architecture
How elevata executes load plans: dependency graphs, retries, failure semantics, load run logging and execution snapshots.
Architecture Catalog
Read-only discovery layer for metadata-defined executable architecture, including dataset search, Portfolio, Catalog Maps, Catalog Insights, lineage entry points, query contract links and execution evidence references.
Architecture Catalog Portfolio
Executive architecture posture lens across readiness, ownership, contracts, health, review state, execution evidence and layer distribution.
Architecture Catalog Data Products
Consumer-readiness perspective for serving-layer datasets, derived from ownership, health, lineage, contracts, review state and execution evidence.
Architecture Control Plane
Deterministic architecture state, change reports, review briefing, promotion reports, approval artifacts, policy decisions and fingerprints for review, CI, controlled approval, and architecture promotion workflows.

💡 Concepts¶

Architecture Overview
High-level view of elevata’s architecture: metadata, lineage, Logical Plan, rendering and execution.
Query Builder & Query Tree
Conceptual introduction to elevata’s Query Builder and Query Tree. Explains why custom query logic exists, when it should be used, and how it integrates with metadata-driven generation.
Determinism & Execution Semantics
Rules and guarantees for deterministic SQL generation, including ORDER BY requirements, window functions, aggregation semantics and error vs warning classification.
Lineage Model & Logical Plan
How datasets depend on each other, how lineage is represented, and how the Logical Plan encodes queries.
Expression DSL & AST
The vendor-neutral expression DSL (Domain Specific Language) (HASH256, CONCAT_WS, COALESCE, COL, …), the AST (Abstract Syntax Tree), and how dialects render it.
Hashing Architecture
Surrogate key and foreign key hashing: deterministic rules, cross-dialect SHA-256, null handling and pepper strategy.
SQL Preview & Rendering Pipeline
How the UI builds previews from metadata, Logical Plan and dialect selection (HTMX-based).
Bizcore - Business Semantics Layer
Bizcore is elevata’s dedicated layer for modeling business meaning and rules as first-class metadata - without introducing a BI semantic layer.
Metadata Health Check
Built-in checks for incomplete or inconsistent metadata, and how to interpret them.

🌐 Source Integration¶

Source Backends
Overview of supported source systems and how to configure them (JDBC/ODBC, file-based, etc.).
Source Metadata Import Review
Deterministic import outcome review for SourceDataset and SourceSystem metadata imports, including created, changed, unchanged, removed and review-needed signals.
Source Ingestion Configuration
(Manual: ingestion_config for Files / REST)

✅ Testing & Quality¶

Test Setup & Guidelines
How the core test suite is structured, how to add tests for new features, and how to reason about coverage.

♟️ Strategy & Architecture¶

elevata Platform Strategy
Conceptual overview of elevata’s metadata-native execution and business semantics.

📦 Project¶

Main Project README
The top-level README from the Git repository (architecture, goals, roadmap).
Changelog
Release history

🧭 Where to start?¶

If you are new to elevata, a good reading path is:

This will give you a mental model for how metadata flows through the platform and becomes executable SQL.

🧭 Advanced Reading Path: Semantic Modeling & Query Logic¶

If you want to understand how elevata models advanced business logic while remaining metadata-native and deterministic, continue with:

🧡 About¶

elevata is an open-source Architecture Runtime for modern data platforms - turning metadata into explicit, discoverable, controllable, executable, and auditable data architecture.

Built for data architects, engineers, and platform teams.
elevata: architecture made explicit through metadata.

👩‍💻 Created and maintained by Ilona Tag
A personal open-source initiative exploring how data architecture can be defined declaratively, governed transparently, and executed deterministically.

Last updated: 2026-06-15