Content
# Sidemantic
The universal metrics layer for consistent metrics across your data stack. Compatible with 15+ semantic model formats.
- **Supported Formats:** Sidemantic (YAML, Python or SQL), Power BI TMDL, Cube, dbt MetricFlow, LookML, Hex, Rill, Superset, Omni, BSL, GoodData LDM, Snowflake Cortex, Malloy, OSI, AtScale SML, ThoughtSpot TML
- **Databases:** DuckDB, MotherDuck, PostgreSQL, BigQuery, Snowflake, ClickHouse, Databricks, Spark SQL (also via ADBC)
[Documentation](https://sidemantic.com) | [GitHub](https://github.com/sidequery/sidemantic) | [Docker Hub](https://hub.docker.com/repository/docker/sidequery/sidemantic) | [Discord](https://discord.com/invite/7MZ4UgSVvF) | [Demo](https://sidemantic.com/demo) (50+ MB data download, runs in your browser with Pyodide + DuckDB)
Sidemantic ships Claude Code and Codex plugin metadata for two skills (`modeler` and `webapp-builder`). See [Agent Plugin](#agent-plugin) below to install.
## Quickstart
Install:
```bash
uv add sidemantic
```
Malloy support (uv):
```bash
uv add "sidemantic[malloy]"
```
DAX and Power BI TMDL support (uv):
```bash
uv add "sidemantic[dax]"
```
HTTP API server (uv):
```bash
uv add "sidemantic[api]"
```
Notebook widget (uv):
```bash
uv add "sidemantic[widget]" jupyterlab
uv run jupyter lab
```
Marimo (uv):
```bash
uv add "sidemantic[widget]" marimo
uv run marimo edit
```
```python
import duckdb
from sidemantic.widget import MetricsExplorer
conn = duckdb.connect(":memory:")
conn.execute("create table t as select 1 as value, 'a' as category, date '2024-01-01' as d")
MetricsExplorer(conn.table("t"), time_dimension="d")
```
Define models in SQL, YAML, or Python:
<details>
<summary><b>SQL</b> (orders.sql)</summary>
```sql
MODEL (name orders, table orders, primary_key order_id);
DIMENSION (name status, type categorical);
DIMENSION (name order_date, type time, granularity day);
METRIC (name revenue, agg sum, sql amount);
METRIC (name order_count, agg count);
```
</details>
<details>
<summary><b>YAML</b> (orders.yml)</summary>
```yaml
models:
- name: orders
table: orders
primary_key: order_id
dimensions:
- name: status
type: categorical
- name: order_date
type: time
granularity: day
metrics:
- name: revenue
agg: sum
sql: amount
- name: order_count
agg: count
```
</details>
<details>
<summary><b>Python</b> (programmatic)</summary>
```python
from sidemantic import Model, Dimension, Metric
orders = Model(
name="orders",
table="orders",
primary_key="order_id",
dimensions=[
Dimension(name="status", type="categorical"),
Dimension(name="order_date", type="time", granularity="day"),
],
metrics=[
Metric(name="revenue", agg="sum", sql="amount"),
Metric(name="order_count", agg="count"),
]
)
```
</details>
Query via CLI:
```bash
sidemantic query "SELECT revenue, status FROM orders" --db data.duckdb
```
Or Python API:
```python
from sidemantic import SemanticLayer, load_from_directory
layer = SemanticLayer(connection="duckdb:///data.duckdb")
load_from_directory(layer, "models/")
result = layer.sql("SELECT revenue, status FROM orders")
```
## DAX And TMDL
DAX/TMDL support lives behind the `dax` extra because it includes a native Rust parser:
```bash
uv add "sidemantic[dax]"
```
Native Sidemantic YAML can preserve DAX expression source text for Power BI interoperability:
```yaml
models:
- name: sales
table: sales
primary_key: id
dimensions:
- name: doubled_amount
type: numeric
dax: "'sales'[amount] * 2"
metrics:
- name: revenue
dax: "SUM('sales'[amount])"
```
Power BI TMDL projects can be loaded from a project root or `definition/` folder. Embedded DAX measures, calculated columns, calculated tables, relationships, and TMDL passthrough metadata are parsed and preserved in model metadata:
```python
from sidemantic import SemanticLayer, load_from_directory
layer = SemanticLayer(connection="duckdb:///warehouse.duckdb")
load_from_directory(layer, "powerbi_project/")
print(layer.describe_models(["Sales"]))
```
TMDL can also round-trip back to disk:
```python
from sidemantic.adapters.tmdl import TMDLAdapter
TMDLAdapter().export(layer.graph, "exported_tmdl/")
```
## CLI
```bash
# Query
sidemantic query "SELECT revenue FROM orders" --db data.duckdb
# Interactive workbench (TUI with SQL editor + charts)
uvx --from "sidemantic[workbench]" sidemantic workbench models/ --db data.duckdb
# PostgreSQL server (connect Tableau, DBeaver, etc.)
uvx --from "sidemantic[serve]" sidemantic serve models/ --port 5433
# HTTP API server (JSON or Arrow)
uvx --from "sidemantic[api]" sidemantic api-serve models/ --port 4400 --auth-token secret
# Validate definitions
sidemantic validate models/
# Model info
sidemantic info models/
# Pre-aggregation recommendations
sidemantic preagg recommend --db data.duckdb
# Migrate SQL queries to semantic layer
sidemantic migrator --queries legacy/ --generate-models output/
```
## Demos
**Workbench** (TUI with SQL editor + charts):
```bash
uvx --from "sidemantic[workbench]" sidemantic workbench --demo
```
**PostgreSQL server** (connect Tableau, DBeaver, etc.):
```bash
uvx --from "sidemantic[serve]" sidemantic serve --demo --port 5433
```
**HTTP API server** (JSON or Arrow):
```bash
uvx --from "sidemantic[api]" sidemantic api-serve --demo --port 4400 --auth-token secret
```
**Colab notebooks:**
[](https://colab.research.google.com/github/sidequery/sidemantic/blob/main/examples/notebooks/sidemantic_sql_duckdb_demo.ipynb) SQL + DuckDB
[](https://colab.research.google.com/github/sidequery/sidemantic/blob/main/examples/notebooks/lookml_multi_entity_duckdb_demo.ipynb) LookML multi-entity
**SQL syntax:**
```bash
uv run https://raw.githubusercontent.com/sidequery/sidemantic/main/examples/sql/sql_syntax_example.py
```
**Comprehensive demo:**
```bash
uv run https://raw.githubusercontent.com/sidequery/sidemantic/main/examples/advanced/comprehensive_demo.py
```
**Symmetric aggregates:**
```bash
uv run https://raw.githubusercontent.com/sidequery/sidemantic/main/examples/features/symmetric_aggregates_example.py
```
**Superset with DuckDB:**
```bash
git clone https://github.com/sidequery/sidemantic.git && cd sidemantic
uv run examples/superset_demo/run_demo.py
```
**Cube Playground:**
```bash
git clone https://github.com/sidequery/sidemantic.git && cd sidemantic
uv run examples/cube_demo/run_demo.py
```
**Rill Developer:**
```bash
git clone https://github.com/sidequery/sidemantic.git && cd sidemantic
uv run examples/rill_demo/run_demo.py
```
**OSI (complex adtech semantic model):**
```bash
git clone https://github.com/sidequery/sidemantic.git && cd sidemantic
uv run examples/osi_demo/run_demo.py
```
**OSI widget notebook (percent-cell Python notebook):**
```bash
git clone https://github.com/sidequery/sidemantic.git && cd sidemantic
uv run examples/osi_demo/osi_widget_notebook.py
```
See `examples/` for more.
## Core Features
- SQL query interface with automatic rewriting
- Automatic joins across models
- Multi-format adapters (Cube, MetricFlow, LookML, Hex, Rill, Superset, Omni, BSL, GoodData LDM, OSI, AtScale SML, ThoughtSpot TML, Graphene GSQL)
- SQLGlot-based SQL generation and transpilation
- Pydantic validation and type safety
- Pre-aggregations with explicit routing
- Predicate pushdown for faster queries
- Segments and metric-level filters
- Jinja2 templating for dynamic SQL
- PostgreSQL wire protocol server for BI tools
- HTTP API with JSON and Arrow IPC responses
## Multi-Format Support
Auto-detects: Sidemantic (SQL/YAML), Power BI TMDL, Cube, MetricFlow (dbt), LookML, Hex, Rill, Superset, Omni, BSL, GoodData LDM, OSI, AtScale SML, ThoughtSpot TML, Graphene GSQL
```bash
sidemantic query "SELECT revenue FROM orders" --models ./my_models
```
```python
from sidemantic import SemanticLayer, load_from_directory
layer = SemanticLayer(connection="duckdb:///data.duckdb")
load_from_directory(layer, "my_models/") # Auto-detects formats
```
## Databases
| Database | Status | Installation |
|----------|:------:|--------------|
| DuckDB | ✅ | built-in |
| MotherDuck | ✅ | built-in |
| PostgreSQL | ✅ | `uv add sidemantic[postgres]` |
| BigQuery | ✅ | `uv add sidemantic[bigquery]` |
| Snowflake | ✅ | `uv add sidemantic[snowflake]` |
| ClickHouse | ✅ | `uv add sidemantic[clickhouse]` |
| Databricks | ✅ | `uv add sidemantic[databricks]` |
| Spark SQL | ✅ | `uv add sidemantic[spark]` |
## Docker
The published image is [`sidequery/sidemantic`](https://hub.docker.com/r/sidequery/sidemantic) on Docker Hub. Mount your models directory as a volume at `/app/models`:
```bash
docker run -p 5433:5433 -v ./models:/app/models sidequery/sidemantic
```
Demo mode (built-in sample data, no volume needed):
```bash
docker run -p 5433:5433 sidequery/sidemantic --demo
```
See [`examples/docker/`](examples/docker/) for MCP mode, env vars, building from source, and integration test services.
For Cloudflare Worker + Container deployment, see [`examples/cloudflare_containers/`](examples/cloudflare_containers/).
## HTTP API
Start the API server:
```bash
uvx --from "sidemantic[api]" sidemantic api-serve models/ --db data.duckdb --port 4400 --auth-token secret
```
Compile a structured semantic query:
```bash
curl -s http://localhost:4400/compile \
-H "Authorization: Bearer secret" \
-H "Content-Type: application/json" \
-d '{"dimensions":["orders.status"],"metrics":["orders.total_amount"]}'
```
Run a structured query as JSON:
```bash
curl -s http://localhost:4400/query \
-H "Authorization: Bearer secret" \
-H "Content-Type: application/json" \
-d '{"dimensions":["orders.status"],"metrics":["orders.total_amount","orders.order_count"]}'
```
Run a structured query as Arrow IPC:
```bash
curl -s http://localhost:4400/query \
-H "Authorization: Bearer secret" \
-H "Accept: application/vnd.apache.arrow.stream" \
-H "Content-Type: application/json" \
-d '{"metrics":["orders.order_count"]}' \
> result.arrow
```
Execute rewritten SQL over HTTP:
```bash
curl -s http://localhost:4400/sql \
-H "Authorization: Bearer secret" \
-H "Content-Type: application/json" \
-d '{"query":"SELECT status, total_amount FROM orders ORDER BY status"}'
```
## Agent Plugin
Sidemantic ships a [plugin bundle](plugins/sidemantic/) with Claude Code and Codex metadata for two skills:
- **`modeler`** — build, validate, and query semantic models
- **`webapp-builder`** — generate analytics webapps from your models
**Install in Claude Code:**
```bash
claude plugin marketplace add sidequery/sidemantic && claude plugin install sidemantic@sidequery
```
**Install in Codex:**
```bash
codex plugin marketplace add sidequery/sidemantic && codex plugin add sidemantic@sidequery
```
**Use a local clone while developing:**
```bash
claude --plugin-dir ./plugins/sidemantic
codex plugin marketplace add . && codex plugin add sidemantic@sidequery
```
The Claude Code plugin manifest lives at `plugins/sidemantic/.claude-plugin/plugin.json`, and its marketplace lives at `.claude-plugin/marketplace.json`.
The Codex plugin manifest lives at `plugins/sidemantic/.codex-plugin/plugin.json`, and its repo-local marketplace lives at `.agents/plugins/marketplace.json`.
The skills also work with other `SKILL.md`-compatible agents by pointing them at `plugins/sidemantic/skills/`.
## How mature is Sidemantic?
Sidemantic is an ambitious but young semantic layer project. You could encounter rough patches, especially with the more exotic features like converting between semantic model formats or serving semantic layers via the included Postgres protocol server.
## Testing
```bash
uv run pytest -v
```
This prints line coverage for `sidemantic` with missing lines in the terminal.
