Author Image

CEO & Co-founder of Visivo

dbt + BI-as-Code Integration Guide for Scalable BI

Learn how to integrate dbt with BI-as-Code for a scalable, maintainable analytics stack that grows with your organization.

dbt and BI-as-Code integration architecture

dbt™ owns transformation. BI-as-code owns visualization. Put them in the same repository and the same pipeline, and your data models and your dashboards change together, in one pull request. According to the dbt Labs State of Analytics Engineering 2023, 67% of organizations now use dbt™ for data transformation. This guide shows how to connect that dbt™ project to Visivo.

How dbt™ and BI-as-code fit together

dbt™ transforms raw data inside your warehouse into tested, documented models (docs.getdbt.com). BI-as-code turns those models into charts and dashboards defined in code. Together they cover the pipeline from raw table to dashboard, and both halves live in Git.

They share the same habits:

  • dbt™ keeps data quality and definitions consistent
  • BI-as-code keeps visualizations consistent and under version control
  • Both use Git for collaboration (see git-scm.com)
  • Both support CI/CD workflows (see our CI/CD analytics implementation guide)
  • Both support testing and documentation

A McKinsey study reports that data-driven organizations are 23 times more likely to acquire customers, 6 times as likely to retain them, and 19 times as likely to be profitable. Models and dashboards that cannot drift apart are part of how you get there.

Integration strategies

Shared repository structure:

Keep the dbt™ project and the Visivo project in one repository, so a single pull request can change a model and the chart that reads it:

analytics-platform/
├── dbt/
│   ├── models/
│   │   ├── staging/
│   │   └── marts/
│   ├── tests/
│   └── macros/
├── dashboards/
│   ├── executive/
│   ├── operational/
│   └── components/
├── project.visivo.yml
├── .github/
│   └── workflows/
│       └── integrated-pipeline.yml
└── README.md

One deployment pipeline:

One job runs dbt™, then runs and tests the dashboards, then deploys. The dashboard steps share the runner with dbt™ because Visivo reads the manifest that dbt run writes:

# .github/workflows/integrated-pipeline.yml
name: Analytics Platform Deploy

on:
  push:
    branches: [main]

jobs:
  build-and-deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install dbt and Visivo
        run: pip install dbt-core dbt-snowflake visivo

      - name: Run dbt transformations
        run: |
          dbt deps --project-dir dbt --profiles-dir dbt
          dbt run --project-dir dbt --profiles-dir dbt
          dbt test --project-dir dbt --profiles-dir dbt

      - name: Run and test dashboards
        run: |
          visivo run
          visivo test

      - name: Deploy dashboards
        run: visivo deploy -s production
        env:
          VISIVO_TOKEN: ${{ secrets.VISIVO_TOKEN }}

dbt run writes target/manifest.json. Visivo reads that manifest during visivo run and turns each dbt™ model into a Visivo model, so there is no separate sync step.

Dependency management:

Turn on the dbt™ integration in project.visivo.yml and point it at the dbt™ project:

# project.visivo.yml
name: analytics_platform

dbt:
  enabled: true
  dbt_project_yml_location: dbt
  profiles_yml_location: dbt

includes:
  - path: dashboards/

Visivo creates one source for each target in profiles.yml (named <profile>_<target>) and one model for each dbt™ model, with the same name as the dbt™ model. An Insight that references a model name that no longer exists in dbt™ fails at visivo compile, so a broken reference stops the pipeline before it reaches a dashboard. Use visivo run --dbt-target <target> to point the dashboards at a different dbt™ target.

Example workflow

A dashboard on a dbt™ model:

The Insight below references revenue_summary, a dbt™ model, the same way it would reference any Visivo model:

# dashboards/executive/revenue.yml
# revenue_summary is a dbt model. Visivo builds it from the dbt manifest.
insights:
  - name: revenue-trend
    description: Monthly revenue from the revenue_summary dbt model
    props:
      type: scatter
      mode: lines
      x: ?{ ${ref(revenue_summary).month} }
      y: ?{ ${ref(revenue_summary).total_revenue} }
    interactions:
      - sort: ?{ ${ref(revenue_summary).month} ASC }

charts:
  - name: revenue-trend-chart
    insights:
      - ${ref(revenue-trend)}
    layout:
      title:
        text: Revenue Trend

dashboards:
  - name: Executive Dashboard
    rows:
      - height: medium
        items:
          - chart: ${ref(revenue-trend-chart)}

Testing:

Data quality tests stay in dbt™ (dbt test). Visivo tests assert on the values your Insights compute, and visivo test runs them:

# dashboards/executive/revenue.yml (continued)
tests:
  - name: revenue-trend-has-data
    assertions:
      - ">{ len( ${ref(revenue-trend).props.x} ) > 0 }"

  - name: revenue-is-positive
    assertions:
      - ">{ numpy.sum( ${ref(revenue-trend).props.y} ) > 0 }"

How it scales

The same structure holds as the project grows:

Modular growth: Add a new model to dbt™, and it is available as a Visivo model on the next visivo run.

Team scalability: Data engineers work in dbt™, analysts work on dashboards, and the model name is the interface between them.

Performance scalability: dbt™ does the heavy transformation in the warehouse. The BI layer only visualizes the result.

Maintenance scalability: A change in one place reaches everything that depends on it.

The result is:

  • No metric mismatches between the warehouse and the dashboard
  • Faster end-to-end development
  • Consistent definitions
  • A stack that grows with the team

Running dbt™ and BI-as-code from one repository treats analytics as an engineering discipline: models and dashboards are code, reviewed and tested together. /get-started covers the install, and the Visivo documentation has the full reference for the dbt configuration block.

Install command copied