Author Image

CEO & Co-founder of Visivo

Reproducible BI Environments for Reliable Data Workflows

Learn how to create reproducible BI environments that ensure consistent analytics across development, staging, and production.

Reproducible BI environment architecture

"It works on my machine" is the sentence that tells you environments have drifted. The DORA State of DevOps Report found that elite performers deploy 208x more frequently than low performers, with a much lower change failure rate. In BI the symptom is a dashboard that shows one number in development and another in production, and once people see that, they stop trusting the data. A reproducible BI environment is one any team member can recreate, so the same project returns the same results wherever it runs.

What reproducible means in BI

A reproducible environment is one any team member can recreate exactly: the same data connections, transformations, dashboards and results, no matter when or where they set it up. It is the difference between "follow these 47 manual steps and hope" and "run this command."

Reproducibility in BI covers four layers:

  • Data layer: Same data sources, schemas, and sample data
  • Transformation layer: Identical SQL queries and business logic
  • Visualization layer: Same dashboards, charts, and calculations
  • Configuration layer: Identical settings, permissions, and schedules

Without it, the friction is constant. Developers cannot reproduce production issues locally. New hires spend weeks getting a working setup. Small changes become risky because the test environment does not match production. Anaconda's State of Data Science report found that data scientists spend 45% of their time on data preparation and cleaning. Reproducible environments give some of that time back.

What goes wrong without it

According to Gartner analyst Nick Heudecker, 85% of big data projects fail, often because of environment inconsistencies. The failure takes three forms.

Development and production disagree. A dashboard works in development and fails in production because:

  • Database schemas differ between environments
  • Test data doesn't reflect production edge cases
  • Configuration settings were manually adjusted and forgotten
  • Dependencies exist in production that weren't documented
# Common discrepancy example
development:
  database_version: 13.2
  timezone: "America/New_York"
  date_format: "MM/DD/YYYY"

production:
  database_version: 12.8 # Different version!
  timezone: "UTC" # Different timezone!
  date_format: "DD/MM/YYYY" # Different format!


# Result: Date calculations work in dev, fail in prod

Test results cannot be trusted. When environments differ, a passing test says nothing about production:

  • Performance tests in staging show 2-second load times
  • Production takes 30 seconds with the same query
  • Investigation reveals staging has different indexes
  • Months of performance testing becomes worthless

Onboarding takes weeks. A new team member has to:

  1. Install BI tools (but which versions?)
  2. Configure data connections (but to which databases?)
  3. Import dashboard definitions (but from where?)
  4. Set up test data (but how much and which kind?)
  5. Configure permissions (but with which roles?)

Each step has undocumented variations that live only in someone's memory.

Building a reproducible BI environment

The pattern below uses Docker for the database, a pinned CLI for the tool, and env files for everything that differs between environments.

Infrastructure as code with Visivo

A Visivo project is YAML in Git. Sources, models, the semantic layer, Insights, charts and dashboards all live in version-controlled files, which is BI-as-code applied to the whole environment. Wrap the project in a Docker Compose file and the database it runs against comes up the same way on every machine:

# docker-compose.yml - local Postgres plus Visivo for development
services:
  database:
    image: postgres:16
    environment:
      POSTGRES_DB: analytics
      POSTGRES_USER: ${PG_USER:-visivo}
      POSTGRES_PASSWORD: ${PG_PASSWORD:-visivo_dev}
    volumes:
      - ./data/init-scripts:/docker-entrypoint-initdb.d
      - postgres_data:/var/lib/postgresql/data
    ports:
      - "5432:5432"
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U ${PG_USER:-visivo}"]
      interval: 10s
      timeout: 5s
      retries: 5

  visivo:
    build: .
    working_dir: /app
    environment:
      PG_HOST: database
      PG_USER: ${PG_USER:-visivo}
      PG_PASSWORD: ${PG_PASSWORD:-visivo_dev}
    volumes:
      - .:/app
    ports:
      - "8000:8000"
    depends_on:
      database:
        condition: service_healthy

volumes:
  postgres_data:

The source definition in the project file reads the PG_* variables through ${env.VAR_NAME}. That file is shown further down.

A containerized development environment

The Dockerfile pins Python and the Visivo CLI, so every container runs the same versions:

# Dockerfile
FROM python:3.12-slim

RUN apt-get update && apt-get install -y \
    postgresql-client \
    curl \
    git \
    && rm -rf /var/lib/apt/lists/*

WORKDIR /app

# Pin the Visivo CLI so every container runs the same version
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy the project: project.visivo.yml plus any files it includes
COPY . .

HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
  CMD curl -f http://localhost:8000/ || exit 1

# visivo serve listens on port 8000 by default
EXPOSE 8000

CMD ["visivo", "serve"]

Environment management with env files

Visivo keeps environment differences out of the project file. Source connection fields read environment variables with ${env.VAR_NAME}, Visivo loads a .env file from the project root automatically, and visivo --env-file <file> points any command at a different file. One script can then set up any environment:

#!/bin/bash
# setup-visivo-environment.sh

set -e

ENVIRONMENT=${1:-development}
ENV_FILE=".env.$ENVIRONMENT"

echo "Configuring for environment: $ENVIRONMENT"

# Install the pinned Visivo CLI
pip install -r requirements.txt
visivo --version

if [ ! -f "$ENV_FILE" ]; then
    echo "Missing $ENV_FILE. Copy .env.example and fill in the values."
    exit 1
fi

# Parse and validate the project against this environment's variables
visivo --env-file "$ENV_FILE" compile

# Run every model and Insight query, then assert on the results
visivo --env-file "$ENV_FILE" run
visivo --env-file "$ENV_FILE" test

echo "Visivo environment setup complete!"
echo "Start the development server with: visivo --env-file $ENV_FILE serve"

Version locking

Visivo is a Python package, so pin it the way you pin any other dependency:

# requirements.txt
visivo==2.3.0

A project file can also carry a cli_version field, which records the CLI version that created the project.

One project file, many env files

Instead of one project file per environment, keep a single project file and let only the connection values change. Commit a .env.example that lists every variable, and keep the real .env.* files out of Git:

# project.visivo.yml
name: Analytics Platform

sources:
  - name: analytics_db
    type: postgresql
    host: ${env.PG_HOST}
    port: 5432
    database: ${env.PG_DATABASE}
    username: ${env.PG_USER}
    password: ${env.PG_PASSWORD}
    connection_pool_size: 4

defaults:
  source_name: analytics_db
# .env.development
PG_HOST=localhost
PG_DATABASE=analytics_dev
PG_USER=visivo
PG_PASSWORD=visivo_dev
# .env.production (never committed; CI injects these as secrets)
PG_HOST=prod-db.internal.example.com
PG_DATABASE=analytics
PG_USER=visivo_reader
PG_PASSWORD=...

What you get

Once the setup above is in place, the same project behaves the same way in every environment. That changes several everyday workflows.

The same tests in every environment. Every Visivo environment runs the same project, and the same tests check the results. A Visivo test is a Python expression that asserts on the computed values of an Insight, so it runs unchanged against each environment:

tests:
  - name: revenue-trend-has-data
    assertions:
      - ">{ len( ${ref(revenue-trend).props.x} ) > 0 }"

  - name: revenue-grains-match
    assertions:
      - ">{ numpy.sum( ${ref(revenue-per-week).props.y} ) == numpy.sum( ${ref(revenue-per-month).props.y} ) }"
visivo --env-file .env.staging run && visivo --env-file .env.staging test
visivo --env-file .env.production run && visivo --env-file .env.production test

Onboarding in minutes. A new team member clones the repository and runs one command:

# New developer onboarding with Visivo
git clone https://github.com/company/analytics-platform
cd analytics-platform
cp .env.example .env

# One-command setup
docker compose up -d

# Or native setup
./setup-visivo-environment.sh development

# Verify everything works
visivo serve
# Navigate to http://localhost:8000

# Compare to manual BI setup:
# - 2-3 days installing and configuring BI tools
# - 1-2 days debugging database connections
# - 1 week learning proprietary BI interfaces
# - Additional time understanding data models and business logic

Deployments that work locally work in production. The same commands run at every stage:

# Local testing workflow
visivo compile  # Parse and validate the project
visivo run      # Run every model and Insight query
visivo test     # Assert on the computed Insight values
visivo serve    # Review the dashboards locally

# Deploy with confidence
visivo --env-file .env.staging run
visivo deploy -s staging      # Deploy to a staging stage in Visivo Cloud
visivo --env-file .env.production run
visivo deploy -s production   # Deploy to production

Recovery from Git. Everything needed to rebuild the project is in the repository:

# Disaster recovery scenario: the machine that built your dashboards is gone

# Restore the project
git clone https://github.com/company/analytics-platform
cd analytics-platform

# Restore the environment
docker compose down
docker compose up -d

# Rebuild the data and check it
visivo --env-file .env.production run
visivo --env-file .env.production test

# Publish again
visivo deploy -s production

CI runs the real tests. The same visivo run and visivo test run in GitHub Actions against each environment's secrets:

# .github/workflows/visivo-testing.yml
name: Visivo Testing Pipeline

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    env:
      PG_HOST: ${{ secrets.PG_HOST }}
      PG_DATABASE: ${{ secrets.PG_DATABASE }}
      PG_USER: ${{ secrets.PG_USER }}
      PG_PASSWORD: ${{ secrets.PG_PASSWORD }}

    steps:
      - uses: actions/checkout@v4
      - name: Setup Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install Visivo
        run: pip install -r requirements.txt

      - name: Run and test
        run: |
          visivo run
          visivo test

      - name: Deploy
        if: github.ref == 'refs/heads/main'
        env:
          VISIVO_TOKEN: ${{ secrets.VISIVO_TOKEN }}
        run: visivo deploy -s production

Reproducing a colleague's bug. Check out their branch and bring up the same environment:

# Debugging a colleague's Visivo issue
# Colleague: "The revenue dashboard breaks with Q4 data"

# You can reproduce immediately:
git checkout colleague-branch

# Recreate the exact environment, including the seeded sample data
docker compose down
docker compose up -d

# Run the project and start the server
visivo run
visivo serve

# Navigate to the problematic dashboard at http://localhost:8000

# Without reproducible environments:
# "Revenue dashboard works fine on my machine ¯\_(ツ)_/¯"
# "My database has different data than yours"
# "I'm using a different version of Visivo"
# Result: Hours or days to debug simple issues

Further patterns

Multi-environment testing

Because environment differences live only in variables, the same pipeline can test every environment. GitHub environments hold a separate set of secrets for each:

# .github/workflows/multi-environment-test.yml
name: Multi-Environment Visivo Testing

on: [pull_request]

jobs:
  test-environments:
    strategy:
      matrix:
        environment: [development, staging, production-replica]
    runs-on: ubuntu-latest
    environment: ${{ matrix.environment }}
    env:
      PG_HOST: ${{ secrets.PG_HOST }}
      PG_DATABASE: ${{ secrets.PG_DATABASE }}
      PG_USER: ${{ secrets.PG_USER }}
      PG_PASSWORD: ${{ secrets.PG_PASSWORD }}

    steps:
      - uses: actions/checkout@v4
      - name: Setup Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install Visivo
        run: pip install -r requirements.txt

      - name: Test in ${{ matrix.environment }}
        run: |
          visivo run
          visivo test

Configuration drift detection

With one project file, configuration cannot drift between environments: every environment runs the same YAML at the same commit. What can drift is the set of variables each environment defines. A short script catches a variable that one environment is missing:

#!/bin/bash
# detect-config-drift.sh

echo "Comparing variable names across environment files..."

for env_file in .env.development .env.staging .env.production; do
    if ! diff <(cut -d= -f1 .env.example | sort) <(cut -d= -f1 "$env_file" | sort); then
        echo "Drift: $env_file does not match .env.example"
        exit 1
    fi
done

echo "No configuration drift found."

Automated provisioning

An Ansible playbook can bring up a new server from nothing:

# scripts/provision-environment.yml
- name: Provision New Visivo Environment
  hosts: new-servers
  tasks:
    - name: Install Docker
      apt:
        name: docker.io
        state: present

    - name: Clone analytics repository
      git:
        repo: https://github.com/company/analytics-platform.git
        dest: /opt/analytics-platform

    - name: Set up environment variables
      template:
        src: environment.env.j2
        dest: /opt/analytics-platform/.env

    - name: Start Visivo environment
      docker_compose:
        project_src: /opt/analytics-platform
        state: present

    - name: Wait for Visivo to be ready
      uri:
        url: http://localhost:8000/
        status_code: 200
      register: result
      until: result.status == 200
      retries: 30
      delay: 10

    - name: Run environment validation
      command: docker compose exec -T visivo sh -c "visivo run && visivo test"
      args:
        chdir: /opt/analytics-platform

Seeded sample data

A Visivo source can declare seeds: commands whose CSV output Visivo loads into a table on that source before any model queries it. Point a local DuckDB source at sample files in the repository, and every developer gets the same data:

sources:
  - name: local_sample
    type: duckdb
    database: target/local.duckdb
    seeds:
      - table_name: customers
        args: [cat, data/customers_sample.csv]
      - table_name: orders
        args: [cat, data/orders_sample.csv]

Models that do not name a source use the project default. The -s flag overrides that default for one command, so the same models run against the sample data offline and against the warehouse in CI:

# Work offline against the seeded sample data
visivo run -s local_sample
visivo serve -s local_sample

# Run against the warehouse configured in .env.staging
visivo --env-file .env.staging run

A four-week rollout

Phase 1: containerization (week 1)

Start by putting the project and its database in containers:

# Quick start containerization
cd your-visivo-project

# Create Dockerfile
cat > Dockerfile << 'EOF'
FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
EXPOSE 8000
CMD ["visivo", "serve"]
EOF

# Create requirements.txt
echo "visivo==2.3.0" > requirements.txt

# Create docker-compose.yml
cat > docker-compose.yml << 'EOF'
services:
  visivo:
    build: .
    ports:
      - "8000:8000"
    environment:
      - PG_HOST=db
      - PG_DATABASE=analytics
      - PG_USER=user
      - PG_PASSWORD=pass
  db:
    image: postgres:16
    environment:
      - POSTGRES_DB=analytics
      - POSTGRES_USER=user
      - POSTGRES_PASSWORD=pass
    volumes:
      - postgres_data:/var/lib/postgresql/data
volumes:
  postgres_data:
EOF

# Test the setup
docker compose up

Phase 2: environment configuration (week 2)

Move connection details into environment variables:

# Point the source at environment variables
cat > project.visivo.yml << 'EOF'
name: Analytics Platform

includes:
  - path: models/
  - path: dashboards/

sources:
  - name: analytics_db
    type: postgresql
    host: ${env.PG_HOST}
    database: ${env.PG_DATABASE}
    username: ${env.PG_USER}
    password: ${env.PG_PASSWORD}

defaults:
  source_name: analytics_db
EOF

# Document every variable, without values
cat > .env.example << 'EOF'
PG_HOST=
PG_DATABASE=
PG_USER=
PG_PASSWORD=
EOF

# Keep real credentials out of Git
echo ".env*" >> .gitignore
echo "!.env.example" >> .gitignore

Phase 3: automated testing (week 3)

Add tests and run them in CI:

# Create test suite
mkdir tests

cat > tests/environment-validation.visivo.yml << 'EOF'
tests:
  - name: core-metrics-have-data
    assertions:
      - ">{ len( ${ref(core-metrics-trend).props.x} ) > 0 }"

  - name: no-negative-revenue
    assertions:
      - ">{ numpy.min( ${ref(core-metrics-trend).props.y} ) >= 0 }"
EOF

# Add GitHub Actions workflow
mkdir -p .github/workflows

cat > .github/workflows/test-environments.yml << 'EOF'
name: Test Visivo Environments

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        environment: [development, staging]
    environment: ${{ matrix.environment }}
    env:
      PG_HOST: ${{ secrets.PG_HOST }}
      PG_DATABASE: ${{ secrets.PG_DATABASE }}
      PG_USER: ${{ secrets.PG_USER }}
      PG_PASSWORD: ${{ secrets.PG_PASSWORD }}

    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.12'
      - run: pip install -r requirements.txt
      - run: |
          visivo run
          visivo test
EOF

Then add - path: tests/ under includes: in project.visivo.yml so Visivo picks up the new tests.

Phase 4: production deployment (week 4)

Put the checks and the deploy in one script:

# Production deployment script
cat > deploy-production.sh << 'EOF'
#!/bin/bash
set -e

echo "Deploying Visivo to production..."

# Pre-deployment checks
echo "Running pre-deployment validation..."
visivo --env-file .env.production compile
visivo --env-file .env.production run
visivo --env-file .env.production test

# The previous version stays in Git: roll back with git revert and deploy again

# Deploy
echo "Deploying to production..."
visivo deploy -s production

echo "Production deployment complete!"
EOF

chmod +x deploy-production.sh

What to expect

Teams that make their Visivo environments reproducible can expect improvements like these:

  • Fewer "works on my machine" issues, and fewer environment-related bugs in production
  • Onboarding measured in hours instead of days
  • Less time spent debugging environments and more spent on analytics
  • Fewer rollbacks caused by environment differences
  • Any team member can reproduce an issue locally
  • Faster recovery from failures, with every environment change recorded in Git

Conclusion

Reproducible BI environments are not a luxury. They are what makes a data workflow reliable. Visivo's YAML configuration and pinned, pip-installable CLI make reproducibility the default: the project is code, the CLI is a dependency, and the only thing that differs between environments is an env file.

The payoff compounds. Developers deploy analytics changes with confidence, new team members contribute in their first week, stakeholders trust the numbers because every environment computes them the same way, and recovery starts with git clone.

The first step is a decision: no more manual environment setup. Put every configuration, dependency and setting in code. If you want to start from a project that already works this way, get started with Visivo, or browse the examples to see projects built on the same pattern.

Install command copied