Diffly

Quickly compare your SQL data with clarity and style.

⭐ Like what we're doing? Give us a star ⬆️

Diffly is a Rust CLI and library for generating row-level data diffs between SQL databases such as PostgreSQL, MySQL, MariaDB, and SQLite. It produces human-readable tables with color-coded inserts, updates, deletes, and totals, as well as machine-readable JSON output for CI/CD pipelines, automated testing, or reporting. Perfect for QA, migrations, ETL verification, or auditing database changes.

👤 Who is Diffly for?

Diffly is built for developers, data engineers, and teams who need to trust their data, not just their schema.

It is especially useful for:

• Backend and platform engineers validating database migrations
• Data engineers verifying ETL and replication pipelines
• QA teams performing data-level regression testing
• DevOps teams automating checks in CI/CD
• Auditors and compliance teams tracking record-level changes

If your system moves or transforms data between databases, Diffly helps ensure nothing was lost, altered, or corrupted.

⏱️ When should you use Diffly?

Use Diffly whenever you need to answer:

• Did my data change?
• Which rows changed?
• Were rows inserted, updated, or deleted?
• Did my migration or sync behave as expected?

Typical scenarios include:

• After running a migration script
• After syncing or replicating data
• Before deploying to production
• After importing or transforming datasets
• When debugging inconsistencies between environments

Diffly compares actual row data, not SQL dumps or schemas, and reports differences in both human-readable tables and machine-readable JSON.

🎯 What problem does Diffly solve?

Most tools focus on:

• schema diffs
• SQL files
• structure

But production issues usually come from:

• missing rows
• wrong values
• duplicated data
• partial imports

Diffly focuses on what really matters:
the data itself.

It acts as a safety net between:

source database → transformation / migration → target database

by validating the result, not just the process.

🧭 Positioning

Diffly is to databases what git diff is to source code — but for data, not schema.

🧬 3-Way Merge and Conflict Detection

Diffly can perform a 3-way merge to not only see the difference between a source and a target, but to understand how both have changed relative to a common ancestor, making it easy to detect conflicting changes.

This process works in two steps:

Step 1: Create a Snapshot

First, create a snapshot of your database when it's in a known "base" or "common ancestor" state. This command inspects the database defined in your config and saves its state to a snapshot directory.

# Create a snapshot from the database configured as `target`
diffly snapshot --config ./diffly.toml

Step 2: Check for Conflicts

Later, after changes have been made to your databases, run the check-conflicts command. This compares your current source and target databases against the snapshot to identify divergent changes.

# Compare the current source and target databases against the snapshot
diffly check-conflicts --config ./diffly.toml --snapshot ./my-snapshot

The result is a rich diff that shows you not just the final state, but how you got there, clearly identifying rows that were changed in the source, the target, or even conflicting changes made to both.

🚨 IMPORTANT — SNAPSHOT CONSISTENCY REQUIRED

Diffly’s conflict detection is only valid if the snapshot reflects the exact state of the target database at snapshot time.

If the target database changes after the snapshot is created:

• conflicts may be missed
• valid changes may be flagged incorrectly
• generated diffs may be unsafe to apply

Always ensure one of the following:

✅ Snapshot is taken immediately before diff
✅ Target database is locked or frozen during snapshot
✅ No writes occur between snapshot and diff

This constraint is fundamental to safe 3-way merge behavior.

🏛️ Architecture

---
config:
  theme: neutral
  look: handDrawn
---
flowchart LR
    SDB[(Source DB)]
    TDB[(Target DB)]
    Diffly[Diffly<br/>Diff Engine]
    
    subgraph DIFFLY_ENGINE["Diffly Engine"]
      direction TB
      DIFF[Diff<br/>Command]
      JSON[Diff JSON Format]
      SQL[Migration Atomic SQL]
      HTML[Diff Report HTML]
      subgraph MERGE["3 Way Merge"]
          SNAP[Snapshot<br/>Command JSON]
          SNAPJSON[Snapshot JSON]
          CONFLICTS[Check-Conflicts<br/>Command]
      end
    end

    SDB -->|Fetch Info| Diffly
    TDB -->|Fetch Info| Diffly

    Diffly -->|diff| DIFF
    DIFF -->|generates| JSON
    DIFF -->|generates| SQL
    DIFF -->|generates| HTML

    Diffly -->|snapshot| SNAP
    SNAP -->|generates| SNAPJSON
    Diffly -->|check-conflicts| CONFLICTS
    CONFLICTS -->|Uses| SNAPJSON

    style SDB fill:#e1f5ff
    style TDB fill:#f3e5f5
    style Diffly fill:#fff3e0
    style DIFF fill:#fffde7
    style JSON fill:#e8f5e9
    style SQL fill:#fce4ec
    style HTML fill:#f1f8e9
    style SNAP fill:#e3fcec
    style CONFLICTS fill:#ffe0e0
    style SNAPJSON fill:#dcedc8
    style DIFFLY_ENGINE fill:#f7fafc,stroke:#aaa,stroke-width:2px
    style MERGE fill:#e1f0fa,stroke:#7dc5f3,stroke-width:2px

🌿 Dialects

Today, Diffly implements these dialects:

• Postgresql
• Mysql
• MariaDB
• Sqlite

Please feel free to fix or add new dialect by forking this repository and use a pull request, we ❤️ having feedbacks and contributions!

⚡ Quick Start

🐳 Run examples in Docker

Each dialect has a self-contained example that spins up the database, imports fixtures, runs diffly, and tears everything down:

just example-pg       # PostgreSQL
just example-mysql    # MySQL
just example-mariadb  # MariaDB
just example-sqlite   # SQLite
just example-all      # all four sequentially

Results are written to ./output/<driver>/ (configurable in each example's config.toml):

output/postgresql/cs_20260211_***/
  *.json   ← Structured changeset
  *.sql    ← Atomic migration SQL (BEGIN/COMMIT)
  *.html   ← Visual report (open in a browser)

📍 Run locally (with cargo)

# 1. Start only the database (optional — skip if you have your own)
docker compose -f examples/postgresql/docker-compose.yml up postgres -d

# 2. Copy the example config and adjust host / credentials
cp ./examples/postgresql/config.toml ./my-config.toml

# 3. Run
just build && ./target/debug/diffly --config ./my-config.toml diff

# Dry run — summary only, no files written
./target/debug/diffly --config ./my-config.toml diff --dry-run

# Single format
./target/debug/diffly --config ./my-config.toml diff --format html

📚 Run as library

# 1. Start only the database (optional)
docker compose -f examples/postgresql/docker-compose.yml up postgres -d

# 2. Copy and adjust the config
cp ./examples/postgresql/config.toml ./my-config.toml

# 3. Run the library example
cargo run --example diff_as_lib -- ./my-config.toml

🔧 Base configuration (config.toml)

[source]
host = "localhost"
port = 5432
dbname = "diffly"
user = "diffly"
password = "diffly"
schema = "sandbox_admin1"    # Sandbox schema

[target]
host = "localhost"
port = 5432
dbname = "diffly"
user = "diffly"
password = "diffly"
schema = "staging"           # Target schema

[diff]
[[diff.tables]]
name = "pricing_rules"       # Table to compare
primary_key = ["id"]         # Simple PK

[[diff.tables]]
name = "tax_rules"
primary_key = ["region_code", "product_category"]  # Composite PK

[output]
dir = "./output"

Note: SQLite has no schema concept. Source and target must be separate database files. The schema config field is ignored.

➡️ Outputs

Actually Diffly generates 3 formats:

JSON

Complete Changeset with before/after for each modification, PK, modified columns, resume and the sql query to apply for each modification (be careful to run them inside an atomic transaction).

SQL

Atomic transaction BEGIN/COMMIT with DELETE → UPDATE → INSERT (secure order), useful for data migration.

The generated .sql diff file adapts to the target driver automatically:

HTML

Visual report with dark/light-mode made for humans.

We use Sailfish templating internally for better isolation. You can change the way the template engine works by modifying the sailfish.toml file on root directory if necessary (see configuration for more informations).

🈂️ Examples

For more examples, visit our examples repository.

🤝 Contribution

We welcome contributions! Please refer to the contribution guidelines (link needed if available). Join our Github Discussions for questions and discussions.

♥️ Contributor Thanks

Big thanks to everyone who's been part of the Diffly journey, whether you've built a plugin, opened an issue, dropped a pull request, or just helped someone out on GitHub Discussions.

Diffly is a community effort, and it keeps getting better because of people like you.

📄 License

Licensed under the MIT License, Copyright © 2026-present Diffly.