Versioning Cloud-Optimized GeoTIFFs with DVC

Version cloud-optimized GeoTIFF (COG) datasets with DVC and an S3/MinIO remote: dvc add, push, tag a release, and reproduce an exact raster version for ML.

TL;DR Answer

Point DVC at an S3 or MinIO remote, run dvc add on your directory of cloud-optimized GeoTIFFs, commit the generated .dvc pointer to git, then dvc push the bytes to object storage. Tag the release with git, and anyone can reproduce the exact raster version with git checkout <tag> followed by dvc checkout. Because COGs are single content-addressed files, DVC hashes each one to a stable MD5 and deduplicates cleanly across releases. This page is the concrete raster how-to for the Spatial Dataset Versioning with DVC and LakeFS workflow — the parent covers when to reach for DVC versus LakeFS; here we wire up COGs end to end.

dvc init
dvc remote add -d storage s3://geo-ml-data/dvcstore
dvc add data/cog/
git add data/cog.dvc data/.gitignore
git commit -m "Track COG training rasters v1.0.0"
git tag -a cog-v1.0.0 -m "Land-cover COG stack, 2026 growing season"
dvc push

Part of: Spatial Dataset Versioning with DVC and LakeFS

Why COGs and DVC Are a Natural Fit

DVC tracks large files by content, not by name. When you run dvc add, it computes an MD5 hash of each file, moves the bytes into the local .dvc/cache under that hash, and writes a small .dvc pointer that git tracks in its place. The pointer is a few hundred bytes; the multi-gigabyte raster never touches git history. Reproducibility comes from the fact that the same bytes always hash to the same value.

A cloud-optimized GeoTIFF is an ideal payload for this model. It is a single self-contained file with internal tiling and pre-computed overviews, so one raster maps to exactly one hash. There is no sidecar .aux.xml, no external overview .ovr, and no tile directory to keep in sync — all of which would fragment the hash and complicate the pointer. When you version a stack of COGs, DVC deduplicates automatically: rasters whose pixels are unchanged between releases share the same object in the remote and are never re-uploaded.

The catch is that DVC hashes the whole file. It has no concept of GeoTIFF internals, so a header-only edit — rewriting the nodata value, appending a TIFFTAG_DATETIME, or re-running gdal_translate with a different creation profile — produces new bytes and therefore a new MD5. The discipline that keeps a COG dataset stable is to freeze the creation profile and only rewrite pixels when the data genuinely changes. Get that right and the workflow below gives you byte-exact, deduplicated raster versions that a training run can pin to forever.

Set Up the Repository and Remote

Initialise git and DVC in the project root. dvc init creates the .dvc/ directory and a .dvc/config file that git tracks.

git init
dvc init
git add .dvc/.gitignore .dvc/config .dvcignore
git commit -m "Initialise DVC"

Now add a remote. DVC speaks the S3 protocol for both AWS S3 and a self-hosted MinIO server; the only difference is the endpointurl. Set -d to make it the default push/pull target.

# AWS S3
dvc remote add -d storage s3://geo-ml-data/dvcstore

# Or a self-hosted MinIO server
dvc remote add -d storage s3://geo-ml-data/dvcstore
dvc remote modify storage endpointurl http://minio.internal:9000

Keep credentials out of git. DVC reads standard AWS environment variables and profiles, so store them in .dvc/config.local (git-ignored by default) or the environment.

dvc remote modify --local storage access_key_id "$MINIO_ACCESS_KEY"
dvc remote modify --local storage secret_access_key "$MINIO_SECRET_KEY"
git add .dvc/config
git commit -m "Add S3/MinIO DVC remote"

The tracked .dvc/config now records the bucket path and endpoint; secrets live only in config.local. This is the same object-storage backing that container-based serving expects, so it pairs naturally with Containerizing Geospatial Inference Pipelines downstream.

Track a Directory of COGs

Place your rasters under a stable path and add the whole directory in one call. Tracking the directory (rather than each file) means new COGs added later are picked up by re-running dvc add without editing a growing list of pointers.

dvc add data/cog/

DVC writes two things: data/cog.dvc, the pointer that records the MD5 and size of the directory tree, and an entry in data/.gitignore that stops git from tracking the rasters directly. Commit both to git.

git add data/cog.dvc data/.gitignore
git commit -m "Track COG training rasters v1.0.0"

Inspect the pointer to see what git is actually storing. For a directory, the md5 ends in .dir and references a JSON manifest of every file and its individual hash.

# data/cog.dvc
outs:
  - md5: a1b2c3d4e5f60718293a4b5c6d7e8f90.dir
    size: 4823947213
    nfiles: 24
    hash: md5
    path: cog

Push the bytes to the remote. Only objects not already present are uploaded, so re-pushing after a small change is cheap.

dvc push

At this point git holds a 300-byte pointer and object storage holds 4.8 GB of COGs, addressed by content. The workspace and remote are in sync.

The Versioning Lifecycle

The diagram below traces one raster release through DVC and git, and back out again when someone reproduces it.

DVC versioning lifecycle for cloud-optimized GeoTIFFs A workflow showing a COG hashed by dvc add into a cache, a small .dvc pointer committed to git and tagged, the bytes pushed to an S3 or MinIO remote, and reproduction via git checkout plus dvc checkout restoring a byte-identical raster. COG raster data/cog/*.tif dvc add MD5 hash .dvc/cache cog.dvc pointer git + tag commit S3 / MinIO remote store push Reproduce a version git checkout tag dvc checkout same MD5 pull bytes

A dvc.yaml Pipeline for COG Generation

Rather than hand-running gdal_translate, encode COG creation as a DVC pipeline stage. This makes the transformation from raw GeoTIFFs to COGs itself reproducible: DVC records the command, its dependencies, and its outputs, and re-runs the stage only when an input changes.

# dvc.yaml
stages:
  build_cogs:
    cmd: python scripts/make_cogs.py data/raw data/cog
    deps:
      - data/raw
      - scripts/make_cogs.py
    outs:
      - data/cog
    frozen: false

The generation script fixes a single creation profile so the output bytes are deterministic. Pinning COMPRESS, BLOCKSIZE, and PREDICTOR is what stops DVC from re-hashing rasters on every run.

gdal_translate raw.tif cog.tif \
  -of COG \
  -co COMPRESS=DEFLATE \
  -co PREDICTOR=2 \
  -co BLOCKSIZE=512 \
  -co OVERVIEWS=IGNORE_EXISTING \
  -co NUM_THREADS=ALL_CPUS

Run the pipeline with dvc repro. DVC tracks the outputs automatically — you do not call dvc add on data/cog when it is produced by a stage.

dvc repro build_cogs
git add dvc.yaml dvc.lock
git commit -m "Add reproducible COG build stage"
dvc push

The generated dvc.lock pins the exact hash of every input and output, so a colleague running dvc repro gets the identical COG stack or a clear signal that an input drifted. If your raw rasters arrive in a different projection, run the reproject step first — see Reprojecting a Raster with rasterio.warp.reproject — so that CRS is fixed before the byte-level hash is computed.

Validate COGs Before You Commit

A malformed COG (missing overviews, non-tiled layout) will still hash and push fine, but it defeats the purpose of the format for downstream ML tiling. Add a lightweight validation gate before dvc add so broken rasters never enter a version.

"""Validate that every raster in a directory is a well-formed COG."""
from pathlib import Path
import sys

import rasterio


def is_valid_cog(path: Path) -> tuple[bool, str]:
    """Return (ok, reason) for a single raster.

    A COG must be internally tiled and carry at least one overview level.
    """
    with rasterio.open(path) as src:
        if not src.profile.get("tiled", False):
            return False, "not internally tiled"
        if not src.overviews(1):
            return False, "no overviews present"
        if src.crs is None:
            return False, "missing CRS"
    return True, "ok"


def main(directory: str) -> int:
    cogs = sorted(Path(directory).glob("*.tif"))
    if not cogs:
        print(f"No .tif files found in {directory}", file=sys.stderr)
        return 1

    failures = 0
    for cog in cogs:
        ok, reason = is_valid_cog(cog)
        status = "PASS" if ok else "FAIL"
        print(f"[{status}] {cog.name}: {reason}")
        failures += not ok

    return 1 if failures else 0


if __name__ == "__main__":
    raise SystemExit(main(sys.argv[1]))

Wire it in ahead of tracking so a failed check aborts the release before anything is hashed:

python scripts/validate_cogs.py data/cog && dvc add data/cog/

Tag a Release and Reproduce It

A dataset version is just a git tag on the commit that holds the .dvc pointer. Because the pointer pins every raster’s MD5, the tag is a permanent, resolvable reference to an exact set of bytes.

git tag -a cog-v1.1.0 -m "Add 2026 Q2 Sentinel-2 composites"
dvc push
git push origin master --tags

To reproduce the data a model was trained on, check out the tag and let DVC materialise the matching COGs from the remote. git checkout restores the pointer; dvc checkout restores the bytes (pulling first if the cache is cold).

git checkout cog-v1.1.0
dvc pull        # fetch objects for this tag if not cached
dvc checkout    # link the exact COGs into the workspace

Anyone cloning the repository fresh runs the same two commands to hydrate the workspace. The rasters they get are byte-identical to what you pushed, which is the property that makes a training run auditable — and the same property that lets Spatial Cross-Validation Strategies be re-run against a fixed input months later.

Verification

Two checks confirm the version is sound. First, dvc status must report a clean workspace — no changes between the pointer, the cache, and the files on disk.

$ dvc status
Data and pipelines are up to date.

Second, prove that a checkout reproduces identical bytes. Record the directory hash from the .dvc pointer, switch away and back, and confirm DVC recomputes the same MD5.

# Hash recorded at release time
$ grep md5 data/cog.dvc
    md5: a1b2c3d4e5f60718293a4b5c6d7e8f90.dir

# After git checkout <tag> && dvc checkout
$ dvc data status --granular
Not in cache:
(none)

$ md5sum data/cog/*.tif | sort | md5sum
7f3a9c21b8e04d65a1f9c3e7b204d8a1  -

If the .dir hash in the pointer matches after checkout and the per-file md5sum roll-up is stable across machines, the raster version is fully reproducible. If dvc status shows the COGs as modified when you have not touched them, a metadata rewrite has changed the bytes — re-run the build with the frozen profile above and confirm the hash returns to its recorded value.

FAQ

Why do cloud-optimized GeoTIFFs work so well with DVC?

DVC is content-addressed: it hashes each file with MD5 and stores the object under that hash. A COG is a single self-contained file with internal tiling and overviews, so one raster maps to one stable hash. That makes dvc add, dvc push, and dvc pull deduplicate cleanly across dataset releases and lets DVC skip re-uploading rasters whose bytes have not changed. There are no sidecar files to fragment the hash.

How do I stop DVC re-hashing a COG that I only re-tagged?

DVC hashes the whole file, so any byte-level change — including a GDAL metadata or nodata rewrite that touches the header — produces a new MD5 and a new object. Freeze your COG creation profile (compression, blocksize, predictor) and write tags with gdal_edit.py only when the pixels genuinely change. Run dvc status after any metadata edit to confirm whether the hash moved before you commit.

How do I reproduce the exact raster version a model was trained on?

Check out the git commit or tag that recorded the .dvc pointer, then run dvc checkout to materialise the matching COGs from the remote. Because the .dvc file pins the MD5 of every raster, git checkout plus dvc checkout restores byte-identical data. Verify with dvc status, which must report the workspace is up to date with no changes.

Part of: Spatial Dataset Versioning with DVC and LakeFS — Geospatial MLOps and Model Deployment