Pinning GDAL and PROJ Versions in a Container Build

Stop PROJ and GDAL drift breaking reprojection in production: pin system and wheel versions together, control PROJ_DATA and datum grids, and assert the stack at container start.

Pin four things together: the base image digest, the system GDAL and PROJ packages, the Python wheels that bind to them, and the datum grids on disk. Then assert all four at container start. Geospatial containers drift in a way most containers do not, because a PROJ upgrade can silently change where coordinates land — not whether the code runs.

This page is the pinning recipe. For image structure, size and layer strategy, see Containerizing Geospatial Inference Pipelines.

Four pinned layers, one startup assertion A stack of four layers is shown: the base image referenced by digest, the system GDAL and PROJ packages at exact versions, the Python wheels bound to those libraries, and the datum grid files on disk. A startup assertion reads all four at run time and refuses to start if any differs from what the image was built with. Pinning three of the four is the same as pinning none base image @sha256:… libgdal 3.8.4, libproj 9.3.1 rasterio 1.3.10, pyproj 3.6.1 datum grids in PROJ_DATA assert_stack() runs at container start refuses to serve on a mismatch A PROJ minor upgrade does not raise — it moves coordinates.

Why This Fails in Geospatial ML Pipelines

The dangerous failure is not a crash. PROJ chooses a transformation pipeline between two coordinate reference systems, and its choice depends on the library version and on which datum grids are present. Upgrade PROJ from 9.2 to 9.3 in an unpinned apt-get install and a transformation that previously used a Helmert approximation may now use a grid — moving every coordinate by one to two metres. On a 10 m raster that is a fifth of a pixel; on a 1 m lidar product it is two pixels, and every sample in the retrained model has shifted relative to the ones in the deployed model.

The second failure is the double stack. pip install rasterio fetches a manylinux wheel that bundles its own GDAL and PROJ. If the image also ran apt-get install libgdal-dev, two copies exist, and which one the process loads depends on LD_LIBRARY_PATH and the linker’s search order. Two builds of the same Dockerfile, months apart, can differ — and the symptom is a PROJ: proj_create_from_database error that appears on one node and not another.

Third, network-fetched grids. PROJ can download datum grids on demand from a CDN. That is convenient on a laptop and unacceptable in a pipeline: the transformation now depends on network reachability and on what the CDN served that day. The same run, repeated, is not guaranteed to produce the same coordinates.

Core Principles

  • One source of GDAL/PROJ. Wheels or system libraries, never both.
  • Reference the base image by digest, not by tag.
  • Pin system packages to exact versions, and record them in the image.
  • Bake the datum grids and set PROJ_DATA; disable network fetching.
  • Assert the whole stack at start-up and fail closed.
  • Include a coordinate golden test in CI — a known point transformed to a known result.

Production-Ready Code

# syntax=docker/dockerfile:1.7
# Base pinned by DIGEST — a tag is a moving target.
FROM ghcr.io/osgeo/gdal:ubuntu-small-3.8.4@sha256:9f2c1b6a4e5d0c73f1a2b8e4c6d9a0f3b5e7c1d2a4f6b8c0d2e4f6a8b0c2d4e6

ENV DEBIAN_FRONTEND=noninteractive \
    PROJ_DATA=/opt/proj \
    PROJ_NETWORK=OFF \
    GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR \
    GDAL_CACHEMAX=256 \
    PYTHONDONTWRITEBYTECODE=1

# Exact system versions. `apt-get install libproj-dev` without a version is drift.
RUN apt-get update && apt-get install -y --no-install-recommends \
        python3-pip=23.0.1+dfsg-1 \
        libproj25=9.3.1-1~jammy0 \
        proj-data=9.3.1-1~jammy0 \
    && rm -rf /var/lib/apt/lists/*

# Bake the datum grids this pipeline needs, then forbid network fetching.
RUN mkdir -p /opt/proj \
    && cp -r /usr/share/proj/. /opt/proj/ \
    && projsync --target-dir /opt/proj --area-of-use "Europe" --system-directory

# Wheels built against the SYSTEM libraries — no bundled second copy.
COPY requirements.txt /tmp/
RUN pip install --no-cache-dir --no-binary rasterio,pyproj -r /tmp/requirements.txt

# Record what was built, so the runtime can compare against it.
RUN python3 -c "import json, rasterio, pyproj; \
    open('/opt/stack.json','w').write(json.dumps({ \
      'gdal': rasterio.__gdal_version__, \
      'proj': pyproj.proj_version_str, \
      'rasterio': rasterio.__version__, \
      'pyproj': pyproj.__version__}))"

COPY geoml/ /app/geoml/
WORKDIR /app
ENTRYPOINT ["python3", "-m", "geoml.entrypoint"]
# geoml/stack.py — imported first by the entrypoint.
from __future__ import annotations

import json
import logging
import os

logger = logging.getLogger(__name__)


def current_stack() -> dict:
    import pyproj
    import rasterio
    return {"gdal": rasterio.__gdal_version__, "proj": pyproj.proj_version_str,
            "rasterio": rasterio.__version__, "pyproj": pyproj.__version__}


def assert_stack(expected_path: str = "/opt/stack.json") -> dict:
    """Fail closed when the runtime stack differs from the build-time stack."""
    with open(expected_path) as fh:
        expected = json.load(fh)
    actual = current_stack()
    drift = {k: (expected[k], actual[k]) for k in expected if expected[k] != actual[k]}
    if drift:
        raise RuntimeError(
            "geospatial stack drifted since build: "
            + ", ".join(f"{k}: built {b}, running {r}" for k, (b, r) in drift.items()))

    proj_data = os.environ.get("PROJ_DATA")
    if not proj_data or not os.path.isdir(proj_data):
        raise RuntimeError(f"PROJ_DATA is not a directory: {proj_data!r}")
    if os.environ.get("PROJ_NETWORK", "OFF").upper() != "OFF":
        raise RuntimeError("PROJ_NETWORK must be OFF for reproducible transformations")

    logger.info("stack ok: GDAL %s, PROJ %s", actual["gdal"], actual["proj"])
    return actual


def transform_fingerprint(src: str = "EPSG:4326", dst: str = "EPSG:25832") -> str:
    """The transformation PROJ selects — the thing that actually moves coordinates."""
    from pyproj import Transformer
    return Transformer.from_crs(src, dst, always_xy=True).description

Step-by-Step Walkthrough

Step 1 — Discover the exact versions before pinning them

docker run --rm ghcr.io/osgeo/gdal:ubuntu-small-3.8.4 bash -lc \
  'apt-cache policy libproj25 proj-data | grep -E "Installed|Candidate"'

Copy those strings into the apt-get install line verbatim. A pin that says libproj25 with no version pins nothing.

Step 2 — Choose one library source and prove it

docker run --rm geoml-inference:2026.08 bash -lc \
  'python3 -c "import rasterio; print(rasterio.__file__)"; ldd $(python3 -c "
import rasterio, pathlib; print(list(pathlib.Path(rasterio.__file__).parent.glob(\"*.so\"))[0])
") | grep -E "gdal|proj"'

Every .so should resolve to /usr/lib, not to a path inside site-packages/rasterio.libs. If both appear, the image has two stacks.

Step 3 — Record the transformation, not just the version

from geoml.stack import transform_fingerprint

print(transform_fingerprint("EPSG:4326", "EPSG:25832"))
# "Inverse of ETRS89 to WGS 84 (1) + UTM zone 32N"

The description names the pipeline PROJ selected. Store it in the model card next to the library versions — it is the highest-signal single string for reproducibility, and it changes when the grids change even if the versions do not.

Step 4 — Assert at start-up

# geoml/entrypoint.py
from geoml.stack import assert_stack

def main() -> None:
    stack = assert_stack()
    logger.info("starting inference with %s", stack)
    run_pipeline()

Failing closed is the point. A container that starts with the wrong PROJ produces plausible, subtly wrong coordinates for hours before anyone notices.

Step 5 — Keep a coordinate golden test in CI

import pytest
from pyproj import Transformer


def test_known_point_transforms_to_known_coordinates():
    """Frankfurt: EPSG:4326 -> EPSG:25832, to the millimetre."""
    t = Transformer.from_crs("EPSG:4326", "EPSG:25832", always_xy=True)
    x, y = t.transform(8.682127, 50.110924)
    assert x == pytest.approx(476_487.334, abs=0.001)
    assert y == pytest.approx(5_551_320.481, abs=0.001)

This is the test that catches a PROJ change no version comparison would flag — for instance, a grid added or removed in the image without a library upgrade.

A different transformation pipeline moves the same point One geographic coordinate is transformed twice. With a Helmert approximation the result is one easting and northing pair. With a grid based transformation, selected after a PROJ upgrade added the grid, the result differs by about one and a half metres, which is a fifth of a pixel on a ten metre raster and two pixels on a one metre raster. Same input, same code, different PROJ — different output lon 8.682127, lat 50.110924 PROJ 9.2 · Helmert 476 487.334, 5 551 320.481 the values the model was trained on PROJ 9.3 · grid-based 476 488.851, 5 551 321.902 1.5 m away · no error raised 1.5 m is 0.15 px at 10 m resolution and 1.5 px at 1 m — the second one shifts every label. The golden coordinate test catches this; a version diff does not always.

Verification

import json
import os
import pytest


def test_stack_matches_build_manifest():
    assert_stack("/opt/stack.json")


def test_only_one_gdal_is_loaded():
    """A bundled wheel library alongside a system one is the classic double stack."""
    import subprocess, pathlib, rasterio
    so = next(pathlib.Path(rasterio.__file__).parent.glob("*.so"))
    out = subprocess.run(["ldd", str(so)], capture_output=True, text=True).stdout
    libs = [ln for ln in out.splitlines() if "libgdal" in ln]
    assert len(libs) == 1, f"multiple GDAL libraries linked:\n{out}"
    assert "site-packages" not in libs[0], "wheel-bundled GDAL is being used"


def test_proj_network_is_disabled():
    assert os.environ.get("PROJ_NETWORK", "OFF").upper() == "OFF"


def test_proj_data_contains_the_grids():
    files = os.listdir(os.environ["PROJ_DATA"])
    assert "proj.db" in files
    assert any(f.endswith(".tif") for f in files), "no datum grids baked into the image"


def test_transformation_pipeline_is_the_expected_one():
    from geoml.stack import transform_fingerprint
    expected = json.load(open("/opt/stack.json")).get("transform_4326_25832")
    if expected:
        assert transform_fingerprint("EPSG:4326", "EPSG:25832") == expected

Run these as the container’s health check as well as in CI. A health check that only reports “process alive” tells you nothing about whether the coordinates are right.

Two rebuilds, six months apart A drifting build referencing a floating tag and unversioned packages produces a different GDAL and PROJ version when rebuilt six months later. A pinned build referencing a digest and exact package versions produces an identical stack on both dates. docker build, run twice six months apart tags GDAL 3.8.4 · PROJ 9.3.1 GDAL 3.9.2 · PROJ 9.4.0 digests GDAL 3.8.4 · PROJ 9.3.1 GDAL 3.8.4 · PROJ 9.3.1 February August Nothing in the Dockerfile changed in either case. Only the pinned build can reproduce a prediction from February.

FAQ

Why do rasterio wheels and a system GDAL conflict?

Manylinux wheels bundle their own GDAL and PROJ. With a system copy also present, which one loads depends on the linker search order, so identical Dockerfiles can behave differently. Pick one source and verify with ldd, as in Step 2.

What actually changes when PROJ changes version?

The transformation pipeline it selects between two CRSs, which depends on the version and the installed grids. A minor upgrade can switch from a Helmert approximation to a grid and move coordinates by one to two metres — with no error at all.

Should datum grids be baked into the image?

Yes. The alternative is PROJ’s network fetching, which makes the transformation depend on what a CDN served at run time. Bake them, point PROJ_DATA at them, and set PROJ_NETWORK=OFF.

Part of: Containerizing Geospatial Inference Pipelines Part of: Geospatial MLOps and Model Deployment