Target Encoding Administrative Regions Without Leakage

Target-encode municipalities and districts for geospatial ML without leaking the label: out-of-fold encoding, spatial folds, smoothing priors and unseen-region fallbacks.

Replace a region code with the mean target of other rows in that region — never including the row being encoded. In practice that means computing the encoding out-of-fold, smoothing each region’s mean towards the global prior in proportion to how few samples it has, and mapping unseen regions to the prior at inference. Anything simpler leaks, and the leak is invisible in a random-split validation score.

This page covers the leakage-safe recipe. For the wider set of options — one-hot, ordinal, hashing, geographic embeddings — see Encoding Categorical Geographic Features.

Out-of-fold encoding: a row never sees its own label Training rows are divided into four folds. For each fold in turn, the region target means are computed from the other three folds only, and those means are assigned to the held-out fold. The result is an encoding column in which no row's value was influenced by that row's own target. Each fold is encoded from the folds it is not in encoding fold 1 fold 1 fold 2 fold 3 fold 4 means from 2–4 encoding fold 2 fold 1 fold 2 fold 3 fold 4 means from 1, 3, 4 encoding fold 3 fold 1 fold 2 fold 3 fold 4 means from 1, 2, 4 encoding fold 4 fold 1 fold 2 fold 3 fold 4 means from 1–3 assembled encoding The final model is then fitted on the assembled column; inference uses full-data means.

Why This Fails in Geospatial ML Pipelines

The naive version — df.groupby("municipality")["target"].transform("mean") — is a one-liner that every team writes once. It leaks because each row is part of the group it is encoded from. In a municipality with 500 samples the self-contribution is negligible; in one with four samples the encoded value is a quarter of the row’s own label, and the model discovers that it can recover the target from the feature. Training loss collapses, random-split validation looks superb, and held-out performance is no better than dropping the column.

Geospatial data makes this worse in two specific ways. First, administrative units are heavy-tailed: a national dataset has a few metropolitan units with tens of thousands of samples and a long run of rural units with three or four. The leak concentrates exactly in the units where the encoding is least reliable. Second, region membership is itself spatial, so a random validation split places samples from the same municipality on both sides of the boundary — the leakage described in reducing spatial leakage in model training. The two leaks compound: the encoding memorises the region, and the split guarantees the region appears in validation.

The third failure appears only in production. Administrative boundaries are re-drawn, municipalities merge, and new codes appear. A map() against a training-time dictionary returns NaN, which most estimators reject outright or, worse, treat as a legitimate value after a careless fillna(0) — where zero is a perfectly plausible target and the model reads it as a confident low prediction.

Core Principles

  • A row must never contribute to its own encoding. Out-of-fold computation is the only reliable way to guarantee that.
  • Use spatial folds for the encoding, not random ones. Encoding folds should match the model’s evaluation folds, or the whole exercise is optimistic.
  • Smooth towards the global prior by count. (n·mean + m·prior) / (n + m) with m around 10–50 samples for administrative units.
  • Map unseen keys to the prior, and count them. Never NaN, never 0.
  • Fit the final encoding on all training data. Out-of-fold values are for training the model; inference uses the full-data mapping.
  • Persist the mapping as an artifact. It is model state, and it must ship with the model, exactly like a scaler.

Production-Ready Code

from __future__ import annotations

import json
import logging
from dataclasses import dataclass, field

import numpy as np
import pandas as pd

logger = logging.getLogger(__name__)


@dataclass
class SmoothedTargetEncoder:
    """Leakage-safe target encoder for high-cardinality geographic keys.

    Attributes:
        smoothing: Pseudo-count m. Higher pulls small regions harder to the prior.
        mapping_: Region -> smoothed target mean, fitted on all training rows.
        prior_: Global target mean, used for unseen regions.
    """
    smoothing: float = 20.0
    mapping_: dict[str, float] = field(default_factory=dict)
    prior_: float = float("nan")
    unseen_count_: int = 0

    def _smoothed(self, frame: pd.DataFrame, key: str, target: str) -> pd.Series:
        stats = frame.groupby(key)[target].agg(["mean", "count"])
        return ((stats["mean"] * stats["count"] + self.prior_ * self.smoothing)
                / (stats["count"] + self.smoothing))

    def fit(self, frame: pd.DataFrame, key: str, target: str) -> "SmoothedTargetEncoder":
        """Fit the inference-time mapping on ALL training rows."""
        if frame[target].isna().any():
            raise ValueError("target contains NaN — drop or impute before encoding")
        self.prior_ = float(frame[target].mean())
        self.mapping_ = self._smoothed(frame, key, target).to_dict()
        logger.info("fitted encoder: %d regions, prior=%.4f", len(self.mapping_), self.prior_)
        return self

    def fit_transform_oof(self, frame: pd.DataFrame, key: str, target: str,
                          folds: np.ndarray) -> pd.Series:
        """Out-of-fold encoding for TRAINING rows, plus a full-data fit for inference.

        Args:
            folds: Fold id per row. Use spatial folds, not random ones.
        """
        self.fit(frame, key, target)
        out = pd.Series(np.nan, index=frame.index, dtype="float64")

        for fold in np.unique(folds):
            held = folds == fold
            rest = frame.loc[~held]
            prior = float(rest[target].mean())
            stats = rest.groupby(key)[target].agg(["mean", "count"])
            smoothed = ((stats["mean"] * stats["count"] + prior * self.smoothing)
                        / (stats["count"] + self.smoothing))
            out.loc[held] = frame.loc[held, key].map(smoothed).fillna(prior).to_numpy()

        assert out.notna().all(), "out-of-fold encoding left NaNs"
        return out

    def transform(self, frame: pd.DataFrame, key: str) -> pd.Series:
        """Encode new rows with the full-data mapping; unseen regions get the prior."""
        if not self.mapping_:
            raise RuntimeError("encoder is not fitted")
        encoded = frame[key].map(self.mapping_)
        unseen = encoded.isna()
        self.unseen_count_ = int(unseen.sum())
        if self.unseen_count_:
            logger.warning("%d rows in %d unseen region(s): %s",
                           self.unseen_count_, frame.loc[unseen, key].nunique(),
                           sorted(frame.loc[unseen, key].unique())[:5])
        return encoded.fillna(self.prior_)

    def save(self, path: str) -> None:
        with open(path, "w") as fh:
            json.dump({"smoothing": self.smoothing, "prior": self.prior_,
                       "mapping": self.mapping_}, fh)

    @classmethod
    def load(cls, path: str) -> "SmoothedTargetEncoder":
        with open(path) as fh:
            blob = json.load(fh)
        enc = cls(smoothing=blob["smoothing"])
        enc.prior_, enc.mapping_ = blob["prior"], blob["mapping"]
        return enc

Step-by-Step Walkthrough

Step 1 — Build spatial folds from the regions themselves

import geopandas as gpd
import numpy as np

samples = gpd.read_file("parcels.gpkg")
regions = samples["municipality"].astype("string")

# Whole regions go to a fold together, so a region never spans the split.
uniq = regions.dropna().unique()
rng = np.random.default_rng(0)
rng.shuffle(uniq)
fold_of_region = {r: i % 5 for i, r in enumerate(uniq)}
folds = regions.map(fold_of_region).to_numpy()

Assigning whole regions to folds is the strictest option and the right default: it measures how the model behaves in a municipality it has never seen, which is what production asks of it.

Step 2 — Produce the out-of-fold column

encoder = SmoothedTargetEncoder(smoothing=20.0)
samples["muni_te"] = encoder.fit_transform_oof(
    samples, key="municipality", target="yield_t_ha", folds=folds
)
print(samples[["municipality", "yield_t_ha", "muni_te"]].head())

Step 3 — Confirm the leak is gone

from scipy.stats import pearsonr

naive = samples.groupby("municipality")["yield_t_ha"].transform("mean")
r_naive, _ = pearsonr(naive, samples["yield_t_ha"])
r_oof, _ = pearsonr(samples["muni_te"], samples["yield_t_ha"])
print(f"naive corr {r_naive:.3f}  out-of-fold corr {r_oof:.3f}")
assert r_oof < r_naive, "out-of-fold encoding should correlate less than the leaky one"

The gap between the two numbers is a direct measure of how much of the naive encoding was self-reference.

Step 4 — Train, then encode inference data with the full mapping

from xgboost import XGBRegressor

features = ["muni_te", "elevation", "slope_deg", "ndvi_mean"]
model = XGBRegressor(n_estimators=400, max_depth=5, learning_rate=0.05).fit(
    samples[features], samples["yield_t_ha"]
)

encoder.save("artifacts/muni_target_encoder.json")

new = gpd.read_file("parcels_2026.gpkg")
new["muni_te"] = encoder.transform(new, key="municipality")
preds = model.predict(new[features])
print(f"{encoder.unseen_count_} rows fell back to the prior")
Smoothing pulls small regions towards the prior A number line runs from the global prior to a high raw regional mean. A region with four samples is shown moving almost all the way back to the prior after smoothing, while a region with two thousand samples barely moves from its raw mean. (n · mean + m · prior) / (n + m), with m = 20 prior 4.8 raw mean 8.1 n = 4 → 5.3 barely trusted n = 2000 → 8.1 trusted as-is Without smoothing, a four-sample municipality would report 8.1 as confidently as a two-thousand-sample one — and the model would believe it. Tune m inside the spatial folds, alongside the model’s own hyperparameters.

Verification

import numpy as np
import pandas as pd


def test_no_row_contributes_to_its_own_encoding():
    """Changing one row's target must not change that row's encoded value."""
    df = pd.DataFrame({"region": list("aaabbbccc"), "y": [1.0, 2, 3, 4, 5, 6, 7, 8, 9]})
    folds = np.array([0, 1, 2] * 3)
    enc = SmoothedTargetEncoder(smoothing=1.0)
    before = enc.fit_transform_oof(df.copy(), "region", "y", folds).to_numpy()

    tweaked = df.copy()
    tweaked.loc[0, "y"] = 100.0
    after = SmoothedTargetEncoder(smoothing=1.0).fit_transform_oof(
        tweaked, "region", "y", folds).to_numpy()

    assert np.isclose(before[0], after[0]), "row 0's own target leaked into its encoding"


def test_unseen_region_gets_the_prior():
    df = pd.DataFrame({"region": ["a", "a", "b"], "y": [1.0, 3.0, 5.0]})
    enc = SmoothedTargetEncoder(smoothing=5.0).fit(df, "region", "y")
    out = enc.transform(pd.DataFrame({"region": ["zzz"]}), "region")
    assert np.isclose(out.iloc[0], enc.prior_)
    assert enc.unseen_count_ == 1


def test_smoothing_moves_small_regions_more():
    df = pd.DataFrame({"region": ["small"] + ["big"] * 200,
                       "y": [10.0] + [10.0] * 200})
    df.loc[df["region"] == "big", "y"] = 10.0
    enc = SmoothedTargetEncoder(smoothing=20.0).fit(df, "region", "y")
    assert abs(enc.mapping_["small"] - enc.prior_) <= abs(enc.mapping_["big"] - enc.prior_)

The first test is the important one. It is the only check that distinguishes a correct implementation from the leaky one-liner, and it runs in milliseconds.

What the leak looks like in the numbers Two pairs of bars show training error and held-out error. For the naive encoding the training error is almost zero while the held-out error is high. For the out-of-fold encoding the two errors are close together, and the held-out error is lower than the naive version's. A huge train–holdout gap is the signature of a leaking encoder 0.08 1.14 naive groupby mean 0.71 0.83 out-of-fold + smoothing train RMSE held-out RMSE The honest encoder trains worse and generalises better — which is the whole point.

FAQ

Why does target encoding leak even when I fit it on the training set only?

Because each training row is part of the group whose mean it receives. In a four-sample region the encoded value is roughly a quarter of the row’s own label, and the model learns to invert that. Computing each row’s value from folds that exclude it removes the self-reference.

What should an unseen region get at inference time?

The global prior from the training set. Never NaN — most estimators reject it — and never 0, which is a plausible target value in most problems and therefore a confident lie. Count and log unseen keys; a rising rate signals that administrative boundaries have changed.

How much smoothing should I use?

Enough that a handful of samples is pulled most of the way to the prior. m between 10 and 50 suits administrative units whose counts span single digits to thousands. Tune it inside the same spatial folds you use for the model, as described in spatial cross-validation strategies.

Part of: Encoding Categorical Geographic Features Part of: Spatial Feature Engineering for Machine Learning