PBT Core Components¶

Overview¶

This document explains the three core classes that implement Population-Based Training in this project: Worker, Evolution, and Population. Together they realise DeepMind's 2017 PBT algorithm adapted for PostgreSQL configuration tuning.

What PBT does. Maintains a population of $N$ workers, each holding its own configuration $\theta_i$ and score $f(\theta_i)$. At each generation, poor performers exploit elites by copying configurations, then explore by perturbing them. Configurations evolve during training rather than being evaluated independently.

What this implementation adds beyond vanilla PBT:

Lockstep generation barriers so every worker's measurement window experiences identical contention from other workers (see GENERATION_BARRIERS.md).
Physical instance cloning during exploit so an exploited worker takes over the elite's data directory + buffer state, not just the knob values.
Per-worker resource slices (WorkerResources) so the population can run safely on a single host with bounded RAM/CPU budgets.
Dead-worker rescue so a single crashed evaluation doesn't block the generation indefinitely.
Adaptive normalisation re-scoring so scores remain comparable as observed metric ranges grow.

Architecture¶

                        ┌──────────────────────┐
                        │      Population      │
                        │    (orchestrator)    │
                        └──────────┬───────────┘
                                   │
              ┌────────────────────┼────────────────────┐
              │                    │                    │
              ▼                    ▼                    ▼
       ┌────────────┐       ┌────────────┐       ┌────────────┐
       │  Worker 0  │       │  Worker 1  │  ...  │  Worker N  │
       │   (state)  │       │   (state)  │       │   (state)  │
       └─────┬──────┘       └─────┬──────┘       └─────┬──────┘
             │                    │                    │
             │ uses               │ uses               │ uses
             ▼                    ▼                    ▼
       ┌──────────────────────────────────────────────────────┐
       │                     Evolution                        │
       │  truncation_selection / execute_exploit_explore /    │
       │  perturb / convergence / population statistics       │
       └──────────────────────────────────────────────────────┘
                                   │
                                   ▼
                        ┌──────────────────────┐
                        │ GenerationBarrier    │
                        │  (B1–B17 lockstep)   │
                        └──────────┬───────────┘
                                   │
                                   ▼
                        ┌──────────────────────┐
                        │ WorkloadOrchestrator │
                        │ (per-worker eval)    │
                        └──────────────────────┘

Roles:

Worker — individual population member. Owns its config, score, history, and lineage.
Evolution — pure functions for selection, perturbation, statistics, convergence detection. No state.
Population — orchestrator. Manages worker lifecycle, parallel evaluation, exploit-explore triggering, dead-worker rescue, generation logging, score finalisation.

Worker¶

Location: src/tuners/pbt/worker.py (PBTWorker, subclass of BaseWorker in src/tuners/engine/worker.py)

A Worker represents a single member of the population. It encapsulates:

the current PostgreSQL configuration,
the most recent PerformanceMetrics and ScoreBreakdown,
evolutionary state (step count, lineage, generation created),
an environment handle to its own PostgreSQL instance.

@dataclass
class Worker:
    worker_id: int
    knob_space: KnobSpace
    knob_config: Dict[str, Any]
    performance_score: float = 0.0
    metrics: Optional[PerformanceMetrics] = None
    score_breakdown: Optional[ScoreBreakdown] = None
    step_count: int = 0
    ready_interval: int = 1
    parent_id: Optional[int] = None
    generation_created: int = 0
    # ... see source for full attribute set

Key methods¶

Method	Purpose
`is_ready()`	`step_count >= ready_interval`. Workers below this count are not eligible for exploit/explore.
`clone_from(other, current_generation, exclude_knobs=None)`	Exploit step. Copies the elite's `knob_config` (skipping any `exclude_knobs`), sets `parent_id`, records the exploit in `config_history`, and resets `step_count` to 0 so the readiness cooldown re-arms: the adopting worker must complete a full `ready_interval` of fresh evaluations before it is eligible to exploit again (ticket #164). Knob values only — it takes no `environment` argument and performs no I/O; the elite's data directory is cloned separately by the population (see Physical instance cloning during exploit).
`perturb(factors)`	Explore step. Calls `KnobSpace.perturb_config()` and applies dependency repair (memory-budget enforcement).
`update_metrics(metrics, score, breakdown)`	Records evaluation results, increments `step_count`.
`get_config_copy()`	Defensive copy for serialisation.
`reset_to_random(seed=None)`	Used by the dead-worker rescue path.
`to_dict()`	Session-serialisation entry point.

Why "ready interval" matters¶

PBT's ready interval prevents premature exploitation: a worker that just exploited an elite needs a full ready_interval of evaluations under its new configuration before it can be ranked as poor again. clone_from enforces this by resetting the worker's readiness counter (step_count → 0) on adoption, so the interval is measured since the member last became ready — the recurring cooldown from Jaderberg et al. (2017), §4.1. Without it, a worker stayed permanently ready once warmed up and a single noisy measurement could trigger a cascade of exploitations generation after generation (bug B1). Typical values: 1 (aggressive, fast convergence), 3 (balanced default), 5 (conservative, more diversity).

Physical instance cloning during exploit¶

clone_from copies knob values only — it takes no environment argument and performs no I/O. The physical data-directory clone is a separate step driven by the population, not by the worker.

Since commit 4165ceb, exploit no longer inherits just the elite's knob values. After execute_exploit_explore returns its (poor_idx, elite_idx) pairs, Population.train_generation() groups them by the elite's worker_id and calls environment.clone_instances(source_worker_id, target_worker_ids) once per source, copying the elite's PostgreSQL data directory into every worker that exploited it.

Why this matters:

A copied configuration without the underlying database state is "cold" — its buffer cache, page cache, and OS-level state are all empty. The next evaluation includes a long warmup tail that has nothing to do with knob quality.
With cloning, the exploit inherits the elite's warmed-up state and its first measured generation reflects the configuration honestly.

The clone path is implemented per environment backend — see bare_metal.py and docker.py. For the full orchestration and grouping semantics see Environment Backends → Instance cloning during exploit.

Evolution¶

Location: src/tuners/pbt/evolution.py

A module of stateless functions implementing the algorithmic core of PBT. Keeping these as functions (not methods on Population) makes them independently testable — see tests/unit/tuners/pbt/.

Public functions¶

truncation_selection(workers, exploit_quantile=0.2, require_ready=True,
                     dead_config_threshold=6.0) -> list[tuple[int, int]]
execute_exploit_explore(workers, exploit_quantile=0.2, perturbation_factors=(0.8, 1.2),
                        current_generation=0, require_ready=True, dead_config_threshold=6.0,
                        exclude_knobs=None, resample_probability=0.0) -> list[tuple[int, int]]
get_elite_workers(workers, quantile=0.2) -> list[Worker]
get_poor_workers(workers, quantile=0.2) -> list[Worker]
get_best_worker(workers) -> Worker
get_population_statistics(workers) -> dict
check_convergence(workers, threshold) -> bool

Truncation selection¶

Identifies which workers should exploit which elites:

Filter to is_ready() workers if require_ready=True.
Sort by performance_score descending.
Slice top α quantile as elites, bottom α as poor.
Pair each poor worker with a uniformly-sampled elite.

The elite pairing is uniform-random (not best-elite always) to prevent the population collapsing to a single point — different poor workers learn from different elites, preserving diversity.

`execute_exploit_explore`¶

The main entry point called once per generation by Population.train_generation(). Returns the (poor_idx, elite_idx) pairs it exploited; the population uses them both to count exploitations and to drive the physical data-directory clone.

1. pairs = truncation_selection(workers, config.exploit_quantile)
2. for (poor_idx, elite_idx) in pairs:
     workers[poor_idx].clone_from(workers[elite_idx], generation)   # knob values only; no I/O
     workers[poor_idx].perturb(config.perturbation_factors)
3. return pairs   # (poor_idx, elite_idx) pairs; the population then drives the physical clone

Convergence¶

check_convergence(workers, threshold) returns True when the population's score standard deviation drops below threshold. This is one of three stopping signals (the others being max generations and early-stopping patience).

Population¶

Location: src/tuners/pbt/population.py

The orchestrator. Holds the worker pool, the WorkloadOrchestrator, the environment factory, the barriers, and the policy/normalisation state.

`PopulationConfig`¶

@dataclass
class PopulationConfig:
    population_size: int = 8
    ready_interval: int = 3
    exploit_quantile: float = 0.25
    perturbation_factors: tuple[float, float] = (0.8, 1.2)
    convergence_threshold: float = 0.05
    max_generations: int = 100
    early_stopping_patience: int = 10
    enable_snapshots: bool = True
    snapshot_restore_interval: int = 10
    num_parallel_workers: int = 1
    worker_resources: Optional[WorkerResources] = None
    # ... see source for full set

The fields beyond the textbook PBT parameters:

enable_snapshots / snapshot_restore_interval — periodic baseline-snapshot restoration to prevent data drift. Implemented per environment backend.
num_parallel_workers — how many workers run concurrently on this host. The orchestrator uses this to slice WorkerResources.
worker_resources — per-worker resource slice (RAM, CPU cores, disk type). Detected at session start by detect_worker_resources().

`GenerationResult`¶

@dataclass
class GenerationResult:
    generation: int
    best_score: float
    mean_score: float
    std_score: float
    num_exploited: int
    best_worker_id: int
    best_config: Dict[str, Any]
    converged: bool
    exploitations: List[Dict[str, int]]   # elite→poor pairs applied this gen (#165)
    exploit_cohort: Dict[str, int]        # {intended, achieved, shortfall} (#165)
    # ... plus per-worker breakdowns and timing

Public methods¶

Method	Purpose
`initialize(initial_configs=None)`	Create N workers. With LHS sampling by default; with explicit configs for warm-start.
`setup_worker_instances()`	Create one PostgreSQL environment per worker (Docker container or bare-metal data dir).
`setup_snapshots()`	Take the baseline snapshot used by `snapshot_restore_interval`.
`evaluate_generation(orchestrator, parallel=True)`	Run all workers; the orchestrator drives the lockstep barriers internally.
`train_generation(orchestrator, parallel=True)`	One full PBT generation: evaluate, exploit-explore, record, check convergence.
`update_metric_ranges_if_needed()`	Performs the one-time initial calibration once ≥ `max(20, 5·population_size)` non-failure samples have accrued (guarded by `_ranges_calibrated`, so it runs once — not a drift-triggered recalibration); fits the normalizer's anchors. The subsequent rescore is done by `_finalize_scores`.
`rescue_dead_workers(...)`	Replace any worker whose environment has died with a freshly-resampled config and a fresh instance.
`record_generation()`	Aggregate per-worker results into a `GenerationResult` and append to history.
`should_stop()`	Combined check: max generations / early-stopping patience / convergence.
`get_best_configuration()`	Return the best `(config, score)` ever observed.
`get_population_summary()`	Final session summary used by the JSON writer in `main.py`.

Score finalisation¶

_finalize_scores() runs every generation (inside Population.train_generation()), not once at session end. Before calibration it only tracks the overall best; on the just-calibrated generation — or when anchors expand or weights change — it calls orchestrator.reload_scoring_engine() and rescores the current workers plus the historical best against the current anchors, so pre- and post-calibration generations stay comparable. A separate global post-hoc rescore helper lives in src/utils/calibration.py and is used by the evaluation suite.

Lockstep generation flow¶

Every generation goes through a strict lockstep sequence enforced by the GenerationBarrier. Each worker thread waits at every barrier point until all workers have arrived — guaranteeing measurement-window overlap.

Generation N

  ┌─────────────────────────────────────────────────────────────┐
  │              Population.evaluate_generation()               │
  └────────────────────────────┬────────────────────────────────┘
                               │ ThreadPoolExecutor(max_workers=N)
                               ▼
   for each worker, in its own thread:
     orchestrator.evaluate_worker(worker)
       ↓
       B1  connect          ← all N workers wait here
       B2  apply config
       B3  restart (or skip)
       B4  reconnect
       B5  verify config
       B6  pre-stats snapshot
       B7  benchmark ready
       B8  warmup
       B9  measurement       ← every worker measures concurrently
       B10 post-stats snapshot
       B11 io delta
       B12 system metrics
       B13 memory pressure
       B14 reliability gate
       B15 vacuum analyze
       B16 score
       B17 disconnect
                               │
                               ▼
                Population.rescue_dead_workers()
                Population.update_metric_ranges_if_needed()
                Population._finalize_scores()
                Population.record_generation()
                execute_exploit_explore()   # module fn, evolution.py
                environment.clone_instances()

Two graceful-degradation paths handle stuck/crashed workers without deadlocking:

barrier.drain_remaining(start_from) — a worker that catches an exception releases its slots in all barriers it hasn't reached yet.
barrier.abort() — when a worker's exception reaches the population's future.result(), it instantly breaks every barrier (BrokenBarrierError on all waiters).

There is no per-barrier timeout — legitimate workloads (e.g. 5-minute OLAP queries) need to wait indefinitely. Both escape paths are driven by a raised exception, so a worker that hangs without raising still blocks its peers — a deliberately accepted residual detailed in generation-barriers §Path 3.

Full barrier table and rationale: GENERATION_BARRIERS.md.

Exploit-explore details¶

After generation N evaluations:

  Sorted by score (descending):
    Elite (top 25%):   [W3: 0.95, W1: 0.89]
    Middle:            [W5: 0.78, W2: 0.72, W4: 0.69, W7: 0.65]
    Poor  (bottom 25%):[W0: 0.58, W6: 0.52]

  Truncation selection (uniform-random pairing):
    W0 → exploit W3
    W6 → exploit W1

  Exploit step (Worker.clone_from — knob values only, no I/O):
    W0.knob_config = deepcopy(W3.knob_config)
    W0.parent_id   = 3
    W0.step_count  = 0

  Explore step (Worker.perturb):
    For each numeric knob k in W0.knob_config:
      k *= discrete factor drawn from {0.8, 1.2}; when the multiplicative move is
           smaller than one grid step, move exactly one grid step instead (so a
           knob is never frozen and zero is not absorbing); integers round rather
           than truncate, then clamp to bounds        # ADR-009 / #167
    Memory budget repaired (KnobSpace.repair_config_dependencies)

  Physical clone (Population.train_generation, after execute_exploit_explore returns):
    group the exploit pairs by elite worker_id, then once per source elite:
      environment.clone_instances(source=W3, targets=[W0, ...])  # PGDATA copy, driven by
                                                                 # the population, not clone_from

  Generation N+1: evaluate with the new configs

Booleans and enums are perturbed differently — booleans flip with a configurable probability, enums probabilistically jump to a neighbour. Numeric knobs use a discrete multiplicative factor with a guaranteed ≥ 1 grid-step move (ADR-009); the earlier special-cased log-space branch is subsumed, because a discrete factor is geometry-preserving for log- and linear-scale knobs alike. See CONFIGURATION_MANAGEMENT.md.

Dead-worker rescue and convergence¶

Dead-worker rescue¶

Population.rescue_dead_workers() runs inside train_generation, immediately after evaluation and before the normalization update. A worker counts as dead only when both hold: its metrics carry a failure_type, and its score is below dead_config_threshold (default 6.0). A failure-tagged worker that still clears the threshold is not rescued, and neither is a low-scoring worker with no failure_type.

Two branches follow:

Alive donors exist — rescue is deferred to execute_exploit_explore, which pairs each dead worker with a genuine elite and perturbs from there.
No alive donors — each dead worker is resampled from an LHS candidate pool via _choose_diverse_resample_config(), which picks the candidate maximising _config_change_ratio() against the worker's previous config subject to resample_min_change_ratio. The worker's score is reset to 0.0, its metrics and score breakdown are cleared, and force_restart_next_eval is set.

When a rescued worker adopts a new configuration through clone_from (the alive-donor branch, via execute_exploit_explore), its step_count is reset to 0 — the readiness cooldown re-arms, so the worker cannot be re-ranked as poor until it has served a full ready_interval again (ticket #164). The no-alive-donor resample branch clears the worker's score, metrics, and score breakdown but leaves step_count untouched; that branch runs only when every worker has failed, where dead workers are rescued regardless of readiness, so the counter is moot there.

This avoids the failure mode where a single environment crash silently halves the population's effective diversity.

Stopping conditions¶

should_stop() returns True if any of:

generation >= max_generations,
generations_without_improvement >= early_stopping_patience,
check_convergence(workers, convergence_threshold).

All three are checked at the end of every generation.

Design decisions¶

1. Functional Evolution module, not methods on Population¶

Evolution is stateless — truncation_selection, perturb, check_convergence are pure functions. This means:

they can be tested without standing up a Population,
the same logic is reused by analysis tooling and the BO baseline (which uses get_best_worker for incumbent extraction),
there's no temptation to silently mutate population state inside an "algorithm" call.

2. Lockstep barriers around the measurement window¶

Without barriers, a worker that finished restarting early would measure under lower contention than a worker still restarting. The barriers force every measurement window to overlap, eliminating that bias. See GENERATION_BARRIERS.md for the full rationale and the abort/drain semantics.

3. Physical instance cloning during exploit¶

A pure knob-value clone is cheap but produces a "cold" inheritor whose first generation's score reflects warmup, not knob quality. Cloning the data directory at exploit time is more expensive but gives the inheritor an honest starting point. The cost is bounded — exploit happens at most once per generation per poor worker.

4. Per-worker resource slicing¶

WorkerResources is computed once per session and passed into KnobSpace.resolve_hardware_ranges(). Every worker sees the same search space but bounded by its slice of host resources. This is what makes "8 parallel workers on one host" safe on memory.

5. Score finalisation every generation¶

Adaptive normalisation means early-generation scores are anchored against narrower ranges than late-generation scores. Without rescoring, the saved session JSON would show artefactual generation-on-generation deltas. _finalize_scores() runs each generation: before calibration it only tracks the overall best, and once calibration fires (or anchors expand / weights change) it reloads the scoring engine and rescores the current workers and historical best against the current anchors.

6. Dead-worker rescue, not session abort¶

A single instance crash should not kill an N-generation tuning session. rescue_dead_workers() swaps in a fresh instance + resampled config, logs the event, and continues. The session JSON records the rescue events so post-hoc analysis can see when they happened.

7. Three stopping signals¶

Max generations is a hard ceiling. Early-stopping patience saves time on plateaued runs. Convergence catches diversity collapse. All three are required because each fires for a different reason.

Generation Barriers — the B1–B17 lockstep mechanism in detail.
Performance Evaluation — the WorkloadOrchestrator and the PerformanceMetrics contract.
Workload Orchestrator — orchestrator internals, restart policy, executor selection.
Configuration Management — KnobSpace, KnobApplicator, perturbation, repair.
Hardware-Aware Normalization — WorkerResources and warm-start.
Feature-Driven Scoring — the scoring math and policies.
Environment Backends — Docker vs bare-metal, instance cloning, snapshot management.

File locations¶

Worker: src/tuners/pbt/worker.py (PBTWorker, subclass of BaseWorker in src/tuners/engine/worker.py)
Population, PopulationConfig, GenerationResult: src/tuners/pbt/population.py
Evolution algorithms: src/tuners/pbt/evolution.py
Generation barriers: src/tuners/engine/barriers.py
Tests: tests/unit/tuners/pbt/