Skip to content

Visualization

See also: Documentation Index, Knob Importance Analysis, PBT vs BO Comparison

Overview

The src/visualization/ package generates publication-quality figures from PBT session and analysis artifacts. It is a self-contained pipeline that loads JSON results, applies a venue-specific matplotlib theme, and renders figures registered through a central registry.

The package is invoked via the CLI python -m src.visualization and is the canonical way to produce the paper figures from a results tree on disk. It is independent of the tuning loop — it never connects to PostgreSQL, never imports from src/tuners/, and runs offline against saved JSON.

results/                              src/visualization/
├── sessions/{workload}/                   │
│   └── {pbt,bo,lhs}/{tier}/               │ loaders/
│       └── traces/                  ────► │   session.py
│           trace_*.json                   │   baseline.py
├── comparisons/{workload}/          ────► │   comparison.py
│   └── {tier}/                            │   ablation.py
│       comparison_*.json                  │   importance.py
├── analysis/{workload}/             ────► │   multi_seed.py
│   importance_results.json                │
└── ...                                    │ plots/
                                           │   knob_importance.py
                                           │   knob_dependence.py
                                           │   knob_interaction_heatmap.py
                                           │
                                           │ registry.py  ◄── @register_figure
                                           │ theme.py     ◄── PBTuneTheme + VenuePreset
                                           │ export.py
                                           │
                                           ▼
                                       figures/{fig_id}.{pdf|png|svg}

Table of Contents

  1. CLI
  2. The figure registry
  3. Theme engine
  4. Loaders
  5. Built-in plots
  6. Adding a new figure
  7. Related documentation

For the design rationale behind the framework (auto-discovery, loader/renderer separation, why the theme owns sizing), see architecture/visualization.

CLI

# List every registered figure
python -m src.visualization --list

# Generate one figure by ID
python -m src.visualization --figure knob_importance

# Generate all figures in a category
python -m src.visualization --category importance

# Pick a venue preset (sizing + typography)
python -m src.visualization --venue pvldb           # default
python -m src.visualization --venue springer
python -m src.visualization --venue preview         # bigger, on-screen review

# Point at a non-default results tree
python -m src.visualization --data-dir results/ --output-dir figures/

Available venues: pvldb (PVLDB / VLDB single+double column widths, 9 pt serif, LaTeX), springer (Springer LNCS, 10 pt serif), preview (larger sans-serif for on-screen review). The widths and typography are encoded as VenuePreset records in theme.py.

The CLI auto-discovers plot modules: importing src.visualization.plots triggers each module's top-level register_figure(...) call. Adding a new module under src/visualization/plots/ is enough to make it visible to --list.

The figure registry

Location: src/visualization/registry.py

The registry tracks every figure by a stable string ID along with its category, paper section, generator, and data requirements.

@dataclass
class FigureSpec:
    fig_id: str
    paper_label: str
    title: str
    section: str
    category: str
    size_hint: str            # "single" or "double" column
    generator: Callable       # produces AND exports the figure
    data_requirements: list[str]  # e.g. ["session_json"]
    description: str

A plot module registers itself at import time:

from src.visualization.registry import register_figure
from src.visualization.types import FigureSpec

register_figure(FigureSpec(
    fig_id="knob_importance",
    paper_label="fig:importance",
    title="Knob importance overview",
    section="analysis",
    category="importance",
    size_hint="single",
    generator=generate_knob_importance,
    data_requirements=["session_json"],
    description="fANOVA bars with SHAP beeswarm (top 10 knobs).",
))

Public registry methods:

Method Purpose
register(spec) Add a figure (warns on overwrite).
get(fig_id) Retrieve by ID, raises FigureRegistryError if not found.
list_all() / list_by_category(c) / list_by_section(s) Enumerate.
_discover_plots() Walk src.visualization.plots and import every module. Triggered automatically by the CLI.

Auto-discovery means contributors do not edit the registry directly — they write a module under src/visualization/plots/ and the registration happens at import time.

Theme engine

Location: src/visualization/theme.py

PBTuneTheme enforces consistent matplotlib styling across every figure for a chosen venue.

@dataclass
class VenuePreset:
    name: str
    single_col_width_in: float
    double_col_width_in: float
    base_font_size_pt: int
    font_family: str
    use_latex: bool

class PBTuneTheme:
    VENUE_PRESETS: dict[str, VenuePreset] = {
        "pvldb":    VenuePreset(name="pvldb",    single_col_width_in=3.33, ...),
        "springer": VenuePreset(name="springer", single_col_width_in=3.39, ...),
        "preview":  VenuePreset(name="preview",  ...),
    }

    def rc_params(self) -> dict[str, Any]: ...
    def figure(self, size_hint: str = "single", aspect=None, **kwargs) -> tuple[Figure, Axes]: ...
    def subplots(self, nrows, ncols, size_hint="single", aspect=None, **kwargs) -> tuple[Figure, Any]: ...

    @contextmanager
    def apply(self) -> Iterator[None]: ...

The theme owns:

  • Figure sizing — single-column / double-column widths are venue-specific. A generator requests a size by hint (theme.figure(size_hint="single") or "double"), which the theme resolves into a FigureSize — a frozen width_in / height_in dataclass built via FigureSize.single_column(...) / FigureSize.double_column(...).
  • Typography — base font size + family. With use_latex=True the theme switches to LaTeX rendering for all text; the resulting PDFs embed Type 1 fonts compatible with PVLDB / Springer requirements.
  • Color palette — the colorblind-friendly palette from src/visualization/colors.py. Generators should pull colors from colors.py (e.g. get_method_style() / METRIC_COLORS) rather than picking colors directly.
  • Axes styling — uniform tick formatting, spine styles, grid behaviour.

Generators should never mutate plt.rcParams directly. with theme.apply(): installs the theme's rcParams for the plotting block and restores them on exit.

Loaders

Location: src/visualization/loaders/

Each loader knows how to walk a results subtree and build a typed dataclass for the renderer to consume.

Loader Reads Produces
session.py results/sessions/{workload}/{strategy}/{tier}/traces/trace_*.json TuningSession (per-generation history, best config, score breakdown, metadata)
baseline.py Default-PostgreSQL baseline JSONs under results/{workload}/baselines/ Baseline metric distributions
comparison.py results/comparisons/{workload}/{tier}/comparison_*.json ComparisonReport from the post-hoc evaluation suite (see EVALUATION_SUITE.md)
ablation.py Multi-config sweeps for ablation tables Per-condition metric records
importance.py results/analysis/{workload}/importance_results.json fANOVA + TreeSHAP per-knob importance + pairwise interactions
multi_seed.py Multiple seed-tagged session JSONs Per-seed convergence curves with mean / std bands

Loaders are deliberately schema-tolerant: they accept the current session JSON layout and the legacy fixed_v1 layout from older runs. Version migration goes through the same scoring-policy compatibility branch as the post-hoc evaluation suite, so a figure rendered today can include data from a session recorded months ago.

Built-in plots

Location: src/visualization/plots/

Plot module fig_id What it shows
knob_importance.py knob_importance Per-knob fANOVA / SHAP importance bar chart with rank-correlation diagnostic.
knob_dependence.py knob_dependence SHAP dependence plots for the top-K knobs, with hardware-feature coloring.
knob_interaction_heatmap.py knob_interaction_heatmap Pairwise interaction heatmap from fANOVA second-order terms.

Convergence and Pareto plots for the PBT-vs-BO comparison are produced by src/scripts/pbt_vs_bo_comarison.py — see PBT_VS_BO_COMPARISON.md. They predate the registry and are kept in src/scripts/ for now; migrating them into the registry is tracked as a follow-up.

Adding a new figure

  1. Create a loader under src/visualization/loaders/. Return a dataclass; do not pass raw dicts into generators.
  2. Create a plot module under src/visualization/plots/. The module must:
  3. import register_figure and FigureSpec,
  4. define a generate_<your_figure>(*, data_dir, output_dir, theme, formats) -> None function that loads its data, opens a with theme.apply(): block, sizes figures via theme.figure(size_hint=...) / theme.subplots(...), and writes them with export_figure(...),
  5. call register_figure(FigureSpec(...)) at module top level.
  6. Run python -m src.visualization --list to confirm registration.
  7. Run python -m src.visualization --figure <your_id> --venue preview to iterate quickly without LaTeX.

Use the existing knob_importance module as a template — it covers the loader → theme → render → export round-trip in a small file.

File locations